| 1 | # Oversight CHANGELOG |
| 2 | |
| 3 | ## v0.4.12 - 2026-06-17 Security hardening, cross-language parity, JCS canonicalization, and the post-quantum KEM mirror |
| 4 | |
| 5 | - **Minimum supported Rust version raised to 1.85.** The post-quantum KEM |
| 6 | mirror depends on RustCrypto's `ml-kem` crate, which requires Rust 1.85 |
| 7 | (edition 2024). The workspace `rust-version` is bumped from 1.75 to 1.85 |
| 8 | to declare this accurately; the workspace itself stays on edition 2021. |
| 9 | Consumers building from source need a 1.85+ toolchain. |
| 10 | |
| 11 | - **Post-quantum KEM mirror (OSGT-HYBRID-v1), Phase A.** The Rust crypto |
| 12 | crate now implements the ML-KEM-768 half of the hybrid DEK wrap, |
| 13 | byte-identical to the Python reference. New functions |
| 14 | `oversight_crypto::hybrid_wrap_dek`, `hybrid_unwrap_dek`, and |
| 15 | `mlkem768_generate_keypair` reproduce the Python construction exactly: |
| 16 | IKM is `ss_x || ss_pq || x25519_ephemeral_pub || mlkem_ciphertext`, |
| 17 | HKDF-SHA256 with a zero salt and `info=b"oversight-hybrid-v1-dek-wrap"`, |
| 18 | then XChaCha20-Poly1305 with `aad=b"oversight-hybrid-dek"`. The envelope |
| 19 | JSON shape (`suite`, `x25519_ephemeral_pub`, `mlkem_ciphertext`, `nonce`, |
| 20 | `wrapped_dek`, all hex) matches `oversight_core.crypto.hybrid_wrap_dek`, |
| 21 | so a Python-wrapped envelope opens in Rust and vice versa. The primitive |
| 22 | is RustCrypto's pure-Rust `ml-kem` crate (FIPS 203, no C build, chosen |
| 23 | over liboqs because the build environment lacks cmake and the pure-Rust |
| 24 | crate satisfies the same NIST-primitives-via-RustCrypto/liboqs rule). Rust keeps its |
| 25 | decapsulation keys in the recommended 64-byte seed form; only the |
| 26 | 1184-byte public key ever crosses the language boundary, so no deprecated |
| 27 | expanded-key import is needed. Six Rust unit tests mirror |
| 28 | `tests/test_pq.py` (round trip, JSON shape, tamper-classical, |
| 29 | tamper-PQ, wrong recipient, overhead bound), and a new cross-language |
| 30 | conformance vector (`tests/conformance_hybrid_kem.py`) proves both |
| 31 | directions (PY wrap to RS unwrap, RS wrap to PY unwrap) against a live |
| 32 | liboqs, wired into `conformance_cross_lang.sh` as step 5. Scope: KEM |
| 33 | only. The container HYBRID seal/open dispatch and ML-DSA manifest |
| 34 | signing are deferred (Phase B and Phase C respectively); per the owner's |
| 35 | decision, Phase C will use dual Ed25519 AND ML-DSA-65 signatures. |
| 36 | |
| 37 | - **Suppress liboqs-python import-time stdout.** `liboqs-python` attaches a |
| 38 | `StreamHandler(sys.stdout)` at INFO and logs a line when imported, which |
| 39 | contaminated stdout for any caller importing `oversight_core.crypto` and |
| 40 | broke byte-identity conformance capture. `oversight_core/crypto.py` now |
| 41 | redirects stdout during the `import oqs` so the message is dropped. |
| 42 | Behavior of the PQ primitives is unchanged. |
| 43 | |
| 44 | - **Rust `open_sealed` now enforces the `jurisdiction` policy.** The |
| 45 | `oversight-policy::check_policy` function already implemented |
| 46 | jurisdiction matching (and had passing unit tests), but the container |
| 47 | open path never called it: `open_sealed` and `open_sealed_with_provider` |
| 48 | inlined only the `not_after` / `not_before` time checks and skipped |
| 49 | jurisdiction entirely, so the policy crate's enforcement was dead code |
| 50 | on the real open path. Python enforces at `container.py:226` |
| 51 | (`check_policy` before decryption); Rust now mirrors that single |
| 52 | chokepoint. Both Rust open functions now call |
| 53 | `oversight_policy::check_policy(&manifest, policy_ctx)` after issuer |
| 54 | trust and before DEK unwrap, which also retires the duplicated inline |
| 55 | time checks onto the tested canonical path. Observable change: time |
| 56 | violations now surface as `ContainerError::Policy` (typed, matching |
| 57 | Python's `PolicyViolation`) rather than `ContainerError::Precondition`. |
| 58 | Four new container-level tests pin the behavior (mismatch rejected, |
| 59 | match allowed, GLOBAL allowed, no-context skip), and the existing |
| 60 | `expired_file_rejected` assertion is updated to the typed error. |
| 61 | Backward compatibility: GLOBAL manifests (the default) and opens with |
| 62 | no `PolicyContext` (the raw-open path used by the cross-language |
| 63 | conformance harness) are unaffected. Scope honesty: this closes the |
| 64 | open-time gap the review flagged. It does NOT implement the |
| 65 | registry-time MUST in SPEC.md §8.4 (`policy.jurisdiction` mismatch |
| 66 | causing `/register` rejection), which is unimplemented in BOTH |
| 67 | languages and tracked separately. A `--jurisdiction` CLI flag for an |
| 68 | opener asserting a non-default region is a deferred follow-up, not |
| 69 | required for the gap closure (the CLI's default `GLOBAL` context |
| 70 | already rejects a manifest requiring a specific jurisdiction). |
| 71 | |
| 72 | - **Constant-time token comparison via `subtle::ConstantTimeEq`.** The |
| 73 | Oversight Rust registry's operator-token and DNS-secret comparison was |
| 74 | a hand-rolled loop in `oversight-registry/src/auth.rs` with an early |
| 75 | return on length mismatch, which trivially leaked whether an attacker's |
| 76 | guessed length was correct in O(1). Replaced with |
| 77 | `subtle::ConstantTimeEq` (audited RustCrypto crate, already a transitive |
| 78 | dependency via `ed25519-dalek` and `chacha20poly1305`, now declared |
| 79 | explicitly). Four new registry unit tests pin the correctness matrix |
| 80 | (equal inputs across lengths, same-length different content, |
| 81 | mismatched-length no-early-return, single-bit difference). The |
| 82 | hand-rolled early-return leak is closed; the residual |
| 83 | max(supplied, expected) timing observation from `subtle`'s slice impl |
| 84 | is documented in the function-level rationale and is acceptable for |
| 85 | Oversight's high-entropy operator tokens. |
| 86 | |
| 87 | - **Canonicalization unified on RFC 8785 JCS.** The Python reference now |
| 88 | canonicalizes via a vendored `oversight_core.jcs.jcs_dumps` that is |
| 89 | byte-exact with the Rust reference's `serde_jcs::to_vec`. Previously |
| 90 | Python used `json.dumps(sort_keys=True, separators=(",",":"))` with the |
| 91 | default `ensure_ascii=True`, which diverged from Rust for any non-ASCII |
| 92 | string value (Python escaped non-ASCII as `\uXXXX`; Rust emitted raw |
| 93 | UTF-8). The divergence was latent because every committed test fixture |
| 94 | and the conformance harness used ASCII-only content, but any production |
| 95 | manifest with a non-ASCII recipient_id, issuer_id, or filename would |
| 96 | have signed to different bytes across the two implementations and |
| 97 | failed cross-language verification. The port covers manifest signing, |
| 98 | manifest wire form, transparency-log leaf payloads, transparency-log |
| 99 | signed heads, sealed-container wrapped_dek, DSSE statement payloads, |
| 100 | DSSE envelope serialization, registry manifest verification, registry |
| 101 | sidecar comparison, and registry evidence-bundle signing. Standalone |
| 102 | tooling (sample generators, live demo, canary keeper) uses the |
| 103 | `sort_keys=True, ensure_ascii=False` form, which is byte-identical to |
| 104 | JCS for the no-floats subset these tools emit. `conformance_cross_lang.sh` |
| 105 | gains a non-ASCII `recipient_id` round trip that exercises the |
| 106 | divergence end-to-end and would fail under any non-JCS serialization. |
| 107 | Backward compatibility: for ASCII-only content (every committed fixture |
| 108 | and every existing test vector), the new JCS bytes are identical to the |
| 109 | old sort_keys bytes, so existing signatures continue to verify. |
| 110 | |
| 111 | ### Earlier batches in v0.4.12 (pre-JCS, kept for review provenance) |
| 112 | |
| 113 | - **Live registry deployment config.** `docker-compose.yml` now has a `live` |
| 114 | Caddy profile with public TLS routing for the registry, beacon, OCSP-style, |
| 115 | and license-style hostnames. `Caddyfile` covers the full registry v1 |
| 116 | read/evidence/tlog surface plus beacon routes, with all hostnames coming |
| 117 | from environment variables. `.env.example` documents public-safe defaults |
| 118 | and leaves secrets blank. |
| 119 | - **Registry operator token.** The Python reference registry can now require |
| 120 | `OVERSIGHT_OPERATOR_TOKEN` for `POST /register` and `POST /attribute`. |
| 121 | The token is optional so local development and unauthenticated conformance |
| 122 | runs keep working, but production operators can protect write-side APIs |
| 123 | without changing route shapes. The conformance harness sends the token as |
| 124 | a bearer header when `OVERSIGHT_OPERATOR_TOKEN` is set. |
| 125 | - **Rust registry operator-token parity.** The Axum + SQLx registry now reads |
| 126 | `OVERSIGHT_OPERATOR_TOKEN` too and enforces it on `POST /register` and |
| 127 | `POST /attribute` with the same bearer/header contract as the Python |
| 128 | registry. Its DNS event route also accepts either `Authorization: Bearer` |
| 129 | or `X-Oversight-DNS-Secret`, matching the live deployment guide. |
| 130 | - **Rust registry migration tooling.** `oversight-registry` now supports |
| 131 | `--migrate-from <python-registry.sqlite>` plus `--migrate-dry-run`. It |
| 132 | copies the Python reference registry's manifests, beacons, watermarks, |
| 133 | events, and corpus rows into the Rust SQLite schema while preserving event |
| 134 | IDs, corpus metadata, and registry evidence relationships. |
| 135 | - **Rust registry integrity validation.** `oversight-registry --validate-db` |
| 136 | now checks migrated Rust registry databases for orphaned attribution rows, |
| 137 | identity mismatches, malformed manifest JSON, invalid manifest signatures, |
| 138 | and manifest/file ID divergence before operators declare migration burn-in |
| 139 | complete. It also validates event/corpus JSON sidecars and tlog index |
| 140 | uniqueness so corrupted migrated evidence cannot look clean. Rust registry |
| 141 | writes now fail closed if the local transparency log cannot append, and |
| 142 | validation checks missing or out-of-range event tlog indexes against the |
| 143 | on-disk tlog size. Validation also compares event rows to the corresponding |
| 144 | tlog leaf payload so an index cannot point at unrelated evidence and still |
| 145 | pass burn-in checks. Local tlog recovery now rejects malformed records, |
| 146 | non-contiguous indexes, and leaf-hash mismatches instead of silently |
| 147 | ignoring corrupted lines during startup or validation. `/tlog/range` now |
| 148 | reads through the same validated tlog API, so malformed or hash-mismatched |
| 149 | records fail the range request instead of being silently omitted from |
| 150 | monitor responses. The Python reference tlog now matches that behavior: |
| 151 | startup and `/tlog/range` fail closed on corrupt leaf records, and new |
| 152 | leaves carry `leaf_data_hex` so exact leaf bytes survive recovery. |
| 153 | - **Registry v1 error envelope parity.** Python and Rust registry errors now |
| 154 | return the spec envelope `{error: {code, message}}` for registry failures |
| 155 | instead of the framework-native string-only shapes. The conformance harness |
| 156 | now checks `/tlog/range` response shape plus representative |
| 157 | `signature_invalid`, `sidecar_mismatch`, `missing_field`, and `not_found` |
| 158 | error envelopes, raising the live/in-process harness to 38 checks. |
| 159 | - **Registry v1 candidate stability note.** Added |
| 160 | `docs/REGISTRY_V1_STABILITY.md`, which names the candidate-frozen route |
| 161 | surface, JSON field families, error envelope, conformance gate, breaking |
| 162 | change rules, and remaining v1.0 burn-in gates for independent registry |
| 163 | operators. |
| 164 | - **GitHub Actions runtime hygiene.** Main CI workflows opt into the GitHub |
| 165 | Actions Node 24 runtime before the hosted runner default changes. |
| 166 | - **Rust policy test parity.** Fixed the `oversight-policy` crate's manifest |
| 167 | fixture after the v0.4.11 `Recipient.p256_pub` schema addition so the full |
| 168 | Rust workspace test suite compiles again. |
| 169 | - **Deployment docs.** Added `docs/REGISTRY_DEPLOYMENT.md` covering the live |
| 170 | Compose/Caddy flow, route map, token headers, DNS bridge secret, and local |
| 171 | versus live conformance commands. |
| 172 | - **Public description refresh.** Updated README/roadmap/embedding copy to |
| 173 | describe v0.4.11 as the current stable line and the post-tag Rust registry |
| 174 | deployment and migration work on `main`. |
| 175 | - **Code comment style.** Added `CONTRIBUTING.md` guidance to prefer |
| 176 | self-explanatory code and tests over prose-style inline comments, then |
| 177 | removed noisy implementation comments from the Rust registry path. |
| 178 | - **Source comment guard.** Added `scripts/check_source_comments.py`, a pytest |
| 179 | wrapper, and a `source-style` GitHub Actions workflow so strict comment-light |
| 180 | paths fail CI if prose comments are reintroduced. |
| 181 | - **Rust PDF extraction parity.** `oversight-formats` now uses lopdf page text |
| 182 | extraction plus parsed content-stream operations for PDF fingerprint text |
| 183 | instead of raw literal scanning. The fallback handles `Tj`, `TJ`, quote |
| 184 | operators, and array spacing, with new Rust tests covering page-level PDF |
| 185 | text extraction. |
| 186 | - **Rust image DCT parity.** `oversight-formats` now ports the Python image |
| 187 | adapter's DCT mid-band spread-spectrum watermarking path using `rustdct`. |
| 188 | Image watermarking writes the DCT mark and then preserves blind LSB recovery, |
| 189 | with tests for the Python-compatible mark sequence, DCT verification, and |
| 190 | adapter round trip. |
| 191 | - **Outlook add-in hosted pilot page.** `integrations/outlook/index.html` |
| 192 | documents the hosted manifest URL, task pane URL, requested `ReadItem` |
| 193 | permission, same-origin viewer reuse, sideload steps, and remaining tenant |
| 194 | load-test gates for the Outlook read-mode inspector. |
| 195 | |
| 196 | ## v0.4.11 - 2026-05-08 Hardware-keys completion: Python parity, browser support, end-to-end seal |
| 197 | |
| 198 | The `OSGT-HW-P256-v1` suite is now implemented end-to-end across all |
| 199 | three reference implementations: Rust core, Python core, and the |
| 200 | public browser inspector. Every layer of the protocol ships every |
| 201 | suite. The only piece deferred to a follow-up is the `PivKeyProvider` |
| 202 | (PKCS#11 binding to actual hardware tokens) and the matching |
| 203 | `--recipient-hw` CLI flag. |
| 204 | |
| 205 | - **`oversight_core.crypto`: Python parity for `OSGT-HW-P256-v1` |
| 206 | (2026-05-08).** New `wrap_dek_for_recipient_p256` and `unwrap_dek_p256` |
| 207 | mirror the Rust reference byte-for-byte: same HKDF info string |
| 208 | (`"oversight-hw-p256-v1-dek-wrap"`), same AEAD AAD |
| 209 | (`"oversight-hw-p256-dek"`), same SEC1 uncompressed (65 byte) wire |
| 210 | format for the ephemeral public key, same wrapped envelope JSON shape |
| 211 | including the explicit `"suite"` field. `unwrap_dek_p256` accepts |
| 212 | either an `EllipticCurvePrivateKey`, a PKCS#8-encoded private key, or |
| 213 | a raw integer scalar so a future PIV / PKCS#11 binding has a portable |
| 214 | on-ramp. `oversight_core.container` now recognizes `suite_id = 3` and |
| 215 | maps it to `OSGT-HW-P256-v1` in `SUITE_ID_TO_NAME`. New |
| 216 | `tests/test_hw_p256.py` (10 tests) covers the round trip across all |
| 217 | three private-key input forms, the on-wire envelope shape against |
| 218 | `SPEC.md` § 5.2, and the negative paths (wrong recipient, wrong |
| 219 | ephemeral key length, missing fields, AAD binding so a classic |
| 220 | envelope's bytes do not silently decrypt through the hardware path). |
| 221 | |
| 222 | - **`oversight-container`: end-to-end seal/open for `OSGT-HW-P256-v1` |
| 223 | (2026-05-07).** New `seal_hw_p256` mirrors `seal` but consumes a P-256 |
| 224 | SEC1 uncompressed recipient public key and writes a container with |
| 225 | `suite_id = 3`. New `open_sealed_with_provider` is the polymorphic |
| 226 | open path: dispatches on the container's `suite_id` and delegates the |
| 227 | recipient-side ECDH to a `KeyProvider`. Today it supports classic (with |
| 228 | `FileKeyProvider`) and HW P-256 (with `SoftwareP256KeyProvider` or any |
| 229 | future `PivKeyProvider`); a hybrid-aware provider extension lands later. |
| 230 | Cross-suite mismatches (e.g. an X25519 provider on a HW P-256 container) |
| 231 | are refused explicitly. `oversight-manifest::Recipient` gains an |
| 232 | optional `p256_pub: Option<String>` field, gated by `serde(default, |
| 233 | skip_serializing_if = "Option::is_none")` so existing JSON manifests |
| 234 | parse unchanged. Five new round-trip / negative tests; `oversight- |
| 235 | container` now 17/17, workspace builds clean. |
| 236 | |
| 237 | ## v0.4.10 - 2026-05-07 Hardware-keys foundation: KeyProvider trait + OSGT-HW-P256-v1 |
| 238 | |
| 239 | This release lands the abstraction and pure-Rust reference path that the |
| 240 | upcoming `PivKeyProvider` (PKCS#11 against YubiKey / Nitrokey / OnlyKey) |
| 241 | plugs into. Public API is purely additive; all existing v0.4.9 callers |
| 242 | keep working unchanged. |
| 243 | |
| 244 | - **`oversight-container`: `OSGT-HW-P256-v1` recognized by the binary |
| 245 | container (2026-05-07).** Added `SUITE_HW_P256_V1_ID = 3` and extended |
| 246 | `suite_id_for_manifest` to map the new manifest suite. This is the bridge |
| 247 | that lets a future `seal_hw_p256` ride the existing container layout |
| 248 | without reinventing it. New unit test covers the full mapping and asserts |
| 249 | unknown suites still return `None`. 12/12 container tests; workspace |
| 250 | build clean. |
| 251 | |
| 252 | - **`oversight-crypto`: `OSGT-HW-P256-v1` suite implementation (2026-05-07).** |
| 253 | P-256 ECDH wrap/unwrap landed alongside the X25519 path so hardware-backed |
| 254 | recipients (YubiKey / Nitrokey / OnlyKey via PIV) have a complete pure-Rust |
| 255 | reference to plug into. `wrap_dek_for_recipient_p256` accepts SEC1 |
| 256 | uncompressed (65 byte) recipient public keys and produces `WrappedDekP256`. |
| 257 | `SoftwareP256KeyProvider` is the in-memory `KeyProvider` impl that |
| 258 | `PivKeyProvider` will mirror against PKCS#11 next. Cross-suite envelopes |
| 259 | are rejected explicitly: an X25519 provider passed to a P-256 envelope |
| 260 | errors out instead of producing garbage. Eight new unit tests covering |
| 261 | round trips, wrong-recipient rejection, cross-suite rejection, JSON |
| 262 | envelope round-trip, and a regression check that the classic path still |
| 263 | works. `SUITE_HW_P256_V1` constant exported. Adds `p256` (RustCrypto) to |
| 264 | workspace deps with `ecdh` + `arithmetic` features. `oversight-crypto` |
| 265 | passes 21/21; workspace build clean. |
| 266 | |
| 267 | - **`oversight-crypto`: `KeyProvider` trait + `FileKeyProvider` (2026-05-07).** |
| 268 | The recipient-side ECDH path is now abstracted behind `pub trait KeyProvider`, |
| 269 | with `FileKeyProvider` shipping as the X25519 file-backed default. New |
| 270 | `unwrap_dek_with_provider` is byte-identical to `unwrap_dek` for file-backed |
| 271 | keys (asserted by tests) and is the entry point hardware-backed providers |
| 272 | (PIV / PKCS#11) will plug into next, per `docs/HARDWARE_KEYS.md`. Public API |
| 273 | is purely additive: existing `unwrap_dek(wrapped, priv_bytes)` callers are |
| 274 | unchanged. `KeyAlgorithm::P256` reserved for the upcoming `OSGT-HW-P256-v1` |
| 275 | suite. Six new unit tests; workspace build clean; `oversight-crypto` passes |
| 276 | 13/13. |
| 277 | |
| 278 | ## v0.4.9 - 2026-05-07 Hybrid browser decrypt, Rust registry v1, Outlook scaffold |
| 279 | |
| 280 | The browser inspector now decrypts post-quantum sealed files end-to-end, |
| 281 | the Rust registry passes the v1 conformance harness 33/33, and a thin |
| 282 | Outlook task-pane scaffold lets us start a tenant pilot. Format watermark |
| 283 | regressions in `oversight-rust/oversight-formats` are also resolved. |
| 284 | |
| 285 | - **Outlook add-in scaffold landed (2026-05-07).** New `integrations/outlook/` |
| 286 | with the Office add-in 1.1 manifest (`MailApp`, read-mode task pane, |
| 287 | `ReadItem` only), task-pane HTML, and JS that imports the public viewer's |
| 288 | `parseSealed`, `verifyManifestSignature`, and `decryptSealed` directly |
| 289 | from `oversightprotocol.dev/viewer/...` rather than reimplementing crypto. |
| 290 | Decrypts both classic and hybrid suites. Architecture decision recorded in |
| 291 | `docs/OUTLOOK.md`. Status: scaffold; not yet load-tested in an Outlook |
| 292 | tenant. Icons (64/128 px) still pending. |
| 293 | - **Browser inspector: hybrid (post-quantum) decrypt shipped (2026-05-03).** |
| 294 | The viewer at `oversight-protocol.github.io/oversight/viewer/` now decrypts |
| 295 | `OSGT-HYBRID-v1` sealed files end-to-end, in addition to the |
| 296 | `OSGT-CLASSIC-v1` path that shipped earlier. Implementation reuses |
| 297 | WebCrypto X25519 + HKDF-SHA256 and the existing vendored `@noble/ciphers` |
| 298 | XChaCha20-Poly1305, plus a newly vendored `@noble/post-quantum` ML-KEM-768 |
| 299 | for the post-quantum half of the KEM. KEK is bound X-wing-style over both |
| 300 | shared secrets and both ephemeral inputs (`ss_x || ss_pq || eph_pub || |
| 301 | mlkem_ct`), matching `oversight_core.crypto.hybrid_wrap_dek`. New files |
| 302 | on the site: `viewer/vendor/noble-post-quantum-ml-kem-0.6.1.js` (+ three |
| 303 | vendored transitive deps from `@noble/hashes` and `@noble/curves`), |
| 304 | `viewer/samples/tutorial-hybrid.sealed`, and |
| 305 | `viewer/samples/tutorial-hybrid-identity.json`. New "Load hybrid tutorial |
| 306 | identity" button surfaces the test fixture. New tooling: |
| 307 | `tools/gen_hybrid_sample.py` (self-contained sample generator that mirrors |
| 308 | the production hybrid wrap construction, runs anywhere `oqs` and |
| 309 | `cryptography` are available), and `tools/test_hybrid_decrypt_node.mjs` |
| 310 | (Node-based end-to-end smoke test against Node's WebCrypto). |
| 311 | - `oversight-rust/oversight-registry`: added the missing registry v1 |
| 312 | read-only and beacon surface (`/.well-known/oversight-registry`, |
| 313 | `/evidence/{file_id}`, `/tlog/head|proof|range`, `/p/{token_id}.png`, |
| 314 | `/r/{token_id}`, `/v/{token_id}`, `/candidates/semantic`) and tightened |
| 315 | CORS to the public browser-inspector origins with GET/OPTIONS only. The |
| 316 | Axum server now passes the existing 33-check |
| 317 | `tests/test_registry_conformance.py` harness in live-URL mode. |
| 318 | - `oversight-rust/oversight-manifest`: added `canonical_content_hash` and |
| 319 | `l3_policy` to the signed manifest model so Rust verifies Python-signed |
| 320 | v0.4.5+ manifests without dropping signed fields before canonicalization, |
| 321 | while retaining a fallback verification path for older manifests that lack |
| 322 | those default fields. |
| 323 | - `oversight-rust/oversight-formats`: fixed Rust text/image watermark |
| 324 | regressions that were failing the workspace test suite. Text embedding now |
| 325 | keeps L2 trailing-whitespace marks at physical line endings after L1 |
| 326 | zero-width insertion, and image LSB embedding avoids duplicate pixel slots |
| 327 | that could overwrite earlier payload bits. |
| 328 | |
| 329 | ## v0.4.8 - 2026-04-29 Mobile-build portability and rustls-webpki security bump |
| 330 | |
| 331 | Patch release covering two upstream-driven fixes that landed on `main` |
| 332 | since v0.4.7. No new features and no breaking changes. |
| 333 | |
| 334 | - `oversight-rust/oversight-container`: gate the 4 GiB |
| 335 | `MAX_CIPHERTEXT_BYTES` literal to 64-bit targets and fall back to |
| 336 | `usize::MAX` on 32-bit. Required to cross-compile the Rust core for |
| 337 | Android `armv7-linux-androideabi` and `i686-linux-android`, which the |
| 338 | mobile companion (`oversight-protocol/oversight-mobile`, Flutter + |
| 339 | Rust via `flutter_rust_bridge`) embeds unchanged. Behavior is preserved |
| 340 | for any realistic bundle on 32-bit; `usize::MAX` is just under 4 GiB |
| 341 | on those targets. (PR #4, merged 2026-04-26.) |
| 342 | - `oversight-rust` Cargo.lock: bumped `rustls-webpki` from 0.103.12 to |
| 343 | 0.103.13. Patches a reachable panic in CRL parsing |
| 344 | (GHSA-82j2-j2ch-gfr8) and an inverted-meaning URI excluded-subtree |
| 345 | check (rustls/webpki#471). In scope because the Rust registry and |
| 346 | Rekor clients use rustls for TLS. (Dependabot PR #3, merged 2026-04-29.) |
| 347 | |
| 348 | ## v0.4.7 - 2026-04-22 Registry federation hardening and conformance harness |
| 349 | |
| 350 | Federation stops being aspirational when a second operator can prove |
| 351 | compatibility. v0.4.7 hardens the registry v1 interop spec against the |
| 352 | reference implementation and ships a conformance harness that any |
| 353 | operator can point at their deployment. |
| 354 | |
| 355 | - `docs/spec/registry-v1.md`: expanded with the canonicalization algorithm |
| 356 | (`json.dumps(sort_keys=True, separators=(",", ":"))` over UTF-8), the |
| 357 | uniform error envelope and `code` vocabulary, a full endpoint table |
| 358 | including the normative beacon paths (`/p/{token_id}.png`, `/r/{token_id}`, |
| 359 | `/v/{token_id}`), the `/.well-known/oversight-registry` shape, the |
| 360 | `/evidence/{file_id}` bundle fields, and the `/tlog/head|proof|range` |
| 361 | endpoints federated verifiers rely on. Removed a phantom |
| 362 | `/query/{file_id}` endpoint that was in the draft but never shipped. |
| 363 | - `tests/test_registry_conformance.py`: 32-check harness with two modes. |
| 364 | In-process against a FastAPI `TestClient` for CI, or against a live URL |
| 365 | when `OVERSIGHT_REGISTRY_URL` is set. Covers identity, liveness, a full |
| 366 | signed-manifest registration round trip, attribution by token id, |
| 367 | evidence bundle shape, transparency-log head, every beacon endpoint, |
| 368 | and DNS event authentication. |
| 369 | - `docs/ROADMAP.md`: the registry federation item references the harness |
| 370 | as the acceptance gate for federation. |
| 371 | - Version bumped to `0.4.7`. No breaking changes. |
| 372 | |
| 373 | ## v0.4.6 - 2026-04-22 SIEM export: Splunk, Sentinel, and Elastic |
| 374 | |
| 375 | Registry beacon events can now be emitted in three SIEM-native formats so |
| 376 | security teams get Oversight data into the incident pipeline they already |
| 377 | run. Formatters are pure; transport is a thin sink layer. |
| 378 | |
| 379 | - `oversight_core/siem.py`: new module. Normalized `OversightEvent` model |
| 380 | built from the registry `events` table, pure formatters for Splunk HEC, |
| 381 | Elastic Common Schema 8.x, and Microsoft Sentinel (Log Analytics custom |
| 382 | logs), plus `sentinel_authorization()` helper that signs the Data |
| 383 | Collector API `Authorization` header per Microsoft's recipe. |
| 384 | - `cli/oversight.py`: new `oversight siem export` subcommand. Streams |
| 385 | events as JSON lines to stdout, a file, or an HTTPS collector. Supports |
| 386 | `--since`, `--limit`, repeatable `--header`, and Splunk source/sourcetype/ |
| 387 | index overrides. Opens the registry database read-only so it is safe |
| 388 | to run against a live service. |
| 389 | - `docs/SIEM.md`: operator integration guide covering each of the three |
| 390 | SIEMs, the event field dictionary, the Sentinel HMAC signing window, |
| 391 | and the honest beacon-absence caveat. Also surfaced from the website |
| 392 | docs index. |
| 393 | - `tests/test_siem_unit.py`: 11 focused unit tests covering envelope |
| 394 | shape per format, empty-field suppression, SQLite row mapping, |
| 395 | read-only iteration, Sentinel HMAC stability, and action-name |
| 396 | coverage for every beacon kind. |
| 397 | - `oversight_core/__init__.py` and `pyproject.toml`: version bumped to |
| 398 | `0.4.6`. No breaking changes; SIEM is additive. |
| 399 | |
| 400 | ## v0.4.5 - 2026-04-20 L3 safety, GUI, and registry federation docs |
| 401 | |
| 402 | Review-driven hardening from the local security review report. |
| 403 | |
| 404 | - `oversight_core/l3_policy.py`: new L3 safety policy engine. L3 defaults off |
| 405 | for legal, regulatory, technical/spec, source-code, SQL, log, and structured |
| 406 | data classes; explicit `full`, `boilerplate`, and `off` modes are supported. |
| 407 | - `cli/oversight.py` and `cli/oversight_rich.py`: seal-time L3 disclosure now |
| 408 | requires acknowledgement when L3 is enabled, and seal manifests record the |
| 409 | applied L3 policy. |
| 410 | - `oversight_core/manifest.py`: manifests now carry `canonical_content_hash` |
| 411 | so auditors can diff recipient copies against the original source bytes. |
| 412 | - `oversight_core/watermark.py` and `oversight_core/formats/text.py`: high-level |
| 413 | L3 application is opt-in; L1/L2 remain available by default. |
| 414 | - `cli/gui.py`: added a Tkinter desktop GUI for key generation, sealing, and |
| 415 | opening files (`oversight gui`) so non-technical users have a starter path. |
| 416 | - GUI and CLI output writes now fail closed against private-key overwrites, |
| 417 | same-path writes, reserved Windows device names, malformed key files, and |
| 418 | non-UTF-8 watermark attempts. Private-key writes use atomic replacement and |
| 419 | restrictive permissions/ACL hardening where supported. |
| 420 | - `.sealed` parsing now rejects tampered suite IDs, malformed manifest/wrapped-DEK |
| 421 | JSON, unknown manifest fields, and trailing bytes after ciphertext. |
| 422 | - `oversight-rust/oversight-container`: Rust now mirrors the Python parser's |
| 423 | strictness by rejecting suite-byte tamper and trailing bytes after the |
| 424 | authenticated ciphertext region. |
| 425 | - `docs/security.md`: documented L3 collusion/canonicalization limits, layer |
| 426 | survival properties, passive beacon limits, jurisdiction-by-IP limits, and |
| 427 | RFC 3161 timestamp semantics. |
| 428 | - `docs/spec/registry-v1.md`: added a registry federation/interoperability |
| 429 | draft for independent compatible registry operators. |
| 430 | - `docs/ROADMAP.md`: corrected launch sequencing, dropped near-term FedRAMP, |
| 431 | scoped ecosystem plugins to Outlook-first, and prioritized SIEM integration |
| 432 | before SOC 2 / ISO 27001 work. |
| 433 | - Raised vulnerable dependency floors flagged by Dependabot/PyPI advisory |
| 434 | checks: setuptools, cryptography, PyNaCl, pydantic, python-multipart, |
| 435 | Pillow, and pypdf now require patched minimums; Rust manifest floors |
| 436 | now pin patched minima for sqlx, tokio, rand_core, zip, chrono, regex, |
| 437 | once_cell, and tracing-subscriber. |
| 438 | - Added focused regression coverage in `tests/test_l3_policy_unit.py`. |
| 439 | |
| 440 | ## v0.4.4 - 2026-04-20 security hardening |
| 441 | |
| 442 | Security patch line started from the `v0.4.3` Python package baseline |
| 443 | (`0b1a4ab`) and incorporates the Codex review fixes made on 2026-04-20. |
| 444 | This is the current `main` download line. Historical `v0.5.0` Rekor/Rust |
| 445 | work remains in git history and the Rust workspace, but the Python package |
| 446 | metadata now intentionally advances from `0.4.3` to `0.4.4` so users do not |
| 447 | confuse the hardened tree with the vulnerable `v0.4.3` baseline. |
| 448 | |
| 449 | - `oversight_core/container.py`: `max_opens` now increments only after a |
| 450 | successful decrypt, and unsafe `seal_multi()` is disabled until the |
| 451 | manifest format can honestly represent multiple recipients. |
| 452 | - `oversight_core/policy.py`: `LOCAL_ONLY` counter locking now works on |
| 453 | Windows, and `REGISTRY` / `HYBRID` fail closed instead of silently using |
| 454 | local state. |
| 455 | - `oversight_core/rekor.py`: offline verification now rejects DSSE envelopes |
| 456 | whose subject digest does not match the expected content hash. |
| 457 | - `registry/server.py`: Rekor attestations now use real watermark mark IDs |
| 458 | and the manifest's actual `content_hash`, and `/register` now rejects |
| 459 | unsigned beacon / watermark sidecars that do not match the signed manifest. |
| 460 | - `oversight_core/formats/text.py`: text adapter now applies L3 before L2/L1, |
| 461 | matching the core watermark pipeline. |
| 462 | - `oversight_core/tlog.py`: empty-tree roots now use the RFC 6962 Merkle |
| 463 | hash (`SHA-256("")`) instead of an all-zero placeholder. |
| 464 | - `oversight_core/__init__.py`, `pyproject.toml`, and the Rich CLI banner: |
| 465 | version metadata is now `0.4.4`, marking this post-`0.4.3` hardening train. |
| 466 | - `oversight_dns/server.py` and `registry/server.py`: DNS beacon callbacks now |
| 467 | support a shared `OVERSIGHT_DNS_EVENT_SECRET`, and non-loopback callbacks |
| 468 | fail closed when no secret is configured. |
| 469 | - `registry/server.py`: evidence bundles now include local transparency-log |
| 470 | inclusion proofs for recorded events, not just the signed tree head. |
| 471 | - `oversight-rust`: removed the direct `rand` dependency in favor of |
| 472 | `rand_core::OsRng`, clearing the low-severity `rand` advisory path. |
| 473 | - `oversight-rust/oversight-registry`: `/dns_event` now requires |
| 474 | `OVERSIGHT_DNS_EVENT_SECRET` for non-loopback callbacks, signed |
| 475 | beacon/watermark artifacts fail registration when malformed instead of being |
| 476 | silently dropped, and Rekor attestation skips watermarkless registrations |
| 477 | rather than logging `mark:<file_id>`. |
| 478 | - `oversight-rust/oversight-container` and `oversight-rust/oversight-policy`: |
| 479 | Rust opens can now enforce `max_opens` after successful recipient decrypt, |
| 480 | `REGISTRY` / `HYBRID` modes fail closed instead of falling back to local |
| 481 | counters, and Rust `seal_multi()` fails closed until recipient-honest |
| 482 | manifests exist. |
| 483 | - `oversight-rust/oversight-rekor`: offline verification now mirrors Python by |
| 484 | rejecting DSSE envelopes whose subject digest does not match the expected |
| 485 | content hash. |
| 486 | - `oversight-rust/oversight-formats`: DOCX metadata insertion no longer reports |
| 487 | success when `<cp:keywords>` is missing, and PDF processing rejects indirect |
| 488 | Launch / JavaScript / unsafe URI actions before rewriting files. |
| 489 | - Added focused regression coverage in `tests/test_policy_unit.py`, |
| 490 | `tests/test_registry_unit.py`, `tests/test_rekor_unit.py`, |
| 491 | `tests/test_text_format_unit.py`, and `tests/test_tlog_unit.py`. |
| 492 | |
| 493 | Patch sequence on top of `v0.4.3`: |
| 494 | |
| 495 | 1. `0.4.3` / `0b1a4ab`: Rich CLI, anti-stripping defenses, and L3 |
| 496 | integration baseline. |
| 497 | 2. `0.4.4` / `dab6157`: policy and Rekor verification hardening. |
| 498 | 3. `0.4.4` / `4d60e3b`: registry Rekor mark indexing fix. |
| 499 | 4. `0.4.4` / `20a566b`: multi-recipient sealing fails closed until the |
| 500 | manifest can represent multiple recipients honestly. |
| 501 | 5. `0.4.4` / `482f294`: default beacon/registry domain updated from |
| 502 | `oversight.example` to `oversightprotocol.dev`. |
| 503 | 6. `0.4.4` / `7712f98`: signed registry sidecars enforced and RFC 6962 |
| 504 | empty tlog roots fixed. |
| 505 | 7. `0.4.4` / `0a7a2da`: package, core, and CLI version metadata |
| 506 | aligned to the hardened `0.4.4` line. |
| 507 | 8. `0.4.4` / `69e50aa`: public changelog patch chronology documented. |
| 508 | 9. `0.4.4` / `26db8d3`: DNS evidence hardening, Rust RNG dependency |
| 509 | cleanup, and evidence-bundle inclusion proofs. |
| 510 | 10. `0.5.0+` / `b9bee41`: Claude-added Rust format adapters, Axum registry, |
| 511 | and USENIX benchmark scaffolding. |
| 512 | 11. `0.5.0+` / current hardening commit: Codex audit fixes for the new Rust |
| 513 | registry/container/policy/Rekor/format-adapter security regressions. |
| 514 | |
| 515 | ## v0.5.0 - 2026-04-19 |
| 516 | |
| 517 | First release with public-Rekor attestations. Now hosted at |
| 518 | https://github.com/oversight-protocol/oversight (so the v0.5 predicate URI |
| 519 | resolves for any third-party verifier). |
| 520 | |
| 521 | ### Session B (registry wiring + e2e + backcompat) |
| 522 | - `registry/server.py`: `/register` now opt-in attests each registration into |
| 523 | a public Rekor v2 log. Off by default; opt in with |
| 524 | `OVERSIGHT_REKOR_ENABLED=1`. Failures non-fatal - local SQLite tlog stays |
| 525 | authoritative for "list marks for issuer X" queries. |
| 526 | - `oversight_core/rekor.py upload_dsse`: fixed three wire-shape bugs against |
| 527 | current rekor-tiles proto (`verifier`→`verifiers` array, `keyDetails` as |
| 528 | sibling of `publicKey`, `raw_bytes` carries DER not PEM). Verified live |
| 529 | against `log2025-1.rekor.sigstore.dev` - got real `log_index` returned. |
| 530 | - `tests/test_rekor_e2e.py`: 2 live tests, gated behind |
| 531 | `OVERSIGHT_REKOR_E2E=1` so default runs do not append entries to the |
| 532 | public log. |
| 533 | - `tests/test_rekor_backcompat.py`: 5 offline checks of v0.4 contract |
| 534 | preservation. |
| 535 | |
| 536 | ### Session C (Rust crate + cross-language conformance + version bump) |
| 537 | - New crate `oversight-rust/oversight-rekor`: bit-identical port of |
| 538 | `oversight_core.rekor`. 9 inline tests cover PAE byte-exactness, |
| 539 | sign/verify round trip, tamper + wrong-key rejection, statement shape, |
| 540 | canonical envelope JSON, and offline TLE inclusion check. |
| 541 | - New conformance suite `oversight-rust/tests/conformance_rekor.sh`: proves |
| 542 | Python ↔ Rust bit-identity in 4 ways - PAE bytes, Python-signs/Rust-verifies, |
| 543 | Rust-signs/Python-verifies, canonical payload bytes for the same statement. |
| 544 | - Version bumped to 0.5.0 across `oversight-rust/Cargo.toml`, `README.md`, |
| 545 | `docs/SPEC.md`. |
| 546 | |
| 547 | Hard constraints respected: no new crypto primitives (RustCrypto + |
| 548 | `cryptography`'s Ed25519 only), test count additions-only, Python ↔ Rust |
| 549 | bit-identity proven by conformance script. |
| 550 | |
| 551 | ## Unreleased - v0.5 Session A (2026-04-19) |
| 552 | - Added `docs/V05_REKOR_PLAN.md`: full Rekor v2 migration plan, verified |
| 553 | against current upstream API (Rekor v2 GA 2025-10-10, DSSE + hashedrekord |
| 554 | only, tile-backed reads, no online proof API, public log shard rotates |
| 555 | ~6 months). |
| 556 | - Added `oversight_core/rekor.py` (~280 LOC): DSSE statement construction, |
| 557 | PAE-exact signing/verification against the spec, Rekor v2 `/api/v2/log/entries` |
| 558 | upload helper, offline inclusion-check helper, and `build_bundle()` shaper. |
| 559 | - Added `docs/predicates/registration-v1.md`: the URI the predicate type |
| 560 | resolves to, with privacy contract and field schema. |
| 561 | - Added `tests/test_rekor_unit.py` with 10 offline unit tests covering DSSE |
| 562 | PAE, sign/verify, tamper rejection, wrong-key rejection, statement shape, |
| 563 | canonical envelope JSON, offline bundle verification, the recipient-pubkey |
| 564 | privacy guarantee, predicate-version int, and 5-year-replay bundle fields. |
| 565 | - Six desktop-review fixes baked into Session A before commit: |
| 566 | - Recipient X25519 pubkey now SHA-256 hashed before going on-log |
| 567 | (deanonymization fix). |
| 568 | - Predicate URI pinned to git-tagged GitHub path, not `oversight.dev`. |
| 569 | - Bundle gained `bundle_schema: 2` integer + `log_pubkey_pem` + |
| 570 | `checkpoint` + `log_entry_schema` + optional `rfc3161_chain`. |
| 571 | - Conformance script `oversight-rust/tests/conformance_cross_lang.sh` now |
| 572 | derives REPO_ROOT from its own location instead of `/home/claude` hardcode. |
| 573 | - `HANDOFF.md` gained explicit "what NOT to accept from a future Claude |
| 574 | session" section per the v0.4.1 retro. |
| 575 | |
| 576 | Test count: 76 → 86 (additions only, baseline conformance still green). |
| 577 | |
| 578 | ## v0.4.1 - 2026-04-18 |
| 579 | |
| 580 | Cosmetic polish only, no functionality changes. |
| 581 | |
| 582 | ### Fixed |
| 583 | - Removed unused `std::path::Path` import from `oversight-policy` - clean |
| 584 | `cargo build --workspace --release` with zero warnings. |
| 585 | - Rust workspace version bumped to 0.4.1 across all crates via |
| 586 | `version.workspace = true`. |
| 587 | |
| 588 | ### No behavioral changes |
| 589 | All 76 tests (31 Python + 42 Rust + 3 conformance) still green. |
| 590 | |
| 591 | --- |
| 592 | |
| 593 | ## v0.4.0 - 2026-04-17 |
| 594 | |
| 595 | **Rust port expands from core to core+enforcement+semantics.** Three new Rust |
| 596 | crates; Python reference unchanged in functionality but with RFC 6962 fix. |
| 597 | |
| 598 | ### Added |
| 599 | |
| 600 | - **`oversight-tlog`** Rust crate (367 LOC). RFC 6962-compliant Merkle tree |
| 601 | from day one - left-heavy largest-power-of-2 split, not the promote-odd |
| 602 | shortcut from the Python v0.2 tlog. Signed tree heads, inclusion proofs, |
| 603 | durable append (fsync), automatic recovery on reopen. 7 tests. |
| 604 | - **`oversight-policy`** Rust crate (284 LOC). TOCTOU-safe `max_opens` |
| 605 | enforcement via `fs2::FileExt::lock_exclusive` + atomic temp-file rename. |
| 606 | File-ID sanitization against path traversal. Jurisdiction / not_after / |
| 607 | not_before checks. 6 tests. |
| 608 | - **`oversight-semantic`** Rust crate (345 LOC + 156-line dictionary file). |
| 609 | Full port of the 151-class synonym dictionary and L3 watermarking. |
| 610 | Airgap-strip-survivor verified (tests embed, then strip zero-width and |
| 611 | trailing whitespace, then verify - still attributes). URL / email / code |
| 612 | / path / hex / base64 skip regions. 8 tests. |
| 613 | - **Fuzz harness** (`oversight-rust/fuzz/`) - two `cargo-fuzz` targets |
| 614 | hammering the container parser and manifest parser. Excluded from main |
| 615 | workspace so normal builds don't need nightly. README with 24-hour |
| 616 | pre-audit run recommendation. |
| 617 | - **`docs/HARDWARE_KEYS.md`** - vendor-neutral setup guide for YubiKey / |
| 618 | Nitrokey / OnlyKey. Covers PIN/PUK setup, PIV slot 9d provisioning, |
| 619 | Oversight identity-file format for hardware-backed recipients, curve |
| 620 | choice rationale (P-256 for PIV compat vs X25519 file-backed), revocation |
| 621 | procedure, threat model, deployment checklist. |
| 622 | |
| 623 | ### Fixed |
| 624 | |
| 625 | - **`oversight_core/tlog.py`** now RFC 6962 compliant. Replaced the |
| 626 | promote-odd-trailing shortcut with the canonical largest-power-of-2 |
| 627 | left-heavy split. Added `_rfc6962_mth`, `_rfc6962_path`, |
| 628 | `verify_inclusion_proof` helpers. Tested with asymmetric sizes |
| 629 | (n ∈ {1,2,3,4,5,7,8,16,17,100}) - every leaf's proof verifies; |
| 630 | tampered proofs rejected. Old custom Merkle logic removed. |
| 631 | - **Mutex self-deadlock** in `oversight-tlog::inclusion_proof` - was |
| 632 | holding the leaves lock while calling `root()` which also locks. |
| 633 | Fixed by dropping the lock before invoking `root()`. |
| 634 | - **`oversight-semantic` round-trip bug** - `embed_synonyms` could pick |
| 635 | hyphenated variants like `"write-up"` that `WORD_RE` tokenizes as two |
| 636 | separate words, desyncing the verify sequence. Both embed and verify |
| 637 | now explicitly skip non-round-trippable variants (whitespace or hyphen). |
| 638 | |
| 639 | ### Changed |
| 640 | |
| 641 | - **Workspace version** bumped to `0.4.0`. Python reference remains `v0.3` |
| 642 | (unchanged feature set, one correctness fix). |
| 643 | - **SealedFile** gained `#[derive(Debug)]` to support test assertions with |
| 644 | `{:?}` formatting. |
| 645 | |
| 646 | ### Known limitations (unchanged from v0.3) |
| 647 | |
| 648 | - Paraphrasing attack defeats all three watermark levels. |
| 649 | - Airgapped readers leave no network beacon. |
| 650 | - Hardware-backed recipients require v0.5+ `KeyProvider` abstraction (not |
| 651 | yet implemented - currently file-backed only). |
| 652 | - Format adapters (image DCT, PDF, DOCX) remain Python-only until v0.6. |
| 653 | - Registry server (FastAPI) remains Python-only until v1.0. |
| 654 | |
| 655 | ## v0.3.0 - 2026-04-17 |
| 656 | |
| 657 | See earlier commits. Initial Rust core + FreeTSA RFC 3161 + cross-language |
| 658 | conformance + SENTINEL→Oversight rename + Nitro→YubiKey pivot. |
| 659 | |
| 660 | ## v0.2.1 and earlier |
| 661 | |
| 662 | Python-only; see git history. |