| 1 | # Registry Deployment |
| 2 | |
| 3 | This is the public-safe live configuration for the reference Oversight |
| 4 | registry. It keeps secrets in `.env`, keeps the registry process off the public |
| 5 | host interface, and exposes TLS through Caddy. The Python FastAPI registry and |
| 6 | the Rust Axum registry both honor the write-side operator token described here. |
| 7 | |
| 8 | ## Layout |
| 9 | |
| 10 | - `Dockerfile` builds the Python reference registry image. |
| 11 | - `docker-compose.yml` runs the registry on loopback and adds a `live` profile |
| 12 | for Caddy. |
| 13 | - `Caddyfile` routes the registry, beacon, OCSP-style, and license-style |
| 14 | hostnames to the registry container. |
| 15 | - `.env.example` lists every live setting without carrying secret values. |
| 16 | |
| 17 | ## First Run |
| 18 | |
| 19 | ```bash |
| 20 | cp .env.example .env |
| 21 | # Fill OVERSIGHT_DNS_EVENT_SECRET and OVERSIGHT_OPERATOR_TOKEN in .env. |
| 22 | # Use high-entropy random values; never commit .env. |
| 23 | |
| 24 | docker compose up -d oversight-registry |
| 25 | curl http://127.0.0.1:8765/health |
| 26 | ``` |
| 27 | |
| 28 | For public TLS, point DNS for the four configured hostnames at the host, then |
| 29 | start the live profile: |
| 30 | |
| 31 | ```bash |
| 32 | docker compose --profile live up -d |
| 33 | ``` |
| 34 | |
| 35 | ## Public Routes |
| 36 | |
| 37 | `OVERSIGHT_REGISTRY_DOMAIN` serves the registry metadata, evidence, tlog, and |
| 38 | operator routes: |
| 39 | |
| 40 | - `GET /health` |
| 41 | - `GET /.well-known/oversight-registry` |
| 42 | - `GET /evidence/{file_id}` |
| 43 | - `GET /tlog/head` |
| 44 | - `GET /tlog/proof/{index}` |
| 45 | - `GET /tlog/range` |
| 46 | - `GET /candidates/semantic` |
| 47 | - `POST /register` |
| 48 | - `POST /attribute` |
| 49 | - `POST /dns_event` |
| 50 | |
| 51 | The beacon hostnames route only their beacon families: |
| 52 | |
| 53 | - `OVERSIGHT_BEACON_DOMAIN`: `/p/{token}.png` |
| 54 | - `OVERSIGHT_OCSP_DOMAIN`: `/r/{token}` and `/ocsp/r/{token}` |
| 55 | - `OVERSIGHT_LICENSE_DOMAIN`: `/v/{token}` and `/lic/v/{token}` |
| 56 | |
| 57 | Everything else returns `404`. |
| 58 | |
| 59 | ## Operator Authentication |
| 60 | |
| 61 | If `OVERSIGHT_OPERATOR_TOKEN` is set, `POST /register` and `POST /attribute` |
| 62 | require either: |
| 63 | |
| 64 | ```http |
| 65 | Authorization: Bearer <token> |
| 66 | ``` |
| 67 | |
| 68 | or: |
| 69 | |
| 70 | ```http |
| 71 | X-Oversight-Operator-Token: <token> |
| 72 | ``` |
| 73 | |
| 74 | Leaving `OVERSIGHT_OPERATOR_TOKEN` empty keeps the v1 conformance harness and |
| 75 | local development behavior unchanged. Do not leave it empty on a public |
| 76 | operator deployment. Both reference registry implementations use the same |
| 77 | token contract, so live conformance commands work against either backend. |
| 78 | |
| 79 | DNS bridge callbacks are separate. Set `OVERSIGHT_DNS_EVENT_SECRET`; the DNS |
| 80 | bridge must send either `Authorization: Bearer <secret>` or |
| 81 | `X-Oversight-DNS-Secret: <secret>` when posting `/dns_event`. |
| 82 | |
| 83 | ## Conformance |
| 84 | |
| 85 | Local reference check: |
| 86 | |
| 87 | ```bash |
| 88 | python tests/test_registry_conformance.py |
| 89 | ``` |
| 90 | |
| 91 | Live check against a token-protected registry: |
| 92 | |
| 93 | ```bash |
| 94 | OVERSIGHT_REGISTRY_URL=https://registry.example.org \ |
| 95 | OVERSIGHT_OPERATOR_TOKEN=<token> \ |
| 96 | python tests/test_registry_conformance.py |
| 97 | ``` |
| 98 | |
| 99 | The token is read from the environment and sent as a bearer header. Do not put |
| 100 | real token values in shell history on shared machines. |
| 101 | |
| 102 | ## Migrating Python Registry Data To Rust |
| 103 | |
| 104 | The Rust Axum registry can import the Python reference registry's SQLite rows |
| 105 | without mutating the source database. Run a dry run first: |
| 106 | |
| 107 | ```bash |
| 108 | oversight-registry \ |
| 109 | --db /var/lib/oversight/rust-registry.sqlite \ |
| 110 | --migrate-from /var/lib/oversight/python-registry.sqlite \ |
| 111 | --migrate-dry-run |
| 112 | ``` |
| 113 | |
| 114 | The command prints JSON row counts for `manifests`, `beacons`, `watermarks`, |
| 115 | `events`, and `corpus`. If the counts are expected, run the same command |
| 116 | without `--migrate-dry-run`: |
| 117 | |
| 118 | ```bash |
| 119 | oversight-registry \ |
| 120 | --db /var/lib/oversight/rust-registry.sqlite \ |
| 121 | --migrate-from /var/lib/oversight/python-registry.sqlite |
| 122 | ``` |
| 123 | |
| 124 | The migration copies into the Rust target database after running its schema |
| 125 | migrations. It preserves `events.id`, `events.tlog_index`, corpus `metadata`, |
| 126 | and the manifest/beacon/watermark relationships that evidence bundles depend |
| 127 | on. Validate the copied database before switching traffic: |
| 128 | |
| 129 | ```bash |
| 130 | oversight-registry \ |
| 131 | --db /var/lib/oversight/rust-registry.sqlite \ |
| 132 | --validate-db |
| 133 | ``` |
| 134 | |
| 135 | The validation command prints JSON counts plus integrity failures for orphaned |
| 136 | beacons, watermarks, events, corpus rows, identity mismatches, malformed |
| 137 | event `extra` JSON, malformed corpus metadata JSON, duplicate or negative |
| 138 | tlog indexes, missing event tlog indexes, event tlog indexes outside the |
| 139 | on-disk tlog size, event rows whose indexed tlog leaf carries unrelated |
| 140 | evidence, malformed or non-contiguous local tlog leaf records, tlog leaf hash |
| 141 | mismatches, malformed manifest JSON, invalid manifest signatures, and |
| 142 | manifest/file ID divergence. Keep the Python database as a rollback artifact |
| 143 | until validation, live conformance, and evidence-bundle checks pass against |
| 144 | the Rust service. |
| 145 | The Rust `/tlog/range` route also reads through validated tlog records, so |
| 146 | malformed or hash-mismatched local leaf data blocks the range response instead |
| 147 | of disappearing from monitor output. |
| 148 | The Python reference registry uses the same fail-closed local tlog validation |
| 149 | for startup recovery and `/tlog/range`; newly appended records include |
| 150 | `leaf_data_hex` so exact event bytes can be recomputed by monitors. |
| 151 | Both reference registries return the registry v1 error envelope |
| 152 | `{"error":{"code":"...","message":"..."}}` for registry failures, and the |
| 153 | live conformance harness checks representative envelope codes. |
| 154 | `docs/REGISTRY_V1_STABILITY.md` defines the candidate-frozen v1.0 route, |
| 155 | field, and error-envelope surface that operators should burn in against. |
| 156 | |
| 157 | ## Rust Registry Burn-In Checklist |
| 158 | |
| 159 | Run this checklist before switching production traffic from the Python |
| 160 | reference registry to the Rust Axum registry: |
| 161 | |
| 162 | 1. Take a cold copy of the Python SQLite database and keep the original |
| 163 | mounted read-only during migration testing. |
| 164 | 2. Run `--migrate-dry-run` and compare all row counts against the source |
| 165 | database. |
| 166 | 3. Run the real `--migrate-from` into a fresh Rust database. |
| 167 | 4. Run `--validate-db` and treat any nonzero field as a deployment blocker. |
| 168 | 5. Start the Rust registry on loopback with `OVERSIGHT_OPERATOR_TOKEN` and |
| 169 | `OVERSIGHT_DNS_EVENT_SECRET` set. |
| 170 | 6. Run the live registry v1 conformance harness against the Rust endpoint. |
| 171 | 7. Fetch `/.well-known/oversight-registry`, `/tlog/head`, `/tlog/range`, |
| 172 | and at least one `/evidence/{file_id}` bundle, then verify the evidence |
| 173 | bundle with an independent client. |
| 174 | 8. Compare the deployed surface with `docs/REGISTRY_V1_STABILITY.md`. |
| 175 | 9. Keep the Python database and tlog as rollback artifacts until the Rust |
| 176 | service has completed the operator's burn-in window. |