# Docket -  Architecture

## Shape

```

            POLLERS (Python)                 API + WEB (Next.js)

  Congress.gov ─┐                          ┌─ Next.js app (PWA, dark-first)

  GovInfo ──────┤                          │   /register (feed)  /bill/[id]

  USAspending ──┤→ normalize → Postgres ──→│   /spending  /states

  Treasury ─────┤   (actions, bills,       │   /how-we-pick  /who-pays

  House/Senate ─┤    awards, members,      │

  OpenStates ───┘    votes, raw_events)    └─ /api (read) → events feed

            │                                        │

            └── change-detection ──→ events ──→ alerts (ntfy + web-push)

```

## Services

1. **ingest** (Python) -  one poller module per source (`pollers/congress.py`, `usaspending.py`, `treasury.py`, `govinfo.py`, `votes_house.py`, `votes_senate.py`, `openstates.py`), each idempotent, rate-budget aware, with Congress.gov scraper fallback. A scheduler (cron/APScheduler) runs them at the cadences in `DATA_SOURCES.md`.

2. **change-detection** -  diffs each poll against stored `updateDate`/hash, writes normalized `actions` + emits `events`.

3. **summaries** (Phase 2) -  extraction-template pipeline; banned-words + Polarization-Index gate before publish. MVP ships structured fields only.

4. **web/api** (Next.js) -  read API over SQLite/Postgres for the feed + entity pages; the brand-pack UI.

5. **alerts** -  topic/agency/threshold subscriptions → ntfy + web-push. No party-based subscriptions.

## Data model (Postgres, draft)

- `bills` (id, source, externalId, congress/session, number, title, sponsor_id, status, policy_area, updated_at, source_url)

- `actions` (id, bill_id?, source, action_type, action_date, chamber, text, dollar_amount?, affected_estimate?, source_url) -  the spine of the feed

- `votes` (id, bill_id, chamber, date, yea, nay, present, result, source_url) + `vote_positions` (vote_id, member_id, position)

- `members` (id, name, party, state, chamber, source_url)

- `awards` (id, source, externalId, agency, recipient, amount, action_date, program, source_url)

- `fiscal` (metric, date, value, source_url) -  debt/deficit/outlays

- `states` (code, name, openstates_coverage_level, last_seen) -  powers the coverage map

- `events` (id, ref_type, ref_id, action_date, dollar_amount, affected_estimate, action_type) -  the ranked feed source (rule #3: ONLY these fields feed ranking)

- `raw_events` (source, externalId, payload jsonb, fetched_at) -  provenance

- `subscriptions` (user_id, kind[topic|agency|threshold|bill|state|member], value)

### Full-spectrum additions (2026-06-26)

- `amendments` (id, congress, type[HAMDT|SAMDT], number, bill_id, sponsor_id, status, agreed_method[recorded|voice|uc|null], text?, source_url)

- `bill_versions` (id, bill_id, version_code[ih|rh|eh|eas|enr|…], govinfo_package_id, fetched_at, text_ref) -  the diff inputs

- `bill_diffs` (id, bill_id, from_version, to_version, section_id, change[added|removed|amended], snippet, rider_flag bool, source_url) -  powers "what got slipped in"

- `procedural_events` (id, bill_id?, kind[voice_vote|unanimous_consent|cloture|suspension|mtr|discharge], date, chamber, detail, members_jsonb?, source_url) -  parsed from CREC; `members` null when no individual record exists

- `lobbying` (id, registrant, client, covered_bill_id?, amount_range, period, source_url) -  LDA.gov

- `executive_actions` (id, type[eo|nprm|final_rule|notice], title, agency, fr_doc_number, date, source_url) -  Federal Register

- `apportionments` (id, agency, account, appropriated, apportioned, status[released|delayed|withheld], date, source_url→.max.gov) -  the impound tracker

- `state_coverage` (state, source[openstates|legiscan], data_type, last_scrape, confidence[ok|stale|missing]) -  the honesty layer

**Hard constraint:** every user-facing row exposes a non-null `source_url` whose host is allowlisted (neutrality rule #1, enforced by test). For aggregator-sourced rows (OpenStates/LegiScan/OpenOMB/Lambda), `source_url` points to the **underlying official record**, not the aggregator (DATA_SOURCES.md third-party rule).

### What-changed pipeline (the signature feature)

Per bill: `/related/BILLS-{id}` → fetch each version XML → section-parse + align (id + fuzzy heading) → diff → write `bill_diffs` (added/removed/amended), run rider keyword regex on appropriations-bill added sections → `rider_flag`. Surfaced as a "What changed / what was added" view + a per-section diff. Conference reports diffed against last engrossed version. Voice-vote/UC additions labeled "no individual record."

## Ranking (neutrality rule #3)

`score(event) = f(actionDate, dollarAmount, affectedEstimate, actionType)` -  no engagement/news/social inputs. Pure function, unit-tested to reference only those four fields.

## Build/run hosts

- **Frontend:** Next.js dev server; `npm run dev` from the repo root.

- **Backend:** Python ingest; `python -m ingest.run` from `services/ingest/`.

- **Database:** SQLite for dev/staging; Postgres for production (Phase 1+).

- **Deploy target:** Cloudflare front + self-hosted API backend.

## Phases (see PLAN.md §8 for detail)

0. ✅ Brand + clickable shell (fake data) - DONE

1. Federal ingest spine → real feed (read-only) -  DONE backend (USAspending/votes); **NEXT: wire read-API so frontend shows real data**

2. **Full bills/laws**: Congress.gov bills + GovInfo full text → bill pages + lookup/search (federal)

3. **"What got slipped in"**: bill_versions + diff pipeline + rider flags -  the signature feature

4. **Closed-door layer**: CREC parsing (voice votes/UC/procedural) + Federal Register + lobbying + apportionment impound tracker

5. **State granular**: OpenStates + LegiScan + per-state confidence scores

6. **Accessibility modes**: Readable/Light + Plain Mode + plain-language summaries (docs/ACCESSIBILITY.md)

7. Alerts + accounts (topic/state/member/bill follows)

8. Trust pages + launch