| @@ -0,0 +1,55 @@ | ||
| + | # Docket - Accessibility & Plain-Language Spec | |
| + | ||
| + | Hard requirement (PLAN.md §1c): Docket must be usable by **any** citizen - explicitly an 80-year-old with low vision, low tech-literacy, keyboard-only or imprecise motor control. The forensic dark brand is the *default*, not a barrier. The brand wins by being **usable**, not just sharp. | |
| + | ||
| + | ## The 8 rules (enforce in code / CI) | |
| + | | # | Rule | Number | | |
| + | |---|---|---| | |
| + | | 1 | Contrast on every text/bg pair | **≥ 4.5:1** (≥3:1 large). Dark base is `#121316` (never pure black). Verify with WebAIM checker. | | |
| + | | 2 | Body text size | **≥ 16px** light / **≥ 18px** dark (mono reads smaller). Line-height **1.6** dark / 1.5 light. | | |
| + | | 3 | Touch/click targets | **≥ 44×44px** (padding counts). | | |
| + | | 4 | Focus indicator | **2px solid outline, ≥3:1**, never removed. | | |
| + | | 5 | Keyboard nav | All interactive elements via Tab/Enter/arrows. **No hover-only** (tooltips, menus, the Source link affordance). | | |
| + | | 6 | Plain-language summaries | **≤ 8th-grade Flesch-Kincaid**, ≤20 words/sentence, active voice. Tested in CI. | | |
| + | | 7 | Links & state | **underline + color**, never color alone (color-blind/presbyopia safe). | | |
| + | | 8 | Text-size / mode toggle | Always-visible, labeled, 44×44px. Scales text 1.5-2×, no layout break. | | |
| + | ||
| + | ## Three display modes (one toggle, persisted) | |
| + | 1. **Default - Dark forensic (brand):** ink `#121316` base, paper text, brass rationed, mono. Enforce: every pair ≥4.5:1, body 18px, line-height 1.6, accents muted (no vibrating saturated colors on dark). | |
| + | 2. **Readable / Light:** parchment `#F6F4EF` bg, ink text, body 16px, calm saturated accents. The "I just want to read it" mode. | |
| + | 3. **High-contrast "Plain Mode":** system sans (not mono), ≥20px, max contrast (black on `#FFFFCC` or inverted), no accent color, uniform weight. One click from anywhere. | |
| + | ||
| + | **Resolving the brand↔accessibility tension:** keep mono + dark + sparse forensic layout as the default *vibe*; enforce contrast/spacing rigorously; offer Light and Plain modes as frictionless escape hatches. The terminal look is preserved for those who want it; nobody is excluded. | |
| + | ||
| + | ## Plain-language summary template (neutral + legible) | |
| + | ``` | |
| + | [BILL ID]: [Official Short Title] | |
| + | ||
| + | What does this bill do? | |
| + | → ONE sentence, <20 words, active voice, no adjectives. | |
| + | "The bill requires large employers to report pay data by race and gender." | |
| + | ||
| + | Who does it affect? | |
| + | → 2-3 bullets, <10 words each. | |
| + | ||
| + | When does it take effect? | |
| + | → Plain date ("January 1, 2027"), not a legal cite. | |
| + | ||
| + | [ View the full bill on Congress.gov ] ← source link, rule #1 | |
| + | ``` | |
| + | Banned (jargon + spin): "notwithstanding," "shall/may" legalese, undefined acronyms, double negatives, and any "good/bad/burdensome/finally-closes-a-loophole" framing. **Describe what it does, never whether it's good** - this is where accessibility meets the neutrality doctrine (NEUTRALITY.md §2, §5). | |
| + | ||
| + | ## Search / lookup for non-experts (the 80-year-old path) | |
| + | - "Look up my state / my rep / my topic" as labeled drop-downs with type-ahead - not a bare search box. | |
| + | - Suggested pre-built filters as big clickable chips ([My State] · [Topic] · [This Year]); no "did you mean?" cognitive-load prompts. | |
| + | - Search field ≥50px tall, ≥18px font. | |
| + | - Persistent visible **Back**; breadcrumbs with whitespace separators, not `///`. | |
| + | ||
| + | ## CI / definition of done (accessibility) | |
| + | - [ ] axe-core (or equivalent) passes on every page, 0 serious violations | |
| + | - [ ] contrast audit green on all three modes | |
| + | - [ ] keyboard-only walkthrough reaches every interactive element with visible focus | |
| + | - [ ] summaries score ≤ grade 8 (automated Flesch-Kincaid check, runs with the neutrality lint) | |
| + | - [ ] mode toggle persists and never breaks layout | |
| + | ||
| + | *Source basis: WCAG 2.2 AA/AAA (target-size, focus-visible), plainlanguage.gov / Plain Writing Act, GOV.UK design system, WebAIM.* |
| @@ -30,9 +30,22 @@ | ||
| - `states` (code, name, openstates_coverage_level, last_seen) - powers the coverage map | ||
| - `events` (id, ref_type, ref_id, action_date, dollar_amount, affected_estimate, action_type) - the ranked feed source (rule #3: ONLY these fields feed ranking) | ||
| - `raw_events` (source, externalId, payload jsonb, fetched_at) - provenance | ||
| - | - `subscriptions` (user_id, kind[topic|agency|threshold|bill], value) | |
| + | - `subscriptions` (user_id, kind[topic|agency|threshold|bill|state|member], value) | |
| - | **Hard constraint:** every user-facing row exposes a non-null `source_url` whose host is in the allowlist (neutrality rule #1, enforced by test). | |
| + | ### Full-spectrum additions (2026-06-26) | |
| + | - `amendments` (id, congress, type[HAMDT|SAMDT], number, bill_id, sponsor_id, status, agreed_method[recorded|voice|uc|null], text?, source_url) | |
| + | - `bill_versions` (id, bill_id, version_code[ih|rh|eh|eas|enr|…], govinfo_package_id, fetched_at, text_ref) - the diff inputs | |
| + | - `bill_diffs` (id, bill_id, from_version, to_version, section_id, change[added|removed|amended], snippet, rider_flag bool, source_url) - powers "what got slipped in" | |
| + | - `procedural_events` (id, bill_id?, kind[voice_vote|unanimous_consent|cloture|suspension|mtr|discharge], date, chamber, detail, members_jsonb?, source_url) - parsed from CREC; `members` null when no individual record exists | |
| + | - `lobbying` (id, registrant, client, covered_bill_id?, amount_range, period, source_url) - LDA.gov | |
| + | - `executive_actions` (id, type[eo|nprm|final_rule|notice], title, agency, fr_doc_number, date, source_url) - Federal Register | |
| + | - `apportionments` (id, agency, account, appropriated, apportioned, status[released|delayed|withheld], date, source_url→.max.gov) - the impound tracker | |
| + | - `state_coverage` (state, source[openstates|legiscan], data_type, last_scrape, confidence[ok|stale|missing]) - the honesty layer | |
| + | ||
| + | **Hard constraint:** every user-facing row exposes a non-null `source_url` whose host is allowlisted (neutrality rule #1, enforced by test). For aggregator-sourced rows (OpenStates/LegiScan/OpenOMB/Lambda), `source_url` points to the **underlying official record**, not the aggregator (DATA_SOURCES.md third-party rule). | |
| + | ||
| + | ### What-changed pipeline (the signature feature) | |
| + | Per bill: `/related/BILLS-{id}` → fetch each version XML → section-parse + align (id + fuzzy heading) → diff → write `bill_diffs` (added/removed/amended), run rider keyword regex on appropriations-bill added sections → `rider_flag`. Surfaced as a "What changed / what was added" view + a per-section diff. Conference reports diffed against last engrossed version. Voice-vote/UC additions labeled "no individual record." | |
| ## Ranking (neutrality rule #3) | ||
| `score(event) = f(actionDate, dollarAmount, affectedEstimate, actionType)` - no engagement/news/social inputs. Pure function, unit-tested to reference only those four fields. | ||
| @@ -44,9 +57,12 @@ | ||
| - **Canonical source tree:** `/shared/projects/docket` (RAG/cbm-indexed, git). Deploy target later: Cloudflare front + homelab CT API. | ||
| ## Phases (see PLAN.md §8 for detail) | ||
| - | 0. Brand + clickable shell (fake data) - GLM | |
| - | 1. Federal ingest spine → real feed (read-only) - Codex | |
| - | 1.5. State spine (OpenStates) + coverage map | |
| - | 2. Extraction summaries + bill pages + neutrality CI | |
| - | 3. Alerts + accounts | |
| - | 4. Trust pages + launch | |
| + | 0. ✅ Brand + clickable shell (fake data) - GLM · DONE | |
| + | 1. Federal ingest spine → real feed (read-only) - DONE backend (USAspending/votes); **NEXT: wire read-API so frontend shows real data** | |
| + | 2. **Full bills/laws**: Congress.gov bills + GovInfo full text → bill pages + lookup/search (federal) | |
| + | 3. **"What got slipped in"**: bill_versions + diff pipeline + rider flags - the signature feature | |
| + | 4. **Closed-door layer**: CREC parsing (voice votes/UC/procedural) + Federal Register + lobbying + apportionment impound tracker | |
| + | 5. **State granular**: OpenStates + LegiScan + per-state confidence scores | |
| + | 6. **Accessibility modes**: Readable/Light + Plain Mode + plain-language summaries (docs/ACCESSIBILITY.md) | |
| + | 7. Alerts + accounts (topic/state/member/bill follows) | |
| + | 8. Trust pages + launch |
| @@ -42,8 +42,56 @@ All primary sources are free + official. State (OpenStates) is the one paid tier | ||
| ## CBO (cost estimates) · USEFUL, no API | ||
| - Scrape `cbo.gov/cost-estimates` by bill for per-bill scores; macro CSVs at `github.com/us-cbo`. Dollar figures only, no characterization. | ||
| + | --- | |
| + | ||
| + | # FULL-SPECTRUM SOURCES (added 2026-06-26 - the real Docket scope) | |
| + | ||
| + | Spending was only one lens. These power the bills/laws/amendments/riders/closed-door pillars (PLAN.md §1b). | |
| + | ||
| + | ## Amendments & "what got slipped in" · SIGNATURE FEATURE | |
| + | ||
| + | ### Amendments data - Congress.gov API | |
| + | - `GET /v3/amendment/{congress}/{type}/{number}` (+ `/actions`, `/text`, `/amendments`, `/cosponsors`). Types: `HAMDT`, `SAMDT`. Each carries `amendedBill`. Parent bill: `/bill/{c}/{type}/{num}/amendments`. | |
| + | - ⚠️ Coverage: searchable Senate amendment **text only 117th Congress (2021)+**; House + older link to Congressional Record, not machine-readable. Manager's/en-bloc packages are **not** discrete objects anywhere - only inferable from sequence + CREC narrative. | |
| + | ||
| + | ### Bill-version text-diff (the engine of "what changed") | |
| + | - **GovInfo BILLS** XML, 113th Congress (2013)+. Version chain: `IH`→`RH`→`EH`→`EAS`→`ENR` (House; `IS/RS/ES` Senate). PackageId `BILLS-{congress}{chamber}{number}{version}` (e.g. `BILLS-117hr3076ih`). | |
| + | - List versions: `GET /related/BILLS-{id}` · fetch text: `GET /packages/{packageId}/xml`. | |
| + | - **Diff recipe:** fetch consecutive versions → parse `<section>`/`<subsection>` w/ identifiers → align by id (+ fuzzy heading match, the hard part) → `difflib` within aligned sections → classify **added / removed / amended**. New section w/ no prior counterpart = inserted provision. | |
| + | - **Rider detection** (R Street keyword signals on appropriations bills, regex over added sections): `Notwithstanding | Is Amended | Are Authorized | Provided, | Provided Further, | In General | Unless | Until | Determines`. Flags candidates for surfacing, not a verdict. | |
| + | - **Conference reports** (GovInfo `CRPT`, 104th+): the biggest slipped-in vector. No structured diff trail - must diff the conference report text against the last engrossed version ourselves. Explanatory statements (modern omnibus) live in CREC, carry legal force. | |
| + | ||
| + | ### Honest gaps (state these in-product, never hide) | |
| + | - Voice-vote / unanimous-consent additions: **no individual record exists** (~94% of Senate measures; most manager's amendments). Surface "passed by voice vote - no individual record." That absence is itself information. | |
| + | - Conference negotiation records, pre-117th Senate amendment text, "who wrote this rider": not public. | |
| + | ||
| + | ## Closed-door / procedural · GovInfo CREC | |
| + | - `CREC` collection (api.govinfo.gov, free key), sections HOUSE/SENATE/DIGEST/EXTENSIONS. Daily, ~1-day lag, bulk XML 1994+. The single best source to extract voice votes, UC agreements, cloture, suspension, motions-to-recommit, discharge-petition signatories - all via **text parsing** (no structured endpoint). | |
| + | - Caveat: "Extensions of Remarks" = statements never spoken on the floor; speeches can be revised after delivery. Not a verbatim transcript - don't inflate "floor activity." | |
| + | - Committee markups/hearings: Congress.gov schedule (House 2017+, Senate 2019+) via docs.house.gov; transcripts discretionary + months-late (GovInfo `CHRG`). | |
| + | ||
| + | ## Lobbying · LDA.gov · EASY WIN | |
| + | - ⚠️ `lda.senate.gov` retires **2026-06-30** → use **`lda.gov/api`** (REST, free key, JSON). LD-1/LD-2 (covered bills + spend) / LD-203 (contributions). Quarterly. Answers "who lobbied this bill." | |
| + | ||
| + | ## Executive branch (where policy is made between laws) | |
| + | - **Federal Register API** `federalregister.gov/api/v1` · **no key**, JSON/CSV, same-day. Executive Orders, proposed/final rules (NPRM), notices. EASY WIN. | |
| + | - **regulations.gov API** (GSA key) - public comment dockets on rules (citizen-engagement layer). | |
| + | - **OpenOMB** `openomb.org` (JSON, daily) - OMB **apportionments** (executive withholding/release of appropriated funds), FY2022+, mirrors statutory `apportionment-public.max.gov`. Live 2025-26 controversy. → **link source to the official `.max.gov` record, not openomb.org** (see third-party rule below). | |
| + | ||
| + | ## State granular · OpenStates + LegiScan | |
| + | - **LegiScan API** - 50 states + Congress, free tier 30k q/mo, better bill-text + amendment completeness than OpenStates. Use as completeness supplement. | |
| + | - **OpenStates v3** - richer standardized vote/people schema, fully open. Both lag 24-48h; worst states historically MA/NE/KY/AL; ballot measures + interim committee work poorly covered by both. | |
| + | - **Per-state data-confidence score** (required): flag stale scrapes + known-missing data types (votes, committee actions, amendment text). Honesty about gaps is a differentiator. | |
| + | ||
| + | ## Member trades · STOCK Act · HARD | |
| + | - No official API. Official PDFs at `efdsearch.senate.gov` + `disclosures-clerk.house.gov`. Third-party normalized JSON via Lambda Finance (free) - but **display the official disclosure as source**. Amounts are ranges; the trade→disclosure gap is the metric. | |
| + | ||
| + | --- | |
| + | ||
| ## Source allowlist (neutrality rule #1) | ||
| - | `*.gov`, `api.congress.gov`, `govinfo.gov`, `usaspending.gov`, `fiscaldata.treasury.gov`, `cbo.gov`, `clerk.house.gov`, `senate.gov`, OpenStates official-record links (→ state `.gov`). | |
| + | `*.gov` (covers congress/govinfo/usaspending/fiscaldata.treasury/cbo/clerk.house/senate/federalregister/lda/regulations/*.max.gov/efdsearch.senate/disclosures-clerk.house/docs.house), **plus a curated `STATE_LEG_HOSTS` registry** for state legislatures that don't use `.gov` (e.g. `legis.state.pa.us`, `leginfo.legislature.ca.gov`, `legislature.mi.gov`, `*.state.*.us`). | |
| + | ||
| + | **Third-party-normalizer rule (neutrality):** when data comes from an aggregator (OpenStates, LegiScan, OpenOMB, Lambda Finance), the user-facing `source_url` MUST point to the **underlying official government record**, never the aggregator. Aggregators are pipes, not sources. | |
| ## Ingest principles | ||
| - One poller per source, each writes normalized `actions` rows + raw payload to `raw_events`. |