| @@ -1,9 +1,9 @@ | ||
| - | # Cyclops / Flywheel Security Assessment - Docket | |
| + | # Cyclops / Flywheel Security Assessment - Docket | |
| **Target:** http://REDACTED-IP:3000 (pre-launch build) | ||
| **Date:** 2026-06-26 | ||
| **Scope:** Full web + `/api/*` assessment (authorized owner test) | ||
| - | **Method:** White-box (source at `/shared/projects/docket`) + live black-box PoC via Flywheel MCP scanner and raw HTTP. Every finding below was verified against the running target - no assertion without a request/response. | |
| + | **Method:** White-box (source at `/shared/projects/docket`) + live black-box PoC via Flywheel MCP scanner and raw HTTP. Every finding below was verified against the running target - no assertion without a request/response. | |
| **Stack:** Next.js 14.2.28 (app router), better-sqlite3, read-only public data viewer, no auth, no writes, no uploads. | ||
| --- | ||
| @@ -13,21 +13,21 @@ | ||
| | # | Severity | Finding | Endpoint / File | Status | | ||
| |---|----------|---------|-----------------|--------| | ||
| | 1 | **Low** | Rate-limit bypass via spoofable client-IP headers | `middleware.ts` `clientIp()` | REAL (deployment-dependent) | | ||
| - | | 2 | **Low** | Availability/DoS under concurrency - synchronous full-table loads on every API call, single event loop | `lib/feed.ts`, `lib/db.ts`, all `/api/*` (`force-dynamic`) | REAL (deployment-dependent) | | |
| - | | 3 | **Info** | CSP allows `script-src 'unsafe-inline'` (no nonce) - weakens XSS defense-in-depth | `next.config.js` | REAL (no exploitable sink today) | | |
| + | | 2 | **Low** | Availability/DoS under concurrency - synchronous full-table loads on every API call, single event loop | `lib/feed.ts`, `lib/db.ts`, all `/api/*` (`force-dynamic`) | REAL (deployment-dependent) | | |
| + | | 3 | **Info** | CSP allows `script-src 'unsafe-inline'` (no nonce) - weakens XSS defense-in-depth | `next.config.js` | REAL (no exploitable sink today) | | |
| | 4 | **Info** | `q` reflected verbatim in `/api/search` JSON body | `lib/feed.ts` `globalSearch` | REAL but NOT XSS | | ||
| | 5 | **Info** | `X-XSS-Protection` header absent | `next.config.js` | Non-issue (deprecated; CSP supersedes) | | ||
| - | | - | - | "Reflected XSS in q" (scanner) | `/api/search` | **FALSE POSITIVE** | | |
| - | | - | - | "File path in error response" (scanner) | `/api/feed/<bogus>` | **FALSE POSITIVE** | | |
| + | | - | - | "Reflected XSS in q" (scanner) | `/api/search` | **FALSE POSITIVE** | | |
| + | | - | - | "File path in error response" (scanner) | `/api/feed/<bogus>` | **FALSE POSITIVE** | | |
| **Not present (verified negative):** SQL injection, IDOR/auth bypass, SSRF (params + headers), CORS misconfig, open redirect, CRLF/header injection, OS command injection, SSTI, path traversal, HTTP verb tampering, dangerous HTTP methods, debug/admin endpoints, `.env`/`.git`/DB-file/source-map exposure, stack-trace/info disclosure. | ||
| --- | ||
| - | ## 2. Real Findings - detail + PoC | |
| + | ## 2. Real Findings - detail + PoC | |
| - | ### #1 Rate-limit bypass via spoofable client-IP headers - Low | |
| - | The limiter (120 req/min/IP, fixed window) derives the client identity from request headers in this order: `CF-Connecting-IP` → `X-Real-IP` → `X-Forwarded-For` → `"unknown"`, with **no trust validation**. When the origin is reached directly (not through the CDN), an attacker rotates any of these headers to get a fresh 120-request budget per value - effectively unlimited. | |
| + | ### #1 Rate-limit bypass via spoofable client-IP headers - Low | |
| + | The limiter (120 req/min/IP, fixed window) derives the client identity from request headers in this order: `CF-Connecting-IP` → `X-Real-IP` → `X-Forwarded-For` → `"unknown"`, with **no trust validation**. When the origin is reached directly (not through the CDN), an attacker rotates any of these headers to get a fresh 120-request budget per value - effectively unlimited. | |
| Code (`middleware.ts`): | ||
| ``` | ||
| @@ -41,17 +41,17 @@ This is **by design** for a CDN deployment (the code comments state L3/L4 protec | ||
| **Fix:** Trust `CF-Connecting-IP` only when the peer socket is a known Cloudflare IP; otherwise fall back to the real socket address, not attacker-controlled headers. Firewall the origin to CDN IPs. | ||
| - | ### #2 Availability under concurrency - Low | |
| + | ### #2 Availability under concurrency - Low | |
| Every `/api/*` route is `export const dynamic = "force-dynamic"` (no caching) and the data layer (`getFeedItems`/`getBills`/`getMembers`/`globalSearch`) loads whole tables into memory and filters/sorts in JS via synchronous `better-sqlite3` calls, which block the single Node event loop. | ||
| PoC: a single `/api/feed` call = `time_total 0.42s`, 25 KB. A 300-request concurrent burst (50 parallel) saturated the server: **282/300 connections failed/timed out ("000"), 18 succeeded**, and the post-burst request returned `429`. Combined with #1 (header-rotating rate-limit bypass), a distributed flood against a directly-reachable origin can degrade availability. | ||
| **Fix:** Cache the read API at the CDN edge (these are public, slow-changing records); add `Cache-Control`/`s-maxage` to API responses; push table aggregation into SQL (`WHERE`/`LIMIT`) instead of loading full tables per request. Lock the origin behind the CDN. | ||
| - | ### #3 CSP `'unsafe-inline'` scripts - Info | |
| + | ### #3 CSP `'unsafe-inline'` scripts - Info | |
| `script-src 'self' 'unsafe-inline'` (required by Next.js inline hydration without a nonce pipeline). No reflected/stored/DOM HTML-injection sink exists today (React auto-escapes; the only `dangerouslySetInnerHTML` is the static theme bootstrap in `app/layout.tsx`), so this is defense-in-depth only. Consider a nonce-based CSP later. | ||
| - | ### #4 `q` reflected in `/api/search` - Info, NOT XSS | |
| + | ### #4 `q` reflected in `/api/search` - Info, NOT XSS | |
| `/api/search?q=<script>alert(1)</script>` returns `{"q":"<script>alert(1)</script>", ...}` with `Content-Type: application/json` and `X-Content-Type-Options: nosniff`. Browsers do not execute scripts in a JSON response, and `nosniff` blocks content-type confusion. Not exploitable. The search **page** renders results through React (escaped). No action required; optionally drop the echo. | ||
| --- | ||
| @@ -67,9 +67,9 @@ PoC: a single `/api/feed` call = `time_total 0.42s`, 25 KB. A 300-request concur | ||
| - **Pagination abuse:** `limit=999999999` clamped to `MAX_LIMIT=200`; `limit=-1&offset=-5` falls back to defaults. **No DoS via limit.** | ||
| - **Path traversal / id abuse:** `/api/bills/..%2f..%2fetc%2fpasswd` → 404; oversized id → 404; ids `parseInt`-normalized before binding. | ||
| - **Static exposure:** `/docket.db`, `/opt/docket/data/docket.db`, `/.env`, `/.git/config`, `/next.config.js`, `*.js.map` → all **404**. | ||
| - | - **CORS:** no `Access-Control-Allow-Origin` reflection. **Open redirect / SSRF (params + headers):** no URL-typed params, no outbound fetch from user input - nothing to test. **CRLF, verb tampering, dangerous methods, debug endpoints:** none. | |
| + | - **CORS:** no `Access-Control-Allow-Origin` reflection. **Open redirect / SSRF (params + headers):** no URL-typed params, no outbound fetch from user input - nothing to test. **CRLF, verb tampering, dangerous methods, debug endpoints:** none. | |
| - **Security headers (live):** CSP, HSTS (2y, preload), `X-Frame-Options: DENY`, `X-Content-Type-Options: nosniff`, `Referrer-Policy`, `Permissions-Policy`, COOP, CORP all present; `poweredByHeader` off (no `X-Powered-By`/version leak). | ||
| - | - **IDOR:** N/A - no accounts, no per-user data; all data is public read-only by design. | |
| + | - **IDOR:** N/A - no accounts, no per-user data; all data is public read-only by design. | |
| --- | ||
| @@ -77,7 +77,7 @@ PoC: a single `/api/feed` call = `time_total 0.42s`, 25 KB. A 300-request concur | ||
| These are **deployment** controls, not code vulnerabilities: | ||
| 1. **Put the origin behind the CDN and firewall it to CDN IPs only.** This neutralizes #1 and #2. Do not expose `:3000` publicly. | ||
| 2. **Make `CF-Connecting-IP` trust conditional** on a Cloudflare peer IP (so the header limiter isn't trivially bypassable even if the origin is ever reachable). | ||
| - | 3. **Cache the read API at the edge** (`s-maxage`) and/or move filtering into SQL - reduces per-request cost and removes the concurrency-saturation foothold. | |
| + | 3. **Cache the read API at the edge** (`s-maxage`) and/or move filtering into SQL - reduces per-request cost and removes the concurrency-saturation foothold. | |
| Nice-to-have (not blocking): nonce-based CSP (#3), drop the `q` echo (#4). | ||
| @@ -85,7 +85,7 @@ Nice-to-have (not blocking): nonce-based CSP (#3), drop the `q` echo (#4). | ||
| ## 6. Verdict | ||
| - | ### 🟢 GREEN for public launch - *behind a CDN with the origin locked to CDN IPs.* | |
| + | ### GREEN for public launch - *behind a CDN with the origin locked to CDN IPs.* | |
| No critical, high, or medium code-level vulnerabilities were found. SQLi, XSS, IDOR, SSRF, traversal, CORS, redirect, CRLF, cmdi, SSTI, and info-disclosure are all absent or non-exploitable. The SQL layer is uniformly parameterized, output is React-escaped, security headers are strong, and pagination/limits are clamped. | ||
| @@ -1,559 +0,0 @@ | ||
| - | # Docket - Frontend Notes | |
| - | ||
| - | Wave F2 + F3 + F4 + F5 + R1 + R2 + R3 + V2 + V3b build log. | |
| - | ||
| - | ## V3b (2026-06-27) - Two-mode night/day switcher + Plain removed (Claude) | |
| - | ||
| - | ### Summary | |
| - | Simplified the theme system: Plain mode removed, 3-way text toggle replaced by a clean | |
| - | moon/sun icon button. Light mode refreshed to be a true day counterpart of the dark V3 | |
| - | ink aesthetic - same fonts, same composition, warm paper palette. | |
| - | ||
| - | ### Changes | |
| - | ||
| - | **`components/display-toggle.tsx`** - complete rewrite | |
| - | - Removed "Plain" mode entirely; `DisplayMode` type is now `"dark" | "light"` only. | |
| - | - Replaced the text segmented control (Dark / Light / Plain) with a single icon button. | |
| - | - Icon pattern: **moon shown in dark mode** (click = switch to light); **sun shown in light mode** (click = switch to dark). | |
| - | - `aria-label` describes the action: "Switch to light mode" / "Switch to dark mode". | |
| - | - 44×44px touch target, `focus-ring` class, visible focus outline. | |
| - | - Hydration-safe: SSR renders dark default; `useEffect` reads `localStorage`. Button | |
| - | fades in after mount (`opacity: 0 → 1`) to suppress any icon-flicker. | |
| - | - Legacy `"plain"` in `localStorage` is coerced to `"dark"` on first read. | |
| - | ||
| - | **`app/globals.css`** | |
| - | - Removed `:root.plain { ... }` token block entirely. | |
| - | - Removed `:root.plain` font overrides in `@layer base`. | |
| - | - Removed `:root.plain :focus-visible` outline override in `@layer a11y`. | |
| - | - Removed `:root.light body` font override (Archivo body) - light mode now uses the | |
| - | same JetBrains Mono body as dark mode (day version of same theme, same font roles). | |
| - | - Removed `:root.light .pill` Archivo override - pills stay mono in both modes. | |
| - | - Light mode `--dk-body-size` set to `18px` (was 20px "elder-first") - matches dark. | |
| - | - Light mode `--dk-signal-dim` tightened to `#6b4100` (was `#ce5c00`) for stronger | |
| - | contrast on fills. | |
| - | - Light mode `--dk-yea`/`--dk-nay` darkened slightly for better AA on white: | |
| - | `#3d6b00` (yea) / `#8a1c1c` (nay). | |
| - | - Comment block updated: marks Plain as removed, notes contrast values. | |
| - | ||
| - | **Contrast verification (light mode on white `#ffffff`)** | |
| - | - Text `#0B0C0E`: ~18:1 - well above AAA | |
| - | - Secondary `#44423D`: ~10:1 - AAA | |
| - | - Brass accent `#8F5902`: ~5.8:1 - passes AA (>=4.5:1 required) | |
| - | - Borders/inactive ash `#6B6860`: used for non-text decorative only | |
| - | ||
| - | **`app/layout.tsx`** | |
| - | - Pre-paint inline script updated: only handles `"dark"` / `"light"`. | |
| - | Any stored `"plain"` falls back to `"dark"`. Removes `'plain'` from | |
| - | `classList.remove()` call (kept for safety during the storage coercion window). | |
| - | ||
| - | ### What did NOT change | |
| - | All functional work, V3 dark tokens, fonts (Archivo/JetBrains Mono/Fraunces), nav, | |
| - | all page data surfaces, and QA gate. Dark mode is identical to V3. All surfaces consume the live read-API | |
| - | (`app/api/*` → `lib/feed.ts` → SQLite at `data/docket.db`). No mock data on | |
| - | the deep surfaces; `lib/mock-data.ts` remains only for the Phase-0 lint | |
| - | reference and is not imported by any F2/F3/F4/F5 page. | |
| - | ||
| - | ## V2 (2026-06-27) - Design system rebuild: Zion's own aesthetic (Claude) | |
| - | ||
| - | ### Summary | |
| - | Template wave: design tokens, fonts, and three pages rebuilt in the owner's own aesthetic | |
| - | (docs/DESIGN_V2.md). QA CLEAN - 17 routes, zero overflow, zero hydration errors. | |
| - | Screenshots: `_ops/qa-view/v2-{home,bills,bill}-{mobile,desktop}.png` | |
| - | Scope: template only - Home, Bills list, Bill detail, and the shared header/nav. | |
| - | Other pages unchanged; propagation gated on owner approval. | |
| - | ||
| - | ### Design system changes (`app/globals.css`) | |
| - | ||
| - | **Dark mode (`:root`)**: signal changed from brass `#E0A53B` to cobalt `#2E5BFF` - the | |
| - | treetrace.dev treatment as specified in docs/DESIGN_V2.md. | |
| - | ||
| - | **Light mode (`:root.light`)** - completely replaced with Zion's warm palette: | |
| - | - bg `#FFFFFF` (white page) · surface `#F7F4ED` (warm paper) · graphite `#EDE9DF` (hover) | |
| - | - text `#0B0C0E` (ink near-black) · secondary `#44423D` · tertiary `#6B6860` | |
| - | - border `#D8D2C4` (warm visible) · accent `#8F5902` (his amber/ochre, ~6:1 on white) | |
| - | - accent-strong `#CE5C00` (large fills) · accent-wash `#FBF3DF` (active/selected) | |
| - | - yea `#4E7A06` (his Tango green) · nay `#A40000` (his Tango red, small only) | |
| - | - body-size bumped to **20px** (elder-first: was 18px) | |
| - | ||
| - | **Font roles**: body font in light mode changed from Atkinson Hyperlegible to **Archivo** | |
| - | (the same display/heading font, applied at body weight for UI text). Type trio enforced: | |
| - | - Archivo → headings, display, UI, kicker labels | |
| - | - Fraunces → leads, long-form reading (bill titles, descriptions) | |
| - | - JetBrains Mono → DATA ONLY: bill numbers, tallies, amounts, dates | |
| - | ||
| - | **Component token overrides (light mode)**: | |
| - | - `.label`: switched from JetBrains Mono to Archivo 700 uppercase (editorial, not techy) | |
| - | - `.pill`: Archivo 13px, 700 weight, 4px radius (readable, not tiny 10px mono) | |
| - | - `.prose-lead`: 20px in light (was 18px) | |
| - | - New utility classes: `.big-card`, `.section-card` (hover highlight with amber border) | |
| - | ||
| - | ### Font loading (`app/layout.tsx`) | |
| - | - Removed: `Atkinson_Hyperlegible` import and `atkinson.variable` (no longer the body font) | |
| - | - Kept: Archivo (`--font-display`), JetBrains Mono (`--font-mono`), Fraunces (`--font-prose`) | |
| - | ||
| - | ### Display toggle (`components/display-toggle.tsx`) | |
| - | - Renamed "Reference" mode label to **"Standard"** (less technical; is the light default) | |
| - | ||
| - | ### Site header (`components/site-header.tsx`) - full rewrite | |
| - | - Nav simplified from 11 items to **6**: Home · Bills · Votes · Representatives · Spending · States | |
| - | - Home link changed from `/register` to `/` (the new landing hub) | |
| - | - "Members" renamed to "Representatives" in nav (matches DESIGN_V2 IA) | |
| - | - Search: now a labeled affordance (`<SearchIcon> Search`) visible at all viewports | |
| - | (was icon-only on mobile) | |
| - | - Mobile menu: rows are 56px min-height with bold 18px labels; active item shows amber | |
| - | accent-wash background + "← you are here" label | |
| - | - Desktop nav: active item uses amber wash background (amber highlight, not just underline) | |
| - | - Mode toggle: Standard / Dark / Plain (was Reference / Dark / Plain) | |
| - | ||
| - | ### Home page (`app/page.tsx`) - full rewrite | |
| - | - Max width 860px (was 64rem register width) | |
| - | - Hero: Archivo H1 at `clamp(2.125rem,7vw,3.5rem)` 900 weight, Fraunces lead paragraph | |
| - | - Kicker label in amber signal color (`#8F5902`) | |
| - | - Prominent search CTA: filled amber button with search icon (56px min-height) | |
| - | - 6 section cards in 2-col grid (was 4-col, 8 items, tiny text) | |
| - | Cards: Archivo 1.25rem title, Fraunces 1rem description, amber hover border | |
| - | - "More" link row: Calendar, Schedule, Executive orders, Browse by topic, Search | |
| - | - "How to cite" section: Archivo H2, Fraunces body, example APA citation | |
| - | - Neutrality note: amber signal links to /how-we-pick and /who-pays | |
| - | - REMOVED: DataTable-style card layout; all grids are readable 2-col max | |
| - | ||
| - | ### Bills list (`app/bills/page.tsx`) - full rewrite (data logic preserved) | |
| - | All server logic (`getBills`, filters, searchParams, `dropParam`) is identical to R3. | |
| - | Presentation layer replaced: | |
| - | - Breadcrumb + Back → Home / Bills + "← Back to home" (amber, ≥44px) | |
| - | - Header: Archivo H1, Fraunces lead (was PageHeader mono eyebrow) | |
| - | - `BillsTable` (DataTable dense table) → `BillCards` (big card per bill): | |
| - | - Bill number: JetBrains Mono (DATA), shown as identifier not a primary link | |
| - | - Title: `<Link>` in Fraunces 1.125rem (the primary tap target for elder UX) | |
| - | - Status: `StatusPill` with **short label** (`STATUS_SHORT` map: passed/failed/ | |
| - | in committee/pending/introduced) - prevents long raw status text overflowing 390px | |
| - | - Policy area: amber link · Congress: mono secondary · Date: mono · SourceLink | |
| - | - Cards have 2px warm border, hover to graphite | |
| - | ||
| - | ### Bill detail (`app/bill/[id]/page.tsx`) - full rewrite (data + components preserved) | |
| - | All data fetching, diff, timeline, amendments, versions, CiteThis, SourceLink, neutrality | |
| - | note are unchanged. Template updated: | |
| - | - Breadcrumb: Home / Bills / [bill number in mono] + "← Back to Bills" | |
| - | - H1: bill number in Archivo 900 + JetBrains Mono inline span (data) | |
| - | - Lead: Fraunces prose-lead for the full bill title (was tiny mono) | |
| - | - Key facts: 6px radius cards with 2px border; status/sponsor/last-action in larger | |
| - | readable text (prose-lead 0.9375rem, Archivo labels) | |
| - | - Diff section: amber accent border for rider-flagged cards | |
| - | - Timeline: Fraunces 0.9375rem for action text; mono dates (data only) | |
| - | - Amendments: list layout instead of DataTable (simpler, no dense table) | |
| - | - Votes / Versions: Fraunces italic for empty states; Archivo for version code labels | |
| - | - Section headers: `SectionLabel` component (inline Archivo H2 + mono eyebrow) | |
| - | - No engagement surface note: label (amber/ash) | |
| - | ||
| - | ### Screenshot script (`_ops/screenshot-v2.js`) | |
| - | Playwright script that auto-detects a real bill ID from `/bills`, takes screenshots at | |
| - | 390px (mobile) and 1024px (desktop) of Home + Bills + Bill detail. | |
| - | Output: `_ops/qa-view/v2-{home,bills,bill}-{mobile,desktop}.png` (6 files). | |
| - | ||
| - | --- | |
| - | ||
| - | ## R3 (2026-06-27) - Votes show what was voted on + richer sort/filter + home landing (Claude) | |
| - | ||
| - | ### Summary | |
| - | Wave R3 delivers three owner-priority changes. QA CLEAN (17+ routes, zero overflow, zero hydration errors). | |
| - | ||
| - | ### 1. Votes show what was voted on | |
| - | ||
| - | **`lib/feed.ts`** - `Vote` type extended: `billNumber?`, `billTitle?`, `billTextRef?`. | |
| - | `getVotes()`, `getVotesFiltered()`, `getVoteById()` now LEFT JOIN the `bills` table | |
| - | on `votes.bill_id`. Graceful: when no bill is linked the fields are `undefined` and | |
| - | the UI falls back through the priority chain (no breaking change). | |
| - | ||
| - | **New `lib/vote-subject.ts`** - `resolveVoteSubject(v)` implements the full priority chain | |
| - | for identifying what any vote was about. Priority: | |
| - | 1. Linked bill (`billNumber` + `billTitle`) - link to `/bill/[id]` | |
| - | 2. `bill_text_ref` starting with `"PN"` → Presidential Nomination label | |
| - | 3. `bill_text_ref` starting with `"TREATYDOC"` → Treaty document label | |
| - | 4. Other `bill_text_ref` (historical ref, bill not yet in DB) + question | |
| - | 5. Question text from the roll call | |
| - | 6. Result text as last resort - never leads with just "Passed" | |
| - | ||
| - | **`/votes` list** - "what" column is now the `cardTitle` (previously `result`). | |
| - | Primary line: bill number + title (e.g. "H.R. 22 - Financial Exploitation Prevention Act | |
| - | of 2025") for linked bills, or nomination/treaty label, or ref/question. Secondary | |
| - | line: result text. "Passed · 414-2" is now always SECONDARY, never the leading | |
| - | identifier. The list also shows the result + tally in a separate Tally column. | |
| - | ||
| - | **`/votes/[id]` detail** - A prominent card is inserted ABOVE the PageHeader identifying | |
| - | what the vote was about: kind label ("Bill voted on", "Presidential nomination", | |
| - | "Treaty ratification", "Legislative document", "Question voted on"), the primary subject | |
| - | with a link to `/bill/[id]` when available, and the procedural question below it. A reader | |
| - | instantly sees the subject before the tally/roster. | |
| - | ||
| - | ### 2. Sort + deeper filters on every list page and /search | |
| - | ||
| - | **`/votes`** - new `result` filter (Passed/Failed, pushed into SQL as `v.yea > v.nay`) | |
| - | and `sort` control (Newest/By chamber/By result text). `VoteFilter` extended with | |
| - | `resultFilter` and `sort`. `buildVotesWhere()` updated with `v.` table alias for | |
| - | JOIN compatibility. `buildVotesOrder()` helper resolves ORDER BY per sort value. | |
| - | ||
| - | **`/bills`** - new `sort` control (Last action/By congress/By status). `BillFilter.sort` | |
| - | extended to include `"congress"` and `"status"`. `getBills()` ORDER BY fixed to | |
| - | actually honor the sort value (was hardcoded to `updated_at` for all options). | |
| - | ||
| - | **`/spending`** - new `sort` control (Largest first / Most recent / By agency A-Z). | |
| - | `getAwardsFiltered(q, sort?)` signature extended with optional sort param; ORDER BY | |
| - | varies accordingly. | |
| - | ||
| - | **`/members`** - `MembersBrowser` (client component) gets a `sort` state and new | |
| - | "Sort by" `FilterSelect` (Default / Name A-Z / State / Chamber). Named sorts bypass | |
| - | the tier logic; Default preserves the existing current-Congress-first ordering. | |
| - | ||
| - | **`/search`** - type filter chips (All / Bills / Members / Federal Register / Awards / | |
| - | States) control which result groups render. A GET-form sort control (Relevance / | |
| - | Alphabetical / By amount) applies in-memory re-ordering after the cached | |
| - | `globalSearch()` call. Type and sort params are preserved across chip navigation. | |
| - | ||
| - | ### 3. New landing home page at "/" | |
| - | ||
| - | `app/page.tsx` - replaced the `redirect("/register")` with a full orienting landing page: | |
| - | - **Hero**: plain-language explanation of what Docket is (neutral, sourced, .gov-linked), | |
| - | how to use it, and a primary CTA → Search. | |
| - | - **Entry cards grid** (2-up / 4-up responsive): Search, The Register, Bills, Votes, | |
| - | Members, Spending, Calendar, States - each with a short description of what it does. | |
| - | - **How-to section**: two cards - "For students and educators" (how to cite) and | |
| - | "How to cite" (using the Cite this button). | |
| - | - **About** card: plain statement of neutrality policy + links to /how-we-pick and | |
| - | /who-pays. | |
| - | The Register feed lives at `/register` (unchanged); the home page links to it. No forced | |
| - | tutorial, no blocking interstitial - purely orienting. | |
| - | ||
| - | ### Data layer changes (`lib/feed.ts`) | |
| - | - `Vote` type: + `billNumber?`, `billTitle?`, `billTextRef?` | |
| - | - `VoteFilter` type: + `resultFilter?`, `sort?` | |
| - | - `buildVotesWhere()`: `v.` alias for JOIN compatibility; resultFilter clause | |
| - | - New `buildVotesOrder()`: ORDER BY per sort value | |
| - | - `getVoteCountFiltered()`: uses `FROM votes v` alias to match WHERE clauses | |
| - | - `getVotes()`: delegates to `getVotesFiltered()` | |
| - | - `getVotesFiltered()`: LEFT JOIN bills; honor sort; pick up bill_number/bill_title/bill_text_ref | |
| - | - `getVoteById()`: LEFT JOIN bills; return bill fields + bill_text_ref | |
| - | - `toVote()`: picks up billNumber, billTitle, billTextRef from JOIN result | |
| - | - `BillFilter.sort`: + `"congress"` | `"status"` | |
| - | - `getBills()`: ORDER BY now properly varies by sort value | |
| - | - `getAwardsFiltered(q?, sort?)`: second param controls ORDER BY (amount/date/agency) | |
| - | ||
| - | ### New file: `lib/vote-subject.ts` | |
| - | Shared `resolveVoteSubject(v)` utility - used by both `/votes` and `/votes/[id]`. | |
| - | Types exported: `VoteSubjectKind`, `VoteSubject`, `VoteSubjectInput`. | |
| - | ||
| - | ### QA / screenshots | |
| - | - `_ops/qa-view/home-landing-r3.png` - new home page | |
| - | - `_ops/qa-view/votes-with-bill-r3.png` - votes list showing bill identifiers | |
| - | - `_ops/qa-view/vote-detail-r3.png` - vote detail with bill banner above roster | |
| - | ||
| - | ## R2 (2026-06-27) - Search/filter on every list page + first-class Cite This (Claude) | |
| - | ||
| - | ### Summary | |
| - | Wave R2 delivers the two owner-priority features from docs/REDESIGN.md §3 and §2. | |
| - | QA CLEAN (17 routes, zero overflow, zero hydration errors). | |
| - | ||
| - | ### 1. Search/filter bar - every list page | |
| - | ||
| - | **New shared component: `components/search-filter-bar.tsx`** | |
| - | ||
| - | `SearchFilterBar` - RSC-compatible visual shell (no `"use client"`). Wraps filter | |
| - | fields in a Card, provides a responsive 1→2→4 col grid, and a result-count + | |
| - | clear-filters slot. Sub-components: | |
| - | - `FilterField` - labeled wrapper (label above, never placeholder-only; a11y Rule §Search) | |
| - | - `FilterInput` - `min-h-[52px]`, `text-[18px]` search box (ACCESSIBILITY.md Rule #2, §Search) | |
| - | - `FilterSelect` - `min-h-[44px]` select (Rule #3) | |
| - | - `FilterSubmit` - submit button for server-side forms | |
| - | ||
| - | Pages updated: | |
| - | | Page | Filter type | Fields | | |
| - | |---|---|---| | |
| - | | `/members` | Client-side (MembersBrowser) | Name search (18px/52px), state, chamber, party | | |
| - | | `/bills` | Server-side GET form | Title/number search, policy area, congress | | |
| - | | `/votes` | Server-side GET form | Keyword search (result+question), chamber | | |
| - | | `/spending` | Server-side GET form | Agency/recipient/program search | | |
| - | | `/executive` | Client-side (ExecutiveBrowser) | Title/agency search, type (pre-existing, unchanged) | | |
| - | ||
| - | **lib/feed.ts additions:** | |
| - | - `VoteFilter` type + `buildVotesWhere()` (pushes chamber/keyword filter into SQL) | |
| - | - `getVoteCountFiltered(filter)` - cheap COUNT(*) for the result eyebrow | |
| - | - `getVotesFiltered(filter, limit, offset)` - filtered paginated list (35k+ rows) | |
| - | - `getAwardsFiltered(q?)` - keyword filter on agency/recipient/program | |
| - | ||
| - | All filter forms use `method="get"` (server-side, bookmark-safe, no JS required). | |
| - | Members and executive remain client-side (smaller in-memory sets, already loaded). | |
| - | ||
| - | ### 2. Cite This - first-class citation component | |
| - | ||
| - | **New component: `components/cite-this.tsx`** (`"use client"`) | |
| - | ||
| - | Generates APA, MLA, and Chicago citations with: | |
| - | - Tab selector (role="tablist/tab", keyboard arrow-key nav) | |
| - | - Copy-to-clipboard button (aria-label, min-h-[44px], "Copied ✓" confirmation) | |
| - | - Permalink (usdocket.org URL) + official .gov source in every citation (neutrality rule #1) | |
| - | - Accessed date computed entirely client-side via `useEffect` (hydration-safe: SSR shows | |
| - | "Loading citation…" placeholder, client hydrates with real date) | |
| - | - Works in Reference / Dark / Plain themes (CSS tokens only, no hardcoded colors) | |
| - | ||
| - | **Citation formats (example for a Senate vote):** | |
| - | ||
| - | APA: | |
| - | ``` | |
| - | Docket. (2026). SENATE Roll-Call Vote - Cloture on the Motion to Proceed (Jan 10, 2025) [Data set]. U.S. Congress. Retrieved June 27, 2026, from https://usdocket.org/votes/1 | |
| - | Official government source: https://www.senate.gov/... | |
| - | ``` | |
| - | ||
| - | MLA: | |
| - | ``` | |
| - | "SENATE Roll-Call Vote - Cloture on the Motion to Proceed (Jan 10, 2025)." Docket, United States Congress, 2026, usdocket.org/votes/1. Accessed 27 June 2026. | |
| - | Official government source: https://www.senate.gov/... | |
| - | ``` | |
| - | ||
| - | Chicago: | |
| - | ``` | |
| - | Docket. "SENATE Roll-Call Vote - Cloture on the Motion to Proceed (Jan 10, 2025)." United States Congress. 2026. https://usdocket.org/votes/1 (accessed June 27, 2026). | |
| - | Official government source: https://www.senate.gov/... | |
| - | ``` | |
| - | ||
| - | **Placed on:** | |
| - | - `/bill/[id]` - "Cite this record" after the official-record footer | |
| - | - `/votes/[id]` - "Cite this record" in the page footer | |
| - | - `/members/[id]` - "Cite this record" after the policy footer note | |
| - | - `/bills` - "Cite this page" in the footer | |
| - | - `/votes` - "Cite this page" in the footer | |
| - | - `/members` - "Cite this page" in the footer | |
| - | - `/spending` - "Cite this page" in the footer | |
| - | - `/executive` - "Cite this page" in the footer | |
| - | ||
| - | ### Accessibility compliance | |
| - | - Search box: `min-h-[52px]` + `text-[18px]` (ACCESSIBILITY.md §Search: "≥50px tall, ≥18px font") | |
| - | - All controls: `min-h-[44px]` tap targets (Rule #3) | |
| - | - Labels above every input, never placeholder-only (§Search) | |
| - | - Focus rings on all tab buttons and copy button (Rule #4) | |
| - | - Tab bar keyboard navigation: ArrowLeft/ArrowRight cycles through APA/MLA/Chicago | |
| - | - Neutrality: every citation includes the official .gov source (rule #1); no adjectives | |
| - | ||
| - | ## R1 (2026-06-26) - UX/QoL redesign foundation (Claude) | |
| - | ||
| - | ### Summary | |
| - | Wave R1 delivers the four foundation changes from docs/REDESIGN.md. QA CLEAN (17 routes, zero overflow, zero hydration errors). | |
| - | ||
| - | ### 1. Default theme: Reference (light) replaces Dark | |
| - | - `app/layout.tsx`: initial `<html>` class changed from `dark` → `light`; inline pre-paint script defaults to `'light'` instead of `'dark'`; added Atkinson Hyperlegible via `next/font/google` as `--font-body` CSS variable. | |
| - | - `app/globals.css`: `:root.light` token values updated to spec (docs/REDESIGN.md §"Theme system"): | |
| - | - bg `#F8FAFC`, surface `#FFFFFF`, text `#0F172A`, muted `#475569`, border `#E2E8F0` | |
| - | - body-size bumped to **18px** (was 16px), leading to **1.6** (was 1.5) - ACCESSIBILITY.md Rule #2 | |
| - | - Added `:root.light body { font-family: var(--font-body), ... }` - Atkinson Hyperlegible in Reference mode; `.label`/`.pill`/`.font-mono` classes still apply JetBrains Mono explicitly for data tokens. | |
| - | - Muted semantic tokens (`--dk-yea/nay/hold`) adjusted for AA on light bg. | |
| - | - `tailwind.config.ts`: added `font-body` alias for `var(--font-body)`. | |
| - | - Three-mode toggle still works; persisted in localStorage. Dark mode and Plain mode are unchanged. | |
| - | ||
| - | ### 2. Toggle relabeled: Reference / Dark / Plain | |
| - | - `components/display-toggle.tsx`: mode order changed to Reference first, labels updated ("Readable" → "Reference"), default state and localStorage fallback both changed from `"dark"` to `"light"`. | |
| - | ||
| - | ### 3. Header: global search icon added | |
| - | - `components/site-header.tsx`: added a search icon `<Link href="/search">` in the header logo row, visible on all viewports. Inline SVG, `min-h-[44px] min-w-[44px]` (a11y Rule #3). The display toggle and hamburger are unchanged. | |
| - | ||
| - | ### 4. Register home decluttered | |
| - | - `app/register/page.tsx`: plain-language explainer paragraph added above the feed table (≤8th-grade, active voice, source stated). The four secondary cards (composition by type, awards total, recently-changed, browse by topic) moved into a `<details>` / `<summary>` collapsed block labeled "More: composition, awards & recently changed". The ranked feed is now the single primary focus. | |
| - | ||
| - | ### 5. States page: 50 states vs DC & territories | |
| - | - `app/states/page.tsx`: introduced `NON_STATE_CODES = Set(["DC","PR","AS","GU","MP","VI"])`. States split into two separate `DataTable` sections: "50 states" (never produces a "51" count) and "DC & U.S. territories · not states". The DC section copy explicitly states "DC is a federal district" and names each territory. Coverage summary cards label "states" not "jurisdictions". The "51 states" count can never appear. | |
| - | ||
| - | ### Out of scope (R2) | |
| - | Per-page search/filter bars and the "Cite this" component are R2. Stubs may be added later. | |
| - | ||
| - | --- | |
| - | ||
| - | ## F5 (this wave) - mobile-responsive overhaul (GLM start / Claude finish) | |
| - | ||
| - | ### Zero-overflow guarantee at 390px | |
| - | ||
| - | **`DataTable` component** (`components/data-table.tsx`) - the single responsive | |
| - | list primitive used across all table pages. `>=md`: a real `<table>` with | |
| - | `table-fixed` layout (prevents auto-layout column expansion) inside an | |
| - | `overflow-x-auto` scroll wrapper. `<md`: stacked labeled cards with a cardTitle | |
| - | row + a definition list of labeled fields. `[contain:layout]` + `overflow:hidden` | |
| - | on the wrapping `Card` prevent the desktop table from escaping into page layout. | |
| - | ||
| - | **`/members/[id]` page** - the only route that was still failing QA after GLM. | |
| - | Root cause: the desktop vote-history table had `table-layout: auto` and long | |
| - | vote question text, causing `scrollWidth` to balloon to ~1971px on mobile. | |
| - | Fixes: (1) `table-fixed` on all DataTable tables, (2) `overflow-hidden` on each | |
| - | `<details>` year section and its content wrapper, (3) `overflow-x-hidden` added | |
| - | to the page-level container `div` - this turns it into a proper BFC that clips | |
| - | any descendant content before it can inflate `document.documentElement.scrollWidth`. | |
| - | QA result: **QA CLEAN across all 17 routes**. | |
| - | ||
| - | ### Site header | |
| - | Hamburger on mobile (separate from DisplayToggle), `>=44px` tap targets, body | |
| - | scroll lock, Escape key closes menu. Done by GLM; untouched in F5. | |
| - | ||
| - | ### Calendar | |
| - | Desktop month-grid (`hidden md:block`) + `MobileAgenda` day-grouped list | |
| - | (`md:hidden`). Done by GLM; untouched in F5. | |
| - | ||
| - | ### States page | |
| - | "50 states + DC & territories" label; un-ingested coverage shown as muted | |
| - | "Coverage rolling out" (not red). Done by GLM; untouched in F5. | |
| - | ||
| - | ### CSP - photo sources | |
| - | `next.config.js` `img-src` directive extended to allow headshots from | |
| - | `https://bioguide.congress.gov` and `https://upload.wikimedia.org` in addition | |
| - | to the existing `https://unitedstates.github.io` source. | |
| - | ||
| - | ### Members roster UX - current Congress first | |
| - | `MemberSummary.isCurrent` added (`lib/feed.ts`) - `true` for members with | |
| - | numeric IDs ≤ 600 (the 119th-Congress ingest batch, IDs 1-537, with a safe | |
| - | buffer). `MembersBrowser` (`components/members-browser.tsx`) now: | |
| - | - Default sort: current Congress first → historical-with-photo → historical-without-photo | |
| - | - Default visible: current Congress + historical-with-photo (~2,917 members) | |
| - | - Historical members without a photo (~9,850) are hidden behind a | |
| - | "Show all N historical members" toggle; filter bar search always reveals all | |
| - | - When any filter is active, the toggle is suppressed and all matching members | |
| - | are shown (the user is looking for something specific) | |
| - | - Neutral copy: "Showing current Congress and members with a recorded photo" | |
| - | ||
| - | ## F4 (this wave) - faces + schedule + fill the gaps | |
| - | ||
| - | - **Photos everywhere** - `members.photo_url` (official headshot mirror at | |
| - | `unitedstates/images`) is now surfaced through the whole member read model | |
| - | and shown via a new `MemberAvatar` component (lazy `<img>` over a clean | |
| - | INITIALS tile; initials show when `photo_url` is NULL **or** the image | |
| - | fails to load - never a broken image). Avatars appear on: | |
| - | `/members` roster, `/members/[id]` profile header (lg), `/votes/[id]` roster | |
| - | (sm, beside each name + position), `/bill/[id]` sponsor card, `/search` | |
| - | member results, and `/states/[code]` delegation. The photo is **decorative** | |
| - | (`aria-hidden`, `alt=""`); the cited record stays the allowlisted bioguide | |
| - | Source link, so neutrality #1 is untouched. Member names are linked to | |
| - | `/members/[id]` wherever they appear. ~400-500-row rosters stay fast via | |
| - | `loading="lazy"` + `decoding="async"` + a plain `<img>` (no per-row | |
| - | next/image domain config / JS cost). | |
| - | - **`/schedule` + `/api/meetings`** - the congressional meetings tracker. | |
| - | `getMeetings(filter)` reads the `meetings` table (id, chamber, committee, | |
| - | date, time, datetime_iso, title, subject, type, location, status, | |
| - | source_url) **defensively**: while the table is mid-ingest (or absent), | |
| - | `safeAll` swallows the missing-table error and the page shows an honest | |
| - | "No meetings on record yet" - data flows the moment the table lands, no | |
| - | code change. `MeetingsBrowser` (client) filters by search/chamber/type/ | |
| - | committee and toggles **Upcoming / Past / All** (today is computed | |
| - | server-side and passed in to avoid hydration drift), then renders a | |
| - | **day-grouped list**: each day is a header with its meetings as cards | |
| - | (time, chamber, committee, subject, type pill, status, location, Source | |
| - | link). Type is shown as a **text mark** (never color alone) so it reads in | |
| - | Plain mode and for the color-blind. Added to header + footer nav. | |
| - | - **Empty-field honesty** - every member-name surface now carries a headshot | |
| - | or an honest initials tile; the member profile header states "No official | |
| - | headshot on file" when `photo_url` is NULL. Voice/UC vote and absent-data | |
| - | states already carry their honesty notes; no blank/broken UI element ships. | |
| - | ||
| - | ## F3 (this wave) - DATA-layer UI | |
| - | ||
| - | Four new surfaces against the new ingest tables, all reusing the F2 read | |
| - | model + components: | |
| - | ||
| - | - **Calendar** (`/calendar` + `/api/calendar`) - a real month-grid calendar | |
| - | of when laws *take effect*, *become law*, and votes happen. Server-rendered | |
| - | (searchParam-driven for keyboard/SEO), prev/next/today nav, a kind filter, | |
| - | a plain-language legend, and a day panel. Every kind carries a **text mark** | |
| - | (never color alone) so the grid reads clearly in Plain mode and for | |
| - | color-blind readers. Each event links to its source. | |
| - | - **Categories** (`/categories` + `/api/categories`) - topic browse with bill + | |
| - | award counts (join `bills.policy_area` ↔ `awards.category` ↔ `categories`). | |
| - | Clicking a topic filters `/bills?policyArea=`. Topics with no record are | |
| - | shown honestly ("No record yet") rather than hidden. | |
| - | - **Bills** (`/bills`) - a browseable bill list (new) that is the filter | |
| - | target for categories; supports `policyArea`, `q`, `congress`. Reuses | |
| - | `getBills`. | |
| - | - **Member year-index** (`/members/[id]`) - the headline member-page upgrade: | |
| - | every recorded vote grouped **by year** (`votes.year`) into collapsible | |
| - | `<details>` sections (newest first), with a per-year count summary and the | |
| - | member's position on each vote (muted `PositionTag`). | |
| - | - **Recently changed** (`/api/changes` + `ChangesFeed` on `/register`) - a | |
| - | compact added/updated/removed feed from `change_events`, with a Source link | |
| - | per entry. | |
| - | - **Bill topic + subjects** - `/bill/[id]` now shows the `policy_area` (links | |
| - | to the filtered bill list) and the top `bill_subjects` tags. | |
| - | ||
| - | Nav updated: header + footer now include Calendar, Topics, and Bills. | |
| - | ||
| - | ## Data layer (`lib/feed.ts`) | |
| - | ||
| - | Server-only read model. Pages import these functions directly (no fetch-to-self). | |
| - | Every returned fact carries an allowlisted `sourceUrl` (neutrality #1). | |
| - | ||
| - | - `getVotes()` / `getVoteById(id)` - roll calls; the detail includes the full | |
| - | `positions[]` roster (joined `vote_positions` ↔ `members`). Voice/UC votes | |
| - | (`recorded:false`) return an empty roster and the UI shows the honesty note. | |
| - | - `getMembers(filter)` / `getMember(id)` - roster + full record (recent votes | |
| - | with position, sponsored bills, amendments offered). | |
| - | - `getBills(filter)` / `getBillDetail(id)` - list + detail with sponsor | |
| - | (bioguide↔members join), versions, amendments, **diffs** (with rider flags), | |
| - | and the actions timeline. | |
| - | - `getBillDiff(billId, from, to)` - the signature "what got slipped in" view. | |
| - | - `getApportionments()` - OMB impound tracker (released/delayed/withheld). | |
| - | - `getExecutive()` - Federal Register actions. | |
| - | - `getStates()` / `getStateDetail(code)` - coverage aggregated per state | |
| - | (worst-case confidence + missing types); detail adds delegation + sponsored bills. | |
| - | - `globalSearch(q)` - bills / members / executive / awards / states. | |
| - | - **F3 additions:** `getCalendarEvents({from,to,kind})` (calendar_events), | |
| - | `getCategories()` (topic counts), `getPolicyAreaLabels()`, | |
| - | `getBillSubjects(billId)` (bill_subjects), `getChangeEvents(limit)` | |
| - | (change_events), and `MemberDetail.votesByYear` (votes grouped by | |
| - | `votes.year`) + `BillDetail.subjects`. | |
| - | - **F4 additions:** `MemberSummary.photoUrl` + `VotePosition.photoUrl` | |
| - | (members.photo_url wired into every member/vote query), `Meeting` + | |
| - | `MeetingFilter` + `getMeetings(filter)` + `getMeetingCommittees()` | |
| - | (meetings table, defensively read). | |
| - | ||
| - | ## Pages | |
| - | ||
| - | | Route | Surface | Source | | |
| - | |---|---|---| | |
| - | | `/votes` | Browse recorded roll calls | `getVotes` | | |
| - | | `/votes/[id]` | **THE ROSTER** - every member's position (Yea/Nay/Present/Not Voting), name+party+state+district. Default sort by vote then alphabetical, not grouped by party. Voice/UC → honesty note. | `getVoteById` | | |
| - | | `/members` | Browseable roster, search/filter by name/state/chamber/party (labeled selects, a11y §Search) | `getMembers` + `MembersBrowser` | | |
| - | | `/members/[id]` | Full record: recent votes (with position), bills sponsored, amendments. Links each to the bill/vote. | `getMember` | | |
| - | | `/bill/[id]` | Enhanced: **diff view (riders flagged)**, amendments, votes, full status timeline, sponsor + linked cosponsors. Honest "no record exists" where data is voice-vote/absent. | `getBillDetail` | | |
| - | | `/spending` | Awards + **OMB apportionment impound tracker** (released/delayed/withheld, muted colors) + fiscal stats | `getAwards`/`getApportionments`/`getFiscal` | | |
| - | | `/executive` | Federal Register (EOs / proposed & final rules / notices), filterable by type | `getExecutive` + `ExecutiveBrowser` | | |
| - | | `/states` | Per-state confidence map (current/stale/missing badges) | `getStates` | | |
| - | | `/states/[code]` | State detail: delegation + sponsored bills | `getStateDetail` | | |
| - | | `/search` | Global search (bills/members/executive/awards/states). Labeled, ≥52px target, GET-form deep-links | `globalSearch` + `SearchBox` | | |
| - | | `/calendar` | **Month-grid calendar** - what takes effect / becomes law / votes, day by day. Kind filter + legend (text marks, not color alone), day panel, prev/next/today nav. Legible in all 3 modes. | `getCalendarEvents` | | |
| - | | `/schedule` | **Congressional meetings** - committee hearings/markups/business meetings/floor action, day-grouped. Upcoming/Past/All scope + chamber/type/committee filters; each links to its official schedule. Defensive: honest "No meetings on record yet" while the `meetings` table is mid-ingest. | `getMeetings` + `MeetingsBrowser` | | |
| - | | `/categories` | Topic browse - bills + awards per policy area (counts only) | `getCategories` | | |
| - | | `/bills` | Browseable bill list; filter target for topics (`?policyArea=`), congress, text | `getBills` + `getPolicyAreaLabels` | | |
| - | ||
| - | ## Brand + neutrality + accessibility (enforced on all new pages) | |
| - | ||
| - | - **Brand:** brass rationed (rider flag is one of the few accent uses); custody- | |
| - | stamp mono data; dark default. Tokens in `globals.css` / `tailwind.config.ts`. | |
| - | - **Neutrality:** Source link on every fact; no adjectives; muted vote/party | |
| - | colors (`--dk-yea/nay`, `--dk-party-d/r/i`) never vivid red/blue; roster sorted | |
| - | by vote then alphabetical (never grouped by party); "no individual record | |
| - | exists" honesty notes for voice/UC/conference votes; no comment surfaces. | |
| - | - **3-mode toggle:** Dark / Readable Light / Plain - applied pre-paint via the | |
| - | inline script in `layout.tsx`; all new pages inherit the token-driven styling | |
| - | and work in every mode (Plain collapses accents to ink). | |
| - | ||
| - | ## Components added | |
| - | ||
| - | `position-tag.tsx` (PositionTag, muted yea/nay), `rider-flag.tsx` (RiderFlag, | |
| - | ChangeTag), `members-browser.tsx`, `executive-browser.tsx`, `search-box.tsx`. | |
| - | F3: `changes-feed.tsx` (compact added/updated/removed list with Source link). | |
| - | F4: `member-avatar.tsx` (`MemberAvatar` + `initialsOf`, lazy img over initials, | |
| - | decorative), `meetings-browser.tsx` (filter + day-grouped Upcoming/Past/All). | |
| - | ||
| - | ## Member page - year-index | |
| - | ||
| - | `/members/[id]` now carries a "Vote record, year by year" section: every | |
| - | recorded vote grouped by `votes.year` into native `<details>` (newest first, | |
| - | most-recent year open), with a per-year count chip row for jumping and the | |
| - | member's position on each vote (muted `PositionTag`). Undated votes bucket to | |
| - | "Undated" and sort last. | |
| - | ||
| - | ## Build | |
| - | ||
| - | `npm run build` passes (Next 14, strict TS). The `npm run neutrality` script | |
| - | reports JSX `key={...}` false positives and a pre-existing `.pa.us` entry in | |
| - | mock-data; these are not real copy violations and the build gate is clean. |
| @@ -1,4 +1,4 @@ | ||
| - | # Docket - Navigation Latency Audit & Fix Notes | |
| + | # Docket - Navigation Latency Audit & Fix Notes | |
| Session: 2026-06-27. All timings local (`curl -w "%{time_total}"` against | ||
| `http://localhost:3000` on CT247). Cloudflare Tunnel overhead is on top of these. | ||
| @@ -10,7 +10,7 @@ Session: 2026-06-27. All timings local (`curl -w "%{time_total}"` against | ||
| ### 1. No Cache-Control on any API route (CONFIRMED, FIXED) | ||
| Every `/api/*` GET response had zero `Cache-Control` header before this session. | ||
| - | Cloudflare's default for headerless JSON responses is "bypass cache" - meaning | |
| + | Cloudflare's default for headerless JSON responses is "bypass cache" - meaning | |
| every client-side API call (register-hero 15-second poll, MembersBrowser | ||
| "show historical" paginated fetches, MemberVoteYear lazy year panels) made a | ||
| full round-trip over the CF Tunnel to the CT247 origin. | ||
| @@ -22,8 +22,8 @@ override this cleanly with the same value. | ||
| ### 2. `force-dynamic` still on pure static API routes | ||
| - | `/api/categories` and `/api/states` read no search params - they returned | |
| - | the same data every time - but carried `force-dynamic`, preventing Next.js | |
| + | `/api/categories` and `/api/states` read no search params - they returned | |
| + | the same data every time - but carried `force-dynamic`, preventing Next.js | |
| server-side ISR. Changed to `export const revalidate = 300` so the Node | ||
| server caches the response object for 5 min in addition to CF edge caching. | ||
| @@ -33,28 +33,28 @@ Without `loading.tsx`, clicking a nav link caused a noticeable blank/stall | ||
| while the RSC payload loaded over the tunnel. The header stayed in place but | ||
| the content area showed nothing. | ||
| - | Fix: added `loading.tsx` skeletons for the six heaviest routes - register, | |
| + | Fix: added `loading.tsx` skeletons for the six heaviest routes - register, | |
| votes, bills, spending, members, calendar. They render immediately on | ||
| client-side navigation while the RSC payload is in-flight. | ||
| - | ### 4. Pages already on ISR - not a root cause here | |
| + | ### 4. Pages already on ISR - not a root cause here | |
| All pages (`/register`, `/votes`, `/bills`, etc.) already carried | ||
| `export const revalidate = 300` and return `Cache-Control: s-maxage=300, | ||
| stale-while-revalidate`. This was fixed in the previous perf session. | ||
| The `x-nextjs-cache: STALE/HIT` headers confirm ISR is working correctly. | ||
| - | ### 5. Plain `<a href>` links - not present in nav | |
| + | ### 5. Plain `<a href>` links - not present in nav | |
| All internal navigation uses `<Link>` from `next/link`. No regressions here; | ||
| Next.js client-side transitions + prefetch are already wired correctly. | ||
| - | ### 6. Font loading - already correct | |
| + | ### 6. Font loading - already correct | |
| Four fonts (Archivo, JetBrains Mono, Fraunces, Atkinson Hyperlegible) all | ||
| loaded with `display: "swap"` via `next/font/google`. No blocking font fetch. | ||
| - | ### 7. Page payload sizes (gzip) - acceptable after compression | |
| + | ### 7. Page payload sizes (gzip) - acceptable after compression | |
| Raw HTML sizes appear large but gzip brings them to reasonable transfer sizes: | ||
| - `/register`: 1.45 MB raw → 92 KB gzip | ||
| @@ -73,14 +73,14 @@ Raw HTML sizes appear large but gzip brings them to reasonable transfer sizes: | ||
| | `app/api/feed/route.ts` | Removed duplicate handler-level Cache-Control (config default now applies) | | ||
| | `app/api/members/route.ts` | Explicit `Cache-Control` in response (same as config default; documents intent) | | ||
| | `app/api/members/[id]/votes/route.ts` | Explicit `Cache-Control` in response | | ||
| - | | `app/register/loading.tsx` | New - hero + table skeleton | | |
| - | | `app/votes/loading.tsx` | New - filter bar + table skeleton | | |
| - | | `app/bills/loading.tsx` | New - filter bar + table skeleton | | |
| - | | `app/spending/loading.tsx` | New - fiscal cards + table skeleton | | |
| - | | `app/members/loading.tsx` | New - member card grid skeleton | | |
| - | | `app/calendar/loading.tsx` | New - calendar grid skeleton | | |
| + | | `app/register/loading.tsx` | New - hero + table skeleton | | |
| + | | `app/votes/loading.tsx` | New - filter bar + table skeleton | | |
| + | | `app/bills/loading.tsx` | New - filter bar + table skeleton | | |
| + | | `app/spending/loading.tsx` | New - fiscal cards + table skeleton | | |
| + | | `app/members/loading.tsx` | New - member card grid skeleton | | |
| + | | `app/calendar/loading.tsx` | New - calendar grid skeleton | | |
| - | QA after rebuild: **CLEAN** - 17 routes, no overflow, no hydration errors. | |
| + | QA after rebuild: **CLEAN** - 17 routes, no overflow, no hydration errors. | |
| --- | ||
| @@ -97,10 +97,10 @@ shows up for end-users, not on localhost. | ||
| | `/members` | 0.11s | 0.11s | ISR warm; unchanged | | ||
| | `/calendar` | 0.31s | 0.06s | Second-hit ISR | | ||
| | `/spending` | 0.66s | 0.74s | 320 awards still full-table; within reason | | ||
| - | | `/api/votes` | - (no CC) | CC: s-maxage=300 | CF now caches | | |
| - | | `/api/feed` | - (no CC) | CC: s-maxage=300 | CF now caches | | |
| - | | `/api/members` | - (no CC) | CC: s-maxage=300 | CF now caches | | |
| - | | `/api/categories` | - (force-dynamic, no CC) | CC: s-maxage=300 (ISR) | CF + Node cache | | |
| + | | `/api/votes` | - (no CC) | CC: s-maxage=300 | CF now caches | | |
| + | | `/api/feed` | - (no CC) | CC: s-maxage=300 | CF now caches | | |
| + | | `/api/members` | - (no CC) | CC: s-maxage=300 | CF now caches | | |
| + | | `/api/categories` | - (force-dynamic, no CC) | CC: s-maxage=300 (ISR) | CF + Node cache | | |
| --- | ||
| @@ -108,12 +108,12 @@ shows up for end-users, not on localhost. | ||
| Cloudflare by default **bypasses cache for HTML responses** even when the | ||
| origin sends `Cache-Control: s-maxage=300`. Pages in this app return ISR HTML | ||
| - | with `s-maxage=300, stale-while-revalidate` - but CF won't cache HTML without | |
| + | with `s-maxage=300, stale-while-revalidate` - but CF won't cache HTML without | |
| an explicit cache rule. | ||
| Create this rule in the Cloudflare dashboard (or via CF API) for the `usdocket.org` zone: | ||
| - | **Rule:** Cache Everything - match `usdocket.org/*` | |
| + | **Rule:** Cache Everything - match `usdocket.org/*` | |
| - Cache status: `Cache Everything` | ||
| - Edge TTL: `Respect existing headers` (honors the `s-maxage=300` from origin) | ||
| - Browser TTL: `Respect existing headers` | ||
| @@ -140,7 +140,7 @@ curl -X POST "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/rulesets" \ | ||
| -H "Authorization: Bearer $CF_TOKEN" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{ | ||
| - | "name": "Cache Everything - usdocket.org", | |
| + | "name": "Cache Everything - usdocket.org", | |
| "kind": "zone", | ||
| "phase": "http_request_cache_settings", | ||
| "rules": [ |
| @@ -1,4 +1,4 @@ | ||
| - | # Docket - Performance Pass 2: Edge-Cacheable Filter Pages | |
| + | # Docket - Performance Pass 2: Edge-Cacheable Filter Pages | |
| Session 2026-06-27. Performance-only; no visual/design changes. | ||
| @@ -18,7 +18,7 @@ crossed the home Cloudflare Tunnel to the CT247 origin (~0.5-0.8s) instead of | ||
| serving from the CF edge (~50-150ms). The `/api/*` routes were already | ||
| cacheable (`public, s-maxage=300`). | ||
| - | ## Fix - static shell + client-side filtering | |
| + | ## Fix - static shell + client-side filtering | |
| Each filter page was split into: | ||
| @@ -31,7 +31,7 @@ Each filter page was split into: | ||
| Because the shell no longer reads `searchParams`, Next renders it statically | ||
| and CF caches the HTML. The filtered data comes from the `/api/*` routes, which | ||
| - | are themselves CF-cached per query string - so filtered views are fast too. | |
| + | are themselves CF-cached per query string - so filtered views are fast too. | |
| ### Pages converted to static-shell + client-filter | ||
| @@ -49,12 +49,12 @@ stay server-rendered (no `searchParams`). | ||
| ### API routes extended to accept the page filters | ||
| - | - `app/api/votes/route.ts` - now parses `q`, `result` (→ `resultFilter`), | |
| + | - `app/api/votes/route.ts` - now parses `q`, `result` (→ `resultFilter`), | |
| `sort` (was chamber-only); delegates to `getVotesFiltered` / | ||
| `getVoteCountFiltered`. | ||
| - | - `app/api/bills/route.ts` - sort mapping fixed to handle all four values | |
| + | - `app/api/bills/route.ts` - sort mapping fixed to handle all four values | |
| (`updated|action|congress|status`). | ||
| - | - `app/api/spending/awards/route.ts` - added `sort` (`date|agency`, default | |
| + | - `app/api/spending/awards/route.ts` - added `sort` (`date|agency`, default | |
| largest-amount-first) via `getAwardsFiltered(q, sort)`. | ||
| All keep their existing `public, s-maxage=300, stale-while-revalidate=600` | ||
| @@ -83,7 +83,7 @@ Each route hit twice through `https://usdocket.org`; 2nd-hit result reported. | ||
| ### Before (confirmed by owner / pre-fix) | ||
| `/votes` and the other filter pages returned `cf-cache-status: BYPASS` with | ||
| - | `Cache-Control: private, no-store` - every hit crossed the tunnel. | |
| + | `Cache-Control: private, no-store` - every hit crossed the tunnel. | |
| ### After | ||
| @@ -96,10 +96,10 @@ Each route hit twice through `https://usdocket.org`; 2nd-hit result reported. | ||
| | `/executive` | HIT | ~0.29s | | ||
| | `/search` | HIT | ~0.14s | | ||
| | `/votes?chamber=house` | HIT | ~0.13s | | ||
| - | | `/votes?chamber=senate` | HIT | - | | |
| + | | `/votes?chamber=senate` | HIT | - | | |
| | `/bills?policyArea=Taxation` | HIT | ~0.13s | | ||
| - | | `/bills?sort=congress` | HIT | - | | |
| - | | `/spending?sort=date` | HIT | - | | |
| + | | `/bills?sort=congress` | HIT | - | | |
| + | | `/spending?sort=date` | HIT | - | | |
| | `/votes/[id]` (detail) | HIT | ~0.14s | | ||
| | `/bill/[id]` (detail) | HIT | ~0.16s | | ||
| @@ -118,7 +118,7 @@ on repeat. No route emits `private`/`no-store` anymore. | ||
| ## Build / QA | ||
| `bash /shared/projects/docket/_ops/rebuild-qa.sh` → | ||
| - | **QA CLEAN - no overflow, no hydration errors across 17 routes.** | |
| + | **QA CLEAN - no overflow, no hydration errors across 17 routes.** | |
| ## Net effect | ||
| @@ -1,4 +1,4 @@ | ||
| - | # Docket - Performance Audit & Fix Notes | |
| + | # Docket - Performance Audit & Fix Notes | |
| Audit and fix session conducted 2026-06-26/27. All timings are server-side | ||
| response time measured with `curl -o /dev/null -w "%{time_total}"` against | ||
| @@ -20,7 +20,7 @@ cold request; "second hit" is a repeat within the same 5-minute TTL window. | ||
| | `/states` | 0.097 s | 0.025 s | 0.026 s | 4x | | ||
| | `/calendar` | 1.07 s | 0.06 s | 0.05 s | 21x | | ||
| - | QA: **CLEAN** - 17 routes, no overflow, no hydration errors, confirmed on | |
| + | QA: **CLEAN** - 17 routes, no overflow, no hydration errors, confirmed on | |
| every rebuild. | ||
| --- | ||
| @@ -38,7 +38,7 @@ helpers: `buildBillsWhere(filter)`, `getBillsCount(filter)`, `getVoteCount(chamb | ||
| ### 2. Missing indexes | ||
| - | The `bills` table had no index on `sponsor_id` - `getMember()` did a full | |
| + | The `bills` table had no index on `sponsor_id` - `getMember()` did a full | |
| 208k-row scan to find each member's sponsored bills. Added 11 indexes via a | ||
| Python retry script (to survive concurrent write locks from `docket-photos.service`): | ||
| @@ -75,7 +75,7 @@ New API route: `GET /api/members/[id]/votes?year=YYYY`. | ||
| Result: 1.81 s → 0.11 s for A000055. | ||
| - | ### 5. Members roster - 16 MB SSR payload | |
| + | ### 5. Members roster - 16 MB SSR payload | |
| `getMembersDefault()` originally returned `isCurrent || !!photoUrl` = 10,326 | ||
| members (nearly all have photos after the photo-sync service ran). Changed to | ||
| @@ -83,18 +83,18 @@ members (nearly all have photos after the photo-sync service ran). Changed to | ||
| lazily via paginated `/api/members` calls in `MembersBrowser` when the toggle | ||
| is clicked. | ||
| - | ### 6. Calendar page - 6.8 MB event serialisation | |
| + | ### 6. Calendar page - 6.8 MB event serialisation | |
| `getCalendarEvents()` returned 2,737 full `CalendarEvent` objects for the | ||
| current month. The grid only needed one dot-per-kind-per-day. Changed to | ||
| `getCalendarDaySummary(from, to)` which uses `SELECT date, kind, COUNT(*) | ||
| GROUP BY date, kind`. Full events for the selected day are fetched separately. | ||
| - | ### 7. Register page - getAwards + double byKind | |
| + | ### 7. Register page - getAwards + double byKind | |
| `getAwards()` loaded the full awards table just to count rows and sum amounts. | ||
| Replaced with `getAwardsSummary()` (single `SELECT COUNT(*), SUM(amount)`). | ||
| - | `byKind(items)` was computed twice in JSX - now computed once as `kindBreakdown`. | |
| + | `byKind(items)` was computed twice in JSX - now computed once as `kindBreakdown`. | |
| ### 8. force-dynamic → revalidate = 300 | ||
| @@ -115,20 +115,20 @@ awards summary) are now served from memory on repeat calls within the window. | ||
| ## Files Changed | ||
| - | - `lib/feed.ts` - cache helper, SQL pagination, new query functions, | |
| + | - `lib/feed.ts` - cache helper, SQL pagination, new query functions, | |
| `getMemberVotesForYear`, lazy year-summary `votesByYear` | ||
| - | - `app/register/page.tsx` - revalidate, `getAwardsSummary`, single `byKind` | |
| - | - `app/votes/page.tsx` - revalidate, paginated `getVotes(100,0)`, `getVoteCount` | |
| - | - `app/bills/page.tsx` - revalidate, SQL `getBills` + `getBillsCount` | |
| - | - `app/members/page.tsx` - revalidate, `getMemberCounts` + `getMembersDefault` | |
| - | - `app/members/[id]/page.tsx` - revalidate, `MemberVoteYear` lazy panels | |
| + | - `app/register/page.tsx` - revalidate, `getAwardsSummary`, single `byKind` | |
| + | - `app/votes/page.tsx` - revalidate, paginated `getVotes(100,0)`, `getVoteCount` | |
| + | - `app/bills/page.tsx` - revalidate, SQL `getBills` + `getBillsCount` | |
| + | - `app/members/page.tsx` - revalidate, `getMemberCounts` + `getMembersDefault` | |
| + | - `app/members/[id]/page.tsx` - revalidate, `MemberVoteYear` lazy panels | |
| - `app/spending/page.tsx`, `app/states/page.tsx`, `app/executive/page.tsx`, | ||
| `app/states/[code]/page.tsx`, `app/votes/[id]/page.tsx`, `app/bill/[id]/page.tsx`, | ||
| - | `app/categories/page.tsx`, `app/search/page.tsx`, `app/schedule/page.tsx` - revalidate | |
| - | - `app/calendar/page.tsx` - revalidate, condensed `getCalendarDaySummary` | |
| - | - `app/api/bills/route.ts` - SQL pagination | |
| - | - `app/api/votes/route.ts` - SQL pagination | |
| - | - `app/api/members/route.ts` - `getMemberCounts` for cheap total | |
| - | - `app/api/members/[id]/votes/route.ts` - new, lazy year votes | |
| - | - `components/members-browser.tsx` - lazy-load historical members | |
| - | - `components/member-vote-year.tsx` - new, lazy year accordion | |
| + | `app/categories/page.tsx`, `app/search/page.tsx`, `app/schedule/page.tsx` - revalidate | |
| + | - `app/calendar/page.tsx` - revalidate, condensed `getCalendarDaySummary` | |
| + | - `app/api/bills/route.ts` - SQL pagination | |
| + | - `app/api/votes/route.ts` - SQL pagination | |
| + | - `app/api/members/route.ts` - `getMemberCounts` for cheap total | |
| + | - `app/api/members/[id]/votes/route.ts` - new, lazy year votes | |
| + | - `components/members-browser.tsx` - lazy-load historical members | |
| + | - `components/member-vote-year.tsx` - new, lazy year accordion |
| @@ -1,9 +1,9 @@ | ||
| - | # Docket - Member Photos: Source, Attribution, and Implementation Notes | |
| + | # Docket - Member Photos: Source, Attribution, and Implementation Notes | |
| ## Image Source | ||
| **Project:** [unitedstates/images](https://github.com/unitedstates/images) | ||
| - | **License:** Public domain / CC0 - these are official congressional headshots | |
| + | **License:** Public domain / CC0 - these are official congressional headshots | |
| released without restriction. | ||
| **Hosted at:** `https://unitedstates.github.io/images/congress/` | ||
| @@ -19,9 +19,9 @@ https://unitedstates.github.io/images/congress/{size}/{bioguide_id}.jpg | ||
| ``` | ||
| Available sizes: | ||
| - | - `225x275` - thumbnail | |
| - | - `450x550` - standard (Docket stores this size) | |
| - | - `original` - full resolution | |
| + | - `225x275` - thumbnail | |
| + | - `450x550` - standard (Docket stores this size) | |
| + | - `original` - full resolution | |
| **Verified:** HEAD-checked against `A000055` (Robert Aderholt) on 2026-06-26. | ||
| All three sizes return HTTP 200. A non-existent bioguide returns HTTP 404. | ||
| @@ -33,7 +33,7 @@ All three sizes return HTTP 200. A non-existent bioguide returns HTTP 404. | ||
| - Added to `ensure_votes_schema()` in `members.py` as an idempotent | ||
| `ALTER TABLE ADD COLUMN` (guarded by try/except, same pattern as other | ||
| additive columns). | ||
| - | - Column ownership: `member_photos.py` - the member roster pollers (`poll()` | |
| + | - Column ownership: `member_photos.py` - the member roster pollers (`poll()` | |
| and `poll_historical()`) do not touch `photo_url`. | ||
| ### Pollers | ||
| @@ -41,7 +41,7 @@ All three sizes return HTTP 200. A non-existent bioguide returns HTTP 404. | ||
| **`pollers/members.py`** (extended): | ||
| - Added `LEGISLATORS_HISTORICAL_JSON_URL` constant pointing to the `gh-pages` | ||
| branch of `unitedstates/congress-legislators`. | ||
| - | - Added `poll_historical()` - fetches `legislators-historical.json` (~12,230 | |
| + | - Added `poll_historical()` - fetches `legislators-historical.json` (~12,230 | |
| records) and upserts every member who has ever served in Congress using the | ||
| same `_parse_legislators()` / `_upsert_members()` helpers as the current | ||
| roster. Idempotent via `ON CONFLICT(bioguide_id) DO UPDATE`. | ||
| @@ -102,7 +102,7 @@ ssh root@REDACTED-IP "pct exec 247 -- bash -lc \ | ||
| | Current members (before historical poll) | ~537 | | ||
| | Total members after `poll_historical()` | ~12,767 | | ||
| | Members with `photo_url` (current only) | ~530+ | | ||
| - | | Members with `photo_url` (after historical) | varies - unitedstates/images covers most post-1900 members; pre-1900 coverage is sparse | | |
| + | | Members with `photo_url` (after historical) | varies - unitedstates/images covers most post-1900 members; pre-1900 coverage is sparse | | |
| ## Attribution requirement | ||
| @@ -1,29 +1,29 @@ | ||
| - | # DOCKET - Neutral Government Activity Tracker | |
| - | ### Product & Brand Plan (v0, planning only - nothing built) | |
| + | # DOCKET - Neutral Government Activity Tracker | |
| + | ### Product & Brand Plan (v0, planning only - nothing built) | |
| - | *Name: **Docket** (LOCKED). A "docket" is the official register of proceedings - it inherits TreeTrace's evidence/record DNA while reading as civic, not forensic-security.* | |
| + | *Name: **Docket** (LOCKED). A "docket" is the official register of proceedings - it inherits TreeTrace's evidence/record DNA while reading as civic, not forensic-security.* | |
| - | Date: 2026-06-26 · Status: PLAN ONLY · Method: overseer (Opus synth + Sonnet/Haiku research) | |
| + | Date: 2026-06-26 · Status: PLAN ONLY | |
| ### Locked decisions (2026-06-26) | ||
| - **Name:** Docket | ||
| - **Accent:** Brass/amber `#E0A53B` | ||
| - | - **Scope:** Federal **+ state** from day one (OpenStates in the MVP - see cost note in §4) | |
| + | - **Scope:** Federal **+ state** from day one (OpenStates in the MVP - see cost note in §4) | |
| - **Summaries at launch:** Structured fields first (title/status/$/votes from official data); LLM prose added later | ||
| --- | ||
| ## 1. The Idea in One Line | ||
| - | > The unfiltered public record of what government does - **every bill, law, amendment, and vote, including what gets slipped in by either side** - at federal *and* granular state level, made legible to any citizen, with **zero spin** and every fact sourced directly to an official record. | |
| + | > The unfiltered public record of what government does - **every bill, law, amendment, and vote, including what gets slipped in by either side** - at federal *and* granular state level, made legible to any citizen, with **zero spin** and every fact sourced directly to an official record. | |
| - | **Scope correction (2026-06-26, Zion):** Docket is NOT a spending tracker. Spending is one lens. Docket is **full-spectrum government transparency** - all bills, all laws, all amendments/riders, all votes, plus the procedural / closed-door activity that is technically public but buried where no ordinary citizen can find it. | |
| + | **Scope correction (2026-06-26, Zion):** Docket is NOT a spending tracker. Spending is one lens. Docket is **full-spectrum government transparency** - all bills, all laws, all amendments/riders, all votes, plus the procedural / closed-door activity that is technically public but buried where no ordinary citizen can find it. | |
| Two jobs at once: | ||
| - | - **Awareness** - a neutral feed so you catch what's happening (*"this was happening and I had no idea"* - e.g. the CBS Texas clip: an election bill stalling on a House blockade, a floor event most people never hear about). | |
| - | - **Lookup** - a resource where anyone, including an 80-year-old, can dig up exactly what their representatives did on any bill, in any state, on a granular level - unfiltered, in plain English. | |
| + | - **Awareness** - a neutral feed so you catch what's happening (*"this was happening and I had no idea"* - e.g. the CBS Texas clip: an election bill stalling on a House blockade, a floor event most people never hear about). | |
| + | - **Lookup** - a resource where anyone, including an 80-year-old, can dig up exactly what their representatives did on any bill, in any state, on a granular level - unfiltered, in plain English. | |
| - | The product's job is to convert **the complete official record** into calm, neutral, *legible* visibility - and to do it without ever picking a side. | |
| + | The product's job is to convert **the complete official record** into calm, neutral, *legible* visibility - and to do it without ever picking a side. | |
| --- | ||
| @@ -31,20 +31,20 @@ The product's job is to convert **the complete official record** into calm, neut | ||
| Docket must cover, for federal and every state, on a granular look-up-anything basis: | ||
| - | 1. **Bills & laws** - every bill introduced, its full text, status timeline, sponsors/cosponsors, committee path, and final enacted law. | |
| - | 2. **Amendments & riders - "what got slipped in."** *The signature feature.* Every amendment, manager's amendment, en-bloc add, and rider - and a **text-diff view** that shows what was added/removed between bill versions (Introduced → Reported → Engrossed → Enrolled) and between House/Senate/conference versions. This is where provisions get buried; Docket drags them into daylight. | |
| - | 3. **Votes - recorded AND unrecorded.** Roll calls with per-member positions, *and* surfacing when something passed by **voice vote / unanimous consent / suspension** (i.e., no individual record exists) - because "there is no record" is itself information a citizen deserves. | |
| - | 4. **The buried / closed-door layer** - committee markups & hearings, conference committees, procedural maneuvers (cloture, discharge petitions, motions to recommit), the Congressional Record, lobbying disclosures, and executive actions (Executive Orders, agency rulemaking via the Federal Register). Technically public, practically invisible. | |
| - | 5. **Spending** - appropriations, awards, agency outlays, debt/deficit. *One lens among many, not the headline.* | |
| - | 6. **Who** - representatives and their actual records (votes, sponsorships, what they slipped in), so "what did MY rep do" is one search. | |
| + | 1. **Bills & laws** - every bill introduced, its full text, status timeline, sponsors/cosponsors, committee path, and final enacted law. | |
| + | 2. **Amendments & riders - "what got slipped in."** *The signature feature.* Every amendment, manager's amendment, en-bloc add, and rider - and a **text-diff view** that shows what was added/removed between bill versions (Introduced → Reported → Engrossed → Enrolled) and between House/Senate/conference versions. This is where provisions get buried; Docket drags them into daylight. | |
| + | 3. **Votes - recorded AND unrecorded.** Roll calls with per-member positions, *and* surfacing when something passed by **voice vote / unanimous consent / suspension** (i.e., no individual record exists) - because "there is no record" is itself information a citizen deserves. | |
| + | 4. **The buried / closed-door layer** - committee markups & hearings, conference committees, procedural maneuvers (cloture, discharge petitions, motions to recommit), the Congressional Record, lobbying disclosures, and executive actions (Executive Orders, agency rulemaking via the Federal Register). Technically public, practically invisible. | |
| + | 5. **Spending** - appropriations, awards, agency outlays, debt/deficit. *One lens among many, not the headline.* | |
| + | 6. **Who** - representatives and their actual records (votes, sponsorships, what they slipped in), so "what did MY rep do" is one search. | |
| - | **Two modes for every pillar:** the *feed* (what's new, ranked by official action) and *deep look-up* (search/browse anything, any state, any granularity). The look-up mode is co-equal with the feed - Zion's explicit ask. | |
| + | **Two modes for every pillar:** the *feed* (what's new, ranked by official action) and *deep look-up* (search/browse anything, any state, any granularity). The look-up mode is co-equal with the feed - Zion's explicit ask. | |
| ## 1c. Accessibility as a first-class principle | ||
| Docket must be usable by **any** citizen, explicitly including an 80-year-old with low vision and low tech-literacy. This is a hard requirement, not a polish item: | ||
| - A clean, high-contrast **readable mode** alongside the dark forensic brand; large type, big touch targets, no hover-only interactions. | ||
| - | - Plain-language explanations at a low reading grade level - **without** adding spin (extraction, not opinion; see §3 neutrality). | |
| + | - Plain-language explanations at a low reading grade level - **without** adding spin (extraction, not opinion; see §3 neutrality). | |
| - "Look up my state / my rep / my topic" search that doesn't overwhelm. | ||
| - (Targets being researched; will land in `docs/ACCESSIBILITY.md` and fold into BRAND.md.) | ||
| @@ -68,38 +68,38 @@ The civic-data space is structurally empty at the consumer-awareness level: | ||
| --- | ||
| - | ## 3. Neutrality Doctrine - the non-negotiables | |
| + | ## 3. Neutrality Doctrine - the non-negotiables | |
| This is the whole product. If it leans, it dies. Five rules to **hard-code into the codebase**, not just the style guide: | ||
| 1. **Government-source-only.** Every displayed fact links to an official URL (Congress.gov, USAspending.gov, GPO/GovInfo, CBO, Treasury). No secondhand, no think tanks, no media. | ||
| - | 2. **No adjectives.** Strip all qualitative modifiers from generated text before display. Never "slashed/gutted/expanded/massive" - only "reduced by $X," "added $Y," with the number and the source. (USAFacts' rule: *adjectives are partisan.*) | |
| - | 3. **Surface by official action, never by virality.** Ranking = date of legislative action (introduced → markup → floor vote → signed) × dollar magnitude × people affected (per CBO/agency estimate). **Never** by news coverage or social engagement - that just imports media's agenda. | |
| + | 2. **No adjectives.** Strip all qualitative modifiers from generated text before display. Never "slashed/gutted/expanded/massive" - only "reduced by $X," "added $Y," with the number and the source. (USAFacts' rule: *adjectives are partisan.*) | |
| + | 3. **Surface by official action, never by virality.** Ranking = date of legislative action (introduced → markup → floor vote → signed) × dollar magnitude × people affected (per CBO/agency estimate). **Never** by news coverage or social engagement - that just imports media's agenda. | |
| 4. **No comment sections, ever.** The single biggest bias-injection point. If engagement is needed, allow reactions on **summary clarity only** (was this clear? yes/no), never on the bill itself. | ||
| 5. **LLM summaries use extraction templates, not free-form generation.** Pull structured fields from official bill text. Free-form summarization is where documented LLM political lean creeps in (AAAI 2025: models skew pro-Democratic on health/guns). Test high-salience summaries against a Polarization Index before they go live. | ||
| - | **Standing risks to monitor:** (a) LLM bias *drifts* as models update - needs ongoing automated checks, not a one-time test; (b) **topic-selection** is soft bias - proportional, dollar-and-action-weighted surfacing, not editorial "what matters" calls; (c) **partisan co-option** - if one base adopts it heavily it gets socially coded as "their tool"; mitigate with a visible bipartisan advisory board + dual-side launch outreach. | |
| + | **Standing risks to monitor:** (a) LLM bias *drifts* as models update - needs ongoing automated checks, not a one-time test; (b) **topic-selection** is soft bias - proportional, dollar-and-action-weighted surfacing, not editorial "what matters" calls; (c) **partisan co-option** - if one base adopts it heavily it gets socially coded as "their tool"; mitigate with a visible bipartisan advisory board + dual-side launch outreach. | |
| - | **Visualization neutrality note:** when showing party splits (sponsor party / vote breakdown *is* factual data, so it's allowed), **never use saturated red-vs-blue** - the eye codes that as "teams." Use muted/desaturated party tones of equal visual weight, or a neutral yea/nay encoding. | |
| + | **Visualization neutrality note:** when showing party splits (sponsor party / vote breakdown *is* factual data, so it's allowed), **never use saturated red-vs-blue** - the eye codes that as "teams." Use muted/desaturated party tones of equal visual weight, or a neutral yea/nay encoding. | |
| --- | ||
| ## 4. Data Architecture | ||
| - | ### Primary stack (the backbone - all free, all official) | |
| - | 1. **Congress.gov API** (`api.congress.gov/v3/`) - canonical bill status, actions, members, votes. Free api.data.gov key, 5,000 req/hr, same-day freshness. *Poll with `?sort=updateDate&fromDateTime=…` every 15-30 min.* | |
| - | 2. **GovInfo API** (`api.govinfo.gov`) - full bill text, Congressional Record, public laws, appropriations docs. Same key, **36,000 req/hr** (most generous). `/published/{start}/{end}` for delta polling. | |
| - | 3. **USAspending.gov API** (`api.usaspending.gov/v2/`) - award-level contracts/grants/loans, agency outlays. **No auth.** 10k-result cap per call → use bulk download for completeness. | |
| - | 4. **Treasury Fiscal Data API** (`fiscaldata.treasury.gov`) - debt, deficit, daily cash flows (DTS T+1), monthly outlays by agency (MTS). No auth. | |
| + | ### Primary stack (the backbone - all free, all official) | |
| + | 1. **Congress.gov API** (`api.congress.gov/v3/`) - canonical bill status, actions, members, votes. Free api.data.gov key, 5,000 req/hr, same-day freshness. *Poll with `?sort=updateDate&fromDateTime=…` every 15-30 min.* | |
| + | 2. **GovInfo API** (`api.govinfo.gov`) - full bill text, Congressional Record, public laws, appropriations docs. Same key, **36,000 req/hr** (most generous). `/published/{start}/{end}` for delta polling. | |
| + | 3. **USAspending.gov API** (`api.usaspending.gov/v2/`) - award-level contracts/grants/loans, agency outlays. **No auth.** 10k-result cap per call → use bulk download for completeness. | |
| + | 4. **Treasury Fiscal Data API** (`fiscaldata.treasury.gov`) - debt, deficit, daily cash flows (DTS T+1), monthly outlays by agency (MTS). No auth. | |
| ### Supplements | ||
| - | - **Roll-call votes, fastest path:** House Clerk XML (`clerk.house.gov/evs/`) + Senate roll-call XML - posted within hours, no key. Faster than the API for "a vote just happened." | |
| - | - **CBO cost estimates:** no API - scrape `cbo.gov/cost-estimates` by bill; macro projections via the `us-cbo` GitHub CSVs. | |
| - | - **Voteview.com** - bulk vote CSV + ideology scores (use scores internally for QA, not as user-facing labels). | |
| - | - **OpenStates v3** - 50-state + DC/PR coverage, standardized schema, JSON. **IN MVP SCOPE (decision: federal + state from day one).** Free tier is only 500 q/day, 10 req/sec - far too tight for production; **a paid tier is required** (Bronze 5k/day, Silver 50k/day; pricing not public - contact OpenStates). This is the one place the data stack stops being free. State ingest is also messier: freshness and schema completeness vary by state. Budget for: (a) the paid tier, (b) per-state polling cadence tuning, (c) a "coverage map" so the UI is honest about which states lag. | |
| + | - **Roll-call votes, fastest path:** House Clerk XML (`clerk.house.gov/evs/`) + Senate roll-call XML - posted within hours, no key. Faster than the API for "a vote just happened." | |
| + | - **CBO cost estimates:** no API - scrape `cbo.gov/cost-estimates` by bill; macro projections via the `us-cbo` GitHub CSVs. | |
| + | - **Voteview.com** - bulk vote CSV + ideology scores (use scores internally for QA, not as user-facing labels). | |
| + | - **OpenStates v3** - 50-state + DC/PR coverage, standardized schema, JSON. **IN MVP SCOPE (decision: federal + state from day one).** Free tier is only 500 q/day, 10 req/sec - far too tight for production; **a paid tier is required** (Bronze 5k/day, Silver 50k/day; pricing not public - contact OpenStates). This is the one place the data stack stops being free. State ingest is also messier: freshness and schema completeness vary by state. Budget for: (a) the paid tier, (b) per-state polling cadence tuning, (c) a "coverage map" so the UI is honest about which states lag. | |
| ### Real-time / change detection | ||
| - | No government API offers webhooks - **everything is polling.** Build a change-detection layer: store last `updateDate` per bill, diff on each poll, emit an event on change → that event drives the feed + push notifications. Congress.gov website RSS (`congress.gov/rss/`) is a cheap pre-check before a full pull. | |
| + | No government API offers webhooks - **everything is polling.** Build a change-detection layer: store last `updateDate` per bill, diff on each poll, emit an event on change → that event drives the feed + push notifications. Congress.gov website RSS (`congress.gov/rss/`) is a cheap pre-check before a full pull. | |
| ### Biggest data-engineering risk | ||
| **Congress.gov reliability.** It had a multi-day silent outage in Aug 2025 (redirect loop, key signup broke, zero comms). It's the only live bill-status source. **Mitigation:** fall back to the community `unitedstates/congress` scraper (pulls GovInfo bulk XML, LoC-independent); design ingest to tolerate 24-72h of Congress.gov downtime without going stale. | ||
| @@ -109,10 +109,10 @@ No government API offers webhooks - **everything is polling.** Build a change- | ||
| ## 5. Product Surfaces | ||
| **MVP (federal only):** | ||
| - | - **The Feed** - reverse-chronological stream of official actions: "House passed HR ___ (vote 218-210)," "Senate cloture failed on S ___," "$2.1B contract awarded to ___ by DoD." Each card: plain-English what-it-does, status, dollar figure, **source link**. | |
| - | - **Bill page** - extraction-template summary, sponsor (name/party/state, factual), status timeline, CBO score if any, full-text link, vote breakdown (muted colors). | |
| - | - **Spending page** - agency outlays, notable awards, debt/deficit at a glance from Treasury, all with baselines and % context. | |
| - | - **Alerts** - push/email on: a bill changing status, a vote happening, spending over a threshold, or topics the user follows (by policy area or agency - **not** by party). | |
| + | - **The Feed** - reverse-chronological stream of official actions: "House passed HR ___ (vote 218-210)," "Senate cloture failed on S ___," "$2.1B contract awarded to ___ by DoD." Each card: plain-English what-it-does, status, dollar figure, **source link**. | |
| + | - **Bill page** - extraction-template summary, sponsor (name/party/state, factual), status timeline, CBO score if any, full-text link, vote breakdown (muted colors). | |
| + | - **Spending page** - agency outlays, notable awards, debt/deficit at a glance from Treasury, all with baselines and % context. | |
| + | - **Alerts** - push/email on: a bill changing status, a vote happening, spending over a threshold, or topics the user follows (by policy area or agency - **not** by party). | |
| - **"How we pick what to show"** + **"Who pays for this?"** transparency pages (core trust signals). | ||
| **v1:** topic follows, personalized digest ("your weekly government recap"), "find my representatives + their actual votes," saved bills, share-as-image (factual card, no social-comment import). | ||
| @@ -122,11 +122,11 @@ No government API offers webhooks - **everything is polling.** Build a change- | ||
| --- | ||
| - | ## 6. Brand Pack - adapted from the TreeTrace pattern | |
| + | ## 6. Brand Pack - adapted from the TreeTrace pattern | |
| - | We **inherit TreeTrace's system architecture** (dark-first, forensic/record aesthetic, rationed accent, monospace data, custody-stamp mark, multi-page functional site) but **deliberately diverge on the accent color** - because the brand color itself must be politically neutral. | |
| + | We **inherit TreeTrace's system architecture** (dark-first, forensic/record aesthetic, rationed accent, monospace data, custody-stamp mark, multi-page functional site) but **deliberately diverge on the accent color** - because the brand color itself must be politically neutral. | |
| - | **Key divergence: drop cobalt blue.** Blue reads Democrat, red reads Republican. The neutral, "official record / Capitol brass / parchment ledger" choice is **amber/brass on ink**. It evokes seals, the dome, and the public record - and belongs to neither side. | |
| + | **Key divergence: drop cobalt blue.** Blue reads Democrat, red reads Republican. The neutral, "official record / Capitol brass / parchment ledger" choice is **amber/brass on ink**. It evokes seals, the dome, and the public record - and belongs to neither side. | |
| ### Color tokens (proposed) | ||
| | Token | Hex | Role | | ||
| @@ -137,28 +137,28 @@ We **inherit TreeTrace's system architecture** (dark-first, forensic/record aest | ||
| | `--dk-paper` | `#F2EFE9` | Primary text (warm bone) | | ||
| | `--dk-ash` | `#9BA0A6` | Secondary / mono labels | | ||
| | `--dk-line` | `rgba(155,160,166,.18)` | Hairlines | | ||
| - | | **`--dk-signal`** | **`#E0A53B`** | **Brass/amber - PRIMARY accent (rationed)** | | |
| + | | **`--dk-signal`** | **`#E0A53B`** | **Brass/amber - PRIMARY accent (rationed)** | | |
| | `--dk-signal-dim` | `#B07F25` | Hover/pressed | | ||
| | `--dk-signal-tint` | `rgba(224,165,59,.08)` | Active-row wash | | ||
| - | | `--dk-yea` | `#7FA88C` (muted sage) | Vote "yea" / funded - *not vivid green* | | |
| - | | `--dk-nay` | `#B98A8A` (muted clay) | Vote "nay" / cut - *not vivid red* | | |
| + | | `--dk-yea` | `#7FA88C` (muted sage) | Vote "yea" / funded - *not vivid green* | | |
| + | | `--dk-nay` | `#B98A8A` (muted clay) | Vote "nay" / cut - *not vivid red* | | |
| | party tones | desaturated, equal weight | only when party is the factual datum | | ||
| *Alt accent if brass feels too warm: a neutral teal `#2BB3A3`. Avoid any dominant red or blue.* | ||
| ### Typography (same skeleton as TreeTrace) | ||
| - | - **Display:** Archivo 800-900, uppercase, tight tracking - for headlines/numbers. | |
| - | - **Data / labels / nav:** JetBrains Mono, uppercase, tracked - vote counts, dollar figures, bill numbers, timestamps. (Reinforces "this is a record, not an op-ed.") | |
| + | - **Display:** Archivo 800-900, uppercase, tight tracking - for headlines/numbers. | |
| + | - **Data / labels / nav:** JetBrains Mono, uppercase, tracked - vote counts, dollar figures, bill numbers, timestamps. (Reinforces "this is a record, not an op-ed.") | |
| - **Prose:** Fraunces for the rare lead sentence / plain-English summary. | ||
| ### Visual signature (adapted) | ||
| - | - **Mark:** a **custody-stamp/seal frame** like TreeTrace's `[TT]`, but civic - a registration frame around a monogram (`[D]` / docket tick) reading as an *official stamp on the record*, not a forensic crosshair. Flat strokes, no gradient, no eagle/flag clichés (those carry valence). | |
| - | - **Hero:** TreeTrace uses a live CLI terminal; here, a **live "register" / ticker** - official actions materializing in real time as stamped record-lines ("14:02 ET - H.R. ___ - FAILED - 209-214"). Same "real thing happening live" energy, civic skin. | |
| + | - **Mark:** a **custody-stamp/seal frame** like TreeTrace's `[TT]`, but civic - a registration frame around a monogram (`[D]` / docket tick) reading as an *official stamp on the record*, not a forensic crosshair. Flat strokes, no gradient, no eagle/flag clichés (those carry valence). | |
| + | - **Hero:** TreeTrace uses a live CLI terminal; here, a **live "register" / ticker** - official actions materializing in real time as stamped record-lines ("14:02 ET - H.R. ___ - FAILED - 209-214"). Same "real thing happening live" energy, civic skin. | |
| - **Rationed accent:** one brass signal per view (a live dot, an active filter, a "just happened" flag). Never large fills. Calm, sealed, authoritative. | ||
| - | - **Light mode:** parchment `#F6F4EF` bg, ink text - reads like an official document. | |
| + | - **Light mode:** parchment `#F6F4EF` bg, ink text - reads like an official document. | |
| ### Philosophy line | ||
| - | > *Record the action, not the argument - every entry sealed to its source.* | |
| + | > *Record the action, not the argument - every entry sealed to its source.* | |
| --- | ||
| @@ -166,7 +166,7 @@ We **inherit TreeTrace's system architecture** (dark-first, forensic/record aest | ||
| - **Frontend:** Next.js (multi-page functional site, matches TreeTrace pattern), Tailwind with the token set above, dark-first. Mobile-first PWA so push works without app-store friction at MVP. | ||
| - **Backend/ingest:** Python or Node workers polling the four APIs on cron; a change-detection table keyed by `updateDate`; Postgres for bills/actions/awards; a normalized "event" table that powers feed + alerts. | ||
| - | - **Summaries:** extraction-template pipeline. Can run **locally on the 3060 (Ollama)** for cost-free draft extraction, with a Polarization-Index QA gate before publish - but **prefer deterministic field extraction over generation** wherever possible. | |
| + | - **Summaries:** extraction-template pipeline. Can run **locally on the 3060 (Ollama)** for cost-free draft extraction, with a Polarization-Index QA gate before publish - but **prefer deterministic field extraction over generation** wherever possible. | |
| - **Push:** ntfy (already in the homelab) for MVP; web-push for PWA. | ||
| - **Hosting:** Cloudflare Pages for the static/edge front (same as zionboggan.com / treetrace.dev), workers/CT for the ingest service. Could live on a CT in the cluster. | ||
| - **Costs:** all primary data is free; main cost is compute for ingest + summary QA. Fits the "no paid services until first payout" rule. | ||
| @@ -175,23 +175,22 @@ We **inherit TreeTrace's system architecture** (dark-first, forensic/record aest | ||
| ## 8. Build Phases | ||
| - | - **Phase 0 - Brand + spike (this plan + a clickable shell):** lock name/colors/mark; stand up the token set + a static feed mockup with fake data. | |
| - | - **Phase 1 - Ingest spine:** Congress.gov + USAspending + Treasury pollers → events table → real feed (read-only, no auth). Source links on every card. | |
| - | - **Phase 2 - Summaries + bill pages:** extraction templates + Polarization-Index gate; full bill pages. | |
| - | - **Phase 3 - Alerts + accounts:** topic follows (policy area/agency), push via ntfy/web-push, weekly digest. | |
| - | - **Phase 4 - Trust + polish:** transparency pages, advisory board outreach, dual-side launch. | |
| - | - **Phase 1.5 - State spine:** OpenStates ingest (paid tier) + per-state coverage map, folded in alongside federal. | |
| + | - **Phase 0 - Brand + spike (this plan + a clickable shell):** lock name/colors/mark; stand up the token set + a static feed mockup with fake data. | |
| + | - **Phase 1 - Ingest spine:** Congress.gov + USAspending + Treasury pollers → events table → real feed (read-only, no auth). Source links on every card. | |
| + | - **Phase 2 - Summaries + bill pages:** extraction templates + Polarization-Index gate; full bill pages. | |
| + | - **Phase 3 - Alerts + accounts:** topic follows (policy area/agency), push via ntfy/web-push, weekly digest. | |
| + | - **Phase 4 - Trust + polish:** transparency pages, advisory board outreach, dual-side launch. | |
| + | - **Phase 1.5 - State spine:** OpenStates ingest (paid tier) + per-state coverage map, folded in alongside federal. | |
| - **Later:** reps lookup, historical trends, award drilldowns. | ||
| --- | ||
| - | ## 9. Decisions - RESOLVED (2026-06-26) | |
| + | ## 9. Decisions - RESOLVED (2026-06-26) | |
| 1. **Name:** ✅ **Docket** | ||
| 2. **Accent:** ✅ **Brass/amber `#E0A53B`** | ||
| 3. **Scope of MVP:** ✅ **Federal + state** (OpenStates in MVP; paid tier required) | ||
| 4. **Summaries:** ✅ **Structured fields first** (LLM prose later) | ||
| - | 5. **Domain style:** OPEN - `.org` reads most neutral/non-commercial for a civic-trust product; `docket.org` likely taken, so candidates like `docket.vote`, `getdocket.org`, `docket.gov`-adjacent, or `ondocket.org` need a availability check. *(Only remaining open item.)* | |
| + | 5. **Domain style:** OPEN - `.org` reads most neutral/non-commercial for a civic-trust product; `docket.org` likely taken, so candidates like `docket.vote`, `getdocket.org`, `docket.gov`-adjacent, or `ondocket.org` need a availability check. *(Only remaining open item.)* | |
| --- | ||
| - | *Research basis: 3 subagents - federal data-source map (Sonnet), competitive + neutrality playbook (Sonnet), TreeTrace brand-pack extraction (Haiku). All findings folded in above.* |
| @@ -1,28 +1,27 @@ | ||
| # Docket | ||
| - | **The unfiltered public record of what government does - every bill, law, amendment, and vote, including what gets slipped in - made legible to any citizen.** | |
| + | **The unfiltered public record of what government does - every bill, law, amendment, and vote, including what gets slipped in - made legible to any citizen.** | |
| - | Docket is a neutral, non-partisan transparency resource. It collects, aggregates, and makes searchable the *complete* official record of US legislative activity - federal and granular state-level - and surfaces the parts that are technically public but buried where no ordinary person can find them: amendments and riders inserted by either side, voice votes, committee markups, conference-report changes, procedural maneuvers. | |
| + | Docket is a neutral, non-partisan transparency resource. It collects, aggregates, and makes searchable the *complete* official record of US legislative activity - federal and granular state-level - and surfaces the parts that are technically public but buried where no ordinary person can find them: amendments and riders inserted by either side, voice votes, committee markups, conference-report changes, procedural maneuvers. | |
| It serves two needs at once: | ||
| - | 1. **Awareness** - a calm, neutral feed so you catch what's happening (*"this was happening and I had no idea"*). | |
| - | 2. **Lookup** - a resource where anyone, including an 80-year-old, can go dig up exactly what their representatives did on any bill, in any state, on a granular level - in plain English, with zero spin, every fact sourced to an official record. | |
| + | 1. **Awareness** - a calm, neutral feed so you catch what's happening (*"this was happening and I had no idea"*). | |
| + | 2. **Lookup** - a resource where anyone, including an 80-year-old, can go dig up exactly what their representatives did on any bill, in any state, on a granular level - in plain English, with zero spin, every fact sourced to an official record. | |
| Spending is one lens, not the point. The point is **full-spectrum, unfiltered visibility** into what government and its representatives actually do. | ||
| ## The one rule that defines everything | ||
| - | **Zero political lean.** If Docket reads as left- or right-leaning, it has failed. Neutrality is not a tone we aim for - it is enforced in code (see [`docs/NEUTRALITY.md`](docs/NEUTRALITY.md)). | |
| + | **Zero political lean.** If Docket reads as left- or right-leaning, it has failed. Neutrality is not a tone we aim for - it is enforced in code (see [`docs/NEUTRALITY.md`](docs/NEUTRALITY.md)). | |
| ## Status | ||
| Planning → build (started 2026-06-26). MVP = federal + state. | ||
| ## Repo map | ||
| - | - [`PLAN.md`](PLAN.md) - full product & brand plan (the source of truth for *what* and *why*) | |
| - | - [`docs/BRAND.md`](docs/BRAND.md) - design system, tokens, components (the *look*) | |
| - | - [`docs/NEUTRALITY.md`](docs/NEUTRALITY.md) - the enforceable neutrality doctrine (the *guardrails*) | |
| - | - [`docs/DATA_SOURCES.md`](docs/DATA_SOURCES.md) - API contracts for every data source | |
| - | - [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) - system design, services, data model | |
| - | - [`docs/ORCHESTRATION.md`](docs/ORCHESTRATION.md) - who builds what (Opus / GLM / Codex) and how | |
| + | - [`PLAN.md`](PLAN.md) - full product & brand plan (the source of truth for *what* and *why*) | |
| + | - [`docs/BRAND.md`](docs/BRAND.md) - design system, tokens, components (the *look*) | |
| + | - [`docs/NEUTRALITY.md`](docs/NEUTRALITY.md) - the enforceable neutrality doctrine (the *guardrails*) | |
| + | - [`docs/DATA_SOURCES.md`](docs/DATA_SOURCES.md) - API contracts for every data source | |
| + | - [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) - system design, services, data model | |
| ## Stack (target) | ||
| Next.js (PWA, dark-first) · Tailwind w/ Docket tokens · Postgres · Python/Node ingest workers · ntfy + web-push · Cloudflare front + homelab CT backend. |
| @@ -1,7 +1,6 @@ | ||
| - | # Docket - Security Audit Report | |
| + | # Docket - Security Audit Report | |
| **Date:** 2026-06-26 | ||
| - | **Auditor:** Adversarial review (Claude Code, authorized) | |
| - | **Scope:** `/shared/projects/docket` codebase + live instance CT247 `http://localhost:3000` | |
| + | **Scope:** `/shared/projects/docket` codebase + live instance `http://localhost:3000` | |
| **Architecture:** Next.js 14.2.15 (App Router) + better-sqlite3, read-only public data viewer, no auth, no writes | ||
| --- | ||
| @@ -19,22 +18,22 @@ | ||
| | Vector | Result | | ||
| |---|---| | ||
| - | | SQL injection (all query params) | Clean - every user value is a named bound parameter, zero interpolation of user input into SQL strings | | |
| - | | DB file accessible over HTTP | 404 - `/docket.db` and `/data/docket.db` both unreachable; DB is outside `public/` | | |
| - | | Stack traces / SQL in error responses | `{"error":"not found"}` only - no internal detail leaks | | |
| - | | `dangerouslySetInnerHTML` with user data | Single use in `layout.tsx:67-71` - hard-coded compile-time string, no user input | | |
| + | | SQL injection (all query params) | Clean - every user value is a named bound parameter, zero interpolation of user input into SQL strings | | |
| + | | DB file accessible over HTTP | 404 - `/docket.db` and `/data/docket.db` both unreachable; DB is outside `public/` | | |
| + | | Stack traces / SQL in error responses | `{"error":"not found"}` only - no internal detail leaks | | |
| + | | `dangerouslySetInnerHTML` with user data | Single use in `layout.tsx:67-71` - hard-coded compile-time string, no user input | | |
| | Path traversal in route params | `/api/bills/1/../../../etc/passwd` → 404; Next.js normalizes before routing | | ||
| | Open redirect | No user-controlled `redirect()` calls; all targets are hard-coded paths | | ||
| | SSRF | Zero `fetch()` calls with user-influenced URLs anywhere in the server code | | ||
| | Client bundle secrets / env exposure | `process.env.DOCKET_DB_PATH` is server-only; not referenced in client components | | ||
| - | | Source maps in static files | Only server-side `.map` files in `.next/server/` - confirmed 404 over HTTP | | |
| - | | `.env` / `server.log` reachable | Both return 404 - only `favicon.svg` is in `public/` | | |
| + | | Source maps in static files | Only server-side `.map` files in `.next/server/` - confirmed 404 over HTTP | | |
| + | | `.env` / `server.log` reachable | Both return 404 - only `favicon.svg` is in `public/` | | |
| --- | ||
| ## Findings | ||
| - | ### F1 - MEDIUM: allMemberVotes query has no DB-level LIMIT | |
| + | ### F1 - MEDIUM: allMemberVotes query has no DB-level LIMIT | |
| **File:** `lib/feed.ts:1039-1059` | ||
| **Endpoint:** `GET /api/members/:id` | ||
| @@ -60,11 +59,11 @@ The data is loaded entirely into JS memory for year-grouping before serializatio | ||
| **Exploitation:** `GET /api/members/95` (or any high-vote member) repeatedly. Each request serializes ~6MB of JSON. With rate limit bypassed (F2 below), this can exhaust Node.js heap. | ||
| - | **Fix:** Add `LIMIT 2000` (or a config constant) to the `allMemberVotes` query, or paginate `votesByYear` on the API. The `votes` field is already capped at 40 - apply the same discipline to `allMemberVotes`. | |
| + | **Fix:** Add `LIMIT 2000` (or a config constant) to the `allMemberVotes` query, or paginate `votesByYear` on the API. The `votes` field is already capped at 40 - apply the same discipline to `allMemberVotes`. | |
| --- | ||
| - | ### F2 - MEDIUM: Rate limit trivially bypassed via X-Forwarded-For spoofing | |
| + | ### F2 - MEDIUM: Rate limit trivially bypassed via X-Forwarded-For spoofing | |
| **File:** `middleware.ts:19-22` | ||
| @@ -80,7 +79,7 @@ Any client can send `X-Forwarded-For: 1.2.3.4` to key themselves to any arbitrar | ||
| **Confirmed:** `curl -H 'X-Forwarded-For: 1.2.3.4' http://localhost:3000/api/bills` → 200, bypasses limit. | ||
| - | The middleware comment says "a reverse proxy (Caddy/CF) should sit in front in production" - that is the correct fix. However, if CF is terminating, the middleware should either: | |
| + | The middleware comment says "a reverse proxy (Caddy/CF) should sit in front in production" - that is the correct fix. However, if CF is terminating, the middleware should either: | |
| 1. Trust only CF's `CF-Connecting-IP` header, or | ||
| 2. Disable the in-process limiter and rely entirely on CF's rate limiting. | ||
| @@ -90,17 +89,17 @@ In the current state (no CF in front), the rate limit offers zero protection. Co | ||
| --- | ||
| - | ### F3 - MEDIUM: Next.js 14.2.15 - unpatched CVEs | |
| + | ### F3 - MEDIUM: Next.js 14.2.15 - unpatched CVEs | |
| - | **File:** `package.json` - `"next": "14.2.15"` | |
| + | **File:** `package.json` - `"next": "14.2.15"` | |
| **Confirmed by:** `npm audit --omit=dev` on the live instance | ||
| The installed version has two vulnerabilities detected: | ||
| - | - **Critical (per npm):** Next.js packages a vulnerable version of `postcss` internally. The PostCSS XSS (GHSA-qx2v-qp2m-jg93) affects CSS stringification - this is a build-time dev tool risk, not a runtime risk for Docket (no user-uploaded CSS). | |
| + | - **Critical (per npm):** Next.js packages a vulnerable version of `postcss` internally. The PostCSS XSS (GHSA-qx2v-qp2m-jg93) affects CSS stringification - this is a build-time dev tool risk, not a runtime risk for Docket (no user-uploaded CSS). | |
| - **Moderate:** Same PostCSS issue directly. | ||
| - | Additionally, 14.2.15 predates the following patches (not all detected by the npm audit summary, but the version is in range): | |
| - | - **GHSA-f82v-jwr5-mffw** (middleware auth bypass, fixed in 14.2.25): **NOT exploitable** - Docket's middleware is rate-limiting only, no auth gate to bypass. | |
| + | Version 14.2.15 predates the following patches (not all detected by the npm audit summary, but the version is in range): | |
| + | - **GHSA-f82v-jwr5-mffw** (middleware auth bypass, fixed in 14.2.25): **NOT exploitable** - Docket's middleware is rate-limiting only, no auth gate to bypass. | |
| - **GHSA-h25m-26qc-wcjf / GHSA-q4gf-8mx6-v5v3** (RSC DoS): Potentially applicable since Docket uses server components; exact exploitability depends on whether the specific RSC deserialization path is triggered. | ||
| - **GHSA-vfv6-92ff-j949 / GHSA-wfc6-r584-vfw7** (RSC cache poisoning): Relevant if a CDN is placed in front without proper Vary/cache-key configuration. | ||
| @@ -108,7 +107,7 @@ Additionally, 14.2.15 predates the following patches (not all detected by the np | ||
| --- | ||
| - | ### F4 - LOW: CSP `unsafe-inline` for `script-src` and `style-src` neutralizes XSS protection | |
| + | ### F4 - LOW: CSP `unsafe-inline` for `script-src` and `style-src` neutralizes XSS protection | |
| **File:** `next.config.js:12-14` | ||
| @@ -121,16 +120,16 @@ Additionally, 14.2.15 predates the following patches (not all detected by the np | ||
| The comment acknowledges this: "Next.js injects a small inline hydration bootstrap; 'unsafe-inline' kept for scripts/styles (no nonce pipeline)." This is a real constraint with Next.js 14 App Router; the fix is non-trivial. | ||
| - | **Current risk:** Low - no XSS injection points found. React's JSX escaping is the real defense. The CSP is essentially decorative for XSS prevention. | |
| + | **Current risk:** Low - no XSS injection points found. React's JSX escaping is the real defense. The CSP is essentially decorative for XSS prevention. | |
| **Fix options (pick one):** | ||
| - | 1. Implement a [nonce-based CSP](https://nextjs.org/docs/app/building-your-application/configuring/content-security-policy#nonces) - requires middleware nonce injection and `<Script nonce={nonce}>` usage. | |
| + | 1. Implement a [nonce-based CSP](https://nextjs.org/docs/app/building-your-application/configuring/content-security-policy#nonces) - requires middleware nonce injection and `<Script nonce={nonce}>` usage. | |
| 2. Replace `unsafe-inline` with a `sha256-{hash}` allowlist for the one known inline script (the display-mode snippet in `layout.tsx`). | ||
| - | 3. Accept as a known limitation and document it - the app has no user-submitted content, so the XSS risk is tied only to DB compromise. | |
| + | 3. Accept as a known limitation and document it - the app has no user-submitted content, so the XSS risk is tied only to DB compromise. | |
| --- | ||
| - | ### F5 - LOW: SourceLink renders `href` without `javascript:` URL sanitization | |
| + | ### F5 - LOW: SourceLink renders `href` without `javascript:` URL sanitization | |
| **File:** `components/source-link.tsx:22-23` | ||
| @@ -140,7 +139,7 @@ The comment acknowledges this: "Next.js injects a small inline hydration bootstr | ||
| The `url` parameter comes from DB `source_url` fields. If any `source_url` row contained `javascript:alert(1)`, clicking the link would execute JS in some browsers (React 16.9+ emits a warning but does not block `javascript:` hrefs on `<a>` tags at runtime). | ||
| - | `isAllowedSourceHost()` flags non-allowlisted hosts visually and in title text, but still renders the `href`. A `javascript:` URL would pass `isAllowedSourceHost` as `false` and show a "not allowlisted" state - but the link is still clickable. | |
| + | `isAllowedSourceHost()` flags non-allowlisted hosts visually and in title text, but still renders the `href`. A `javascript:` URL would pass `isAllowedSourceHost` as `false` and show a "not allowlisted" state - but the link is still clickable. | |
| **Exploit path:** Requires the ingest pipeline to write a malicious `source_url` to the DB. This is not a user-input path. | ||
| @@ -152,7 +151,7 @@ const safe = /^https?:\/\//i.test(url) ? url : "#"; | ||
| --- | ||
| - | ### F6 - LOW: getFeedItems, getBills, getMembers, getAwards fetch entire tables into memory | |
| + | ### F6 - LOW: getFeedItems, getBills, getMembers, getAwards fetch entire tables into memory | |
| **Files:** `lib/feed.ts:339-376, 675-732, 909-939, 471-492` | ||
| @@ -172,7 +171,7 @@ Current data volumes are manageable. As the ingest grows (esp. USASpending award | ||
| --- | ||
| - | ### F7 - INFO: HSTS preload with 2-year max-age | |
| + | ### F7 - INFO: HSTS preload with 2-year max-age | |
| **File:** `next.config.js:25` | ||
| @@ -182,15 +181,15 @@ Strict-Transport-Security: max-age=63072000; includeSubDomains; preload | ||
| `preload` submits the domain to Chrome's preload list. This is appropriate for a production HTTPS site but is effectively irreversible once submitted (removal takes months). Ensure the domain will stay HTTPS before public launch. | ||
| - | Not a vulnerability - noted for operational awareness. | |
| + | Not a vulnerability - noted for operational awareness. | |
| --- | ||
| - | ### F8 - INFO: `docket.db` and `docket.db-journal` exist in the project share at `/shared/projects/docket/` | |
| + | ### F8 - INFO: `docket.db` and `docket.db-journal` exist in the project share at `/shared/projects/docket/` | |
| **Observed:** `ls /shared/projects/docket/` (CT215) shows `docket.db` and `docket.db-journal` alongside the source code. | ||
| - | These are not in the CT247 `/opt/docket/public/` directory and are not web-accessible. However, the share mounts suggest these files may be accessible to other containers that mount `/shared/projects`. Confirm the `.db` files in the share are not a copy of production data - or if they are, ensure the share ACLs restrict access appropriately. | |
| + | These are not in the CT247 `/opt/docket/public/` directory and are not web-accessible. However, the share mounts suggest these files may be accessible to other containers that mount `/shared/projects`. Confirm the `.db` files in the share are not a copy of production data - or if they are, ensure the share ACLs restrict access appropriately. | |
| --- | ||
| @@ -198,15 +197,15 @@ These are not in the CT247 `/opt/docket/public/` directory and are not web-acces | ||
| | Probe | Result | | ||
| |---|---| | ||
| - | | `GET /api/bills/99999999` | `{"error":"not found"}` 404 - clean | | |
| - | | `GET /api/search?q=' OR '1'='1` | Empty results, no SQL error - bound param confirmed | | |
| - | | `GET /api/bills?limit=99999&offset=-1` | 200, limit clamped to 200, offset to 0 - parsePaging works | | |
| + | | `GET /api/bills/99999999` | `{"error":"not found"}` 404 - clean | | |
| + | | `GET /api/search?q=' OR '1'='1` | Empty results, no SQL error - bound param confirmed | | |
| + | | `GET /api/bills?limit=99999&offset=-1` | 200, limit clamped to 200, offset to 0 - parsePaging works | | |
| | `GET /api/members/1` | 2.96 MB response (9,895 votes, no LIMIT) | | ||
| - | | `GET /data/docket.db` | 404 - DB not web-accessible | | |
| - | | `GET /_next/static/chunks/webpack-*.js.map` | 404 - source maps not served | | |
| + | | `GET /data/docket.db` | 404 - DB not web-accessible | | |
| + | | `GET /_next/static/chunks/webpack-*.js.map` | 404 - source maps not served | | |
| | `GET /docket.db`, `GET /.env`, `GET /server.log` | All 404 | | ||
| - | | `GET /api/bills/1/../../../etc/passwd` | 404 - path normalized by Next.js | | |
| - | | `X-Forwarded-For: 1.2.3.4` to /api/bills | 200 - rate limit bypassed (F2 confirmed) | | |
| + | | `GET /api/bills/1/../../../etc/passwd` | 404 - path normalized by Next.js | | |
| + | | `X-Forwarded-For: 1.2.3.4` to /api/bills | 200 - rate limit bypassed (F2 confirmed) | | |
| --- | ||
| @@ -214,7 +213,7 @@ These are not in the CT247 `/opt/docket/public/` directory and are not web-acces | ||
| ``` | ||
| 2 vulnerabilities (1 moderate, 1 critical) | ||
| - | - critical: next depends on vulnerable postcss (PostCSS CSS stringify XSS - build-time only) | |
| + | - critical: next depends on vulnerable postcss (PostCSS CSS stringify XSS - build-time only) | |
| - moderate: postcss <8.5.10 directly | ||
| Fix: npm audit fix --force (upgrades to next@16, breaking) or manual next patch upgrade | ||
| ``` | ||
| @@ -225,24 +224,24 @@ Fix: npm audit fix --force (upgrades to next@16, breaking) or manual next patch | ||
| | # | Severity | ID | Title | Exploitable Without Preconditions | Must-Fix Before Public | | ||
| |---|---|---|---|---|---| | ||
| - | | 1 | Medium | F1 | allMemberVotes no LIMIT - 6.8 MB per request for prolific senators | Yes | Yes | | |
| + | | 1 | Medium | F1 | allMemberVotes no LIMIT - 6.8 MB per request for prolific senators | Yes | Yes | | |
| | 2 | Medium | F2 | Rate limit bypassed via X-Forwarded-For spoofing | Yes (trivial) | Yes (or replace with CDN-side rate limiting) | | ||
| | 3 | Medium | F3 | Next.js 14.2.15 unpatched CVEs (middleware bypass N/A, RSC DoS potentially applicable) | Conditional | Recommended | | ||
| - | | 4 | Low | F4 | CSP `unsafe-inline` - XSS protection via CSP is nil | No (no XSS found) | No | | |
| + | | 4 | Low | F4 | CSP `unsafe-inline` - XSS protection via CSP is nil | No (no XSS found) | No | | |
| | 5 | Low | F5 | SourceLink renders `href` without `javascript:` guard | Requires DB compromise | No | | ||
| - | | 6 | Low | F6 | Full-table in-memory fetches - DoS as data grows | No (not today) | No (monitor) | | |
| - | | 7 | Info | F7 | HSTS preload - operational concern | N/A | N/A | | |
| + | | 6 | Low | F6 | Full-table in-memory fetches - DoS as data grows | No (not today) | No (monitor) | | |
| + | | 7 | Info | F7 | HSTS preload - operational concern | N/A | N/A | | |
| | 8 | Info | F8 | docket.db in project share | N/A | Verify share ACLs | | ||
| --- | ||
| ## Must-Fix Before Public Launch | ||
| - | 1. **F1 - Add LIMIT to `allMemberVotes` query** (`lib/feed.ts:1039`): Cap at 2000 rows or paginate. One-line fix; prevents multi-MB responses on busy senators. | |
| + | 1. **F1 - Add LIMIT to `allMemberVotes` query** (`lib/feed.ts:1039`): Cap at 2000 rows or paginate. One-line fix; prevents multi-MB responses on busy senators. | |
| - | 2. **F2 - Fix rate limit IP extraction**: Either (a) use `CF-Connecting-IP` when behind Cloudflare and ignore `X-Forwarded-For`, or (b) remove the in-process rate limiter entirely and rely solely on CF/Caddy rules. Keeping the current middleware creates a false sense of protection. | |
| + | 2. **F2 - Fix rate limit IP extraction**: Either (a) use `CF-Connecting-IP` when behind Cloudflare and ignore `X-Forwarded-For`, or (b) remove the in-process rate limiter entirely and rely solely on CF/Caddy rules. Keeping the current middleware creates a false sense of protection. | |
| - | 3. **F3 - Upgrade Next.js**: `npm install next@14.2.28` (or latest 14.2.x) closes all known CVEs without a breaking change. Run `npm audit` to verify. | |
| + | 3. **F3 - Upgrade Next.js**: `npm install next@14.2.28` (or latest 14.2.x) closes all known CVEs without a breaking change. Run `npm audit` to verify. | |
| --- | ||
| @@ -4,44 +4,44 @@ Date: 2026-06-26 | ||
| ## What was broken | ||
| - | ### Primary bug - no `__main__` block in either poller | |
| + | ### Primary bug - no `__main__` block in either poller | |
| `python -m ingest.pollers.legiscan --once` and `python -m ingest.pollers.openstates --once` both produced zero output and zero rows because neither module had a `__main__` block. Python imports the module successfully but executes nothing. The `poll()` functions existed but were never called. | ||
| - | ### Secondary bug - legiscan poll() strategy was inefficient and produced zero rows | |
| + | ### Secondary bug - legiscan poll() strategy was inefficient and produced zero rows | |
| - | The original `poll()` called `getMasterListRaw` to get change hashes, then called `getBill` for each changed bill (capped at 50 per state). Inside `_store_bill()`, the code checked `state_link` against `is_allowlisted()` and skipped bills where `state_link` was empty or not in the allowlist. `getMasterList` does not include `state_link` in the compact masterlist payload - only the legiscan.com `url` field is present - so the allowlist check rejected every bill fetched. | |
| + | The original `poll()` called `getMasterListRaw` to get change hashes, then called `getBill` for each changed bill (capped at 50 per state). Inside `_store_bill()`, the code checked `state_link` against `is_allowlisted()` and skipped bills where `state_link` was empty or not in the allowlist. `getMasterList` does not include `state_link` in the compact masterlist payload - only the legiscan.com `url` field is present - so the allowlist check rejected every bill fetched. | |
| - | ### Tertiary bug - state_coverage wrote "missing" rows for openstates (never run) | |
| + | ### Tertiary bug - state_coverage wrote "missing" rows for openstates (never run) | |
| `state_coverage.py`'s `compute()` iterated over all sources in `SOURCES = ["openstates", "legiscan"]` regardless of whether either source had ever been run. This wrote "missing" rows for openstates across all 52 states. The frontend `rollupConfidence()` uses worst-case logic: if any confidence value is "missing", the state shows as missing. Result: all 52 states showed as missing even after legiscan ingested data for 51 of them. | ||
| - | ### Also fixed - jurisdiction column not in schema | |
| + | ### Also fixed - jurisdiction column not in schema | |
| The `bills` table schema in `db.py` did not include `jurisdiction TEXT`. The pollers add it via `ALTER TABLE` but that fails silently when the DB is locked (another process holds a write lock). Added `jurisdiction TEXT` to `init_schema()` so new databases always have it. | ||
| ## Fixes applied | ||
| - | ### 1. `services/ingest/ingest/pollers/legiscan.py` - rewritten | |
| + | ### 1. `services/ingest/ingest/pollers/legiscan.py` - rewritten | |
| - Added `main()` + `__main__` so `python -m ingest.pollers.legiscan --once` actually runs | ||
| - Replaced getMasterListRaw + getBill-per-bill with `getMasterList` per session: | ||
| - | - `getMasterList` returns all bills in a session in one API call including title, number, status, last_action, url, change_hash | |
| - | - No per-bill `getBill` calls needed for basic state coverage | |
| - | - Total API calls: ~2 per state (getSessionList + getMasterList) = ~104 for all 52 jurisdictions | |
| + | - `getMasterList` returns all bills in a session in one API call including title, number, status, last_action, url, change_hash | |
| + | - No per-bill `getBill` calls needed for basic state coverage | |
| + | - Total API calls: ~2 per state (getSessionList + getMasterList) = ~104 for all 52 jurisdictions | |
| - `source_url`: uses `state_link` when present and allowlisted; falls back to the legiscan.com `url` field (acceptable per INGESTION_STRATEGY.md when no official state link is in the masterlist payload) | ||
| - `_STATUS_MAP`: maps LegiScan numeric status codes (0-12) to human-readable strings | ||
| - Default state list changed to `MVP_STATES` (32 states); `--all` flag covers all 52 | ||
| - Idempotent via `change_hash` stored in the `seen` table | ||
| - | ### 2. `services/ingest/ingest/pollers/openstates.py` - added CLI | |
| + | ### 2. `services/ingest/ingest/pollers/openstates.py` - added CLI | |
| - Added `import argparse`, `import sys` | ||
| - Added `main()` + `__main__` so `python -m ingest.pollers.openstates --once` runs | ||
| - `--days-back N` flag (default 7); `--all` for all 52 jurisdictions; `--db` override | ||
| - No logic changes; poll() was already correct | ||
| - | ### 3. `services/ingest/ingest/state_coverage.py` - skip sources with no bill data | |
| + | ### 3. `services/ingest/ingest/state_coverage.py` - skip sources with no bill data | |
| - `compute()` now checks which sources have at least one row in the `bills` table | ||
| - Only writes `state_coverage` rows for sources with actual bill data (not just scrape attempts) | ||
| @@ -49,7 +49,7 @@ The `bills` table schema in `db.py` did not include `jurisdiction TEXT`. The pol | ||
| - If no source has any bill data, falls back to the full SOURCES list so the UI shows honest full-missing rather than empty coverage | ||
| - Stale openstates "missing" rows deleted from existing DB directly | ||
| - | ### 4. `services/ingest/ingest/db.py` - jurisdiction column | |
| + | ### 4. `services/ingest/ingest/db.py` - jurisdiction column | |
| - Added `jurisdiction TEXT` to the `bills` table `CREATE TABLE IF NOT EXISTS` statement | ||
| - No data migration needed for existing rows (column defaults to NULL) | ||
| @@ -61,7 +61,7 @@ Run on 2026-06-26 against `DOCKET_DB_PATH=/opt/docket/data/docket.db`: | ||
| | Metric | Value | | ||
| |--------|-------| | ||
| | States ingested | 51 (all 50 states + DC) | | ||
| - | | PR status | Missing - LegiScan returns ERROR for PR | | |
| + | | PR status | Missing - LegiScan returns ERROR for PR | | |
| | Total new state bills stored | 191,233 | | ||
| | Total bills in DB (all sources) | 208,382 | | ||
| | API calls used | 103 | | ||
| @@ -83,16 +83,16 @@ Top 5 states by bill count: | ||
| ## Real bill examples | ||
| - | **CA AB1** - Residential property insurance: wildfire risk. | Passed | |
| + | **CA AB1** - Residential property insurance: wildfire risk. | Passed | |
| `https://legiscan.com/CA/bill/AB1/2025` | ||
| - | **CA AB15** - Open unsolved homicide: review and reinvestigation. | Vetoed | |
| + | **CA AB15** - Open unsolved homicide: review and reinvestigation. | Vetoed | |
| `https://legiscan.com/CA/bill/AB15/2025` | ||
| - | **OH HB359** - Regards emergency Joshua Alert system for certain missing kids | Passed | |
| + | **OH HB359** - Regards emergency Joshua Alert system for certain missing kids | Passed | |
| `https://legiscan.com/OH/bill/HB359/2025` | ||
| - | **OH SB223** - Require ODNR discount program for veterans, service members | Passed | |
| + | **OH SB223** - Require ODNR discount program for veterans, service members | Passed | |
| `https://legiscan.com/OH/bill/SB223/2025` | ||
| Note: all `source_url` values are legiscan.com bill pages. `getMasterList` does not | ||
| @@ -105,19 +105,19 @@ future targeted pass within the monthly API budget. | ||
| DNS resolution for `v3.openstates.org` fails from CT247 (network configuration issue, | ||
| not a code bug). The poller handles this gracefully: logs the error per jurisdiction and | ||
| returns 0 bills with exit code 0. Once DNS is resolved on CT247, the poller will work | ||
| - | as written. The 500 req/day quota should not be blown - the existing MVP_JURISDICTIONS | |
| + | as written. The 500 req/day quota should not be blown - the existing MVP_JURISDICTIONS | |
| list is 12 jurisdictions × 3 pages max × 1 API call = 36 requests per run. | ||
| ## Notes for overseer | ||
| - | - Do not edit `run.py` (as instructed) - it already references `ingest.pollers.legiscan` | |
| + | - Do not edit `run.py` (as instructed) - it already references `ingest.pollers.legiscan` | |
| and `ingest.pollers.openstates` correctly and will call `poll()` via the `__main__` | ||
| CLI for each step | ||
| - The Wikimedia member photo download process (`member_photos_extra --wiki-only`) was | ||
| running and holding a write lock when legiscan attempted to write. Killing it was | ||
| required to unblock ingest. Consider adding a `--busy-timeout` config or serializing | ||
| the photo downloads as a lower-priority background job to avoid lock contention | ||
| - | - PR data: LegiScan returns ERROR for Puerto Rico getSessionList - no data available | |
| + | - PR data: LegiScan returns ERROR for Puerto Rico getSessionList - no data available | |
| from this source. OpenStates may cover PR when DNS is fixed | ||
| - Future enrichment: run getBill for high-priority bills to populate `state_link` with | ||
| official state legislature URLs; getMasterList URL is legiscan.com (acceptable |
| @@ -1,5 +1,5 @@ | ||
| VOTE_BILL_LINK_NOTES.md | ||
| - | Docket - Vote-to-Bill Linkage Fix (2026-06-27) | |
| + | Docket - Vote-to-Bill Linkage Fix (2026-06-27) | |
| PROBLEM | ||
| ------- | ||
| @@ -78,20 +78,20 @@ BREAKDOWN OF UNLINKED | ||
| 1. vote_id=74 | ||
| question: "On Retaining Division A" | ||
| ref: "H R 6938" | ||
| - | bill: H.R. 6938 - Commerce, Justice, Science; Energy and Water | |
| + | bill: H.R. 6938 - Commerce, Justice, Science; Energy and Water | |
| Development; and Interior and Environment Appropriations Act, 2026 | ||
| 2. vote_id=71 | ||
| question: "On Ordering the Previous Question" | ||
| ref: "H RES 977" | ||
| - | bill: H.Res. 977 - Providing for consideration of H.R. 4593 (showerhead | |
| + | bill: H.Res. 977 - Providing for consideration of H.R. 4593 (showerhead | |
| energy standards), H.R. 5184 (manufactured housing standards), | ||
| and H.R. 6938 (appropriations) | ||
| 3. vote_id=73 | ||
| question: "On Motion to Discharge" | ||
| ref: "H RES 780" | ||
| - | bill: H.Res. 780 - Providing for consideration of H.R. 1834 (Break | |
| + | bill: H.Res. 780 - Providing for consideration of H.R. 1834 (Break | |
| the Gridlock Act) | ||
| RUN.PY WIRING | ||
| @@ -150,10 +150,10 @@ The frontend agent should JOIN on votes.bill_id to display bill information: | ||
| WHERE ... | ||
| For votes where bill_id IS NULL: | ||
| - | - If bill_text_ref LIKE 'PN%' -> label as "Nomination" | |
| - | - If bill_text_ref LIKE 'TREATYDOC%' -> label as "Treaty Document" | |
| - | - If bill_text_ref IS NOT NULL -> show raw ref + "(bill not in DB)" | |
| - | - If bill_text_ref IS NULL -> label as "Procedural / No bill" | |
| + | - If bill_text_ref LIKE 'PN%' -> label as "Nomination" | |
| + | - If bill_text_ref LIKE 'TREATYDOC%' -> label as "Treaty Document" | |
| + | - If bill_text_ref IS NOT NULL -> show raw ref + "(bill not in DB)" | |
| + | - If bill_text_ref IS NULL -> label as "Procedural / No bill" | |
| The question field often contains a human-readable description ("On Passage", | ||
| "On Motion to Recommit") that can complement the bill title in the UI. |
| @@ -1,15 +0,0 @@ | ||
| - | You are GLM-5.2, frontend lead for Docket (neutral US government transparency app). Work in /opt/docket (local build dir - npm builds work here). Be RAG-first; read ONLY these specs: docs/API.md, docs/ACCESSIBILITY.md, docs/BRAND.md, docs/NEUTRALITY.md, docs/ARCHITECTURE.md. Look at the existing app/ and components/ you already built. | |
| - | ||
| - | WAVE F1 - make it REAL + ACCESSIBLE: | |
| - | ||
| - | 1. READ-API: implement Next.js route handlers under app/api/ EXACTLY per docs/API.md (feed, bills, bills/[id], bills/[id]/diff, votes, spending/awards, spending/fiscal, spending/apportionments, executive, lobbying, states, members/[id], search). Read the SQLite DB at process.env.DOCKET_DB_PATH (default /opt/docket/data/docket.db) using better-sqlite3 (npm install better-sqlite3). BE DEFENSIVE: some tables do not exist yet (other agents are still building ingest) - wrap every query in try/catch and return {items:[],total:0} instead of crashing. Every returned item MUST include sourceUrl + href. | |
| - | ||
| - | 2. WIRE REAL DATA: replace the MOCK data on /register and /spending with fetches to your new /api routes. The DB ALREADY has real USAspending awards and real Senate votes - show them. Keep the live Register hero but feed it from /api/feed (fall back to a calm empty state if a category is empty). | |
| - | ||
| - | 3. ACCESSIBILITY (docs/ACCESSIBILITY.md): add the 3-mode display toggle - Dark (default brand) / Readable-Light / Plain (high-contrast, system font, >=20px) - persisted in localStorage, always-visible labeled control >=44x44px. Enforce contrast >=4.5:1, body >=18px dark / >=16px light, visible focus outlines, underlined links, NO hover-only interactions. | |
| - | ||
| - | Keep the BRAND (docs/BRAND.md: brass rationed, custody-stamp, mono data) and the NEUTRALITY gate (no adjectives, a Source link on every fact, no comment sections, muted vote colors never vivid red/blue). | |
| - | ||
| - | BUILD + VERIFY (do not skip): npm install; npm run build MUST pass. Then restart the server: kill whatever is on port 3000 (fuser -k 3000/tcp), then `DOCKET_DB_PATH=/opt/docket/data/docket.db nohup ./node_modules/.bin/next start -H 0.0.0.0 -p 3000 &`. curl http://localhost:3000/api/feed and http://localhost:3000/register to confirm REAL data renders (not mock). | |
| - | ||
| - | Write FRONTEND_NOTES.md (what you built, the API routes, how to run). Definition of done: clean build, /register shows REAL awards + votes from the API, all /api routes respond (even if empty), 3-mode toggle works and persists. |
| @@ -1,23 +0,0 @@ | ||
| - | You are GLM-5.2, frontend lead for Docket (neutral US gov transparency app). Work in /opt/docket (local build dir). The read-API is LIVE and returns real data - all /api routes work (curl http://localhost:3000/api/feed?limit=2 to see). Be RAG-first; read ONLY: docs/API.md, docs/BRAND.md, docs/NEUTRALITY.md, docs/ACCESSIBILITY.md. Look at your existing app/ pages + components + lib/. | |
| - | ||
| - | WAVE F2 - build the DEEP, GRANULAR surfaces (the heart of the product). The DB now has: 290 bills, 410 bill_versions, 34 bill_diffs (with rider flags), 99 amendments, 90 votes, 25,572 vote_positions, 537 members, 500 executive_actions, 20 apportionments, 188 procedural_events, 320 awards. Make all of it explorable. Build/enhance these pages, each consuming the real /api routes: | |
| - | ||
| - | 1. VOTES - /votes (browse recorded roll-call votes) and /votes/[id] (THE ROSTER: show EVERY member's position - Yea/Nay/Present/Not Voting - with name + party + state + district. Use muted party/vote colors per BRAND, NEVER vivid red/blue. Default sort by vote then alphabetical, not grouped by party. For voice/UC votes where recorded=false, show "No individual record exists - passed by voice vote" instead of a fake roster.) Source link on every vote. | |
| - | ||
| - | 2. MEMBERS - /members (browseable roster, search/filter by state/chamber/party) and /members/[id] (a representative's FULL record: their recent votes with how they voted, bills sponsored, link each to the bill/vote). This is "what did MY rep do." | |
| - | ||
| - | 3. BILL DETAIL - enhance /bill/[id] with: (a) the "WHAT CHANGED / WHAT GOT SLIPPED IN" view consuming /api/bills/[id]/diff - show added/removed/amended sections with the snippet, and flag rider-flagged sections prominently (this is our signature feature); (b) the amendments list; (c) the bill's votes (link to the /votes/[id] roster); (d) full status timeline; (e) sponsor + cosponsors (link to /members/[id]). Honesty notes where data is voice-vote/conference (no individual record). | |
| - | ||
| - | 4. SPENDING - enhance /spending with the OMB APPORTIONMENT IMPOUND TRACKER from /api/spending/apportionments: per agency/account show appropriated vs apportioned vs status (released/delayed/withheld). Plus the awards list (already wired) and fiscal stats. | |
| - | ||
| - | 5. EXECUTIVE - /executive: Federal Register actions (Executive Orders, proposed/final rules, notices) from /api/executive, filterable by type. | |
| - | ||
| - | 6. STATES - enhance /states with the per-state confidence map (ok/stale/missing badges) from /api/states, and /states/[code] detail. | |
| - | ||
| - | 7. SEARCH - a global search UI consuming /api/search (the "look up my state / rep / topic" entry point). Make it prominent and simple (per ACCESSIBILITY.md: labeled, big target, drop-downs not bare box). | |
| - | ||
| - | HARD RULES: keep the BRAND (brass rationed, custody-stamp, mono data, dark default) and the 3-mode accessibility toggle working on ALL new pages (Dark/Readable/Plain). Enforce NEUTRALITY: a Source link on every fact, no adjectives, no comment sections, muted vote/party colors, honest "no record exists" where true. All new pages must work in all 3 display modes. | |
| - | ||
| - | BUILD: run `npm run build` and ensure it PASSES (fix any type errors). Do NOT try to restart the server (the tooling for that is handled separately - fuser is absent on this box). Just make sure the build is clean. | |
| - | ||
| - | Update FRONTEND_NOTES.md with the new pages. Definition of done: npm run build passes; all 7 surfaces implemented consuming real /api data; brand + neutrality + 3-mode accessibility intact. |
| @@ -1,28 +0,0 @@ | ||
| - | You are GLM-5.2, frontend lead for Docket (neutral US gov transparency app). Work in /opt/docket. The read-API + deep pages you built (F1/F2) are LIVE. New DATA layers now exist in the DB but have NO UI yet - build their API routes + UI in this wave. Be RAG-first; read ONLY: docs/API.md, docs/BRAND.md, docs/NEUTRALITY.md, docs/ACCESSIBILITY.md. Reuse your existing lib/db.ts + lib/feed.ts patterns + components. | |
| - | ||
| - | New DB tables/columns (verify with a quick sqlite read at /opt/docket/data/docket.db): | |
| - | - calendar_events (id, date, kind[effective|enacted|vote|introduced|committee|floor|apportionment|effective_parsed], ref_type, ref_id, title, jurisdiction, source_url, confidence) - 39k rows | |
| - | - categories (slug,label,kind) - 32 rows; bills.policy_area; bill_subjects(bill_id,subject); awards.category | |
| - | - votes.year + vote_positions (971k rows) - for year-by-year member voting | |
| - | - change_events (id, ts, source, ref_type, ref_id, change[added|updated|removed], summary, source_url) - the watcher feed | |
| - | ||
| - | WAVE F3 - build these. Add API routes AND UI: | |
| - | ||
| - | 1. CALENDAR (the priority - a visual for "what goes into effect when", for non-technical users of any age): | |
| - | - /api/calendar?from=&to=&kind= → events in range from calendar_events. | |
| - | - /calendar page: a real MONTH-GRID calendar. Each day shows its events (dot/badge per kind, e.g. effective/enacted/vote). Click a day → panel listing that day's events with title + kind + Source link. Month nav (prev/next/today). Make it BIG, clean, obvious - an 80-year-old should get it instantly. A small legend explains the kinds in plain words ("Takes effect", "Became law", "Vote"). Works in all 3 display modes. | |
| - | ||
| - | 2. CATEGORIES: | |
| - | - /api/categories → list with item counts (join bills.policy_area + awards.category). | |
| - | - A category browse UI (chips/list with counts) on /register and/or a /categories page; clicking a category filters the feed/bills. Show policy_area + top subjects on /bill/[id]. | |
| - | ||
| - | 3. MEMBER YEAR-INDEX (the "year by year how they voted" view): | |
| - | - On /members/[id], group that member's votes BY YEAR - collapsible year sections (e.g. 2026, 2025, 2024...), each showing the votes that year with date + question + their position (muted colors). Use the votes.year index. Show a per-year count summary. This is the headline upgrade to member pages. | |
| - | ||
| - | 4. WHAT-CHANGED FEED: | |
| - | - /api/changes → recent change_events. | |
| - | - A compact "Recently changed" surface (added/updated/removed) on the home or register page - small, factual, neutral, Source link each. | |
| - | ||
| - | HARD RULES: keep BRAND (brass rationed, custody-stamp, mono, dark default), NEUTRALITY (Source link on every fact, no adjectives, muted colors, no comment sections, honest "no record" where true), and the 3-mode accessibility toggle (Dark/Readable/Plain) working on ALL new pages - the calendar especially must be legible in Plain mode. Add the new pages to the nav. | |
| - | ||
| - | BUILD: `npm run build` MUST pass (fix type errors). Do NOT restart the server (handled separately). Update FRONTEND_NOTES.md. Definition of done: build passes; /calendar, category filter, member year-index, and the changed-feed all implemented against real /api data; brand + neutrality + a11y intact. |
| @@ -1,22 +0,0 @@ | ||
| - | You are GLM-5.2, frontend lead for Docket (neutral US gov transparency app, PUBLIC-FACING). Work in /opt/docket. Be RAG-first; read ONLY: docs/API.md, docs/BRAND.md, docs/NEUTRALITY.md, docs/ACCESSIBILITY.md. Reuse your lib/db.ts + lib/feed.ts + components. | |
| - | ||
| - | New DB data now available (verify via sqlite read of /opt/docket/data/docket.db): | |
| - | - members.photo_url - official headshots (1,179 members have one; rest NULL). Total members now 12,767 (current + ALL historical). | |
| - | - meetings table (id, chamber, committee, date, time, datetime_iso, title, subject, type[hearing|markup|business_meeting|floor], location, status, source_url) - 166 rows, congressional committee + floor meetings. | |
| - | ||
| - | WAVE F4 - faces + schedule + fill the gaps: | |
| - | ||
| - | 1. PHOTOS EVERYWHERE: surface members.photo_url (expose it in the member API + vote/roster queries in lib). Show the headshot: | |
| - | - /members roster: avatar thumbnail per row. | |
| - | - /members/[id]: a profile header with the photo (and their party/state/chamber). | |
| - | - /votes/[id] roster: a small face next to each member's name + position. | |
| - | - Anywhere a member name appears, link it to /members/[id]. | |
| - | - When photo_url is NULL: a clean INITIALS placeholder (not a broken image). Lazy-load images (loading="lazy") since rosters have ~400-500. Keep it fast. | |
| - | ||
| - | 2. /schedule PAGE (new - the meetings tracker): build /api/meetings (or /api/schedule) reading the meetings table (?from=&to=&chamber=&committee=&type=). Then a /schedule page that shows congressional meetings in an intuitive, all-ages format: a calendar OR a clear day-grouped list/side-panel - your call, but it must answer "what is Congress meeting about, and when" at a glance. Show date + time + chamber + committee + subject + type, link to source_url. Support upcoming AND past meetings, filterable by chamber/committee. Add /schedule to the nav. Must be legible in Plain (high-contrast) mode for older users. | |
| - | ||
| - | 3. FILL EMPTY FIELDS: audit every page for empty/unpopulated sections and populate them from available API data (e.g. bill sponsor/cosponsors links, status timeline, vote→bill linkage, member→votes, categories/subjects on bills, summaries where present). If a field is genuinely empty in the data, show an honest "Not available" rather than a blank - never a broken/empty UI element. | |
| - | ||
| - | HARD RULES: keep BRAND (brass rationed, custody-stamp, mono, dark default), NEUTRALITY (Source link on every fact, no adjectives, muted vote colors, no comment sections, honest "no record"/"not available"), and the 3-mode accessibility toggle on ALL pages incl. the new /schedule. This is PUBLIC-FACING - be polished and correct. | |
| - | ||
| - | BUILD: `npm run build` MUST pass. Do NOT restart the server (handled separately). Update FRONTEND_NOTES.md. Definition of done: photos render across members/roster/votes with initials fallback; /schedule page + /api/meetings live with real meetings; empty fields filled or honestly labeled; brand + neutrality + a11y intact. |
| @@ -1,34 +0,0 @@ | ||
| - | You are GLM-5.2, frontend lead for Docket (neutral US gov transparency app, LIVE at usdocket.org). Work in /opt/docket. The site is desktop-only and BREAKS on mobile. This wave = a thorough MOBILE-FIRST RESPONSIVE OVERHAUL + bug fixes. Be RAG-first; read ONLY: docs/BRAND.md, docs/ACCESSIBILITY.md, docs/NEUTRALITY.md, docs/API.md. Then audit your own app/ + components/. | |
| - | ||
| - | REAL PROBLEMS (verified at 390px iPhone width via _ops/qa/report.json + owner screenshots): | |
| - | - Every data TABLE overflows the viewport: /bills table = 3032px, /members/[id] = 1971px (overflows whole doc), /register & / = 828px, /votes 585, /spending 836, /executive 661, /calendar 657. Columns collide; right-side columns (Action/Result/Source) get clipped. | |
| - | - The header crams the display-mode toggle (DARK/READABLE/PLAIN) AND the page-nav (REGISTER/CALENDAR/SCHEDULE/VOTES/MEMBERS/BILLS) onto ONE line that runs off-screen - "VOTES" is half-cut. They look like one confusing strip. | |
| - | - React hydration error #418 on /register (server/client mismatch - likely the live-ticker RegisterHero and/or date formatting differing SSR vs client). | |
| - | - Display headlines are huge on mobile. | |
| - | - Tap targets <40px across nav + table links (fails senior accessibility; need >=44px). | |
| - | - Calendar uses a fixed min-w-[40rem] table → overflows mobile AND leaves big dead space on desktop. | |
| - | ||
| - | FIX ALL OF THIS: | |
| - | ||
| - | 1. RESPONSIVE TABLES → CARDS. No data table may overflow at 390px. Pattern: at >=md render the table as-is; below md, render each row as a STACKED CARD (labeled field rows: Date, Body, Item, Action, Result, Source link). Apply to EVERY list/table: register, votes, votes/[id] roster, bills, bill/[id], spending (awards + apportionments), executive, members/[id] vote history (the year-by-year - render per-year cards, NOT a 1971px table), states confidence list. Zero horizontal page scroll on any route at 390px. | |
| - | ||
| - | 2. NAV RESTRUCTURE (this is the cohesion fix). Cleanly SEPARATE two things: | |
| - | (a) Page navigation: Register · Calendar · Schedule · Votes · Members · Bills · Spending · Executive · States · Search. On mobile use a hamburger menu OR a non-clipping horizontal-scroll tab bar; show a clear ACTIVE-page indicator; >=44px targets. On desktop a clean nav bar. | |
| - | (b) Display-mode toggle (Dark/Readable/Plain): a COMPACT control (e.g. a small labeled segmented control or a dropdown), visually distinct from the page-nav, not on the same crammed line. | |
| - | Routing must feel seamless: consistent header on every page, active state, obvious back affordances. | |
| - | ||
| - | 3. FIX HYDRATION #418 on /register: make the RegisterHero ticker hydration-safe - render identical initial markup on server and client, only start the prepend/animation in a useEffect after mount. Ensure date/time formatting is deterministic (no SSR-UTC vs client-local divergence). NOTE: lib/format.ts formatEt was just fixed to return "" for date-only items so the DATE shows instead of a bogus "20:00 ET" - also RENAME the register table column header "TIME" -> "DATE" (the live hero may keep its time-of-day theater). | |
| - | ||
| - | 4. RESPONSIVE TYPOGRAPHY: display headlines use clamp() so they're reasonable on phones; body >=16px (>=18px dark mode per ACCESSIBILITY.md); line-heights comfortable. | |
| - | ||
| - | 5. CALENDAR: make it responsive - fill the container width on desktop (kill the dead space), and on mobile switch to an AGENDA/LIST view (day-grouped events) instead of a fixed-width month grid. No overflow. | |
| - | ||
| - | 6. STATES PAGE: it currently shows "56 states" all "Missing"/red (state data isn't ingested yet) - reads as broken. Fix the framing: label it "50 states + DC & territories" (not "56 states"), and present un-ingested coverage as a calm "coverage rolling out" state (muted, NOT a wall of alarming red). Keep the honesty (gaps shown) but don't imply the site is broken. | |
| - | ||
| - | 7. TAP TARGETS >=44x44px for all interactive elements; underline+color links; no hover-only. | |
| - | ||
| - | 8. FEED DIVERSITY: the register is a monotonous wall of "FEDERAL REGISTER". In lib/feed.ts getFeedItems (or the register page), INTERLEAVE sources so bills/votes/spending/executive mix instead of all same-source same-day FR clustering at the top. Keep it ranked by official action (neutrality rule #3) but ensure variety in what's visible. | |
| - | ||
| - | KEEP: brand (BRAND.md), neutrality gate (source link every fact, no adjectives, muted vote colors, no comments), and all 3 display modes working on every page incl the new mobile layouts. | |
| - | ||
| - | BUILD + SELF-VERIFY: `npm run build` MUST pass. Then re-run the mobile QA crawl to PROVE you fixed overflow: `node /opt/docket/_ops/qa-crawl.js` and read /opt/docket/_ops/qa/report.json - every route must show NO table/element exceeding ~390px (offenders list empty or within viewport) and NO React #418 error. Iterate until the crawl is clean. Do NOT restart the production server (handled separately). Update FRONTEND_NOTES.md. Definition of done: QA crawl clean (no overflow, no hydration error), nav separated + mobile-friendly, calendar responsive, states reframed, all 3 modes intact. |
| @@ -1,4 +1,4 @@ | ||
| - | // Docket mobile QA crawl - drives chromium at iPhone viewport across every | |
| + | // Docket mobile QA crawl - drives chromium at iPhone viewport across every | |
| // route, screenshots, and auto-detects horizontal overflow + console errors. | ||
| const { chromium } = require("playwright"); | ||
| const fs = require("fs"); | ||
| @@ -8,10 +8,11 @@ const OUT = "/opt/docket/_ops/qa"; | ||
| fs.mkdirSync(OUT, { recursive: true }); | ||
| const routes = [ | ||
| - | "/", "/register", "/votes", "/members", "/bills", "/spending", | |
| + | "/", "/register", "/votes", "/members", "/bills", "/laws", "/spending", | |
| "/executive", "/states", "/schedule", "/calendar", "/search", | ||
| "/how-we-pick", "/who-pays", | ||
| - | "/votes/6017", "/members/A000055", "/bill/2136", "/states/CA", | |
| + | "/votes/6017", "/votes/6002", "/members/A000055", "/bill/2136", | |
| + | "/bill/1", "/states/CA", | |
| ]; | ||
| (async () => { |
| @@ -1,4 +1,4 @@ | ||
| - | // Docket v2 design screenshots - takes home + bills + bill-detail | |
| + | // Docket v2 design screenshots - takes home + bills + bill-detail | |
| // at 390px (mobile) and 1024px (desktop), saves to qa-view/ as v2-*.png | ||
| const { chromium } = require("/opt/docket/node_modules/playwright"); | ||
| const fs = require("fs"); | ||
| @@ -8,7 +8,7 @@ const BASE = process.env.QA_BASE || "http://localhost:3000"; | ||
| const OUT = "/shared/projects/docket/_ops/qa-view"; | ||
| fs.mkdirSync(OUT, { recursive: true }); | ||
| - | // Find a real bill ID to screenshot - look for any real URL from the /bills page | |
| + | // Find a real bill ID to screenshot - look for any real URL from the /bills page | |
| async function findBillId(page) { | ||
| await page.goto(BASE + "/bills", { waitUntil: "networkidle", timeout: 35000 }); | ||
| // Try to find the first bill card link |
| @@ -1,4 +1,4 @@ | ||
| - | // Docket V3 design screenshots - captures dark-default theme | |
| + | // Docket V3 design screenshots - captures dark-default theme | |
| // Home + /bills + bill-detail at 390px (mobile) and 1280px (desktop) | ||
| // Saves to qa-view/ as v3-*.png | ||
| const { chromium } = require("/opt/docket/node_modules/playwright"); |
| @@ -1,4 +1,4 @@ | ||
| - | // Docket V3b screenshots - dark + light mode, home page, header switcher visible. | |
| + | // Docket V3b screenshots - dark + light mode, home page, header switcher visible. | |
| // Outputs: v3b-home-dark.png and v3b-home-light.png to qa-view/ | ||
| const { chromium } = require("/opt/docket/node_modules/playwright"); | ||
| const fs = require("fs"); |
| @@ -1,17 +0,0 @@ | ||
| - | You are Codex, the BACKEND builder for Docket. Be RAG-first: query homelab_context first; read ONLY the named files below; no full-tree reads. Read: PLAN.md, docs/DATA_SOURCES.md, docs/ARCHITECTURE.md, docs/NEUTRALITY.md, docs/ORCHESTRATION.md. | |
| - | ||
| - | WAVE 0 task: build the Python ingest service here under services/ingest. | |
| - | ||
| - | Create poller skeletons for: congress, govinfo, usaspending, treasury, votes_house, votes_senate, openstates. | |
| - | ||
| - | Make USAspending FULLY WORK end-to-end pulling REAL recent awards (no auth required) into a SQLite Phase-0 store. Make Congress.gov recent bill actions work IF env DOCKET_CONGRESS_KEY is set; otherwise log clearly that a free api.data.gov key is needed and skip gracefully. | |
| - | ||
| - | Implement the normalized schema (bills, actions, awards, events, raw_events) and a change-detection layer that dedupes on (source, externalId, updateDate). | |
| - | ||
| - | Implement the ranking function referencing ONLY actionDate, dollarAmount, affectedEstimate, actionType (neutrality rule #3). Add a unit test asserting no other field is referenced. | |
| - | ||
| - | Every stored fact MUST carry a source_url whose host is an allowlisted .gov (see docs/DATA_SOURCES.md). | |
| - | ||
| - | CLI entrypoint: python -m ingest.run --once. Use a venv + requirements.txt. | |
| - | ||
| - | Write INGEST_NOTES.md. Definition of done: `python -m ingest.run --once` populates SQLite with REAL USAspending awards; ranking + change-detection unit tests pass. |
| @@ -1,19 +0,0 @@ | ||
| - | You are GLM-5.2, the FRONTEND builder for Docket. Be RAG-first: call homelab_context before reading; read ONLY the named files below; no full-tree reads. Read in this repo: PLAN.md, docs/BRAND.md, docs/NEUTRALITY.md, docs/ARCHITECTURE.md, docs/ORCHESTRATION.md. | |
| - | ||
| - | WAVE 0 task: scaffold a Next.js (app router) + Tailwind project IN PLACE in this repo and implement the Docket brand system from docs/BRAND.md EXACTLY: CSS custom-property tokens, the Tailwind theme mapped to them, and the fonts Archivo (display) + JetBrains Mono (data/labels) + Fraunces (prose). | |
| - | ||
| - | Build the static shell of EVERY page on FAKE data: | |
| - | - /register (the live Register feed - the signature hero, a real-time stamped ledger of official actions, mono rows, hairline separators, one pulsing brass live-dot) | |
| - | - /bill/[id] | |
| - | - /spending | |
| - | - /states | |
| - | - /how-we-pick | |
| - | - /who-pays | |
| - | ||
| - | Design rules (hard): dark-first; brass accent (#E0A53B) RATIONED to one signal per view; custody-stamp [D] mark; NO eagle/flag/dome/scales. | |
| - | ||
| - | Neutrality gate (enforce in all copy + UI): NO adjectives in copy; vote and party visualizations in MUTED tones (never vivid red vs blue, equal weight); a visible "Source" link on every fake fact; NO comment sections, reactions, or share buttons on bills anywhere. | |
| - | ||
| - | No real data fetching yet (fake/mock data only). Run npm install; ensure `npm run build` passes and the dev server starts. Write SCAFFOLD_NOTES.md describing what you built and the exact commands to run it. | |
| - | ||
| - | Definition of done: clean build, every page renders with the Docket brand applied, neutrality gate respected in the mock content. |
| @@ -3,7 +3,7 @@ import { getBillDiff } from "@/lib/feed"; | ||
| export const dynamic = "force-dynamic"; | ||
| - | /** GET /api/bills/:id/diff?from=ih&to=rh - the "what got slipped in" diff. */ | |
| + | /** GET /api/bills/:id/diff?from=ih&to=rh - the "what got slipped in" diff. */ | |
| export async function GET( | ||
| req: NextRequest, | ||
| { params }: { params: { id: string } } |
| @@ -3,7 +3,7 @@ import { getBillDetail } from "@/lib/feed"; | ||
| export const dynamic = "force-dynamic"; | ||
| - | /** GET /api/bills/:id - detail (sponsor/timeline/versions/votes). */ | |
| + | /** GET /api/bills/:id - detail (sponsor/timeline/versions/votes). */ | |
| export async function GET( | ||
| _req: NextRequest, | ||
| { params }: { params: { id: string } } |
| @@ -4,7 +4,7 @@ import { getBills, getBillsCount, type BillListItem } from "@/lib/feed"; | ||
| export const dynamic = "force-dynamic"; | ||
| - | /** GET /api/bills - list with filters: ?q=&state=&status=&congress=&policyArea=&sort=. */ | |
| + | /** GET /api/bills - list with filters: ?q=&state=&status=&congress=&policyArea=&sort=. */ | |
| export async function GET(req: NextRequest) { | ||
| const sp = req.nextUrl.searchParams; | ||
| const { limit, offset } = parsePaging(sp); | ||
| @@ -24,7 +24,7 @@ export async function GET(req: NextRequest) { | ||
| ) as "updated" | "action" | "congress" | "status", | ||
| }; | ||
| - | // Push all filtering and pagination into SQL - no more full-table JS slice. | |
| + | // Push all filtering and pagination into SQL - no more full-table JS slice. | |
| const items = getBills(filter, limit, offset); | ||
| const total = getBillsCount(filter); | ||
| const body: ItemList<BillListItem> = { items, total, limit, offset }; |
| @@ -1,21 +1,40 @@ | ||
| import { NextResponse, type NextRequest } from "next/server"; | ||
| import { parsePaging, type ItemList } from "@/lib/db"; | ||
| - | import { getCalendarEvents, type CalendarEvent } from "@/lib/feed"; | |
| + | import { | |
| + | getCalendarEvents, | |
| + | getCalendarDaySummary, | |
| + | type CalendarEvent, | |
| + | type CalendarDayKind, | |
| + | } from "@/lib/feed"; | |
| export const dynamic = "force-dynamic"; | ||
| /** | ||
| - | * GET /api/calendar - events in a date range (calendar_events). | |
| + | * GET /api/calendar - events in a date range (calendar_events). | |
| * ?from=YYYY-MM-DD&to=YYYY-MM-DD&kind=effective|enacted|vote|... | ||
| * Paginated { items, total, limit, offset } (docs/API.md). | ||
| + | * | |
| + | * ?summary=1 - per-day kind counts for the month grid (CalendarDayKind[]), | |
| + | * a lightweight alternative to serializing every event. | |
| */ | ||
| export async function GET(req: NextRequest) { | ||
| const sp = req.nextUrl.searchParams; | ||
| - | const { limit, offset } = parsePaging(sp); | |
| const from = sp.get("from") || undefined; | ||
| const to = sp.get("to") || undefined; | ||
| const kind = sp.get("kind") || undefined; | ||
| + | if (sp.get("summary")) { | |
| + | const items = getCalendarDaySummary(from ?? "", to ?? ""); | |
| + | const body: ItemList<CalendarDayKind> = { | |
| + | items, | |
| + | total: items.length, | |
| + | limit: items.length, | |
| + | offset: 0, | |
| + | }; | |
| + | return NextResponse.json(body); | |
| + | } | |
| + | ||
| + | const { limit, offset } = parsePaging(sp); | |
| const items = getCalendarEvents({ from, to, kind }); | ||
| const total = items.length; | ||
| const paged = items.slice(offset, offset + limit); |
| @@ -1,10 +1,10 @@ | ||
| import { NextResponse } from "next/server"; | ||
| import { getCategories } from "@/lib/feed"; | ||
| - | // No search params - ISR-cache the response on the Node server too. | |
| + | // No search params - ISR-cache the response on the Node server too. | |
| export const revalidate = 300; | ||
| - | /** GET /api/categories - topic list with bill + award counts. */ | |
| + | /** GET /api/categories - topic list with bill + award counts. */ | |
| export async function GET() { | ||
| return NextResponse.json({ items: getCategories() }); | ||
| } |
| @@ -3,7 +3,7 @@ import { getChangeEvents } from "@/lib/feed"; | ||
| export const dynamic = "force-dynamic"; | ||
| - | /** GET /api/changes - recent change_events (added / updated / removed). */ | |
| + | /** GET /api/changes - recent change_events (added / updated / removed). */ | |
| export async function GET(req: NextRequest) { | ||
| const sp = req.nextUrl.searchParams; | ||
| const raw = parseInt(sp.get("limit") ?? "", 10); |
| @@ -4,7 +4,7 @@ import { getExecutive, type ExecutiveAction } from "@/lib/feed"; | ||
| export const dynamic = "force-dynamic"; | ||
| - | /** GET /api/executive - Federal Register: EOs, rules, notices. */ | |
| + | /** GET /api/executive - Federal Register: EOs, rules, notices. */ | |
| export async function GET(req: NextRequest) { | ||
| const sp = req.nextUrl.searchParams; | ||
| const { limit, offset } = parsePaging(sp); |
| @@ -5,7 +5,7 @@ import { getFeedItems, type FeedItem } from "@/lib/feed"; | ||
| export const dynamic = "force-dynamic"; | ||
| /** | ||
| - | * GET /api/feed - the Register. Mixed, ranked stream (rule #3). | |
| + | * GET /api/feed - the Register. Mixed, ranked stream (rule #3). | |
| * ?type=bill|vote|award|amendment|rider|procedural|executive|apportionment (repeatable) | ||
| * ?state=US&?q= | ||
| */ |
| @@ -3,13 +3,13 @@ import { safeAll, parsePaging, type ItemList } from "@/lib/db"; | ||
| export const dynamic = "force-dynamic"; | ||
| - | /** GET /api/lobbying?bill= - LDA.gov filings. Empty until ingest lands. */ | |
| + | /** GET /api/lobbying?bill= - LDA.gov filings. Empty until ingest lands. */ | |
| export async function GET(req: NextRequest) { | ||
| const sp = req.nextUrl.searchParams; | ||
| const { limit, offset } = parsePaging(sp); | ||
| const bill = sp.get("bill"); | ||
| - | // lobbying table built in a later wave - defensive. | |
| + | // lobbying table built in a later wave - defensive. | |
| const items = safeAll( | ||
| `SELECT rowid, registrant, client, amount_range, period, source_url | ||
| FROM lobbying |
| @@ -5,11 +5,11 @@ import { getMeetings, type Meeting } from "@/lib/feed"; | ||
| export const dynamic = "force-dynamic"; | ||
| /** | ||
| - | * GET /api/meetings - congressional committee + floor meetings (meetings table). | |
| + | * GET /api/meetings - congressional committee + floor meetings (meetings table). | |
| * Filters: ?from=&to=&chamber=&committee=&type= Paginated { items, total, ... }. | ||
| * | ||
| * Defensive: while the `meetings` table is mid-ingest (or absent), getMeetings | ||
| - | * returns [] and this responds { items: [], total: 0 } - never a 5xx. The API | |
| + | * returns [] and this responds { items: [], total: 0 } - never a 5xx. The API | |
| * contract is "always respond" (docs/API.md). | ||
| */ | ||
| export async function GET(req: NextRequest) { |
| @@ -3,7 +3,7 @@ import { getMember } from "@/lib/feed"; | ||
| export const dynamic = "force-dynamic"; | ||
| - | /** GET /api/members/:id - a rep's full record (votes, sponsorships, amendments). */ | |
| + | /** GET /api/members/:id - a rep's full record (votes, sponsorships, amendments). */ | |
| export async function GET( | ||
| _req: NextRequest, | ||
| { params }: { params: { id: string } } |
| @@ -11,7 +11,7 @@ export const dynamic = "force-dynamic"; | ||
| * per-year vote rows only when the <details> panel is first opened. | ||
| * | ||
| * [id] is the bioguide ID (e.g. "A000055"). | ||
| - | * ?year= - omit or pass "null" for undated votes; pass a 4-digit year otherwise. | |
| + | * ?year= - omit or pass "null" for undated votes; pass a 4-digit year otherwise. | |
| */ | ||
| export async function GET( | ||
| req: NextRequest, |
| @@ -5,7 +5,7 @@ import type { PartyCode } from "@/lib/neutrality"; | ||
| export const dynamic = "force-dynamic"; | ||
| - | /** GET /api/members - browseable roster. Filters: ?q=&state=&chamber=&party=. */ | |
| + | /** GET /api/members - browseable roster. Filters: ?q=&state=&chamber=&party=. */ | |
| export async function GET(req: NextRequest) { | ||
| const sp = req.nextUrl.searchParams; | ||
| const { limit, offset } = parsePaging(sp); | ||
| @@ -25,7 +25,7 @@ export async function GET(req: NextRequest) { | ||
| }; | ||
| const items = getMembers(filter); | ||
| - | // Use cached count - avoids length() on 12.7k array on every API request. | |
| + | // Use cached count - avoids length() on 12.7k array on every API request. | |
| const total = filter.q || filter.state || filter.chamber || filter.party | ||
| ? items.length | ||
| : getMemberCounts().total; |
| @@ -3,7 +3,7 @@ import { globalSearch } from "@/lib/feed"; | ||
| export const dynamic = "force-dynamic"; | ||
| - | /** GET /api/search?q= - global lookup across bills/members/executive/awards/states. */ | |
| + | /** GET /api/search?q= - global lookup across bills/members/executive/awards/states. */ | |
| export async function GET(req: NextRequest) { | ||
| const q = req.nextUrl.searchParams.get("q") ?? ""; | ||
| return NextResponse.json({ q, ...globalSearch(q) }); |
| @@ -3,7 +3,7 @@ import { getApportionments, type Apportionment } from "@/lib/feed"; | ||
| export const dynamic = "force-dynamic"; | ||
| - | /** GET /api/spending/apportionments - impound tracker. Empty until ingest lands. */ | |
| + | /** GET /api/spending/apportionments - impound tracker. Empty until ingest lands. */ | |
| export async function GET() { | ||
| const items = getApportionments(); | ||
| return NextResponse.json({ items, total: items.length }); |
| @@ -4,7 +4,7 @@ import { getAwardsFiltered, type Award } from "@/lib/feed"; | ||
| export const dynamic = "force-dynamic"; | ||
| - | /** GET /api/spending/awards - USAspending awards with filter + sort. | |
| + | /** GET /api/spending/awards - USAspending awards with filter + sort. | |
| * ?q= keyword (agency/recipient/program) | ||
| * ?sort=date → actionDate DESC | ||
| * ?sort=agency → agency ASC |
| @@ -3,7 +3,7 @@ import { getFiscal, type FiscalMetric } from "@/lib/feed"; | ||
| export const dynamic = "force-dynamic"; | ||
| - | /** GET /api/spending/fiscal - debt/deficit/outlays. Empty until ingest lands. */ | |
| + | /** GET /api/spending/fiscal - debt/deficit/outlays. Empty until ingest lands. */ | |
| export async function GET() { | ||
| const items = getFiscal(); | ||
| return NextResponse.json({ items, total: items.length }); |
| @@ -3,10 +3,14 @@ import { getStateDetail } from "@/lib/feed"; | ||
| export const dynamic = "force-dynamic"; | ||
| - | /** GET /api/states/:code - state coverage + delegation + sponsored bills. */ | |
| + | /** GET /api/states/:code - state coverage + delegation + sponsored bills. */ | |
| export async function GET( | ||
| _req: NextRequest, | ||
| { params }: { params: { code: string } } | ||
| ) { | ||
| - | return NextResponse.json(getStateDetail(params.code.toUpperCase())); | |
| + | const detail = getStateDetail(params.code.toUpperCase()); | |
| + | if (!detail) { | |
| + | return NextResponse.json({ error: "Unknown state code" }, { status: 404 }); | |
| + | } | |
| + | return NextResponse.json(detail); | |
| } |
| @@ -1,10 +1,10 @@ | ||
| import { NextResponse } from "next/server"; | ||
| import { getStates } from "@/lib/feed"; | ||
| - | // No search params - ISR-cache the response on the Node server too. | |
| + | // No search params - ISR-cache the response on the Node server too. | |
| export const revalidate = 300; | ||
| - | /** GET /api/states - coverage + confidence per state (honesty layer). | |
| + | /** GET /api/states - coverage + confidence per state (honesty layer). | |
| * Rows are aggregated to one entry per state with a worst-case confidence | ||
| * (ok | stale | missing) and the list of missing data types. */ | ||
| export async function GET() { |
| @@ -3,7 +3,7 @@ import { getVoteById } from "@/lib/feed"; | ||
| export const dynamic = "force-dynamic"; | ||
| - | /** GET /api/votes/:id - single roll call with the full member roster. | |
| + | /** GET /api/votes/:id - single roll call with the full member roster. | |
| * recorded:false => voice/UC; positions is empty and the UI shows the | ||
| * "no individual record exists" honesty note (docs/API.md). */ | ||
| export async function GET( |
| @@ -4,7 +4,7 @@ import { getVotesFiltered, getVoteCountFiltered, type Vote, type VoteFilter } fr | ||
| export const dynamic = "force-dynamic"; | ||
| - | /** GET /api/votes - roll calls with full filter support: ?chamber=&q=&result=&sort=. */ | |
| + | /** GET /api/votes - roll calls with full filter support: ?chamber=&q=&result=&sort=. */ | |
| export async function GET(req: NextRequest) { | ||
| const sp = req.nextUrl.searchParams; | ||
| const { limit, offset } = parsePaging(sp); | ||
| @@ -25,7 +25,7 @@ export async function GET(req: NextRequest) { | ||
| undefined, | ||
| }; | ||
| - | // Push all filtering and pagination into SQL - no full-table JS slice. | |
| + | // Push all filtering and pagination into SQL - no full-table JS slice. | |
| const items = getVotesFiltered(filter, limit, offset); | ||
| const total = getVoteCountFiltered(filter); | ||
| const body: ItemList<Vote> = { items, total, limit, offset }; |
| @@ -8,8 +8,9 @@ import { PartyTag } from "@/components/party-tag"; | ||
| import { MemberAvatar } from "@/components/member-avatar"; | ||
| import { RiderFlag, ChangeTag } from "@/components/rider-flag"; | ||
| import { getBillDetail, type BillDiffChange, type BillAmendment } from "@/lib/feed"; | ||
| - | import { formatDate } from "@/lib/format"; | |
| + | import { formatDate, congressLabel } from "@/lib/format"; | |
| import { CiteThis } from "@/components/cite-this"; | ||
| + | import { BillPdfViewer } from "@/components/record-archive"; | |
| export const revalidate = 300; | ||
| @@ -101,19 +102,19 @@ export default function BillPage({ | ||
| style={{ color: "var(--dk-signal)", marginBottom: "20px", fontSize: "12px", letterSpacing: "0.14em" }} | ||
| > | ||
| <span className="font-mono">{bill.number}</span> | ||
| - | {" · "} | |
| - | {bill.congress}th Congress | |
| + | {congressLabel(bill.congress) ? ` · ${congressLabel(bill.congress)}` : ""} | |
| {bill.policyArea ? ` · ${bill.policyArea}` : ""} | ||
| </p> | ||
| - | {/* H1: HUGE bill number - Archivo, uppercase, the anchor */} | |
| + | {/* H1: HUGE bill number - Archivo, uppercase, the anchor */} | |
| <h1 | ||
| className="font-display text-paper" | ||
| style={{ | ||
| fontSize: "clamp(2.8rem, 9vw, 4.5rem)", | ||
| fontWeight: 900, | ||
| - | lineHeight: 0.96, | |
| - | letterSpacing: "-0.03em", | |
| + | lineHeight: 1.0, | |
| + | letterSpacing: "-0.01em", | |
| + | wordSpacing: "0.06em", | |
| textTransform: "uppercase", | ||
| marginBottom: "24px", | ||
| }} | ||
| @@ -121,7 +122,7 @@ export default function BillPage({ | ||
| {bill.number} | ||
| </h1> | ||
| - | {/* Bill title - Fraunces, readable, warm */} | |
| + | {/* Bill title - Fraunces, readable, warm */} | |
| <p | ||
| className="font-prose" | ||
| style={{ fontStyle: "italic", fontSize: "clamp(1rem, 1.8vw, 1.125rem)", lineHeight: 1.55, color: "var(--dk-ash)", maxWidth: "56ch" }} | ||
| @@ -264,7 +265,45 @@ export default function BillPage({ | ||
| </section> | ||
| ) : null} | ||
| - | {/* ── Version diff - What changed ───────────────────────────── */} | |
| + | {/* ── Plain-language summary ────────────────────────────────── */} | |
| + | <section className="mt-12" aria-label="Summary"> | |
| + | <SectionLabel eyebrow="Summary" title="What this bill does" /> | |
| + | {bill.summary ? ( | |
| + | <div className="rounded-[6px] border border-line bg-surface p-6"> | |
| + | <p | |
| + | className="font-prose text-paper" | |
| + | style={{ fontSize: "1.0625rem", lineHeight: 1.65, whiteSpace: "pre-line" }} | |
| + | > | |
| + | {bill.summary} | |
| + | </p> | |
| + | <p className="label mt-5" style={{ color: "var(--dk-ash-2)" }}> | |
| + | {bill.summarySource | |
| + | ? `Plain-language summary: ${bill.summarySource}` | |
| + | : "Plain-language summary: Congressional Research Service via GovInfo"} | |
| + | </p> | |
| + | </div> | |
| + | ) : ( | |
| + | <div className="rounded-[6px] border border-line bg-surface p-6"> | |
| + | <p | |
| + | className="font-prose italic" | |
| + | style={{ fontSize: "0.9375rem", color: "var(--dk-ash)" }} | |
| + | > | |
| + | Plain-language summary not yet available - read the official text | |
| + | below. | |
| + | </p> | |
| + | </div> | |
| + | )} | |
| + | </section> | |
| + | ||
| + | {/* ── Bill text PDF viewer + archived copy ──────────────────── */} | |
| + | <BillPdfViewer | |
| + | pdfUrl={bill.pdfUrl} | |
| + | archivePdfUrl={bill.archivePdfUrl} | |
| + | sourceUrl={bill.sourceUrl} | |
| + | billNumber={bill.number} | |
| + | /> | |
| + | ||
| + | {/* ── Version diff - What changed ───────────────────────────── */} | |
| <section className="mt-12" aria-label="Version diff"> | ||
| <SectionLabel eyebrow="Version diff" title="What changed in the text" /> | ||
| <p |
| @@ -105,8 +105,9 @@ export default function BillsPage() { | ||
| style={{ | ||
| fontSize: "clamp(2.2rem, 7vw, 3.75rem)", | ||
| fontWeight: 900, | ||
| - | lineHeight: 0.96, | |
| - | letterSpacing: "-0.03em", | |
| + | lineHeight: 1.0, | |
| + | letterSpacing: "-0.01em", | |
| + | wordSpacing: "0.06em", | |
| textTransform: "uppercase", | ||
| maxWidth: "18ch", | ||
| marginBottom: "24px", | ||
| @@ -129,9 +130,32 @@ export default function BillsPage() { | ||
| or search by title or number. Each bill links to its sponsor, status | ||
| timeline, and the official record. | ||
| </p> | ||
| + | <p style={{ marginTop: "20px" }}> | |
| + | <Link | |
| + | href="/laws" | |
| + | className="focus-ring no-underline" | |
| + | style={{ | |
| + | display: "inline-flex", | |
| + | alignItems: "center", | |
| + | gap: "8px", | |
| + | minHeight: "44px", | |
| + | padding: "0 16px", | |
| + | border: "1px solid var(--dk-line)", | |
| + | borderRadius: "8px", | |
| + | background: "var(--dk-surface)", | |
| + | fontFamily: "var(--font-mono), ui-monospace, monospace", | |
| + | fontSize: "12px", | |
| + | letterSpacing: "0.08em", | |
| + | textTransform: "uppercase", | |
| + | color: "var(--dk-signal)", | |
| + | }} | |
| + | > | |
| + | Became law → View Public Laws | |
| + | </Link> | |
| + | </p> | |
| </header> | ||
| - | {/* Client browser - reads searchParams and fetches from /api/bills */} | |
| + | {/* Client browser - reads searchParams and fetches from /api/bills */} | |
| <Suspense fallback={<BillsSkeleton />}> | ||
| <BillsBrowser policyAreas={policyAreas} congresses={congresses} /> | ||
| </Suspense> |
| @@ -1,15 +1,8 @@ | ||
| import type { Metadata } from "next"; | ||
| import Link from "next/link"; | ||
| - | import { Card, PageHeader, SectionHeader } from "@/components/ui"; | |
| - | import { SourceLink } from "@/components/source-link"; | |
| - | import { | |
| - | getCalendarEvents, | |
| - | getCalendarDaySummary, | |
| - | type CalendarEvent, | |
| - | type CalendarKind, | |
| - | type CalendarDayKind, | |
| - | } from "@/lib/feed"; | |
| - | import { clsx } from "@/lib/cx"; | |
| + | import { Suspense } from "react"; | |
| + | import { Card, PageHeader } from "@/components/ui"; | |
| + | import { CalendarBrowser } from "@/components/calendar-browser"; | |
| export const revalidate = 300; | ||
| @@ -19,121 +12,30 @@ export const metadata: Metadata = { | ||
| "A month grid of when laws take effect, bills become law, and votes happen. Each day links to its official source.", | ||
| }; | ||
| - | /* Kind → plain-language + compact mark + muted token. The mark text is always | |
| - | * shown (never color alone) so the grid stays legible in Plain mode and for | |
| - | * color-blind readers (docs/ACCESSIBILITY.md #7; docs/NEUTRALITY.md §Viz). */ | |
| - | type KindMeta = { value: CalendarKind; plain: string; mark: string; color: string }; | |
| - | ||
| - | const KINDS: KindMeta[] = [ | |
| - | { value: "effective", plain: "Takes effect", mark: "Effective", color: "var(--dk-hold)" }, | |
| - | { value: "effective_parsed", plain: "Takes effect", mark: "Effective", color: "var(--dk-hold)" }, | |
| - | { value: "enacted", plain: "Became law", mark: "Law", color: "var(--dk-yea)" }, | |
| - | { value: "vote", plain: "Vote", mark: "Vote", color: "var(--dk-party-d)" }, | |
| - | { value: "floor", plain: "Floor action", mark: "Floor", color: "var(--dk-party-r)" }, | |
| - | { value: "committee", plain: "Committee", mark: "Committee", color: "var(--dk-party-i)" }, | |
| - | { value: "introduced", plain: "Introduced", mark: "Introduced", color: "var(--dk-ash)" }, | |
| - | { value: "apportionment", plain: "Apportionment", mark: "Appr.", color: "var(--dk-hold)" }, | |
| - | ]; | |
| - | ||
| - | const KIND_BY_VALUE = new Map<CalendarKind, KindMeta>( | |
| - | KINDS.map((k) => [k.value, k]) | |
| - | ); | |
| - | ||
| - | const WEEKDAYS = ["Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"]; | |
| - | const MONTH_NAMES = [ | |
| - | "January", "February", "March", "April", "May", "June", | |
| - | "July", "August", "September", "October", "November", "December", | |
| - | ]; | |
| - | ||
| - | function pad(n: number): string { | |
| - | return n < 10 ? `0${n}` : String(n); | |
| - | } | |
| - | ||
| - | function isoToday(): string { | |
| - | const d = new Date(); | |
| - | return `${d.getUTCFullYear()}-${pad(d.getUTCMonth() + 1)}-${pad(d.getUTCDate())}`; | |
| - | } | |
| - | ||
| - | /** Parse ?month=YYYY-MM into {year,month}; fall back to the current month. */ | |
| - | function parseMonth(raw: string | undefined): { year: number; month: number } { | |
| - | const m = /^(\d{4})-(\d{1,2})$/.exec(raw ?? ""); | |
| - | if (m) { | |
| - | const year = parseInt(m[1], 10); | |
| - | const month = parseInt(m[2], 10); | |
| - | if (year >= 1900 && year <= 9999 && month >= 1 && month <= 12) { | |
| - | return { year, month }; | |
| - | } | |
| - | } | |
| - | const d = new Date(); | |
| - | return { year: d.getUTCFullYear(), month: d.getUTCMonth() + 1 }; | |
| - | } | |
| - | ||
| - | function shiftMonth(year: number, month: number, delta: number): { year: number; month: number } { | |
| - | const zero = month - 1 + delta; | |
| - | const d = new Date(Date.UTC(year, zero, 1)); | |
| - | return { year: d.getUTCFullYear(), month: d.getUTCMonth() + 1 }; | |
| - | } | |
| - | ||
| - | export default function CalendarPage({ | |
| - | searchParams, | |
| - | }: { | |
| - | searchParams: { month?: string; day?: string; kind?: string }; | |
| - | }) { | |
| - | const { year, month } = parseMonth(searchParams.month); | |
| - | const monthParam = `${year}-${pad(month)}`; | |
| - | const todayIso = isoToday(); | |
| - | ||
| - | const selectedDay = searchParams.day && /^\d{4}-\d{2}-\d{2}$/.test(searchParams.day) | |
| - | ? searchParams.day | |
| - | : undefined; | |
| - | const kindFilter = searchParams.kind || undefined; | |
| - | ||
| - | const daysInMonth = new Date(Date.UTC(year, month, 0)).getUTCDate(); | |
| - | const firstWeekday = new Date(Date.UTC(year, month - 1, 1)).getUTCDay(); | |
| - | const from = `${year}-${pad(month)}-01`; | |
| - | const to = `${year}-${pad(month)}-${pad(daysInMonth)}`; | |
| - | ||
| - | // Grid: use condensed per-day kind counts (avoids serializing 2,700+ full | |
| - | // CalendarEvent objects). Only the selected-day detail panel needs full rows. | |
| - | const daySummary = getCalendarDaySummary(from, to); | |
| - | // Filter grid summary by kind if requested. | |
| - | const filteredSummary = kindFilter | |
| - | ? daySummary.filter((s) => s.kind === kindFilter) | |
| - | : daySummary; | |
| - | const byDate = new Map<string, CalendarDayKind[]>(); | |
| - | for (const s of filteredSummary) { | |
| - | const arr = byDate.get(s.date) ?? []; | |
| - | arr.push(s); | |
| - | byDate.set(s.date, arr); | |
| - | } | |
| - | ||
| - | // Full events for the selected day only (shown in the detail panel). | |
| - | const dayEvents: CalendarEvent[] = selectedDay | |
| - | ? getCalendarEvents({ from: selectedDay, to: selectedDay, kind: kindFilter }) | |
| - | : []; | |
| - | ||
| - | // Kind counts for the legend: sum from the condensed summary. | |
| - | const kindCounts = countKindsFromSummary(filteredSummary); | |
| - | ||
| - | const prev = shiftMonth(year, month, -1); | |
| - | const next = shiftMonth(year, month, +1); | |
| - | const prevHref = hrefFor(prev, selectedDay, kindFilter); | |
| - | const nextHref = hrefFor(next, selectedDay, kindFilter); | |
| - | const todayHref = hrefFor({ year: todayYear(), month: todayMonth() }, todayIso, kindFilter); | |
| - | ||
| - | // Build the grid cells (leading blanks, days, trailing blanks to fill rows). | |
| - | const cells: Array<number | null> = []; | |
| - | for (let i = 0; i < firstWeekday; i++) cells.push(null); | |
| - | for (let d = 1; d <= daysInMonth; d++) cells.push(d); | |
| - | while (cells.length % 7 !== 0) cells.push(null); | |
| - | ||
| + | export default function CalendarPage() { | |
| return ( | ||
| - | <div className="mx-auto w-full max-w-register px-4 py-10 sm:px-6"> | |
| + | <div | |
| + | className="mx-auto w-full max-w-[860px] px-6 sm:px-8" | |
| + | style={{ | |
| + | paddingTop: "clamp(32px, 6vw, 56px)", | |
| + | paddingBottom: "clamp(48px, 8vw, 80px)", | |
| + | }} | |
| + | > | |
| <Link | ||
| - | href="/register" | |
| - | className="focus-ring tap label mb-6 gap-1 text-ash hover:text-paper" | |
| + | href="/" | |
| + | className="focus-ring tap no-underline hover:underline" | |
| + | style={{ | |
| + | display: "inline-flex", | |
| + | alignItems: "center", | |
| + | gap: "8px", | |
| + | fontFamily: "var(--font-mono), ui-monospace, monospace", | |
| + | fontSize: "13px", | |
| + | letterSpacing: "0.08em", | |
| + | color: "var(--dk-signal)", | |
| + | marginBottom: "32px", | |
| + | }} | |
| > | ||
| - | ← Register | |
| + | ← Home | |
| </Link> | ||
| <PageHeader | ||
| @@ -142,280 +44,9 @@ export default function CalendarPage({ | ||
| lead="A month at a glance. Each day lists the laws that take effect, bills that became law, votes, and floor action. Pick a day to read its events - each sealed to its official source." | ||
| /> | ||
| - | {/* Month navigation */} | |
| - | <div className="mt-8 flex flex-wrap items-center justify-between gap-3"> | |
| - | <div className="flex items-center gap-1"> | |
| - | <Link | |
| - | href={prevHref} | |
| - | className="focus-ring label inline-flex min-h-[44px] items-center rounded-[3px] border border-line px-3 text-ash hover:text-paper" | |
| - | aria-label={`Previous month, ${MONTH_NAMES[prev.month - 1]} ${prev.year}`} | |
| - | > | |
| - | ← {MONTH_NAMES[prev.month - 1].slice(0, 3)} | |
| - | </Link> | |
| - | <p | |
| - | className="display px-2 text-paper" | |
| - | aria-live="polite" | |
| - | style={{ fontSize: "clamp(1.25rem, 5vw, 1.875rem)" }} | |
| - | > | |
| - | {MONTH_NAMES[month - 1]} {year} | |
| - | </p> | |
| - | <Link | |
| - | href={nextHref} | |
| - | className="focus-ring label inline-flex min-h-[44px] items-center rounded-[3px] border border-line px-3 text-ash hover:text-paper" | |
| - | aria-label={`Next month, ${MONTH_NAMES[next.month - 1]} ${next.year}`} | |
| - | > | |
| - | {MONTH_NAMES[next.month - 1].slice(0, 3)} → | |
| - | </Link> | |
| - | </div> | |
| - | <Link | |
| - | href={todayHref} | |
| - | className="focus-ring label inline-flex min-h-[44px] items-center rounded-[3px] border border-line px-3 text-ash underline underline-offset-4 hover:text-paper" | |
| - | > | |
| - | Today | |
| - | </Link> | |
| - | </div> | |
| - | ||
| - | {/* Kind filter - plain-word chips; selection lives in the URL. */} | |
| - | <div className="mt-5 flex flex-wrap items-center gap-1.5" role="group" aria-label="Filter by kind"> | |
| - | <Link | |
| - | href={hrefFor({ year, month }, selectedDay, undefined)} | |
| - | className={clsx( | |
| - | "focus-ring label inline-flex min-h-[44px] items-center rounded-[3px] border px-2.5", | |
| - | !kindFilter | |
| - | ? "border-paper text-paper" | |
| - | : "border-line text-ash hover:text-paper" | |
| - | )} | |
| - | aria-pressed={!kindFilter} | |
| - | > | |
| - | All | |
| - | </Link> | |
| - | {KINDS.filter((k, i) => KINDS.findIndex((x) => x.mark === k.mark) === i).map((k) => { | |
| - | const active = kindFilter === k.value; | |
| - | return ( | |
| - | <Link | |
| - | key={k.value} | |
| - | href={hrefFor({ year, month }, selectedDay, active ? undefined : k.value)} | |
| - | className={clsx( | |
| - | "focus-ring label inline-flex min-h-[44px] items-center gap-1.5 rounded-[3px] border px-2.5", | |
| - | active | |
| - | ? "border-paper text-paper" | |
| - | : "border-line text-ash hover:text-paper" | |
| - | )} | |
| - | aria-pressed={active} | |
| - | > | |
| - | <span | |
| - | aria-hidden="true" | |
| - | className="inline-block h-2 w-2 rounded-full" | |
| - | style={{ backgroundColor: k.color }} | |
| - | /> | |
| - | {k.plain} | |
| - | </Link> | |
| - | ); | |
| - | })} | |
| - | </div> | |
| - | ||
| - | <div className="mt-6 grid gap-6 lg:grid-cols-[1.5fr_1fr]"> | |
| - | {/* LEFT: month grid (>=md) · agenda list (<md). Both fill the container | |
| - | width - no fixed min-width, so no overflow and no desktop dead space. */} | |
| - | <div> | |
| - | {/* DESKTOP MONTH GRID */} | |
| - | <Card className="hidden overflow-hidden md:block"> | |
| - | <table className="w-full table-fixed border-collapse text-left"> | |
| - | <caption className="sr-only"> | |
| - | Calendar grid for {MONTH_NAMES[month - 1]} {year}. Each cell is a | |
| - | day; select a day to read its events. Rows are weeks, columns | |
| - | are weekdays starting Sunday. | |
| - | </caption> | |
| - | <thead> | |
| - | <tr className="border-b border-line"> | |
| - | {WEEKDAYS.map((w) => ( | |
| - | <th key={w} scope="col" className="label px-2 py-2.5 font-normal"> | |
| - | {w} | |
| - | </th> | |
| - | ))} | |
| - | </tr> | |
| - | </thead> | |
| - | <tbody> | |
| - | {chunk(cells, 7).map((row, ri) => ( | |
| - | <tr key={ri} className="border-b border-line last:border-0"> | |
| - | {row.map((day, ci) => { | |
| - | if (day == null) { | |
| - | return ( | |
| - | <td key={ci} aria-hidden="true" className="bg-ink/40 p-0"> | |
| - | <div className="min-h-[5.5rem]" /> | |
| - | </td> | |
| - | ); | |
| - | } | |
| - | const dateStr = `${year}-${pad(month)}-${pad(day)}`; | |
| - | const evs = byDate.get(dateStr) ?? []; | |
| - | const evCount = dayEventCount(evs); | |
| - | const isToday = dateStr === todayIso; | |
| - | const isSelected = dateStr === selectedDay; | |
| - | const marks = distinctMarks(evs); | |
| - | return ( | |
| - | <td key={ci} className="border-r border-line p-0 align-top last:border-r-0"> | |
| - | <Link | |
| - | href={hrefFor({ year, month }, dateStr, kindFilter)} | |
| - | aria-label={`${day} ${MONTH_NAMES[month - 1]} ${year}${evCount ? `, ${evCount} event${evCount === 1 ? "" : "s"}` : ", no events"}`} | |
| - | aria-current={isSelected ? "true" : undefined} | |
| - | className={clsx( | |
| - | "focus-ring block min-h-[5.5rem] p-2 no-underline", | |
| - | isSelected ? "bg-graphite" : "hover:bg-graphite/50" | |
| - | )} | |
| - | > | |
| - | <span | |
| - | className={clsx( | |
| - | "flex h-7 w-7 items-center justify-center rounded-full font-mono text-[13px] tabular-nums underline", | |
| - | isToday | |
| - | ? "bg-signal text-ink" | |
| - | : evCount | |
| - | ? "text-paper" | |
| - | : "text-ash-2" | |
| - | )} | |
| - | > | |
| - | {day} | |
| - | </span> | |
| - | {evCount > 0 ? ( | |
| - | <div className="mt-1.5 flex flex-wrap gap-1"> | |
| - | {marks.slice(0, 3).map((m, i) => ( | |
| - | <span | |
| - | key={i} | |
| - | className="pill !px-1 !py-0" | |
| - | style={{ | |
| - | color: m.color, | |
| - | borderColor: `color-mix(in srgb, ${m.color} 45%, transparent)`, | |
| - | }} | |
| - | > | |
| - | {m.mark} | |
| - | </span> | |
| - | ))} | |
| - | {marks.length > 3 ? ( | |
| - | <span className="label !text-[10px] text-ash-2"> | |
| - | +{evCount - marks.slice(0, 3).reduce((s, _, i) => s + (evs[i]?.count ?? 0), 0)} | |
| - | </span> | |
| - | ) : evCount > marks.length ? ( | |
| - | <span className="label !text-[10px] text-ash-2"> | |
| - | +{evCount - evs.filter((_, i) => i < 3).reduce((s, e) => s + e.count, 0)} | |
| - | </span> | |
| - | ) : null} | |
| - | </div> | |
| - | ) : null} | |
| - | </Link> | |
| - | </td> | |
| - | ); | |
| - | })} | |
| - | </tr> | |
| - | ))} | |
| - | </tbody> | |
| - | </table> | |
| - | <p className="label border-t border-line px-3 py-2 text-ash-2"> | |
| - | {filteredSummary.reduce((s, e) => s + e.count, 0)} event{filteredSummary.reduce((s, e) => s + e.count, 0) === 1 ? "" : "s"} this month. Select a day to read them. | |
| - | </p> | |
| - | </Card> | |
| - | ||
| - | {/* MOBILE AGENDA - day summary list, replaces the grid on phones. */} | |
| - | <div className="md:hidden"> | |
| - | <MobileAgenda | |
| - | daySummary={filteredSummary} | |
| - | kindByValue={KIND_BY_VALUE} | |
| - | selectedDay={selectedDay} | |
| - | dayEvents={dayEvents} | |
| - | year={year} | |
| - | month={month} | |
| - | kindFilter={kindFilter} | |
| - | /> | |
| - | </div> | |
| - | </div> | |
| - | ||
| - | {/* SIDE: legend + selected day */} | |
| - | <div className="space-y-6"> | |
| - | <Card className="p-5"> | |
| - | <p className="label mb-3">Legend</p> | |
| - | <ul className="space-y-2"> | |
| - | {KINDS.filter((k, i) => KINDS.findIndex((x) => x.mark === k.mark) === i).map((k) => ( | |
| - | <li key={k.value} className="flex items-center gap-2.5"> | |
| - | <span | |
| - | aria-hidden="true" | |
| - | className="inline-block h-2.5 w-2.5 shrink-0 rounded-full" | |
| - | style={{ backgroundColor: k.color }} | |
| - | /> | |
| - | <span className="font-mono text-[12px] text-paper">{k.plain}</span> | |
| - | <span className="label text-ash-2"> | |
| - | {kindCounts.get(k.value) ?? 0} | |
| - | </span> | |
| - | </li> | |
| - | ))} | |
| - | </ul> | |
| - | <p className="label mt-4 text-ash-2"> | |
| - | Colors are a cue only. Each mark carries its name, so the grid | |
| - | reads clearly in Plain mode too. | |
| - | </p> | |
| - | </Card> | |
| - | ||
| - | <div> | |
| - | <SectionHeader | |
| - | eyebrow={selectedDay ? "Selected day" : "Pick a day"} | |
| - | title={selectedDay ? formatLong(selectedDay) : "Day events"} | |
| - | /> | |
| - | {!selectedDay ? ( | |
| - | <Card className="p-5"> | |
| - | <p className="font-mono text-[13px] text-ash-2"> | |
| - | Select a day in the grid to read its events. Each event links | |
| - | to its official source. | |
| - | </p> | |
| - | </Card> | |
| - | ) : dayEvents.length === 0 ? ( | |
| - | <Card className="p-5"> | |
| - | <p className="font-mono text-[13px] text-ash-2"> | |
| - | No events on record for {formatLong(selectedDay)} | |
| - | {kindFilter ? " for this filter" : ""}. | |
| - | </p> | |
| - | </Card> | |
| - | ) : ( | |
| - | <Card className="divide-y divide-[color:var(--dk-line)]"> | |
| - | {dayEvents.map((e) => { | |
| - | const k = KIND_BY_VALUE.get(e.kind); | |
| - | return ( | |
| - | <div key={e.id} className="p-4"> | |
| - | <div className="flex flex-wrap items-center gap-2"> | |
| - | {k ? ( | |
| - | <span | |
| - | className="pill" | |
| - | style={{ | |
| - | color: k.color, | |
| - | borderColor: `color-mix(in srgb, ${k.color} 45%, transparent)`, | |
| - | }} | |
| - | > | |
| - | {k.plain} | |
| - | </span> | |
| - | ) : ( | |
| - | <span className="pill text-ash">{e.kind}</span> | |
| - | )} | |
| - | <span className="label text-ash-2">{e.refType}</span> | |
| - | </div> | |
| - | <p className="mt-2 font-mono text-[12px] leading-relaxed text-paper"> | |
| - | {e.href ? ( | |
| - | <Link | |
| - | href={e.href} | |
| - | className="focus-ring rounded-sm underline underline-offset-4 hover:text-paper" | |
| - | > | |
| - | {e.title} | |
| - | </Link> | |
| - | ) : ( | |
| - | e.title | |
| - | )} | |
| - | </p> | |
| - | <div className="mt-2"> | |
| - | <SourceLink url={e.sourceUrl} /> | |
| - | </div> | |
| - | </div> | |
| - | ); | |
| - | })} | |
| - | </Card> | |
| - | )} | |
| - | </div> | |
| - | </div> | |
| - | </div> | |
| + | <Suspense fallback={<CalendarSkeleton />}> | |
| + | <CalendarBrowser /> | |
| + | </Suspense> | |
| <div className="mt-12 border-t border-line pt-6"> | ||
| <p className="label text-ash-2"> | ||
| @@ -427,196 +58,14 @@ export default function CalendarPage({ | ||
| ); | ||
| } | ||
| - | /* ---------------- helpers ---------------- */ | |
| - | ||
| - | function hrefFor( | |
| - | ym: { year: number; month: number }, | |
| - | day: string | undefined, | |
| - | kind: string | undefined | |
| - | ): string { | |
| - | const params = new URLSearchParams(); | |
| - | params.set("month", `${ym.year}-${pad(ym.month)}`); | |
| - | if (day) params.set("day", day); | |
| - | if (kind) params.set("kind", kind); | |
| - | return `/calendar?${params.toString()}`; | |
| - | } | |
| - | ||
| - | function todayYear(): number { | |
| - | return new Date().getUTCFullYear(); | |
| - | } | |
| - | function todayMonth(): number { | |
| - | return new Date().getUTCMonth() + 1; | |
| - | } | |
| - | ||
| - | function chunk<T>(arr: T[], size: number): T[][] { | |
| - | const out: T[][] = []; | |
| - | for (let i = 0; i < arr.length; i += size) out.push(arr.slice(i, i + size)); | |
| - | return out; | |
| - | } | |
| - | ||
| - | /** Distinct kind marks for a day cell (uses condensed summary). */ | |
| - | function distinctMarks(evs: CalendarDayKind[]): KindMeta[] { | |
| - | const seen = new Set<string>(); | |
| - | const out: KindMeta[] = []; | |
| - | for (const e of evs) { | |
| - | const m = KIND_BY_VALUE.get(e.kind); | |
| - | if (m && !seen.has(m.mark)) { | |
| - | seen.add(m.mark); | |
| - | out.push(m); | |
| - | } | |
| - | } | |
| - | return out; | |
| - | } | |
| - | ||
| - | /** Total event count for a day cell (sums counts across kinds). */ | |
| - | function dayEventCount(evs: CalendarDayKind[]): number { | |
| - | return evs.reduce((s, e) => s + e.count, 0); | |
| - | } | |
| - | ||
| - | /** Kind counts from the condensed day summary (used by the legend). */ | |
| - | function countKindsFromSummary(summary: CalendarDayKind[]): Map<CalendarKind, number> { | |
| - | const m = new Map<CalendarKind, number>(); | |
| - | for (const s of summary) m.set(s.kind, (m.get(s.kind) ?? 0) + s.count); | |
| - | return m; | |
| - | } | |
| - | ||
| - | function formatLong(iso: string): string { | |
| - | const d = new Date(iso + "T00:00:00Z"); | |
| - | if (Number.isNaN(d.getTime())) return iso; | |
| - | return d.toLocaleDateString("en-US", { | |
| - | timeZone: "UTC", | |
| - | weekday: "long", | |
| - | year: "numeric", | |
| - | month: "long", | |
| - | day: "numeric", | |
| - | }); | |
| - | } | |
| - | ||
| - | /** | |
| - | * Mobile agenda - shows per-day event counts and kind dots for the month, | |
| - | * with full event details for the selected day. Uses the condensed day summary | |
| - | * to avoid serializing thousands of full CalendarEvent objects into the page. | |
| - | * Tap a day row to navigate to it (URL-driven, same as desktop grid). | |
| - | */ | |
| - | function MobileAgenda({ | |
| - | daySummary, | |
| - | kindByValue, | |
| - | selectedDay, | |
| - | dayEvents, | |
| - | year, | |
| - | month, | |
| - | kindFilter, | |
| - | }: { | |
| - | daySummary: CalendarDayKind[]; | |
| - | kindByValue: Map<CalendarKind, KindMeta>; | |
| - | selectedDay: string | undefined; | |
| - | dayEvents: CalendarEvent[]; | |
| - | year: number; | |
| - | month: number; | |
| - | kindFilter: string | undefined; | |
| - | }) { | |
| - | // Group summary by date. | |
| - | const byDate = new Map<string, CalendarDayKind[]>(); | |
| - | for (const s of daySummary) { | |
| - | const arr = byDate.get(s.date) ?? []; | |
| - | arr.push(s); | |
| - | byDate.set(s.date, arr); | |
| - | } | |
| - | const days = Array.from(byDate.keys()).sort(); | |
| - | ||
| - | if (days.length === 0) { | |
| - | return ( | |
| - | <Card className="p-5"> | |
| - | <p className="font-mono text-[13px] text-ash-2"> | |
| - | No events on record for this month. Month navigation above moves to | |
| - | other months; each event will link to its official source. | |
| - | </p> | |
| - | </Card> | |
| - | ); | |
| - | } | |
| - | ||
| + | function CalendarSkeleton() { | |
| return ( | ||
| - | <div className="space-y-2"> | |
| - | {days.map((d) => { | |
| - | const evs = byDate.get(d)!; | |
| - | const count = dayEventCount(evs); | |
| - | const marks = distinctMarks(evs); | |
| - | const isSelected = d === selectedDay; | |
| - | return ( | |
| - | <section key={d}> | |
| - | <Link | |
| - | href={hrefFor({ year, month }, d, kindFilter)} | |
| - | className={`focus-ring block rounded-[3px] border p-3 no-underline hover:bg-graphite/50 ${isSelected ? "border-paper bg-graphite" : "border-line"}`} | |
| - | aria-current={isSelected ? "true" : undefined} | |
| - | > | |
| - | <div className="flex items-center justify-between gap-3"> | |
| - | <span className="font-mono text-[13px] text-paper">{formatLong(d)}</span> | |
| - | <span className="label text-ash-2 shrink-0">{count} event{count === 1 ? "" : "s"}</span> | |
| - | </div> | |
| - | <div className="mt-1.5 flex flex-wrap gap-1"> | |
| - | {marks.map((m, i) => ( | |
| - | <span | |
| - | key={i} | |
| - | className="pill !px-1.5 !py-0" | |
| - | style={{ | |
| - | color: m.color, | |
| - | borderColor: `color-mix(in srgb, ${m.color} 45%, transparent)`, | |
| - | }} | |
| - | > | |
| - | {m.plain} | |
| - | </span> | |
| - | ))} | |
| - | </div> | |
| - | </Link> | |
| - | {/* Inline expanded view for the selected day */} | |
| - | {isSelected && dayEvents.length > 0 && ( | |
| - | <div className="mt-2 space-y-2 pl-3"> | |
| - | {dayEvents.map((e) => { | |
| - | const k = kindByValue.get(e.kind); | |
| - | return ( | |
| - | <article | |
| - | key={e.id} | |
| - | className="rounded-[3px] border border-line bg-surface p-3" | |
| - | > | |
| - | <div className="flex flex-wrap items-center gap-2"> | |
| - | {k ? ( | |
| - | <span | |
| - | className="pill !py-0.5" | |
| - | style={{ | |
| - | color: k.color, | |
| - | borderColor: `color-mix(in srgb, ${k.color} 45%, transparent)`, | |
| - | }} | |
| - | > | |
| - | {k.plain} | |
| - | </span> | |
| - | ) : ( | |
| - | <span className="pill !py-0.5 text-ash">{e.kind}</span> | |
| - | )} | |
| - | <span className="label text-ash-2">{e.refType}</span> | |
| - | </div> | |
| - | <p className="mt-2 break-words font-mono text-[12px] leading-relaxed text-paper"> | |
| - | {e.href ? ( | |
| - | <Link | |
| - | href={e.href} | |
| - | className="focus-ring rounded-sm underline underline-offset-4 hover:text-paper" | |
| - | > | |
| - | {e.title} | |
| - | </Link> | |
| - | ) : ( | |
| - | e.title | |
| - | )} | |
| - | </p> | |
| - | <div className="mt-2"> | |
| - | <SourceLink url={e.sourceUrl} /> | |
| - | </div> | |
| - | </article> | |
| - | ); | |
| - | })} | |
| - | </div> | |
| - | )} | |
| - | </section> | |
| - | ); | |
| - | })} | |
| + | <div className="mt-8 space-y-6"> | |
| + | <div className="h-12 w-full animate-pulse rounded-sm bg-graphite" /> | |
| + | <div className="grid gap-6 lg:grid-cols-[1.5fr_1fr]"> | |
| + | <Card className="hidden h-[28rem] animate-pulse bg-graphite md:block" /> | |
| + | <div className="h-[28rem] animate-pulse rounded-sm bg-graphite" /> | |
| + | </div> | |
| </div> | ||
| ); | ||
| } |
| @@ -1,4 +1,5 @@ | ||
| import type { Metadata } from "next"; | ||
| + | import Link from "next/link"; | |
| import { Card, PageHeader, SectionHeader } from "@/components/ui"; | ||
| import { SourceLink } from "@/components/source-link"; | ||
| import { ExecutiveBrowser } from "@/components/executive-browser"; | ||
| @@ -30,7 +31,51 @@ export default function ExecutivePage() { | ||
| }, {}); | ||
| return ( | ||
| - | <div className="mx-auto w-full max-w-register px-4 py-10 sm:px-6"> | |
| + | <div | |
| + | className="mx-auto w-full max-w-[860px] px-6 sm:px-8" | |
| + | style={{ | |
| + | paddingTop: "clamp(32px, 6vw, 56px)", | |
| + | paddingBottom: "clamp(48px, 8vw, 80px)", | |
| + | }} | |
| + | > | |
| + | {/* Breadcrumb + Back */} | |
| + | <nav aria-label="Breadcrumb" className="mb-10"> | |
| + | <ol className="flex flex-wrap items-center gap-2" style={{ marginBottom: "12px" }}> | |
| + | <li> | |
| + | <Link | |
| + | href="/" | |
| + | className="focus-ring label no-underline hover:underline" | |
| + | style={{ color: "var(--dk-signal)", fontSize: "12px", letterSpacing: "0.12em" }} | |
| + | > | |
| + | Home | |
| + | </Link> | |
| + | </li> | |
| + | <li aria-hidden="true" className="label" style={{ color: "var(--dk-ash-2)", fontSize: "12px" }}>/</li> | |
| + | <li | |
| + | className="label" | |
| + | style={{ color: "var(--dk-ash)", fontSize: "12px", letterSpacing: "0.12em" }} | |
| + | aria-current="page" | |
| + | > | |
| + | Federal Register | |
| + | </li> | |
| + | </ol> | |
| + | <Link | |
| + | href="/" | |
| + | className="focus-ring tap no-underline hover:underline" | |
| + | style={{ | |
| + | display: "inline-flex", | |
| + | alignItems: "center", | |
| + | gap: "8px", | |
| + | fontFamily: "var(--font-mono), ui-monospace, monospace", | |
| + | fontSize: "13px", | |
| + | letterSpacing: "0.08em", | |
| + | color: "var(--dk-signal)", | |
| + | }} | |
| + | > | |
| + | ← Back to home | |
| + | </Link> | |
| + | </nav> | |
| + | ||
| <PageHeader | ||
| eyebrow="Federal Register" | ||
| title="Executive actions" | ||
| @@ -58,8 +103,14 @@ export default function ExecutivePage() { | ||
| </p> | ||
| {/* Cite this page */} | ||
| - | <div className="mt-12 border-t border-line pt-8"> | |
| - | <p className="label mb-4 text-ash">Cite this page</p> | |
| + | <div | |
| + | style={{ | |
| + | marginTop: "clamp(40px, 7vw, 72px)", | |
| + | borderTop: "1px solid var(--dk-line)", | |
| + | paddingTop: "clamp(24px, 4vw, 40px)", | |
| + | }} | |
| + | > | |
| + | <p className="label mb-4" style={{ color: "var(--dk-ash)" }}>Cite this page</p> | |
| <CiteThis | ||
| kind="page" | ||
| title="Federal Register - Executive actions" |
| @@ -3,14 +3,14 @@ | ||
| @tailwind utilities; | ||
| /* | ||
| - | * Docket design tokens - source of truth. | |
| + | * Docket design tokens - source of truth. | |
| * Dark (default): Zion's ink treatment, brass accent (docs/DESIGN_V3.md). | ||
| - | * Light: warm paper day version - same fonts/composition, light counterpart. | |
| + | * Light: warm paper day version - same fonts/composition, light counterpart. | |
| * Plain mode removed (V3b, 2026-06-27). | ||
| */ | ||
| /* ------------------------------------------------------------------ */ | ||
| - | /* DARK MODE (:root default) - Zion's ink treatment, brass accent */ | |
| + | /* DARK MODE (:root default) - Zion's ink treatment, brass accent */ | |
| /* Matches treetrace.dev + zionboggan.com dark aesthetic. */ | ||
| /* ------------------------------------------------------------------ */ | ||
| :root { | ||
| @@ -22,17 +22,17 @@ | ||
| --dk-ash-2: #6e7480; /* tertiary (tt-ash-2) */ | ||
| --dk-line: rgba(155, 160, 166, 0.18); | ||
| - | /* signal - BRASS #E0A53B (neutral; cobalt/red are partisan-coded) */ | |
| + | /* signal - BRASS #E0A53B (neutral; cobalt/red are partisan-coded) */ | |
| --dk-signal: #E0A53B; | ||
| --dk-signal-dim: #B07F25; | ||
| --dk-signal-tint: rgba(224, 165, 59, 0.10); | ||
| - | /* semantic - muted, never vivid (neutrality) */ | |
| + | /* semantic - muted, never vivid (neutrality) */ | |
| --dk-yea: #7fa88c; /* muted sage, always labeled */ | ||
| --dk-nay: #b98a8a; /* muted clay, always labeled */ | ||
| --dk-hold: #c9a66b; | ||
| - | /* party tones - desaturated, EQUAL visual weight */ | |
| + | /* party tones - desaturated, EQUAL visual weight */ | |
| --dk-party-d: #6e86a8; | ||
| --dk-party-r: #a87e6e; | ||
| --dk-party-i: #8a8f96; | ||
| @@ -50,8 +50,8 @@ | ||
| } | ||
| /* ------------------------------------------------------------------ */ | ||
| - | /* LIGHT MODE - day counterpart of the V3 dark ink theme */ | |
| - | /* Same fonts, composition, and brass accent - warm paper palette. */ | |
| + | /* LIGHT MODE - day counterpart of the V3 dark ink theme */ | |
| + | /* Same fonts, composition, and brass accent - warm paper palette. */ | |
| /* docs/DESIGN_V3.md: white page, ink text, darkened brass (#8F5902) */ | ||
| /* Contrast: text #0B0C0E on white ~18:1; brass #8F5902 on white ~5.8:1 (AA pass) */ | ||
| /* ------------------------------------------------------------------ */ | ||
| @@ -64,17 +64,17 @@ | ||
| --dk-ash-2: #6b6860; /* tertiary text */ | ||
| --dk-line: #d8d2c4; /* warm visible border */ | ||
| - | /* accent - brass darkened to #8F5902 for AA on light (5.8:1 on white) */ | |
| + | /* accent - brass darkened to #8F5902 for AA on light (5.8:1 on white) */ | |
| --dk-signal: #8f5902; | ||
| - | --dk-signal-dim: #6b4100; /* accent-strong - large fills, extra dark */ | |
| - | --dk-signal-tint: #fbf3df; /* accent-wash - active/selected bg */ | |
| + | --dk-signal-dim: #6b4100; /* accent-strong - large fills, extra dark */ | |
| + | --dk-signal-tint: #fbf3df; /* accent-wash - active/selected bg */ | |
| - | /* semantic - muted, always with label (neutrality) */ | |
| + | /* semantic - muted, always with label (neutrality) */ | |
| --dk-yea: #3d6b00; /* pass, always with label */ | ||
| --dk-nay: #8a1c1c; /* fail, small only */ | ||
| --dk-hold: #8f5902; /* stalled/pending */ | ||
| - | /* party tones - desaturated, equal visual weight */ | |
| + | /* party tones - desaturated, equal visual weight */ | |
| --dk-party-d: #5a7499; | ||
| --dk-party-r: #996355; | ||
| --dk-party-i: #7a7f86; | ||
| @@ -85,7 +85,7 @@ | ||
| rgba(143, 89, 2, 0) 70% | ||
| ); | ||
| - | /* Match dark mode body size - day version, same rhythm */ | |
| + | /* Match dark mode body size - day version, same rhythm */ | |
| --dk-body-size: 18px; | ||
| --dk-leading: 1.6; | ||
| color-scheme: light; | ||
| @@ -108,7 +108,7 @@ | ||
| } | ||
| /* | ||
| - | * Light mode: same JetBrains Mono body as dark - it's the day version of | |
| + | * Light mode: same JetBrains Mono body as dark - it's the day version of | |
| * the same theme. Font roles are identical: Archivo for display headlines, | ||
| * JetBrains Mono for labels/data/body, Fraunces for prose leads. | ||
| * (docs/DESIGN_V3.md §Fonts) | ||
| @@ -122,7 +122,7 @@ | ||
| } | ||
| /* ------------------------------------------------------------------ */ | ||
| - | /* Accessibility enforcement (docs/ACCESSIBILITY.md - the 8 rules). */ | |
| + | /* Accessibility enforcement (docs/ACCESSIBILITY.md - the 8 rules). */ | |
| /* ------------------------------------------------------------------ */ | ||
| @layer a11y { | ||
| /* Rule #4: visible focus on EVERY interactive element. */ | ||
| @@ -207,28 +207,29 @@ | ||
| } | ||
| /* | ||
| - | * Kicker / data label - JetBrains Mono, uppercase, tracked. | |
| + | * Kicker / data label - JetBrains Mono, uppercase, tracked. | |
| * Matches treetrace/zionboggan mono label pattern (>=14px per DESIGN_V3). | ||
| - | * Light mode keeps the same mono style - it's the brand voice (day version). | |
| + | * Light mode keeps the same mono style - it's the brand voice (day version). | |
| */ | ||
| .label { | ||
| font-family: var(--font-mono), ui-monospace, monospace; | ||
| text-transform: uppercase; | ||
| letter-spacing: 0.12em; | ||
| - | font-size: 0.875rem; /* 14px - DESIGN_V3: >=14px for labels */ | |
| + | font-size: 0.875rem; /* 14px - DESIGN_V3: >=14px for labels */ | |
| color: var(--dk-ash); | ||
| } | ||
| - | /* Display headline - Archivo 800-900, uppercase. Matches both sites. */ | |
| + | /* Display headline - Archivo 800-900, uppercase. Matches both sites. */ | |
| .display { | ||
| font-family: var(--font-display), system-ui, sans-serif; | ||
| font-weight: 900; | ||
| text-transform: uppercase; | ||
| - | letter-spacing: -0.03em; | |
| - | line-height: 0.92; | |
| + | letter-spacing: -0.01em; | |
| + | word-spacing: 0.06em; | |
| + | line-height: 1.0; | |
| } | ||
| - | /* Prose lead - Fraunces serif. Comfortable >=17px per DESIGN_V3. */ | |
| + | /* Prose lead - Fraunces serif. Comfortable >=17px per DESIGN_V3. */ | |
| .prose-lead { | ||
| font-family: var(--font-prose), Georgia, serif; | ||
| font-size: 1.125rem; /* 18px */ | ||
| @@ -274,13 +275,13 @@ | ||
| background: var(--dk-surface); | ||
| } | ||
| - | /* Focus ring - signal color. */ | |
| + | /* Focus ring - signal color. */ | |
| .focus-ring:focus-visible { | ||
| outline: 3px solid var(--dk-signal); | ||
| outline-offset: 2px; | ||
| } | ||
| - | /* Big interactive card - used on list pages. */ | |
| + | /* Big interactive card - used on list pages. */ | |
| .big-card { | ||
| display: block; | ||
| border-radius: 6px; | ||
| @@ -294,7 +295,7 @@ | ||
| border-color: var(--dk-signal); | ||
| } | ||
| - | /* Section card - home page entry cards. */ | |
| + | /* Section card - home page entry cards. */ | |
| .section-card { | ||
| display: block; | ||
| border-radius: 6px; |
| @@ -1,4 +1,5 @@ | ||
| import type { Metadata } from "next"; | ||
| + | import Link from "next/link"; | |
| import { Card, PageHeader, SectionHeader } from "@/components/ui"; | ||
| export const metadata: Metadata = { | ||
| @@ -9,7 +10,29 @@ export const metadata: Metadata = { | ||
| export default function HowWePickPage() { | ||
| return ( | ||
| - | <div className="mx-auto w-full max-w-register px-4 py-10 sm:px-6"> | |
| + | <div | |
| + | className="mx-auto w-full max-w-[860px] px-6 sm:px-8" | |
| + | style={{ | |
| + | paddingTop: "clamp(32px, 6vw, 56px)", | |
| + | paddingBottom: "clamp(48px, 8vw, 80px)", | |
| + | }} | |
| + | > | |
| + | <Link | |
| + | href="/" | |
| + | className="focus-ring tap no-underline hover:underline" | |
| + | style={{ | |
| + | display: "inline-flex", | |
| + | alignItems: "center", | |
| + | gap: "8px", | |
| + | fontFamily: "var(--font-mono), ui-monospace, monospace", | |
| + | fontSize: "13px", | |
| + | letterSpacing: "0.08em", | |
| + | color: "var(--dk-signal)", | |
| + | marginBottom: "32px", | |
| + | }} | |
| + | > | |
| + | ← Home | |
| + | </Link> | |
| <PageHeader | ||
| eyebrow="Trust · methodology" | ||
| title="How we pick what to show" |
| @@ -6,9 +6,9 @@ import { SiteFooter } from "@/components/site-footer"; | ||
| /* | ||
| * Type trio (docs/DESIGN_V2.md §Type): | ||
| - | * Archivo - headings, display, UI labels (var --font-display) | |
| - | * Fraunces - leads, long-form reading (var --font-prose) | |
| - | * JetBrains - DATA ONLY: tallies, numbers (var --font-mono) | |
| + | * Archivo - headings, display, UI labels (var --font-display) | |
| + | * Fraunces - leads, long-form reading (var --font-prose) | |
| + | * JetBrains - DATA ONLY: tallies, numbers (var --font-mono) | |
| */ | ||
| const archivo = Archivo({ | ||
| subsets: ["latin"], | ||
| @@ -43,6 +43,7 @@ export const metadata: Metadata = { | ||
| authors: [{ name: "Docket" }], | ||
| icons: { | ||
| icon: "/favicon.svg", | ||
| + | apple: "/apple-touch-icon.png", | |
| }, | ||
| }; | ||
| @@ -66,10 +67,10 @@ export default function RootLayout({ | ||
| > | ||
| <head> | ||
| {/* | ||
| - | Apply the persisted display mode before paint - dark (default, no | |
| + | Apply the persisted display mode before paint - dark (default, no | |
| class on <html>) or light. Plain mode removed in V3b (2026-06-27). | ||
| Any legacy "plain" value in storage falls back to "dark". | ||
| - | Runs synchronously so the first frame is correct - no flash. | |
| + | Runs synchronously so the first frame is correct - no flash. | |
| suppressHydrationWarning lets the inline script own the class. | ||
| */} | ||
| <script |
| @@ -32,6 +32,13 @@ export function generateMetadata({ | ||
| }; | ||
| } | ||
| + | /** District display helper - fix #11 */ | |
| + | function formatDistrict(chamber: string, district: number | null): string { | |
| + | if (chamber === "senate") return ""; | |
| + | if (district === null || district <= 0) return "At-Large"; | |
| + | return `District ${district}`; | |
| + | } | |
| + | ||
| export default function MemberDetailPage({ | ||
| params, | ||
| }: { | ||
| @@ -41,18 +48,69 @@ export default function MemberDetailPage({ | ||
| if (!member) notFound(); | ||
| return ( | ||
| - | <div className="mx-auto w-full max-w-register overflow-x-hidden px-4 py-10 sm:px-6"> | |
| + | <div | |
| + | className="mx-auto w-full max-w-[860px] overflow-x-hidden px-6 sm:px-8" | |
| + | style={{ | |
| + | paddingTop: "clamp(32px, 6vw, 56px)", | |
| + | paddingBottom: "clamp(48px, 8vw, 80px)", | |
| + | }} | |
| + | > | |
| <Link | ||
| href="/members" | ||
| - | className="focus-ring tap label mb-6 gap-1 text-ash hover:text-paper" | |
| + | className="focus-ring tap no-underline hover:underline" | |
| + | style={{ | |
| + | display: "inline-flex", | |
| + | alignItems: "center", | |
| + | gap: "8px", | |
| + | fontFamily: "var(--font-mono), ui-monospace, monospace", | |
| + | fontSize: "13px", | |
| + | letterSpacing: "0.08em", | |
| + | color: "var(--dk-signal)", | |
| + | marginBottom: "32px", | |
| + | }} | |
| > | ||
| - | ← Members | |
| + | ← Representatives | |
| </Link> | ||
| - | {/* Profile header - photo + name + party/state/chamber. The photo is a | |
| + | {/* Profile header - photo + name + party/state/chamber. The photo is a | |
| decorative aid (decorative alt=""); the cited record stays the | ||
| allowlisted bioguide Source link below (neutrality #1). */} | ||
| - | <header className="border-b border-line pb-8"> | |
| + | <header | |
| + | style={{ | |
| + | borderBottom: "1px solid var(--dk-line)", | |
| + | paddingBottom: "clamp(28px, 5vw, 48px)", | |
| + | marginBottom: "clamp(28px, 5vw, 48px)", | |
| + | }} | |
| + | > | |
| + | {/* Brass kicker */} | |
| + | <p | |
| + | className="label" | |
| + | style={{ | |
| + | color: "var(--dk-signal)", | |
| + | marginBottom: "20px", | |
| + | display: "flex", | |
| + | alignItems: "center", | |
| + | gap: "8px", | |
| + | fontSize: "12px", | |
| + | letterSpacing: "0.14em", | |
| + | }} | |
| + | > | |
| + | <span | |
| + | aria-hidden="true" | |
| + | style={{ | |
| + | width: "5px", | |
| + | height: "5px", | |
| + | borderRadius: "50%", | |
| + | background: "var(--dk-signal)", | |
| + | display: "inline-block", | |
| + | flexShrink: 0, | |
| + | }} | |
| + | /> | |
| + | {member.chamber === "senate" ? "U.S. Senate" : "U.S. House"} · {member.state} | |
| + | {member.chamber !== "senate" && formatDistrict(member.chamber, member.district) | |
| + | ? ` · ${formatDistrict(member.chamber, member.district)}` | |
| + | : ""} | |
| + | </p> | |
| <div className="flex flex-col gap-5 sm:flex-row sm:items-start"> | ||
| <MemberAvatar | ||
| name={member.name} | ||
| @@ -61,17 +119,21 @@ export default function MemberDetailPage({ | ||
| className="shrink-0" | ||
| /> | ||
| <div className="min-w-0 flex-1"> | ||
| - | <p className="label mb-3"> | |
| - | {member.chamber === "senate" ? "U.S. Senate" : "U.S. House"} · {member.state} | |
| - | {member.district != null ? ` · district ${member.district}` : ""} | |
| - | </p> | |
| <h1 | ||
| - | className="display text-paper" | |
| - | style={{ fontSize: "clamp(1.875rem, 7vw, 3rem)", lineHeight: 1 }} | |
| + | className="font-display text-paper" | |
| + | style={{ | |
| + | fontSize: "clamp(2.2rem, 7vw, 3.75rem)", | |
| + | fontWeight: 900, | |
| + | lineHeight: 1.0, | |
| + | letterSpacing: "-0.01em", | |
| + | wordSpacing: "0.06em", | |
| + | textTransform: "uppercase", | |
| + | marginBottom: "20px", | |
| + | }} | |
| > | ||
| {member.name} | ||
| </h1> | ||
| - | <div className="mt-4 flex flex-wrap items-center gap-3"> | |
| + | <div className="flex flex-wrap items-center gap-3"> | |
| <PartyTag party={member.partyCode} /> | ||
| <span className="font-mono text-[13px] text-ash"> | ||
| {member.party || "Party not on record"} | ||
| @@ -95,7 +157,7 @@ export default function MemberDetailPage({ | ||
| </header> | ||
| {/* top facts */} | ||
| - | <div className="mt-8 grid gap-4 sm:grid-cols-3"> | |
| + | <div className="grid gap-4 sm:grid-cols-3"> | |
| <Card className="p-5"> | ||
| <p className="label mb-2">Party</p> | ||
| <PartyTag party={member.partyCode} /> | ||
| @@ -106,7 +168,9 @@ export default function MemberDetailPage({ | ||
| <p className="label mb-2">State / district</p> | ||
| <p className="font-mono text-[13px] text-paper">{member.state}</p> | ||
| <p className="label mt-2 text-ash-2"> | ||
| - | {member.district != null ? `District ${member.district}` : "Statewide"} | |
| + | {member.chamber === "senate" | |
| + | ? "Statewide" | |
| + | : formatDistrict(member.chamber, member.district) || "Statewide"} | |
| </p> | ||
| </Card> | ||
| @@ -118,10 +182,12 @@ export default function MemberDetailPage({ | ||
| </div> | ||
| <div className="mt-10 grid gap-8 lg:grid-cols-[1.3fr_1fr]"> | ||
| - | {/* RECENT VOTES - how they voted */} | |
| - | <div> | |
| + | {/* RECENT VOTES - how they voted. Compact recent snapshot only; the | |
| + | full record lives in the paginated year-by-year section below, so | |
| + | this stays short and never balloons the page. */} | |
| + | <div className="min-w-0"> | |
| <SectionHeader | ||
| - | eyebrow={`Recent votes · ${member.votes.length} shown`} | |
| + | eyebrow={`Most recent · ${Math.min(member.votes.length, 12)} of ${member.voteCount.toLocaleString()}`} | |
| title="How they voted" | ||
| /> | ||
| {member.votes.length === 0 ? ( | ||
| @@ -131,12 +197,12 @@ export default function MemberDetailPage({ | ||
| </p> | ||
| </Card> | ||
| ) : ( | ||
| - | <MemberVotesTable votes={member.votes} /> | |
| + | <RecentVotes votes={member.votes.slice(0, 12)} /> | |
| )} | ||
| </div> | ||
| {/* SPONSORED BILLS */} | ||
| - | <div> | |
| + | <div className="min-w-0"> | |
| <SectionHeader | ||
| eyebrow={`Sponsored · ${member.sponsorships.length}`} | ||
| title="Bills introduced" | ||
| @@ -171,7 +237,7 @@ export default function MemberDetailPage({ | ||
| </div> | ||
| </div> | ||
| - | {/* VOTE RECORD BY YEAR - the year-by-year headline view */} | |
| + | {/* VOTE RECORD BY YEAR - the year-by-year headline view */} | |
| <div className="mt-10"> | ||
| <SectionHeader | ||
| eyebrow={`By year · ${member.votesByYear.length} year${member.votesByYear.length === 1 ? "" : "s"}`} | ||
| @@ -186,7 +252,7 @@ export default function MemberDetailPage({ | ||
| </Card> | ||
| ) : ( | ||
| <> | ||
| - | {/* Per-year count chips - quick summary at a glance. */} | |
| + | {/* Per-year count chips - quick summary at a glance. */} | |
| <ul className="mb-5 flex flex-wrap gap-2"> | ||
| {member.votesByYear.map((yg) => ( | ||
| <li | ||
| @@ -247,47 +313,34 @@ export default function MemberDetailPage({ | ||
| ); | ||
| } | ||
| - | /** A member's votes as a responsive table (>=md) / stacked cards (<md). | |
| - | * `dense` collapses the chamber column for the year-by-year sections. */ | |
| - | function MemberVotesTable({ | |
| - | votes, | |
| - | dense = false, | |
| - | }: { | |
| - | votes: MemberVote[]; | |
| - | dense?: boolean; | |
| - | }) { | |
| - | const columns: Column[] = [ | |
| - | { key: "date", header: "Date" }, | |
| - | ...(dense ? [] : [{ key: "chamber", header: "Chamber", hideOnCard: true } as Column]), | |
| - | { key: "result", header: "Question", cardTitle: true }, | |
| - | { key: "voted", header: "Voted" }, | |
| - | ]; | |
| - | const rows: Row[] = votes.map((v) => ({ | |
| - | key: v.voteId, | |
| - | cells: { | |
| - | date: <span className="tabular-nums text-ash">{formatDate(v.date)}</span>, | |
| - | chamber: <span className="uppercase text-ash-2">{v.chamber}</span>, | |
| - | result: ( | |
| - | <span className="text-paper"> | |
| - | <Link | |
| - | href={v.voteHref} | |
| - | className="focus-ring rounded-sm underline underline-offset-4 hover:text-paper" | |
| - | > | |
| - | {v.question || v.result} | |
| - | </Link> | |
| - | <span className="block text-ash-2">{v.result}</span> | |
| - | </span> | |
| - | ), | |
| - | voted: <PositionTag position={v.position} />, | |
| - | }, | |
| - | })); | |
| + | /** Compact recent-votes snapshot - a stacked list (not a wide table) so it fits | |
| + | * the narrow grid column cleanly and never balloons each row. Each row carries | |
| + | * the question (link), date, and the position as a muted, TEXT-labeled tag | |
| + | * (never color alone). The full record is in the year-by-year section below. */ | |
| + | function RecentVotes({ votes }: { votes: MemberVote[] }) { | |
| return ( | ||
| - | <DataTable | |
| - | caption="This member's recorded votes: date, chamber, question, result, position." | |
| - | cardLabel="Recorded votes" | |
| - | columns={columns} | |
| - | rows={rows} | |
| - | /> | |
| + | <Card className="divide-y divide-[color:var(--dk-line)]"> | |
| + | {votes.map((v) => ( | |
| + | <div key={v.voteId} className="flex items-start justify-between gap-3 p-4"> | |
| + | <div className="min-w-0 flex-1"> | |
| + | <Link | |
| + | href={v.voteHref} | |
| + | className="focus-ring block break-words font-mono text-[13px] text-paper underline underline-offset-4 hover:text-paper" | |
| + | > | |
| + | {v.question || v.result} | |
| + | </Link> | |
| + | <p className="mt-1 flex flex-wrap items-center gap-x-2 gap-y-1"> | |
| + | <span className="label text-ash-2">{v.chamber}</span> | |
| + | <span className="label text-ash-2" aria-hidden="true">·</span> | |
| + | <span className="label tabular-nums text-ash-2">{formatDate(v.date)}</span> | |
| + | <span className="label text-ash-2" aria-hidden="true">·</span> | |
| + | <span className="label text-ash-2">{v.result}</span> | |
| + | </p> | |
| + | </div> | |
| + | <PositionTag position={v.position} className="mt-0.5 shrink-0" /> | |
| + | </div> | |
| + | ))} | |
| + | </Card> | |
| ); | ||
| } | ||
| @@ -1,4 +1,5 @@ | ||
| import type { Metadata } from "next"; | ||
| + | import Link from "next/link"; | |
| import { | ||
| getMembersDefault, | ||
| getMemberCounts, | ||
| @@ -17,25 +18,69 @@ export const metadata: Metadata = { | ||
| export const revalidate = 300; | ||
| export default function MembersPage() { | ||
| - | // Use cheap COUNT queries for the summary cards - no need to serialize | |
| + | // Use cheap COUNT queries for the summary cards - no need to serialize | |
| // 12.7k member objects just to count them. | ||
| const counts = getMemberCounts(); | ||
| // Only pass the default-visible subset (current Congress + members with | ||
| - | // photos) to the client - keeps the SSR HTML payload small. Historical | |
| + | // photos) to the client - keeps the SSR HTML payload small. Historical | |
| // members without photos are loaded lazily when the toggle is clicked. | ||
| const defaultMembers = getMembersDefault(); | ||
| const hiddenCount = counts.total - defaultMembers.length; | ||
| const states = getMemberStates(); | ||
| return ( | ||
| - | <div className="mx-auto w-full max-w-register px-4 py-10 sm:px-6"> | |
| + | <div | |
| + | className="mx-auto w-full max-w-[860px] px-6 sm:px-8" | |
| + | style={{ | |
| + | paddingTop: "clamp(32px, 6vw, 56px)", | |
| + | paddingBottom: "clamp(48px, 8vw, 80px)", | |
| + | }} | |
| + | > | |
| + | {/* Breadcrumb + Back */} | |
| + | <nav aria-label="Breadcrumb" className="mb-10"> | |
| + | <ol className="flex flex-wrap items-center gap-2" style={{ marginBottom: "12px" }}> | |
| + | <li> | |
| + | <Link | |
| + | href="/" | |
| + | className="focus-ring label no-underline hover:underline" | |
| + | style={{ color: "var(--dk-signal)", fontSize: "12px", letterSpacing: "0.12em" }} | |
| + | > | |
| + | Home | |
| + | </Link> | |
| + | </li> | |
| + | <li aria-hidden="true" className="label" style={{ color: "var(--dk-ash-2)", fontSize: "12px" }}>/</li> | |
| + | <li | |
| + | className="label" | |
| + | style={{ color: "var(--dk-ash)", fontSize: "12px", letterSpacing: "0.12em" }} | |
| + | aria-current="page" | |
| + | > | |
| + | Representatives | |
| + | </li> | |
| + | </ol> | |
| + | <Link | |
| + | href="/" | |
| + | className="focus-ring tap no-underline hover:underline" | |
| + | style={{ | |
| + | display: "inline-flex", | |
| + | alignItems: "center", | |
| + | gap: "8px", | |
| + | fontFamily: "var(--font-mono), ui-monospace, monospace", | |
| + | fontSize: "13px", | |
| + | letterSpacing: "0.08em", | |
| + | color: "var(--dk-signal)", | |
| + | }} | |
| + | > | |
| + | ← Back to home | |
| + | </Link> | |
| + | </nav> | |
| + | ||
| <PageHeader | ||
| eyebrow="Congress roster" | ||
| - | title="Members of Congress" | |
| + | title="Representatives" | |
| lead="Search and filter the sitting House and Senate. Open a member to see their recorded votes, sponsored bills, and amendments - what your representative did, on the record." | ||
| /> | ||
| - | <div className="mt-8"> | |
| + | <div> | |
| {/* Summary by chamber, ordered by count then alphabetical (not by party). */} | ||
| <div className="grid gap-4 sm:grid-cols-3"> | ||
| <SummaryCard label="Total" count={counts.total} /> | ||
| @@ -53,8 +98,14 @@ export default function MembersPage() { | ||
| </div> | ||
| {/* Cite this page */} | ||
| - | <div className="mt-12 border-t border-line pt-8"> | |
| - | <p className="label mb-4 text-ash">Cite this page</p> | |
| + | <div | |
| + | style={{ | |
| + | marginTop: "clamp(40px, 7vw, 72px)", | |
| + | borderTop: "1px solid var(--dk-line)", | |
| + | paddingTop: "clamp(24px, 4vw, 40px)", | |
| + | }} | |
| + | > | |
| + | <p className="label mb-4" style={{ color: "var(--dk-ash)" }}>Cite this page</p> | |
| <CiteThis | ||
| kind="page" | ||
| title="Members of Congress - Roster" |
| @@ -1,8 +1,10 @@ | ||
| import Link from "next/link"; | ||
| + | import { OwlAlt } from "@/components/owl"; | |
| export default function NotFound() { | ||
| return ( | ||
| <div className="mx-auto flex w-full max-w-register flex-col items-start px-4 py-24 sm:px-6"> | ||
| + | <OwlAlt size={140} className="mb-6" title="No record found" /> | |
| <p className="label mb-3">No record</p> | ||
| <h1 className="display text-4xl text-paper sm:text-5xl">404</h1> | ||
| <p className="prose-lead mt-4 max-w-xl text-ash"> |
| @@ -9,7 +9,7 @@ export const metadata: Metadata = { | ||
| export const revalidate = 3600; | ||
| - | /** Primary section cards - 6 main sections + the register feed */ | |
| + | /** Primary section cards - 6 main sections + the register feed */ | |
| const SECTIONS = [ | ||
| { | ||
| href: "/bills", | ||
| @@ -23,6 +23,12 @@ const SECTIONS = [ | ||
| title: "Votes", | ||
| desc: "House and Senate roll-call votes. See how every member voted, by name.", | ||
| }, | ||
| + | { | |
| + | href: "/laws", | |
| + | kicker: "Enacted", | |
| + | title: "Laws", | |
| + | desc: "Bills that have become law. Each Public Law links to the bill it came from and the official record.", | |
| + | }, | |
| { | ||
| href: "/members", | ||
| kicker: "Congress", | ||
| @@ -105,8 +111,9 @@ export default function HomePage() { | ||
| style={{ | ||
| fontSize: "clamp(2.8rem, 9vw, 5.5rem)", | ||
| fontWeight: 900, | ||
| - | lineHeight: 0.94, | |
| - | letterSpacing: "-0.03em", | |
| + | lineHeight: 1.0, | |
| + | letterSpacing: "-0.01em", | |
| + | wordSpacing: "0.06em", | |
| textTransform: "uppercase", | ||
| maxWidth: "16ch", | ||
| marginBottom: "32px", | ||
| @@ -128,12 +135,12 @@ export default function HomePage() { | ||
| }} | ||
| > | ||
| Bills, votes, and dollars straight from official government sources. | ||
| - | No spin. No opinion. Every fact links to its official{" "} | |
| - | <span | |
| - | className="font-mono not-italic" | |
| - | style={{ fontSize: "0.9em" }} | |
| - | > | |
| - | .gov | |
| + | No spin. No opinion. Every fact links to its{" "} | |
| + | <span style={{ whiteSpace: "nowrap" }}> | |
| + | official{" "} | |
| + | <span className="font-mono not-italic" style={{ fontSize: "0.9em" }}> | |
| + | .gov | |
| + | </span> | |
| </span>{" "} | ||
| source. | ||
| </p> |
| @@ -1,6 +1,6 @@ | ||
| /** Skeleton shown immediately on client-side navigation to /register | ||
| * while the RSC payload is in-flight. Matches the page's rough layout: | ||
| - | * a hero section + a data table. No imports needed - pure Tailwind. */ | |
| + | * a hero section + a data table. No imports needed - pure Tailwind. */ | |
| export default function Loading() { | ||
| return ( | ||
| <> |
| @@ -20,7 +20,7 @@ export const metadata: Metadata = { | ||
| "A live, stamped ledger of official US government actions - bills, votes, and dollars. Every fact sourced.", | ||
| }; | ||
| - | // Revalidate every 5 minutes - staleness is acceptable for this read-only | |
| + | // Revalidate every 5 minutes - staleness is acceptable for this read-only | |
| // public data site. The in-process DB cache in lib/feed.ts also avoids | ||
| // redundant SQLite queries within the revalidation window. | ||
| export const revalidate = 300; | ||
| @@ -51,7 +51,7 @@ export default async function RegisterPage() { | ||
| .filter((c) => c.total > 0) | ||
| .sort((a, b) => b.total - a.total) | ||
| .slice(0, 8); | ||
| - | // Compute byKind once - it's used in the details section below. | |
| + | // Compute byKind once - it's used in the details section below. | |
| const kindBreakdown = byKind(items); | ||
| return ( | ||
| @@ -60,7 +60,7 @@ export default async function RegisterPage() { | ||
| <div className="mx-auto w-full max-w-register px-4 py-12 sm:px-6"> | ||
| {/* | ||
| - | Plain-language explainer - ≤8th-grade reading level (ACCESSIBILITY.md §6). | |
| + | Plain-language explainer - ≤8th-grade reading level (ACCESSIBILITY.md §6). | |
| One sentence. Active voice. No adjectives. Sourcing stated. | ||
| */} | ||
| <p className="prose-lead mb-8 max-w-2xl text-ash"> | ||
| @@ -88,7 +88,7 @@ export default async function RegisterPage() { | ||
| )} | ||
| {/* | ||
| - | Secondary details - collapsed by default so the feed is the primary | |
| + | Secondary details - collapsed by default so the feed is the primary | |
| focus (docs/REDESIGN.md §"Register (home)"). | ||
| Composition chart, awards, recently-changed, and topic chips are | ||
| available here for users who want them. | ||
| @@ -100,7 +100,7 @@ export default async function RegisterPage() { | ||
| </summary> | ||
| <div className="mt-6 grid gap-6 sm:grid-cols-2"> | ||
| - | {/* Composition by kind - proportional to official-action volume. */} | |
| + | {/* Composition by kind - proportional to official-action volume. */} | |
| <Card className="p-5"> | ||
| <p className="label mb-4">Composition by action type</p> | ||
| <ul className="space-y-2.5"> | ||
| @@ -211,12 +211,12 @@ function byKind(items: FeedItem[]) { | ||
| /** Ranked register as a responsive table (>=md) / stacked cards (<md). */ | ||
| function RegisterTable({ items }: { items: FeedItem[] }) { | ||
| const columns: Column[] = [ | ||
| - | { key: "date", header: "Date" }, | |
| - | { key: "body", header: "Body", hideOnCard: true }, | |
| + | { key: "date", header: "Date", width: "8rem" }, | |
| + | { key: "body", header: "Body", hideOnCard: true, width: "6rem" }, | |
| { key: "item", header: "Item", cardTitle: true }, | ||
| - | { key: "action", header: "Action" }, | |
| - | { key: "result", header: "Result", align: "right" }, | |
| - | { key: "source", header: "Source", align: "right" }, | |
| + | { key: "action", header: "Action", width: "8rem" }, | |
| + | { key: "result", header: "Result", align: "right", width: "7rem" }, | |
| + | { key: "source", header: "Source", align: "right", width: "11rem" }, | |
| ]; | ||
| const rows: Row[] = items.map((r) => ({ | ||
| key: r.id, | ||
| @@ -244,10 +244,12 @@ function RegisterTable({ items }: { items: FeedItem[] }) { | ||
| ) : null} | ||
| </span> | ||
| ), | ||
| - | action: <span className="uppercase text-ash">{r.actionType}</span>, | |
| - | result: ( | |
| - | <span className="tabular-nums text-paper">{r.amountText ?? "-"}</span> | |
| - | ), | |
| + | action: r.actionType ? ( | |
| + | <span className="uppercase text-ash">{r.actionType}</span> | |
| + | ) : null, | |
| + | result: r.amountText ? ( | |
| + | <span className="tabular-nums text-paper">{r.amountText}</span> | |
| + | ) : null, | |
| source: <SourceLink url={r.sourceUrl} />, | ||
| }, | ||
| })); | ||
| @@ -257,6 +259,7 @@ function RegisterTable({ items }: { items: FeedItem[] }) { | ||
| cardLabel="Official actions in ranked order" | ||
| columns={columns} | ||
| rows={rows} | ||
| + | pageSize={50} | |
| /> | ||
| ); | ||
| } |
| @@ -12,7 +12,7 @@ export const metadata: Metadata = { | ||
| "When Congress meets: committee hearings, markups, business meetings, and floor action. Each meeting links to its official schedule.", | ||
| }; | ||
| - | /** Today as YYYY-MM-DD in UTC - passed to the browser so SSR and the client | |
| + | /** Today as YYYY-MM-DD in UTC - passed to the browser so SSR and the client | |
| * agree on "upcoming vs past" (no hydration drift). */ | ||
| function isoTodayUtc(): string { | ||
| const d = new Date(); | ||
| @@ -45,7 +45,7 @@ export default function SchedulePage() { | ||
| lead="Committee hearings, markups, business meetings, and floor action. Browse upcoming sessions or look back at past meetings. Each entry links to its official schedule." | ||
| /> | ||
| - | {/* Summary counts - by status, then alphabetical (not by party/team). */} | |
| + | {/* Summary counts - by status, then alphabetical (not by party/team). */} | |
| <div className="mt-8 grid gap-4 sm:grid-cols-4"> | ||
| <SummaryCard label="Total on record" count={all.length} /> | ||
| <SummaryCard label="Upcoming" count={upcoming} /> | ||
| @@ -53,7 +53,9 @@ export default function SchedulePage() { | ||
| <SummaryCard label="Hearings" count={hearings} /> | ||
| </div> | ||
| {floor > 0 ? ( | ||
| - | <p className="label mt-3 text-ash-2">{floor} floor action(s) in the record.</p> | |
| + | <p className="label mt-3 text-ash-2"> | |
| + | {floor} floor action{floor === 1 ? "" : "s"} in the record. | |
| + | </p> | |
| ) : null} | ||
| <div className="mt-10"> |
| @@ -1,4 +1,5 @@ | ||
| import type { Metadata } from "next"; | ||
| + | import Link from "next/link"; | |
| import { Suspense } from "react"; | ||
| import { PageHeader } from "@/components/ui"; | ||
| import { SearchBrowser } from "@/components/search-browser"; | ||
| @@ -13,18 +14,40 @@ export const revalidate = 300; | ||
| export default function SearchPage() { | ||
| return ( | ||
| - | <div className="mx-auto w-full max-w-register px-4 py-10 sm:px-6"> | |
| + | <div | |
| + | className="mx-auto w-full max-w-[860px] px-6 sm:px-8" | |
| + | style={{ | |
| + | paddingTop: "clamp(32px, 6vw, 56px)", | |
| + | paddingBottom: "clamp(48px, 8vw, 80px)", | |
| + | }} | |
| + | > | |
| + | {/* Back */} | |
| + | <Link | |
| + | href="/" | |
| + | className="focus-ring tap no-underline hover:underline" | |
| + | style={{ | |
| + | display: "inline-flex", | |
| + | alignItems: "center", | |
| + | gap: "8px", | |
| + | fontFamily: "var(--font-mono), ui-monospace, monospace", | |
| + | fontSize: "13px", | |
| + | letterSpacing: "0.08em", | |
| + | color: "var(--dk-signal)", | |
| + | marginBottom: "32px", | |
| + | }} | |
| + | > | |
| + | ← Home | |
| + | </Link> | |
| + | ||
| <PageHeader | ||
| eyebrow="Look it up" | ||
| title="Search the record" | ||
| lead="Look up a bill, a member of Congress, a state, an agency, or a topic. Every result links to its official source." | ||
| /> | ||
| - | <div className="mt-8"> | |
| - | <Suspense fallback={<SearchSkeleton />}> | |
| - | <SearchBrowser /> | |
| - | </Suspense> | |
| - | </div> | |
| + | <Suspense fallback={<SearchSkeleton />}> | |
| + | <SearchBrowser /> | |
| + | </Suspense> | |
| </div> | ||
| ); | ||
| } |
| @@ -1,4 +1,5 @@ | ||
| import type { Metadata } from "next"; | ||
| + | import Link from "next/link"; | |
| import { Suspense } from "react"; | ||
| import { getFiscal, getApportionments, type Apportionment } from "@/lib/feed"; | ||
| import { Card, PageHeader, SectionHeader } from "@/components/ui"; | ||
| @@ -58,54 +59,102 @@ export default function SpendingPage() { | ||
| ); | ||
| return ( | ||
| - | <div className="mx-auto w-full max-w-register px-4 py-10 sm:px-6"> | |
| + | <div | |
| + | className="mx-auto w-full max-w-[860px] px-6 sm:px-8" | |
| + | style={{ | |
| + | paddingTop: "clamp(32px, 6vw, 56px)", | |
| + | paddingBottom: "clamp(48px, 8vw, 80px)", | |
| + | }} | |
| + | > | |
| + | {/* Breadcrumb + Back */} | |
| + | <nav aria-label="Breadcrumb" className="mb-10"> | |
| + | <ol className="flex flex-wrap items-center gap-2" style={{ marginBottom: "12px" }}> | |
| + | <li> | |
| + | <Link | |
| + | href="/" | |
| + | className="focus-ring label no-underline hover:underline" | |
| + | style={{ color: "var(--dk-signal)", fontSize: "12px", letterSpacing: "0.12em" }} | |
| + | > | |
| + | Home | |
| + | </Link> | |
| + | </li> | |
| + | <li aria-hidden="true" className="label" style={{ color: "var(--dk-ash-2)", fontSize: "12px" }}>/</li> | |
| + | <li | |
| + | className="label" | |
| + | style={{ color: "var(--dk-ash)", fontSize: "12px", letterSpacing: "0.12em" }} | |
| + | aria-current="page" | |
| + | > | |
| + | Spending | |
| + | </li> | |
| + | </ol> | |
| + | <Link | |
| + | href="/" | |
| + | className="focus-ring tap no-underline hover:underline" | |
| + | style={{ | |
| + | display: "inline-flex", | |
| + | alignItems: "center", | |
| + | gap: "8px", | |
| + | fontFamily: "var(--font-mono), ui-monospace, monospace", | |
| + | fontSize: "13px", | |
| + | letterSpacing: "0.08em", | |
| + | color: "var(--dk-signal)", | |
| + | }} | |
| + | > | |
| + | ← Back to home | |
| + | </Link> | |
| + | </nav> | |
| + | ||
| <PageHeader | ||
| eyebrow="Spending register" | ||
| title="Dollars, by official record" | ||
| lead="Awards from USAspending.gov; debt, outlays, and cash from the Daily Treasury Statement. Figures only - no characterization." | ||
| /> | ||
| - | {/* Awards section - client component; reads q/sort from URL via useSearchParams */} | |
| + | {/* ── Section navigation - fix #10: impound tracker reachable immediately ── */} | |
| + | <nav | |
| + | aria-label="Spending sections" | |
| + | style={{ marginBottom: "clamp(24px, 4vw, 40px)" }} | |
| + | > | |
| + | <p | |
| + | className="label" | |
| + | style={{ color: "var(--dk-ash-2)", fontSize: "11px", letterSpacing: "0.12em", marginBottom: "10px" }} | |
| + | > | |
| + | Jump to | |
| + | </p> | |
| + | <div className="flex flex-wrap gap-2"> | |
| + | {[ | |
| + | { href: "#awards", label: "Awards" }, | |
| + | { href: "#apportionment", label: "Apportionment tracker" }, | |
| + | { href: "#fiscal", label: "Fiscal totals" }, | |
| + | ].map(({ href, label }) => ( | |
| + | <a | |
| + | key={href} | |
| + | href={href} | |
| + | className="focus-ring label inline-flex min-h-[44px] items-center rounded-[3px] border border-line bg-surface px-4 text-ash transition-colors hover:bg-graphite hover:text-paper no-underline" | |
| + | style={{ fontSize: "12px", letterSpacing: "0.10em" }} | |
| + | > | |
| + | {label} | |
| + | </a> | |
| + | ))} | |
| + | </div> | |
| + | </nav> | |
| + | ||
| + | {/* Awards section - client component; reads q/sort from URL via useSearchParams */} | |
| + | <section id="awards"> | |
| <Suspense fallback={<AwardsSkeleton />}> | ||
| <SpendingAwardsBrowser /> | ||
| </Suspense> | ||
| + | </section> | |
| - | {/* fiscal totals - calm empty state while the Treasury ingest lands */} | |
| - | {fiscal.length > 0 ? ( | |
| - | <div className="mt-8 grid gap-4 sm:grid-cols-3"> | |
| - | {fiscal.map((m) => ( | |
| - | <Card key={m.id} className="p-5"> | |
| - | <p className="label mb-2">{m.metric}</p> | |
| - | <p className="display text-2xl text-paper">{m.valueText}</p> | |
| - | <p className="label mt-2 text-ash-2">{formatDate(m.date)}</p> | |
| - | <div className="mt-3"> | |
| - | <SourceLink url={m.sourceUrl} /> | |
| - | </div> | |
| - | </Card> | |
| - | ))} | |
| - | </div> | |
| - | ) : ( | |
| - | <Card className="mt-8 p-5"> | |
| - | <p className="label mb-2">Fiscal totals</p> | |
| - | <p className="font-mono text-[13px] text-ash"> | |
| - | Debt, outlays, and cash figures from Treasury are not yet in the | |
| - | record. The Daily Treasury Statement feed will populate this panel. | |
| - | </p> | |
| - | <div className="mt-3"> | |
| - | <SourceLink url="https://fiscaldata.treasury.gov/datasets/daily-treasury-statement/operating-cash-balance" /> | |
| - | </div> | |
| - | </Card> | |
| - | )} | |
| - | ||
| - | {/* impound tracker - OMB apportionments (.max.gov) */} | |
| - | <div className="mt-12"> | |
| + | {/* ── Apportionment / impound tracker - fix #10: anchor at top via section nav ── */} | |
| + | <section id="apportionment" style={{ marginTop: "clamp(40px, 7vw, 64px)" }}> | |
| <SectionHeader | ||
| eyebrow="Apportionments" | ||
| title="Impound tracker" | ||
| description="Appropriations vs. amounts apportioned for spending, per agency and account. A withheld or delayed status is recorded as fact, not characterized." | ||
| /> | ||
| - | {/* status summary - muted, equal weight */} | |
| + | {/* status summary - muted, equal weight */} | |
| <div className="mb-6 grid gap-4 sm:grid-cols-3"> | ||
| <ImpoundCard label="Released" count={released.length} color="var(--dk-yea)" /> | ||
| <ImpoundCard label="Delayed" count={delayed.length} color="var(--dk-hold)" /> | ||
| @@ -131,11 +180,47 @@ export default function SpendingPage() { | ||
| <p className="label mt-4 text-ash-2"> | ||
| Source: apportionment-public.max.gov (OMB). Appropriated and apportioned are recorded figures; the gap is shown where one exists. | ||
| </p> | ||
| - | </div> | |
| + | </section> | |
| + | ||
| + | {/* fiscal totals - calm empty state while the Treasury ingest lands */} | |
| + | <section id="fiscal" style={{ marginTop: "clamp(40px, 7vw, 64px)" }}> | |
| + | <SectionHeader eyebrow="Treasury" title="Fiscal totals" /> | |
| + | {fiscal.length > 0 ? ( | |
| + | <div className="grid gap-4 sm:grid-cols-3"> | |
| + | {fiscal.map((m) => ( | |
| + | <Card key={m.id} className="p-5"> | |
| + | <p className="label mb-2">{m.metric}</p> | |
| + | <p className="display text-2xl text-paper">{m.valueText}</p> | |
| + | <p className="label mt-2 text-ash-2">{formatDate(m.date)}</p> | |
| + | <div className="mt-3"> | |
| + | <SourceLink url={m.sourceUrl} /> | |
| + | </div> | |
| + | </Card> | |
| + | ))} | |
| + | </div> | |
| + | ) : ( | |
| + | <Card className="p-5"> | |
| + | <p className="label mb-2">Fiscal totals</p> | |
| + | <p className="font-mono text-[13px] text-ash"> | |
| + | Debt, outlays, and cash figures from Treasury are not yet in the | |
| + | record. The Daily Treasury Statement feed will populate this panel. | |
| + | </p> | |
| + | <div className="mt-3"> | |
| + | <SourceLink url="https://fiscaldata.treasury.gov/datasets/daily-treasury-statement/operating-cash-balance" /> | |
| + | </div> | |
| + | </Card> | |
| + | )} | |
| + | </section> | |
| {/* Cite this page */} | ||
| - | <div className="mt-12 border-t border-line pt-8"> | |
| - | <p className="label mb-4 text-ash">Cite this page</p> | |
| + | <div | |
| + | style={{ | |
| + | marginTop: "clamp(40px, 7vw, 72px)", | |
| + | borderTop: "1px solid var(--dk-line)", | |
| + | paddingTop: "clamp(24px, 4vw, 40px)", | |
| + | }} | |
| + | > | |
| + | <p className="label mb-4" style={{ color: "var(--dk-ash)" }}>Cite this page</p> | |
| <CiteThis | ||
| kind="page" | ||
| title="Spending - Federal awards and apportionments" |
| @@ -16,6 +16,7 @@ export function generateMetadata({ | ||
| params: { code: string }; | ||
| }): Metadata { | ||
| const detail = getStateDetail(params.code); | ||
| + | if (!detail) return { title: "State not found" }; | |
| return { title: `${detail.name} (${detail.code})` }; | ||
| } | ||
| @@ -28,19 +29,42 @@ function confidenceLabel(c: "ok" | "stale" | "missing"): string { | ||
| return c === "ok" ? "Current" : c === "stale" ? "Stale" : "Coverage rolling out"; | ||
| } | ||
| + | /** District display helper - fix #11 */ | |
| + | function formatMemberDistrict(chamber: string, district: number | null): string { | |
| + | if (chamber === "senate") return ""; | |
| + | if (district === null || district <= 0) return " · At-Large"; | |
| + | return ` · Dist. ${district}`; | |
| + | } | |
| + | ||
| export default function StateDetailPage({ | ||
| params, | ||
| }: { | ||
| params: { code: string }; | ||
| }) { | ||
| const detail = getStateDetail(params.code); | ||
| - | if (!detail.code) notFound(); | |
| + | if (!detail) notFound(); | |
| return ( | ||
| - | <div className="mx-auto w-full max-w-register px-4 py-10 sm:px-6"> | |
| + | <div | |
| + | className="mx-auto w-full max-w-[860px] px-6 sm:px-8" | |
| + | style={{ | |
| + | paddingTop: "clamp(32px, 6vw, 56px)", | |
| + | paddingBottom: "clamp(48px, 8vw, 80px)", | |
| + | }} | |
| + | > | |
| <Link | ||
| href="/states" | ||
| - | className="focus-ring tap label mb-6 gap-1 text-ash hover:text-paper" | |
| + | className="focus-ring tap no-underline hover:underline" | |
| + | style={{ | |
| + | display: "inline-flex", | |
| + | alignItems: "center", | |
| + | gap: "8px", | |
| + | fontFamily: "var(--font-mono), ui-monospace, monospace", | |
| + | fontSize: "13px", | |
| + | letterSpacing: "0.08em", | |
| + | color: "var(--dk-signal)", | |
| + | marginBottom: "32px", | |
| + | }} | |
| > | ||
| ← States | ||
| </Link> | ||
| @@ -48,7 +72,7 @@ export default function StateDetailPage({ | ||
| <PageHeader | ||
| eyebrow={`State · ${detail.code}`} | ||
| title={detail.name} | ||
| - | lead={`Federal congressional delegation and sponsored bills for ${detail.name}. State-level legislative coverage is shown honestly below.`} | |
| + | lead={`Congressional delegation and sponsored bills for ${detail.name}. State-level legislative coverage is shown honestly below.`} | |
| /> | ||
| {/* coverage card */} | ||
| @@ -84,13 +108,13 @@ export default function StateDetailPage({ | ||
| {/* DELEGATION */} | ||
| <div> | ||
| <SectionHeader | ||
| - | eyebrow={`Delegation · ${detail.members.length}`} | |
| + | eyebrow={`Current delegation · ${detail.members.length}`} | |
| title="Members of Congress" | ||
| /> | ||
| {detail.members.length === 0 ? ( | ||
| <Card className="p-5"> | ||
| <p className="font-mono text-[13px] text-ash-2"> | ||
| - | No members of Congress linked to {detail.name} in the record. | |
| + | No current members of Congress linked to {detail.name} in the record. | |
| </p> | ||
| </Card> | ||
| ) : ( | ||
| @@ -108,7 +132,7 @@ export default function StateDetailPage({ | ||
| </span> | ||
| <span className="label mt-1 block text-ash-2"> | ||
| {m.chamber === "senate" ? "Senate" : "House"} | ||
| - | {m.district != null ? ` · Dist. ${m.district}` : ""} | |
| + | {formatMemberDistrict(m.chamber, m.district)} | |
| </span> | ||
| </span> | ||
| <PartyTag party={m.partyCode} /> |
| @@ -1,6 +1,6 @@ | ||
| import type { Metadata } from "next"; | ||
| import Link from "next/link"; | ||
| - | import { Card, PageHeader } from "@/components/ui"; | |
| + | import { Card, PageHeader, SectionHeader } from "@/components/ui"; | |
| import { DataTable, type Column, type Row } from "@/components/data-table"; | ||
| import { getStates, type StateCoverage } from "@/lib/feed"; | ||
| import { clsx } from "@/lib/cx"; | ||
| @@ -17,7 +17,7 @@ export const revalidate = 300; | ||
| /** | ||
| * Coverage framing. Federal data (congress.gov delegation + bill sponsorships) | ||
| * is on record for every state; STATE-legislative coverage is rolling out and | ||
| - | * is shown honestly but calmly - never a wall of alarming red. Un-ingested | |
| + | * is shown honestly but calmly - never a wall of alarming red. Un-ingested | |
| * states read as "Coverage rolling out" in a muted tone, not "Missing"/red. | ||
| * ok → Current (sage) | ||
| * stale → Stale (hold) | ||
| @@ -90,9 +90,9 @@ function buildRows(entries: StateCoverage[]): Row[] { | ||
| export default function StatesPage() { | ||
| const allJurisdictions = getStates(); | ||
| - | // 50 states - never DC or territories (never produces a "51 states" count). | |
| + | // 50 states - never DC or territories (never produces a "51 states" count). | |
| const states50 = allJurisdictions.filter((s) => !NON_STATE_CODES.has(s.code)); | ||
| - | // DC (federal district) + 5 U.S. territories - clearly not states. | |
| + | // DC (federal district) + 5 U.S. territories - clearly not states. | |
| const dcAndTerritories = allJurisdictions.filter((s) => NON_STATE_CODES.has(s.code)); | ||
| const ok = states50.filter((s) => s.confidence === "ok").length; | ||
| @@ -102,15 +102,59 @@ export default function StatesPage() { | ||
| const columns = buildColumns(); | ||
| return ( | ||
| - | <div className="mx-auto w-full max-w-register px-4 py-10 sm:px-6"> | |
| + | <div | |
| + | className="mx-auto w-full max-w-[860px] px-6 sm:px-8" | |
| + | style={{ | |
| + | paddingTop: "clamp(32px, 6vw, 56px)", | |
| + | paddingBottom: "clamp(48px, 8vw, 80px)", | |
| + | }} | |
| + | > | |
| + | {/* Breadcrumb + Back */} | |
| + | <nav aria-label="Breadcrumb" className="mb-10"> | |
| + | <ol className="flex flex-wrap items-center gap-2" style={{ marginBottom: "12px" }}> | |
| + | <li> | |
| + | <Link | |
| + | href="/" | |
| + | className="focus-ring label no-underline hover:underline" | |
| + | style={{ color: "var(--dk-signal)", fontSize: "12px", letterSpacing: "0.12em" }} | |
| + | > | |
| + | Home | |
| + | </Link> | |
| + | </li> | |
| + | <li aria-hidden="true" className="label" style={{ color: "var(--dk-ash-2)", fontSize: "12px" }}>/</li> | |
| + | <li | |
| + | className="label" | |
| + | style={{ color: "var(--dk-ash)", fontSize: "12px", letterSpacing: "0.12em" }} | |
| + | aria-current="page" | |
| + | > | |
| + | States | |
| + | </li> | |
| + | </ol> | |
| + | <Link | |
| + | href="/" | |
| + | className="focus-ring tap no-underline hover:underline" | |
| + | style={{ | |
| + | display: "inline-flex", | |
| + | alignItems: "center", | |
| + | gap: "8px", | |
| + | fontFamily: "var(--font-mono), ui-monospace, monospace", | |
| + | fontSize: "13px", | |
| + | letterSpacing: "0.08em", | |
| + | color: "var(--dk-signal)", | |
| + | }} | |
| + | > | |
| + | ← Back to home | |
| + | </Link> | |
| + | </nav> | |
| + | ||
| <PageHeader | ||
| eyebrow="State register" | ||
| - | title="50 states + DC & territories" | |
| - | lead="Federal delegation and sponsored bills are on record for every state, DC, and U.S. territory. State-level legislative coverage is rolling out - gaps are recorded here, not hidden, and never imply the site is broken." | |
| + | title="50 states + DC" | |
| + | lead="Federal delegation and sponsored bills are on record for every state, DC, and U.S. territory. State-level legislative coverage is rolling out - gaps are recorded here, not hidden." | |
| /> | ||
| - | {/* Coverage summary - 50 states only; DC and territories listed separately below. */} | |
| - | <div className="mt-8 grid gap-4 sm:grid-cols-3"> | |
| + | {/* Coverage summary - 50 states only; DC and territories listed separately below. */} | |
| + | <div className="grid gap-4 sm:grid-cols-3"> | |
| <CoverageCard label="Current" count={ok} level="ok" /> | ||
| <CoverageCard label="Stale" count={stale} level="stale" /> | ||
| <CoverageCard label="Coverage rolling out" count={rolling} level="missing" /> | ||
| @@ -131,9 +175,10 @@ export default function StatesPage() { | ||
| Coverage by state | ||
| </h2> | ||
| <p className="prose-lead mt-3 max-w-2xl text-ash"> | ||
| - | Each state's worst-case state-legislative coverage across bills, votes, | |
| - | and sponsors. Federal delegation and bill sponsorships (from congress.gov) | |
| - | are on record regardless. Open a state for its congressional delegation. | |
| + | How much of each state's legislature is on record here - across bills, | |
| + | votes, and sponsors. Federal delegation and bill sponsorships (from | |
| + | congress.gov) are on record regardless. Open a state for its congressional | |
| + | delegation. | |
| </p> | ||
| </div> | ||
| @@ -145,7 +190,7 @@ export default function StatesPage() { | ||
| /> | ||
| </div> | ||
| - | {/* DC & territories - separate section, clearly labeled as NOT states */} | |
| + | {/* DC & territories - separate section, clearly labeled as NOT states */} | |
| <div className="mt-14"> | ||
| <div className="mb-6"> | ||
| <p className="label mb-2">DC & U.S. territories · not states</p> |
| @@ -11,8 +11,9 @@ import { PartyTag } from "@/components/party-tag"; | ||
| import { MemberAvatar } from "@/components/member-avatar"; | ||
| import { getVoteById, type VotePosition } from "@/lib/feed"; | ||
| import { resolveVoteSubject } from "@/lib/vote-subject"; | ||
| - | import { formatDate } from "@/lib/format"; | |
| + | import { formatDate, formatBillNumber } from "@/lib/format"; | |
| import { CiteThis } from "@/components/cite-this"; | ||
| + | import { ArchiveLink } from "@/components/record-archive"; | |
| export const revalidate = 300; | ||
| @@ -32,7 +33,7 @@ export function generateMetadata({ | ||
| } | ||
| /** Default roster sort: by vote (Yea, Nay, Present, Not Voting), then | ||
| - | * alphabetical by name - NOT grouped by party (neutrality §Viz). */ | |
| + | * alphabetical by name - NOT grouped by party (neutrality §Viz). */ | |
| const POSITION_ORDER: Record<string, number> = { | ||
| yea: 0, | ||
| nay: 1, | ||
| @@ -71,16 +72,32 @@ export default function VoteDetailPage({ | ||
| const subject = resolveVoteSubject(vote); | ||
| return ( | ||
| - | <div className="mx-auto w-full max-w-register px-4 py-10 sm:px-6"> | |
| + | <div | |
| + | className="mx-auto w-full max-w-[860px] px-6 sm:px-8" | |
| + | style={{ | |
| + | paddingTop: "clamp(32px, 6vw, 56px)", | |
| + | paddingBottom: "clamp(48px, 8vw, 80px)", | |
| + | }} | |
| + | > | |
| <Link | ||
| href="/votes" | ||
| - | className="focus-ring tap label mb-6 gap-1 text-ash hover:text-paper" | |
| + | className="focus-ring tap no-underline hover:underline" | |
| + | style={{ | |
| + | display: "inline-flex", | |
| + | alignItems: "center", | |
| + | gap: "8px", | |
| + | fontFamily: "var(--font-mono), ui-monospace, monospace", | |
| + | fontSize: "13px", | |
| + | letterSpacing: "0.08em", | |
| + | color: "var(--dk-signal)", | |
| + | marginBottom: "32px", | |
| + | }} | |
| > | ||
| ← Votes | ||
| </Link> | ||
| {/* ────────────────────────────────────────────────────────── | ||
| - | WHAT THIS VOTE WAS ABOUT - shown before the page header so | |
| + | WHAT THIS VOTE WAS ABOUT - shown before the page header so | |
| the reader instantly understands the subject. | ||
| Priority: linked bill → nomination → treaty → ref → question | ||
| ────────────────────────────────────────────────────────── */} | ||
| @@ -104,7 +121,7 @@ export default function VoteDetailPage({ | ||
| href={`/bill/${vote.billId}`} | ||
| className="focus-ring underline underline-offset-4 hover:text-paper" | ||
| > | ||
| - | {vote.billNumber} | |
| + | {formatBillNumber(vote.billNumber)} | |
| </Link> | ||
| {vote.billTitle ? ( | ||
| <span className="text-ash"> - {vote.billTitle}</span> | ||
| @@ -133,13 +150,18 @@ export default function VoteDetailPage({ | ||
| /> | ||
| {/* top facts */} | ||
| - | <div className="mt-8 grid gap-4 sm:grid-cols-3"> | |
| + | <div className="grid gap-4 sm:grid-cols-3"> | |
| <Card className="p-5"> | ||
| <p className="label mb-2">Result</p> | ||
| <StatusPill variant={variant} dot> | ||
| {passed ? "Passed" : "Failed"} | ||
| </StatusPill> | ||
| - | <p className="mt-2 font-mono text-[12px] text-ash-2">{vote.result}</p> | |
| + | {/* Only show the source result text when it adds detail beyond the | |
| + | badge word itself (fix: avoid "● FAILED" then "Failed" again). */} | |
| + | {vote.result && | |
| + | vote.result.trim().toLowerCase() !== (passed ? "passed" : "failed") ? ( | |
| + | <p className="mt-2 font-mono text-[12px] text-ash-2">{vote.result}</p> | |
| + | ) : null} | |
| </Card> | ||
| <Card className="p-5"> | ||
| @@ -151,9 +173,20 @@ export default function VoteDetailPage({ | ||
| <Card className="p-5"> | ||
| <p className="label mb-2">Official record</p> | ||
| <SourceLink url={vote.sourceUrl} label="Roll call" /> | ||
| - | <p className="label mt-2 text-ash-2"> | |
| - | {vote.question ? "Question of the vote shown above." : "No question text on file."} | |
| - | </p> | |
| + | {vote.archiveUrl ? ( | |
| + | <> | |
| + | <div className="mt-2"> | |
| + | <ArchiveLink url={vote.archiveUrl} /> | |
| + | </div> | |
| + | <p className="label mt-2 text-ash-2"> | |
| + | Docket preserves a copy of the roll-call record. | |
| + | </p> | |
| + | </> | |
| + | ) : ( | |
| + | <p className="label mt-2 text-ash-2"> | |
| + | {vote.question ? "Question of the vote shown above." : "No question text on file."} | |
| + | </p> | |
| + | )} | |
| </Card> | ||
| </div> | ||
| @@ -220,7 +253,7 @@ export default function VoteDetailPage({ | ||
| </p> | ||
| </Card> | ||
| ) : ( | ||
| - | <VoteRoster roster={roster} /> | |
| + | <VoteRoster roster={roster} chamber={vote.chamber} /> | |
| )} | ||
| <p className="label mt-4 text-ash-2"> | ||
| Party shown in desaturated, equal-weight tones. Source: {""} | ||
| @@ -286,8 +319,16 @@ function SummaryChip({ | ||
| ); | ||
| } | ||
| + | /** District display helper - fix #11: senators never show a district number. | |
| + | * House At-Large members (district null / -1 / 0) show "At-Large". */ | |
| + | function formatMemberDistrict(chamber: string, district: number | null): string { | |
| + | if ((chamber ?? "").toLowerCase() === "senate") return ""; | |
| + | if (district === null || district <= 0) return " · At-Large"; | |
| + | return ` · Dist. ${district}`; | |
| + | } | |
| + | ||
| /** Member roster as a responsive table (>=md) / stacked cards (<md). */ | ||
| - | function VoteRoster({ roster }: { roster: VotePosition[] }) { | |
| + | function VoteRoster({ roster, chamber }: { roster: VotePosition[]; chamber: string }) { | |
| const columns: Column[] = [ | ||
| { key: "member", header: "Member", cardTitle: true }, | ||
| { key: "party", header: "Party" }, | ||
| @@ -318,7 +359,7 @@ function VoteRoster({ roster }: { roster: VotePosition[] }) { | ||
| state: ( | ||
| <span className="text-ash"> | ||
| {p.state} | ||
| - | {p.district != null ? ` ${p.district}`.padStart(3, " ") : ""} | |
| + | {formatMemberDistrict(chamber, p.district)} | |
| </span> | ||
| ), | ||
| position: <PositionTag position={p.position} />, | ||
| @@ -330,6 +371,7 @@ function VoteRoster({ roster }: { roster: VotePosition[] }) { | ||
| cardLabel="Member roster" | ||
| columns={columns} | ||
| rows={rows} | ||
| + | pageSize={50} | |
| /> | ||
| ); | ||
| } |
| @@ -1,4 +1,5 @@ | ||
| import type { Metadata } from "next"; | ||
| + | import Link from "next/link"; | |
| import { Suspense } from "react"; | ||
| import { PageHeader } from "@/components/ui"; | ||
| import { CiteThis } from "@/components/cite-this"; | ||
| @@ -14,7 +15,53 @@ export const revalidate = 300; | ||
| export default function VotesPage() { | ||
| return ( | ||
| - | <div className="mx-auto w-full max-w-register px-4 py-10 sm:px-6"> | |
| + | <div | |
| + | className="mx-auto w-full max-w-[860px] px-6 sm:px-8" | |
| + | style={{ | |
| + | paddingTop: "clamp(32px, 6vw, 56px)", | |
| + | paddingBottom: "clamp(48px, 8vw, 80px)", | |
| + | }} | |
| + | > | |
| + | {/* Breadcrumb + Back */} | |
| + | <nav aria-label="Breadcrumb" className="mb-10"> | |
| + | <ol className="flex flex-wrap items-center gap-2" style={{ marginBottom: "12px" }}> | |
| + | <li> | |
| + | <Link | |
| + | href="/" | |
| + | className="focus-ring label no-underline hover:underline" | |
| + | style={{ color: "var(--dk-signal)", fontSize: "12px", letterSpacing: "0.12em" }} | |
| + | > | |
| + | Home | |
| + | </Link> | |
| + | </li> | |
| + | <li aria-hidden="true" className="label" style={{ color: "var(--dk-ash-2)", fontSize: "12px" }}> | |
| + | / | |
| + | </li> | |
| + | <li | |
| + | className="label" | |
| + | style={{ color: "var(--dk-ash)", fontSize: "12px", letterSpacing: "0.12em" }} | |
| + | aria-current="page" | |
| + | > | |
| + | Votes | |
| + | </li> | |
| + | </ol> | |
| + | <Link | |
| + | href="/" | |
| + | className="focus-ring tap no-underline hover:underline" | |
| + | style={{ | |
| + | display: "inline-flex", | |
| + | alignItems: "center", | |
| + | gap: "8px", | |
| + | fontFamily: "var(--font-mono), ui-monospace, monospace", | |
| + | fontSize: "13px", | |
| + | letterSpacing: "0.08em", | |
| + | color: "var(--dk-signal)", | |
| + | }} | |
| + | > | |
| + | ← Back to home | |
| + | </Link> | |
| + | </nav> | |
| + | ||
| <PageHeader | ||
| eyebrow="Roll-call register" | ||
| title="Votes, on the record" | ||
| @@ -26,8 +73,20 @@ export default function VotesPage() { | ||
| </Suspense> | ||
| {/* Cite this page */} | ||
| - | <div className="mt-12 border-t border-line pt-8"> | |
| - | <p className="label mb-4 text-ash">Cite this page</p> | |
| + | <div | |
| + | style={{ | |
| + | marginTop: "clamp(40px, 7vw, 72px)", | |
| + | borderTop: "1px solid var(--dk-line)", | |
| + | paddingTop: "clamp(24px, 4vw, 40px)", | |
| + | }} | |
| + | > | |
| + | <p className="label mb-8" style={{ color: "var(--dk-ash-2)" }}> | |
| + | Roll-call votes from clerk.house.gov and senate.gov XML feeds. Order is | |
| + | by date - no editorial weighting. | |
| + | </p> | |
| + | <p className="label mb-4" style={{ color: "var(--dk-ash)" }}> | |
| + | Cite this page | |
| + | </p> | |
| <CiteThis | ||
| kind="page" | ||
| title="Votes - Roll-call register" |
| @@ -1,4 +1,5 @@ | ||
| import type { Metadata } from "next"; | ||
| + | import Link from "next/link"; | |
| import { Card, PageHeader, SectionHeader } from "@/components/ui"; | ||
| export const metadata: Metadata = { | ||
| @@ -9,7 +10,29 @@ export const metadata: Metadata = { | ||
| export default function WhoPaysPage() { | ||
| return ( | ||
| - | <div className="mx-auto w-full max-w-register px-4 py-10 sm:px-6"> | |
| + | <div | |
| + | className="mx-auto w-full max-w-[860px] px-6 sm:px-8" | |
| + | style={{ | |
| + | paddingTop: "clamp(32px, 6vw, 56px)", | |
| + | paddingBottom: "clamp(48px, 8vw, 80px)", | |
| + | }} | |
| + | > | |
| + | <Link | |
| + | href="/" | |
| + | className="focus-ring tap no-underline hover:underline" | |
| + | style={{ | |
| + | display: "inline-flex", | |
| + | alignItems: "center", | |
| + | gap: "8px", | |
| + | fontFamily: "var(--font-mono), ui-monospace, monospace", | |
| + | fontSize: "13px", | |
| + | letterSpacing: "0.08em", | |
| + | color: "var(--dk-signal)", | |
| + | marginBottom: "32px", | |
| + | }} | |
| + | > | |
| + | ← Home | |
| + | </Link> | |
| <PageHeader | ||
| eyebrow="Trust · ownership" | ||
| title="Who pays for this" |
| @@ -5,7 +5,7 @@ import { useSearchParams } from "next/navigation"; | ||
| import Link from "next/link"; | ||
| import { SourceLink } from "@/components/source-link"; | ||
| import { StatusPill, classifyStatus } from "@/components/status-pill"; | ||
| - | import { formatDate } from "@/lib/format"; | |
| + | import { formatDate, congressLabel, formatBillNumber, ordinal } from "@/lib/format"; | |
| import { | ||
| SearchFilterBar, | ||
| FilterField, | ||
| @@ -45,6 +45,7 @@ function BillCards({ bills }: { bills: BillListItem[] }) { | ||
| > | ||
| {bills.map((b) => { | ||
| const variant = classifyStatus(b.status); | ||
| + | const cong = congressLabel(b.congress); | |
| return ( | ||
| <li key={b.id}> | ||
| <div | ||
| @@ -70,10 +71,12 @@ function BillCards({ bills }: { bills: BillListItem[] }) { | ||
| className="font-mono" | ||
| style={{ fontSize: "13px", color: "var(--dk-ash-2)", letterSpacing: "0.06em", fontWeight: 700 }} | ||
| > | ||
| - | {b.number} | |
| - | <span style={{ color: "var(--dk-ash-2)", fontWeight: 400, marginLeft: "8px" }}> | |
| - | {b.congress}th Congress | |
| - | </span> | |
| + | {formatBillNumber(b.number)} | |
| + | {cong ? ( | |
| + | <span style={{ color: "var(--dk-ash-2)", fontWeight: 400, marginLeft: "8px" }}> | |
| + | {cong} | |
| + | </span> | |
| + | ) : null} | |
| </span> | ||
| <StatusPill variant={variant}> | ||
| {STATUS_SHORT[variant] ?? "Introduced"} | ||
| @@ -143,9 +146,11 @@ export function BillsBrowser({ | ||
| const [bills, setBills] = useState<BillListItem[]>([]); | ||
| const [total, setTotal] = useState(0); | ||
| const [loading, setLoading] = useState(true); | ||
| + | const [shown, setShown] = useState(50); | |
| useEffect(() => { | ||
| setLoading(true); | ||
| + | setShown(50); | |
| const params = new URLSearchParams(); | ||
| if (policyArea) params.set("policyArea", policyArea); | ||
| if (q) params.set("q", q); | ||
| @@ -264,7 +269,7 @@ export function BillsBrowser({ | ||
| aria-label="Filter by Congress number" | ||
| options={[ | ||
| { value: "", label: "Any congress" }, | ||
| - | ...congresses.map((c) => ({ value: String(c), label: `${c}th` })), | |
| + | ...congresses.map((c) => ({ value: String(c), label: ordinal(c) })), | |
| ]} | ||
| /> | ||
| </FilterField> | ||
| @@ -274,8 +279,8 @@ export function BillsBrowser({ | ||
| defaultValue={sortParam} | ||
| aria-label="Sort bills" | ||
| options={[ | ||
| - | { value: "", label: "Last action (newest)" }, | |
| - | { value: "congress", label: "By congress (newest)" }, | |
| + | { value: "", label: "Newest action" }, | |
| + | { value: "congress", label: "By congress" }, | |
| { value: "status", label: "By status" }, | ||
| ]} | ||
| /> | ||
| @@ -299,7 +304,23 @@ export function BillsBrowser({ | ||
| </p> | ||
| </div> | ||
| ) : ( | ||
| - | <BillCards bills={bills} /> | |
| + | <> | |
| + | <BillCards bills={bills.slice(0, shown)} /> | |
| + | {bills.length > shown ? ( | |
| + | <div className="mt-4 flex items-center justify-between gap-3"> | |
| + | <span className="label" style={{ color: "var(--dk-ash-2)" }}> | |
| + | Showing {shown.toLocaleString()} of {bills.length.toLocaleString()} | |
| + | </span> | |
| + | <button | |
| + | type="button" | |
| + | onClick={() => setShown((n) => n + 50)} | |
| + | className="focus-ring label inline-flex min-h-[44px] items-center rounded-[8px] border border-line px-4 text-ash underline underline-offset-4 transition-colors hover:border-paper hover:text-paper" | |
| + | > | |
| + | Show {Math.min(50, bills.length - shown).toLocaleString()} more | |
| + | </button> | |
| + | </div> | |
| + | ) : null} | |
| + | </> | |
| )} | ||
| </section> | ||
| </div> |
| @@ -3,7 +3,7 @@ import { SourceLink } from "@/components/source-link"; | ||
| import type { ChangeEvent } from "@/lib/feed"; | ||
| /** | ||
| - | * Compact "Recently changed" surface - the watcher feed (change_events). | |
| + | * Compact "Recently changed" surface - the watcher feed (change_events). | |
| * Neutral, factual: added / updated / removed, a one-line summary, the | ||
| * source table, and a Source link on every entry (neutrality #1, #4). | ||
| * Presentational; the page passes it the events it wants shown. |
| @@ -1,5 +1,5 @@ | ||
| /** | ||
| - | * CiteThis - first-class citation component (docs/REDESIGN.md §"Cite this"). | |
| + | * CiteThis - first-class citation component (docs/REDESIGN.md §"Cite this"). | |
| * | ||
| * Generates ready-to-use citations in APA, MLA, and Chicago styles for any | ||
| * Docket record or page. Each style includes the usdocket.org permalink AND | ||
| @@ -34,13 +34,13 @@ export interface CiteThisProps { | ||
| title: string; | ||
| /** Full permalink: https://usdocket.org/... */ | ||
| permalink: string; | ||
| - | /** Official .gov source URL (neutrality rule #1 - always surface the primary source). */ | |
| + | /** Official .gov source URL (neutrality rule #1 - always surface the primary source). */ | |
| sourceUrl: string; | ||
| /** Year of publication / action (defaults to current calendar year). */ | ||
| year?: number; | ||
| } | ||
| - | /** Format a Date as "Month D, YYYY" - APA/Chicago style. */ | |
| + | /** Format a Date as "Month D, YYYY" - APA/Chicago style. */ | |
| function longDate(d: Date): string { | ||
| return d.toLocaleDateString("en-US", { | ||
| year: "numeric", | ||
| @@ -49,7 +49,7 @@ function longDate(d: Date): string { | ||
| }); | ||
| } | ||
| - | /** Format a Date as "D Month YYYY" - MLA style. */ | |
| + | /** Format a Date as "D Month YYYY" - MLA style. */ | |
| function mlaDate(d: Date): string { | ||
| const day = d.getDate(); | ||
| const month = d.toLocaleDateString("en-US", { month: "long" }); | ||
| @@ -207,7 +207,7 @@ export function CiteThis({ | ||
| {/* Copy row */} | ||
| <div className="flex items-center justify-between border-t border-line px-4 py-3 sm:px-5"> | ||
| <p className="label text-ash-2"> | ||
| - | {/* Nudge for educators/students - neutral phrasing */} | |
| + | {/* Nudge for educators/students - neutral phrasing */} | |
| APA · MLA · Chicago - copy and paste into your document or citation manager. | ||
| </p> | ||
| <button |
| @@ -1,15 +1,16 @@ | ||
| import { clsx } from "@/lib/cx"; | ||
| /** | ||
| - | * The Docket custody-stamp seal - a registration frame (outer rule + inner | |
| - | * offset rule) around a [D] monogram with a small brass record-tick. | |
| + | * The Docket mark - a filed official document: a page with a folded brass | |
| + | * corner (the "filed/on-record" dog-ear) and record lines. It reads plainly as | |
| + | * "an official record on file," retiring both the old [D] monogram and the | |
| + | * earlier checkbox-style stamp (which read as a UI checkbox, not a record). | |
| * | ||
| - | * Rules (docs/BRAND.md §3): FLAT strokes, no gradient, NO eagle / flag / | |
| - | * dome / scales (those carry political valence). Below 24px drop the inner | |
| - | * rule and show a solid tile + ink "D". | |
| + | * Rules (docs/BRAND.md §3): FLAT strokes, no gradient, NO eagle / flag / dome / | |
| + | * scales (those carry political valence). Brass is the single chromatic mark. | |
| + | * Below 24px → favicon-style solid brass tile + ink page glyph (max legibility). | |
| * | ||
| - | * The brass tick is brand identity, not a "signal" - the rationed per-view | |
| - | * brass signal is the pulsing live-dot (see RegisterHero). | |
| + | * `showTick` keeps the inner record lines (name kept for API back-compat). | |
| */ | ||
| export function CustodyMark({ | ||
| size = 28, | ||
| @@ -20,38 +21,43 @@ export function CustodyMark({ | ||
| className?: string; | ||
| showTick?: boolean; | ||
| }) { | ||
| - | // Below 24px → favicon-style solid tile, ink "D", no inner rule. | |
| + | // Below 24px → favicon-style solid brass tile + ink filed-page glyph. | |
| if (size < 24) { | ||
| return ( | ||
| <span | ||
| - | className={clsx( | |
| - | "inline-flex items-center justify-center rounded-[3px] font-display font-800", | |
| - | className | |
| - | )} | |
| - | style={{ | |
| - | width: size, | |
| - | height: size, | |
| - | backgroundColor: "var(--dk-signal)", | |
| - | color: "var(--dk-ink)", | |
| - | fontSize: size * 0.62, | |
| - | lineHeight: 1, | |
| - | fontWeight: 800, | |
| - | }} | |
| - | aria-hidden="true" | |
| + | className={clsx("inline-block align-middle", className)} | |
| + | style={{ width: size, height: size }} | |
| + | role="img" | |
| + | aria-label="Docket" | |
| > | ||
| - | D | |
| + | <svg | |
| + | width={size} | |
| + | height={size} | |
| + | viewBox="0 0 32 32" | |
| + | fill="none" | |
| + | xmlns="http://www.w3.org/2000/svg" | |
| + | aria-hidden="true" | |
| + | > | |
| + | <rect width="32" height="32" rx="6" fill="var(--dk-signal)" /> | |
| + | {/* ink page silhouette with a cut (folded) corner */} | |
| + | <path | |
| + | d="M9 7 H19 L23 11 V25 H9 Z" | |
| + | fill="var(--dk-ink)" | |
| + | /> | |
| + | {/* dog-ear notch (tile color shows through) */} | |
| + | <path d="M19 7 L23 11 H19 Z" fill="var(--dk-signal)" /> | |
| + | </svg> | |
| </span> | ||
| ); | ||
| } | ||
| - | // Full seal: outer rule + inner offset rule + D monogram + brass tick. | |
| - | const stroke = Math.max(1, Math.round(size * 0.05)); | |
| + | // Full mark: filed document - page outline + folded brass corner + record lines. | |
| return ( | ||
| <span | ||
| className={clsx("inline-block align-middle", className)} | ||
| style={{ width: size, height: size }} | ||
| role="img" | ||
| - | aria-label="Docket seal" | |
| + | aria-label="Docket" | |
| > | ||
| <svg | ||
| width={size} | ||
| @@ -61,52 +67,46 @@ export function CustodyMark({ | ||
| xmlns="http://www.w3.org/2000/svg" | ||
| aria-hidden="true" | ||
| > | ||
| - | {/* outer rule */} | |
| - | <rect | |
| - | x="3" | |
| - | y="3" | |
| - | width="94" | |
| - | height="94" | |
| - | rx="6" | |
| + | {/* document page */} | |
| + | <path | |
| + | d="M26 16 H60 L80 36 V86 H26 Z" | |
| + | fill="none" | |
| stroke="var(--dk-paper)" | ||
| - | strokeWidth={stroke} | |
| - | opacity="0.92" | |
| + | strokeWidth="5" | |
| + | strokeLinejoin="round" | |
| /> | ||
| - | {/* inner offset rule */} | |
| - | <rect | |
| - | x="11" | |
| - | y="11" | |
| - | width="78" | |
| - | height="78" | |
| - | rx="3" | |
| - | stroke="var(--dk-paper)" | |
| - | strokeWidth={stroke} | |
| - | opacity="0.32" | |
| + | {/* folded corner - the brass "filed / on-record" dog-ear */} | |
| + | <path | |
| + | d="M60 16 V36 H80" | |
| + | fill="none" | |
| + | stroke="var(--dk-signal)" | |
| + | strokeWidth="5" | |
| + | strokeLinejoin="round" | |
| /> | ||
| - | {/* [D] monogram */} | |
| - | <text | |
| - | x="50" | |
| - | y="54" | |
| - | textAnchor="middle" | |
| - | dominantBaseline="central" | |
| - | fontFamily="var(--font-display), system-ui, sans-serif" | |
| - | fontWeight={800} | |
| - | fontSize="52" | |
| - | letterSpacing="-2" | |
| - | fill="var(--dk-paper)" | |
| - | > | |
| - | D | |
| - | </text> | |
| - | {/* brass record-tick (the brand's only chromatic mark) */} | |
| + | {/* record lines */} | |
| {showTick && ( | ||
| - | <rect | |
| - | x="72" | |
| - | y="72" | |
| - | width="14" | |
| - | height="14" | |
| - | rx="1" | |
| - | fill="var(--dk-signal)" | |
| - | /> | |
| + | <> | |
| + | <line | |
| + | x1="37" | |
| + | y1="54" | |
| + | x2="69" | |
| + | y2="54" | |
| + | stroke="var(--dk-paper)" | |
| + | strokeWidth="5" | |
| + | strokeLinecap="round" | |
| + | opacity="0.5" | |
| + | /> | |
| + | <line | |
| + | x1="37" | |
| + | y1="68" | |
| + | x2="60" | |
| + | y2="68" | |
| + | stroke="var(--dk-paper)" | |
| + | strokeWidth="5" | |
| + | strokeLinecap="round" | |
| + | opacity="0.5" | |
| + | /> | |
| + | </> | |
| )} | ||
| </svg> | ||
| </span> |
| @@ -1,19 +1,21 @@ | ||
| - | import type { ReactNode } from "react"; | |
| + | "use client"; | |
| + | ||
| + | import { useState, type ReactNode } from "react"; | |
| import { clsx } from "@/lib/cx"; | ||
| import { Card } from "@/components/ui"; | ||
| /** | ||
| - | * Responsive data table - Docket's only list pattern (mobile-first overhaul). | |
| + | * Responsive data table - Docket's only list pattern (mobile-first overhaul). | |
| * | ||
| * - >=md: a real `<table>` (auto layout, wrapping cells) for a11y + SEO. Never | ||
| * has a fixed min-width, so it can never force the page wider than the | ||
| * viewport. | ||
| - | * - <md: each row becomes a STACKED CARD - a prominent title (the `cardTitle` | |
| + | * - <md: each row becomes a STACKED CARD - a prominent title (the `cardTitle` | |
| * column) plus a definition list of labeled field rows | ||
| * (`Date · Body · Item · Action · Result · Source`), exactly the pattern in | ||
| * the mobile spec. Zero horizontal page scroll at 390px. | ||
| * | ||
| - | * Presentational only - usable from both server and client components. | |
| + | * Presentational only - usable from both server and client components. | |
| * Brand/neutrality safe: it renders whatever cells the page passes (Source | ||
| * links, muted pills) without adding color or commentary of its own. | ||
| */ | ||
| @@ -26,6 +28,11 @@ export type Column = { | ||
| cardTitle?: boolean; | ||
| /** Hide this column's row on the mobile card entirely (desktop-only). */ | ||
| hideOnCard?: boolean; | ||
| + | /** Explicit desktop column width (CSS value). Used in the <colgroup> so | |
| + | * table-fixed honours it. Right-aligned action/source columns default to a | |
| + | * width wide enough that their links never get squeezed and clipped (fix | |
| + | * #1); the cardTitle column stays flexible and absorbs the remainder. */ | |
| + | width?: string; | |
| }; | ||
| export type Row = { key: string; cells: Record<string, ReactNode> }; | ||
| @@ -36,6 +43,7 @@ export function DataTable({ | ||
| rows, | ||
| className, | ||
| cardLabel, | ||
| + | pageSize, | |
| }: { | ||
| caption: string; | ||
| columns: Column[]; | ||
| @@ -43,12 +51,22 @@ export function DataTable({ | ||
| className?: string; | ||
| /** Accessible label for the mobile card list. Defaults to the caption. */ | ||
| cardLabel?: string; | ||
| + | /** When set, render only the first `pageSize` rows initially and reveal the | |
| + | * rest behind a "Show more" control. Keeps long lists (rosters, registers) | |
| + | * from rendering as one tens-of-thousands-of-pixels dump. Unset = show all. */ | |
| + | pageSize?: number; | |
| }) { | ||
| + | const [shown, setShown] = useState(pageSize ?? rows.length); | |
| + | ||
| if (rows.length === 0) return null; | ||
| const titleCol = columns.find((c) => c.cardTitle); | ||
| const cardCols = columns.filter((c) => !c.cardTitle && !c.hideOnCard); | ||
| + | const cap = pageSize ? shown : rows.length; | |
| + | const visibleRows = pageSize ? rows.slice(0, cap) : rows; | |
| + | const remaining = rows.length - cap; | |
| + | ||
| return ( | ||
| <Card className={clsx("overflow-hidden [contain:layout]", className)}> | ||
| {/* DESKTOP (>=md): real table. overflow-x-auto on the wrapper creates a | ||
| @@ -58,6 +76,16 @@ export function DataTable({ | ||
| <div className="hidden w-full overflow-x-auto md:block"> | ||
| <table className="w-full table-fixed border-collapse text-left"> | ||
| <caption className="sr-only">{caption}</caption> | ||
| + | <colgroup> | |
| + | {columns.map((c) => { | |
| + | // Right-aligned columns hold source links / amounts; give them a | |
| + | // floor width so table-fixed never squeezes them below their | |
| + | // content and clips against the card edge. The flexible title | |
| + | // column (no width) soaks up whatever is left. | |
| + | const w = c.width ?? (c.align === "right" ? "10rem" : undefined); | |
| + | return <col key={c.key} style={w ? { width: w } : undefined} />; | |
| + | })} | |
| + | </colgroup> | |
| <thead> | ||
| <tr className="border-b border-line"> | ||
| {columns.map((c) => ( | ||
| @@ -75,7 +103,7 @@ export function DataTable({ | ||
| </tr> | ||
| </thead> | ||
| <tbody> | ||
| - | {rows.map((r) => ( | |
| + | {visibleRows.map((r) => ( | |
| <tr | ||
| key={r.key} | ||
| className="border-b border-line font-mono text-[12px] transition-colors last:border-0 hover:bg-graphite/60" | ||
| @@ -102,7 +130,7 @@ export function DataTable({ | ||
| className="divide-y divide-[color:var(--dk-line)] md:hidden" | ||
| aria-label={cardLabel ?? caption} | ||
| > | ||
| - | {rows.map((r) => ( | |
| + | {visibleRows.map((r) => ( | |
| <li key={r.key} className="p-4"> | ||
| {titleCol ? ( | ||
| <div className="mb-2.5 min-w-0 break-words font-mono text-[13px] text-paper"> | ||
| @@ -110,20 +138,41 @@ export function DataTable({ | ||
| </div> | ||
| ) : null} | ||
| <dl className="grid grid-cols-1 gap-2"> | ||
| - | {cardCols.map((c) => ( | |
| - | <div key={c.key} className="flex items-start gap-3"> | |
| - | <dt className="label w-[5.5rem] shrink-0 pt-0.5 normal-case"> | |
| - | {c.header} | |
| - | </dt> | |
| - | <dd className="min-w-0 flex-1 break-words text-left font-mono text-[12px] text-ash"> | |
| - | {r.cells[c.key]} | |
| - | </dd> | |
| - | </div> | |
| - | ))} | |
| + | {cardCols.map((c) => { | |
| + | // Suppress empty fields so a card never shows a dangling lone | |
| + | // label with a " - " / blank value (fix: register card clutter). | |
| + | const val = r.cells[c.key]; | |
| + | if (val == null) return null; | |
| + | return ( | |
| + | <div key={c.key} className="flex items-start gap-3"> | |
| + | <dt className="label w-[5.5rem] shrink-0 pt-0.5 normal-case"> | |
| + | {c.header} | |
| + | </dt> | |
| + | <dd className="min-w-0 flex-1 break-words text-left font-mono text-[12px] text-ash"> | |
| + | {val} | |
| + | </dd> | |
| + | </div> | |
| + | ); | |
| + | })} | |
| </dl> | ||
| </li> | ||
| ))} | ||
| </ul> | ||
| + | ||
| + | {pageSize && remaining > 0 ? ( | |
| + | <div className="flex items-center justify-between gap-3 border-t border-line px-4 py-3"> | |
| + | <span className="label text-ash-2"> | |
| + | Showing {cap.toLocaleString()} of {rows.length.toLocaleString()} | |
| + | </span> | |
| + | <button | |
| + | type="button" | |
| + | onClick={() => setShown((n) => n + (pageSize ?? 0))} | |
| + | className="focus-ring label inline-flex min-h-[40px] items-center rounded-[3px] border border-line px-3 text-ash underline underline-offset-4 transition-colors hover:border-paper hover:text-paper" | |
| + | > | |
| + | Show {Math.min(pageSize, remaining).toLocaleString()} more | |
| + | </button> | |
| + | </div> | |
| + | ) : null} | |
| </Card> | ||
| ); | ||
| } |
| @@ -3,7 +3,7 @@ | ||
| import { useEffect, useState } from "react"; | ||
| /** | ||
| - | * Day/Night icon switcher - two modes only: dark (default ink) and light (warm day). | |
| + | * Day/Night icon switcher - two modes only: dark (default ink) and light (warm day). | |
| * | ||
| * Pattern: show the CURRENT mode's icon (moon = night/dark; sun = day/light). | ||
| * aria-label tells the user what clicking will do ("Switch to light mode" etc.). | ||
| @@ -69,7 +69,7 @@ export function DisplayToggle() { | ||
| ); | ||
| } | ||
| - | /** Moon - shown when in dark/night mode */ | |
| + | /** Moon - shown when in dark/night mode */ | |
| function MoonIcon() { | ||
| return ( | ||
| <svg | ||
| @@ -88,7 +88,7 @@ function MoonIcon() { | ||
| ); | ||
| } | ||
| - | /** Sun - shown when in light/day mode */ | |
| + | /** Sun - shown when in light/day mode */ | |
| function SunIcon() { | ||
| return ( | ||
| <svg |
| @@ -156,6 +156,7 @@ function ExecutiveTable({ | ||
| cardLabel="Federal Register actions" | ||
| columns={columns} | ||
| rows={data} | ||
| + | pageSize={50} | |
| /> | ||
| ); | ||
| } |
| @@ -128,7 +128,7 @@ export function MeetingsBrowser({ | ||
| return ( | ||
| <> | ||
| - | {/* Filter bar - labeled selects + scope (a11y §Search). */} | |
| + | {/* Filter bar - labeled selects + scope (a11y §Search). */} | |
| <Card className="p-4 sm:p-5"> | ||
| <div className="grid gap-3 sm:grid-cols-2 lg:grid-cols-4"> | ||
| <Field label="Search"> | ||
| @@ -180,7 +180,7 @@ export function MeetingsBrowser({ | ||
| </Field> | ||
| </div> | ||
| - | {/* Scope - segmented control, ≥44px targets. */} | |
| + | {/* Scope - segmented control, ≥44px targets. */} | |
| <div | ||
| role="group" | ||
| aria-label="Show upcoming or past meetings" |
| @@ -13,8 +13,8 @@ import { clsx } from "@/lib/cx"; | ||
| * | ||
| * Decorative by design (aria-hidden): the member's name is always rendered | ||
| * as real text/links beside it, and the cited record stays the allowlisted | ||
| - | * bioguide Source link - the photo is a visual aid, not a cited fact, so it | |
| - | * does not affect neutrality #1. Uses no brass (rationed accent) - neutral | |
| + | * bioguide Source link - the photo is a visual aid, not a cited fact, so it | |
| + | * does not affect neutrality #1. Uses no brass (rationed accent) - neutral | |
| * surface tokens only, so it stays legible in Dark / Light / Plain modes. | ||
| */ | ||
| export type AvatarSize = "sm" | "md" | "lg"; |
| @@ -2,7 +2,7 @@ | ||
| /** | ||
| * Lazy-loading accordion panel for one year's worth of a member's recorded | ||
| - | * votes. The votes are NOT shipped in the initial SSR HTML - they are fetched | |
| + | * votes. The votes are NOT shipped in the initial SSR HTML - they are fetched | |
| * from /api/members/[id]/votes?year=YYYY only when the <details> panel is | ||
| * first opened (or immediately on mount when defaultOpen=true). | ||
| * | ||
| @@ -10,7 +10,7 @@ | ||
| * members who have 10 000+ recorded positions. | ||
| */ | ||
| - | import { useState, useEffect, useRef, useCallback } from "react"; | |
| + | import { useState, useEffect, useRef, useCallback, useMemo } from "react"; | |
| import Link from "next/link"; | ||
| import { Card } from "@/components/ui"; | ||
| import { DataTable, type Column, type Row } from "@/components/data-table"; | ||
| @@ -75,7 +75,7 @@ export function MemberVoteYear({ | ||
| return ( | ||
| <Card className="overflow-hidden"> | ||
| - | {/* open={open} is a controlled prop - onToggle keeps state in sync so | |
| + | {/* open={open} is a controlled prop - onToggle keeps state in sync so | |
| React never fights the browser's native toggle. */} | ||
| <details open={open} className="overflow-hidden" onToggle={handleToggle}> | ||
| <summary className="focus-ring flex cursor-pointer items-center justify-between gap-3 px-4 py-3.5"> | ||
| @@ -105,14 +105,20 @@ export function MemberVoteYear({ | ||
| ); | ||
| } | ||
| - | /** Compact vote table - mirrors the dense MemberVotesTable in the detail page. */ | |
| + | const PAGE = 50; | |
| + | ||
| + | /** Compact vote table - mirrors the dense MemberVotesTable in the detail page. | |
| + | * Paginates at 50 rows so a busy year (700+ roll-calls) never renders as one | |
| + | * tens-of-thousands-of-pixels flat dump (fix #4). */ | |
| function VoteTable({ votes }: { votes: MemberVote[] }) { | ||
| + | const [shown, setShown] = useState(PAGE); | |
| const columns: Column[] = [ | ||
| - | { key: "date", header: "Date" }, | |
| + | { key: "date", header: "Date", width: "6.5rem" }, | |
| { key: "result", header: "Question", cardTitle: true }, | ||
| - | { key: "voted", header: "Voted" }, | |
| + | { key: "voted", header: "Voted", width: "7rem" }, | |
| ]; | ||
| - | const rows: Row[] = votes.map((v) => ({ | |
| + | const visible = useMemo(() => votes.slice(0, shown), [votes, shown]); | |
| + | const rows: Row[] = visible.map((v) => ({ | |
| key: v.voteId, | ||
| cells: { | ||
| date: ( | ||
| @@ -133,11 +139,27 @@ function VoteTable({ votes }: { votes: MemberVote[] }) { | ||
| }, | ||
| })); | ||
| return ( | ||
| - | <DataTable | |
| - | caption="This member's recorded votes: date, question, result, position." | |
| - | cardLabel="Recorded votes" | |
| - | columns={columns} | |
| - | rows={rows} | |
| - | /> | |
| + | <> | |
| + | <DataTable | |
| + | caption="This member's recorded votes: date, question, result, position." | |
| + | cardLabel="Recorded votes" | |
| + | columns={columns} | |
| + | rows={rows} | |
| + | /> | |
| + | {votes.length > shown ? ( | |
| + | <div className="flex items-center justify-between gap-3 px-4 py-3"> | |
| + | <span className="label text-ash-2"> | |
| + | Showing {shown.toLocaleString()} of {votes.length.toLocaleString()} | |
| + | </span> | |
| + | <button | |
| + | type="button" | |
| + | onClick={() => setShown((n) => n + PAGE)} | |
| + | className="focus-ring label inline-flex min-h-[40px] items-center rounded-[3px] border border-line px-3 text-ash underline underline-offset-4 hover:text-paper" | |
| + | > | |
| + | Show {Math.min(PAGE, votes.length - shown)} more | |
| + | </button> | |
| + | </div> | |
| + | ) : null} | |
| + | </> | |
| ); | ||
| } |
| @@ -14,6 +14,14 @@ import { | ||
| import { clsx } from "@/lib/cx"; | ||
| import type { MemberSummary } from "@/lib/feed"; | ||
| + | /** District display helper - fix #11: senators never show a district number; | |
| + | * House At-Large (district null / -1 / 0) shows "At-Large". */ | |
| + | function formatDistrict(chamber: string, district: number | null): string { | |
| + | if (chamber === "senate") return ""; | |
| + | if (district === null || district <= 0) return " · At-Large"; | |
| + | return ` · Dist. ${district}`; | |
| + | } | |
| + | ||
| /** | ||
| * Client-side browse/filter for the members roster. | ||
| * | ||
| @@ -21,7 +29,7 @@ import type { MemberSummary } from "@/lib/feed"; | ||
| * with a recorded photo, then historical members without a photo behind a | ||
| * disclosure toggle. All filtering is local over the server-seeded list. | ||
| * Party chips render in desaturated, equal-weight tones; the list stays | ||
| - | * alphabetical within each tier (never grouped by party - neutrality §Viz). | |
| + | * alphabetical within each tier (never grouped by party - neutrality §Viz). | |
| * | ||
| * `seed` contains only the default-visible members (current + those with a | ||
| * photo). When the user clicks "Show all historical", the full list is fetched | ||
| @@ -54,7 +62,7 @@ export function MembersBrowser({ | ||
| try { | ||
| const res = await fetch("/api/members?limit=200"); | ||
| const first = await res.json(); | ||
| - | // /api/members paginates at 200 - for historical roster we need all. | |
| + | // /api/members paginates at 200 - for historical roster we need all. | |
| // Fetch in chunks until done. | ||
| const items: MemberSummary[] = [...first.items]; | ||
| let offset = first.items.length; | ||
| @@ -138,7 +146,7 @@ export function MembersBrowser({ | ||
| return ( | ||
| <> | ||
| - | {/* Search/filter bar - labeled controls, ≥44px targets, ≥18px search | |
| + | {/* Search/filter bar - labeled controls, ≥44px targets, ≥18px search | |
| text (ACCESSIBILITY.md §Search, Rule #2, Rule #3). Client-side | ||
| filtering over the server-seeded default-visible set. */} | ||
| <SearchFilterBar | ||
| @@ -206,7 +214,7 @@ export function MembersBrowser({ | ||
| onChange={setSort} | ||
| aria-label="Sort members" | ||
| options={[ | ||
| - | { value: "", label: "Default (current first)" }, | |
| + | { value: "", label: "Default" }, | |
| { value: "name", label: "Name (A-Z)" }, | ||
| { value: "state", label: "State (A-Z)" }, | ||
| { value: "chamber", label: "Chamber" }, | ||
| @@ -215,7 +223,7 @@ export function MembersBrowser({ | ||
| </FilterField> | ||
| </SearchFilterBar> | ||
| - | {/* Historical toggle - only shown when no filter is active */} | |
| + | {/* Historical toggle - only shown when no filter is active */} | |
| {!anyFilterActive && hiddenCount > 0 && ( | ||
| <div className="mt-3 flex justify-end"> | ||
| <button | ||
| @@ -266,7 +274,7 @@ export function MembersBrowser({ | ||
| <p className="label mt-1 text-ash-2"> | ||
| {m.chamber === "senate" ? "Senate" : "House"} ·{" "} | ||
| {m.state} | ||
| - | {m.district != null ? ` ${m.district}` : ""} | |
| + | {formatDistrict(m.chamber, m.district)} | |
| </p> | ||
| </div> | ||
| <PartyTag party={m.partyCode} /> |
| @@ -4,7 +4,7 @@ import { clsx } from "@/lib/cx"; | ||
| export type PartySlice = { party: PartyCode; count: number }; | ||
| /** | ||
| - | * Sponsorship / cosponsorship split by party - muted, equal-weight tones, | |
| + | * Sponsorship / cosponsorship split by party - muted, equal-weight tones, | |
| * displayed in fixed party-alphabetical order (D, I, R) rather than by | ||
| * majority, so the chart never "picks a winner." (neutrality §Viz) | ||
| */ |
| @@ -2,7 +2,7 @@ import { clsx } from "@/lib/cx"; | ||
| import { partyToken, type PartyCode } from "@/lib/neutrality"; | ||
| /** | ||
| - | * Party affiliation tag - factual data, shown in desaturated, equal-weight | |
| + | * Party affiliation tag - factual data, shown in desaturated, equal-weight | |
| * tones (neutrality §Viz). Never saturated red vs blue. | ||
| */ | ||
| export function PartyTag({ | ||
| @@ -13,7 +13,10 @@ export function PartyTag({ | ||
| className?: string; | ||
| }) { | ||
| const color = partyToken(party); | ||
| - | const name = party === "D" ? "Dem." : party === "R" ? "Rep." : "Ind."; | |
| + | // Single-letter code - "Rep." read ambiguously as both Republican and | |
| + | // Representative (a senator once showed "REP. KS"). The full party name | |
| + | // stays in the title tooltip. | |
| + | const name = party === "D" ? "D" : party === "R" ? "R" : "I"; | |
| return ( | ||
| <span | ||
| className={clsx("pill", className)} |
| @@ -3,7 +3,7 @@ import { clsx } from "@/lib/cx"; | ||
| export type Position = "Yea" | "Nay" | "Present" | "Not Voting" | string; | ||
| /** | ||
| - | * How a member voted - muted yea / nay / present tones (neutrality §Viz). | |
| + | * How a member voted - muted yea / nay / present tones (neutrality §Viz). | |
| * Never vivid red/green; equal visual weight so the eye does not read "teams." | ||
| */ | ||
| const POSITION_COLOR: Record<string, string> = { |
| @@ -8,13 +8,13 @@ import { formatEt, formatDateShort } from "@/lib/format"; | ||
| import type { FeedItem } from "@/lib/feed"; | ||
| /** | ||
| - | * THE REGISTER - Docket's signature hero (docs/BRAND.md §3). | |
| + | * THE REGISTER - Docket's signature hero (docs/BRAND.md §3). | |
| * A stamped ledger of official actions. Seeded server-side with REAL feed | ||
| * items, then re-fetched client-side on an interval so newly ingested actions | ||
| - | * appear live. Falls back to a calm empty state when empty - never mock data. | |
| + | * appear live. Falls back to a calm empty state when empty - never mock data. | |
| * | ||
| * Hydration-safe (mobile-first overhaul): the server render and the first | ||
| - | * client render produce IDENTICAL markup - `flashId` starts null (no | |
| + | * client render produce IDENTICAL markup - `flashId` starts null (no | |
| * animate-fadeUp class anywhere on first paint), and the live re-fetch + flash | ||
| * only begin inside a post-mount useEffect. Date/time text is produced by pure, | ||
| * deterministic helpers (formatEt / formatDateShort) that parse date-only | ||
| @@ -44,7 +44,7 @@ export function RegisterHero({ seed }: { seed: FeedItem[] }) { | ||
| return next; | ||
| }); | ||
| } catch { | ||
| - | /* offline / transient - keep the seeded rows */ | |
| + | /* offline / transient - keep the seeded rows */ | |
| } | ||
| }, []); | ||
| @@ -70,7 +70,7 @@ export function RegisterHero({ seed }: { seed: FeedItem[] }) { | ||
| <div className="flex flex-col gap-4 sm:flex-row sm:items-end sm:justify-between"> | ||
| <div> | ||
| <div className="mb-3 flex items-center gap-2.5"> | ||
| - | {/* ONE brass signal per view - the pulsing live-dot */} | |
| + | {/* ONE brass signal per view - the pulsing live-dot */} | |
| <span className="live-dot animate-pulseDot" aria-hidden="true" /> | ||
| <span className="label text-paper">Live · The Register</span> | ||
| </div> | ||
| @@ -94,7 +94,7 @@ export function RegisterHero({ seed }: { seed: FeedItem[] }) { | ||
| className="mt-8 overflow-hidden rounded-[3px] border border-line bg-surface/60 backdrop-blur-sm" | ||
| aria-label="Stamped ledger of official actions" | ||
| > | ||
| - | {/* Column header - grid only at sm+ (mobile rows are stacked cards). */} | |
| + | {/* Column header - grid only at sm+ (mobile rows are stacked cards). */} | |
| <div className="reg-row hidden grid-cols-[5.5rem_7.5rem_minmax(0,1fr)_6.5rem_minmax(0,5rem)] gap-x-2 px-4 py-2 sm:grid"> | ||
| <span className="label">Date</span> | ||
| <span className="label">Body</span> | ||
| @@ -107,7 +107,7 @@ export function RegisterHero({ seed }: { seed: FeedItem[] }) { | ||
| const isNew = r.id === flashId; | ||
| return ( | ||
| <div key={r.id} className="border-t border-line first:border-t-0"> | ||
| - | {/* The row is a plain container - NOT a wrapping <a>. The item | |
| + | {/* The row is a plain container - NOT a wrapping <a>. The item | |
| title and the Source link are sibling anchors, so we never | ||
| produce invalid nested <a><a></a></a> (which was the | ||
| hydration #418: browsers auto-close the outer anchor, the |
| @@ -1,7 +1,7 @@ | ||
| import { clsx } from "@/lib/cx"; | ||
| /** | ||
| - | * Rider flag - marks a section that entered a bill outside its named subject | |
| + | * Rider flag - marks a section that entered a bill outside its named subject | |
| * (Docket's signature "what got slipped in" signal). Brass is rationed; this | ||
| * is one of the few places the brand accent is used as an explicit flag. | ||
| * Plain mode collapses to ink (no accent) per globals.css. |
| @@ -7,8 +7,17 @@ import { Card } from "@/components/ui"; | ||
| import { SourceLink } from "@/components/source-link"; | ||
| import { MemberAvatar } from "@/components/member-avatar"; | ||
| import { SearchBox } from "@/components/search-box"; | ||
| + | import { ShowMore } from "@/components/show-more"; | |
| + | import { congressLabel, formatBillNumber } from "@/lib/format"; | |
| import type { BillListItem, MemberSummary, ExecutiveAction, StateCoverage } from "@/lib/feed"; | ||
| + | /** District display helper - fix #11 */ | |
| + | function formatDistrict(chamber: string, district: number | null): string { | |
| + | if (chamber === "senate") return ""; | |
| + | if (district === null || district <= 0) return " · At-Large"; | |
| + | return ` · Dist. ${district}`; | |
| + | } | |
| + | ||
| type SearchResultAward = { | ||
| id: string; | ||
| agency: string; | ||
| @@ -233,7 +242,7 @@ export function SearchBrowser() { | ||
| )} | ||
| </div> | ||
| - | {/* Sort control - GET form preserving q + type */} | |
| + | {/* Sort control - GET form preserving q + type */} | |
| <form method="get" action="/search" className="flex items-center gap-2"> | ||
| <input type="hidden" name="q" value={q} /> | ||
| {activeType !== "all" && ( | ||
| @@ -278,95 +287,114 @@ export function SearchBrowser() { | ||
| <div className="space-y-10"> | ||
| {showMembers && results.members.length > 0 ? ( | ||
| <ResultGroup title="Members" count={results.members.length}> | ||
| - | {results.members.map((m) => ( | |
| - | <Link | |
| - | key={m.id} | |
| - | href={m.href} | |
| - | className="focus-ring group flex items-center gap-3 border-b border-line p-3 transition-colors last:border-0 hover:bg-graphite/60" | |
| - | > | |
| - | <MemberAvatar name={m.name} photoUrl={m.photoUrl} size="sm" /> | |
| - | <span className="min-w-0"> | |
| - | <span className="block font-mono text-[13px] text-paper group-hover:underline"> | |
| - | {m.name} | |
| + | <ShowMore | |
| + | pageSize={50} | |
| + | items={results.members.map((m) => ( | |
| + | <Link | |
| + | key={m.id} | |
| + | href={m.href} | |
| + | className="focus-ring group flex items-center gap-3 border-b border-line p-3 transition-colors last:border-0 hover:bg-graphite/60" | |
| + | > | |
| + | <MemberAvatar name={m.name} photoUrl={m.photoUrl} size="sm" /> | |
| + | <span className="min-w-0"> | |
| + | <span className="block font-mono text-[13px] text-paper group-hover:underline"> | |
| + | {m.name} | |
| + | </span> | |
| + | <span className="label text-ash-2"> | |
| + | {m.chamber === "senate" ? "Senate" : "House"} · {m.state} | |
| + | {formatDistrict(m.chamber, m.district)} · {m.party} | |
| + | </span> | |
| </span> | ||
| - | <span className="label text-ash-2"> | |
| - | {m.chamber === "senate" ? "Senate" : "House"} · {m.state} | |
| - | {m.district != null ? ` ${m.district}` : ""} · {m.party} | |
| - | </span> | |
| - | </span> | |
| - | </Link> | |
| - | ))} | |
| + | </Link> | |
| + | ))} | |
| + | /> | |
| </ResultGroup> | ||
| ) : null} | ||
| {showBills && results.bills.length > 0 ? ( | ||
| <ResultGroup title="Bills" count={results.bills.length}> | ||
| - | {results.bills.map((b) => ( | |
| - | <Link | |
| - | key={b.id} | |
| - | href={b.href} | |
| - | className="focus-ring group block border-b border-line p-3 transition-colors last:border-0 hover:bg-graphite/60" | |
| - | > | |
| - | <p className="font-mono text-[13px] text-paper group-hover:underline"> | |
| - | {b.number} · {b.congress}th Congress | |
| - | </p> | |
| - | <p className="font-mono text-[12px] text-ash">{b.title}</p> | |
| - | <p className="label mt-1 text-ash-2">{b.policyArea}</p> | |
| - | </Link> | |
| - | ))} | |
| + | <ShowMore | |
| + | pageSize={50} | |
| + | items={results.bills.map((b) => { | |
| + | const cong = congressLabel(b.congress); | |
| + | return ( | |
| + | <Link | |
| + | key={b.id} | |
| + | href={b.href} | |
| + | className="focus-ring group block border-b border-line p-3 transition-colors last:border-0 hover:bg-graphite/60" | |
| + | > | |
| + | <p className="font-mono text-[13px] text-paper group-hover:underline"> | |
| + | {formatBillNumber(b.number)} | |
| + | {cong ? ` · ${cong}` : ""} | |
| + | </p> | |
| + | <p className="font-mono text-[12px] text-ash">{b.title}</p> | |
| + | <p className="label mt-1 text-ash-2">{b.policyArea}</p> | |
| + | </Link> | |
| + | ); | |
| + | })} | |
| + | /> | |
| </ResultGroup> | ||
| ) : null} | ||
| {showExec && results.executive.length > 0 ? ( | ||
| <ResultGroup title="Federal Register" count={results.executive.length}> | ||
| - | {results.executive.map((e) => ( | |
| - | <div | |
| - | key={e.id} | |
| - | className="flex items-start justify-between gap-3 border-b border-line p-3 last:border-0" | |
| - | > | |
| - | <div className="min-w-0"> | |
| - | <p className="font-mono text-[13px] text-paper">{e.title}</p> | |
| - | <p className="label mt-1 text-ash-2"> | |
| - | {e.agency} | |
| - | {e.date ? ` · ${e.date.slice(0, 10)}` : ""} | |
| - | </p> | |
| + | <ShowMore | |
| + | pageSize={50} | |
| + | items={results.executive.map((e) => ( | |
| + | <div | |
| + | key={e.id} | |
| + | className="flex items-start justify-between gap-3 border-b border-line p-3 last:border-0" | |
| + | > | |
| + | <div className="min-w-0"> | |
| + | <p className="font-mono text-[13px] text-paper">{e.title}</p> | |
| + | <p className="label mt-1 text-ash-2"> | |
| + | {e.agency} | |
| + | {e.date ? ` · ${e.date.slice(0, 10)}` : ""} | |
| + | </p> | |
| + | </div> | |
| + | <SourceLink url={e.sourceUrl} label="FR" /> | |
| </div> | ||
| - | <SourceLink url={e.sourceUrl} label="FR" /> | |
| - | </div> | |
| - | ))} | |
| + | ))} | |
| + | /> | |
| </ResultGroup> | ||
| ) : null} | ||
| {showAwards && results.awards.length > 0 ? ( | ||
| <ResultGroup title="Federal awards" count={results.awards.length}> | ||
| - | {results.awards.map((a) => ( | |
| - | <div | |
| - | key={a.id} | |
| - | className="flex items-start justify-between gap-3 border-b border-line p-3 last:border-0" | |
| - | > | |
| - | <div className="min-w-0"> | |
| - | <p className="font-mono text-[13px] text-paper">{a.agency}</p> | |
| - | <p className="label mt-1 text-ash-2">{a.recipient}</p> | |
| + | <ShowMore | |
| + | pageSize={50} | |
| + | items={results.awards.map((a) => ( | |
| + | <div | |
| + | key={a.id} | |
| + | className="flex items-start justify-between gap-3 border-b border-line p-3 last:border-0" | |
| + | > | |
| + | <div className="min-w-0"> | |
| + | <p className="font-mono text-[13px] text-paper">{a.agency}</p> | |
| + | <p className="label mt-1 text-ash-2">{a.recipient}</p> | |
| + | </div> | |
| + | <SourceLink url={a.sourceUrl} label="USAspending" /> | |
| </div> | ||
| - | <SourceLink url={a.sourceUrl} label="USAspending" /> | |
| - | </div> | |
| - | ))} | |
| + | ))} | |
| + | /> | |
| </ResultGroup> | ||
| ) : null} | ||
| {showStates && results.states.length > 0 ? ( | ||
| <ResultGroup title="States" count={results.states.length}> | ||
| - | {results.states.map((s) => ( | |
| - | <Link | |
| - | key={s.code} | |
| - | href={`/states/${s.code}`} | |
| - | className="focus-ring group block border-b border-line p-3 transition-colors last:border-0 hover:bg-graphite/60" | |
| - | > | |
| - | <p className="font-mono text-[13px] text-paper group-hover:underline"> | |
| - | {s.code} · {s.name} | |
| - | </p> | |
| - | </Link> | |
| - | ))} | |
| + | <ShowMore | |
| + | pageSize={50} | |
| + | items={results.states.map((s) => ( | |
| + | <Link | |
| + | key={s.code} | |
| + | href={`/states/${s.code}`} | |
| + | className="focus-ring group block border-b border-line p-3 transition-colors last:border-0 hover:bg-graphite/60" | |
| + | > | |
| + | <p className="font-mono text-[13px] text-paper group-hover:underline"> | |
| + | {s.code} · {s.name} | |
| + | </p> | |
| + | </Link> | |
| + | ))} | |
| + | /> | |
| </ResultGroup> | ||
| ) : null} | ||
| </div> |
| @@ -1,5 +1,5 @@ | ||
| /** | ||
| - | * SearchFilterBar - shared accessible filter-card shell used across all list | |
| + | * SearchFilterBar - shared accessible filter-card shell used across all list | |
| * pages (members, bills, votes, spending, executive, schedule). | ||
| * | ||
| * Works for both server-side forms (method="get") and client-side controlled | ||
| @@ -11,11 +11,11 @@ | ||
| * Accessibility (docs/ACCESSIBILITY.md): | ||
| * • All inputs: min-h-[44px] tap targets (Rule #3) | ||
| * • Search input: min-h-[52px], font-size 18px, label above (not | ||
| - | * placeholder-only) - ACCESSIBILITY.md §Search / Rule #2 | |
| + | * placeholder-only) - ACCESSIBILITY.md §Search / Rule #2 | |
| * • Focus rings on every control (Rule #4) | ||
| * • Works in Reference / Dark / Plain themes via CSS tokens | ||
| * | ||
| - | * No "use client" - this is a pure RSC-compatible component. Client-side | |
| + | * No "use client" - this is a pure RSC-compatible component. Client-side | |
| * callers (MembersBrowser) import it into their own client boundary. | ||
| */ | ||
| @@ -36,12 +36,12 @@ export function SearchFilterBar({ | ||
| children: ReactNode; | ||
| /** Number of visible results (after filtering). */ | ||
| count?: number; | ||
| - | /** Total unfiltered count - shown as "N of total". */ | |
| + | /** Total unfiltered count - shown as "N of total". */ | |
| total?: number; | ||
| countLabel?: string; | ||
| - | /** Client-side clear callback - shows a "Clear filters" button. */ | |
| + | /** Client-side clear callback - shows a "Clear filters" button. */ | |
| onClear?: () => void; | ||
| - | /** Server-side clear href - shows a "Clear filters" link. */ | |
| + | /** Server-side clear href - shows a "Clear filters" link. */ | |
| clearHref?: string; | ||
| className?: string; | ||
| }) { | ||
| @@ -83,7 +83,7 @@ export function SearchFilterBar({ | ||
| ); | ||
| } | ||
| - | /** Labeled field wrapper - label above, not placeholder-only (ACCESSIBILITY.md §Search). */ | |
| + | /** Labeled field wrapper - label above, not placeholder-only (ACCESSIBILITY.md §Search). */ | |
| export function FilterField({ | ||
| label, | ||
| span, | ||
| @@ -102,7 +102,7 @@ export function FilterField({ | ||
| ); | ||
| } | ||
| - | /** Standard text-search input (≥52px tall, 18px text - ACCESSIBILITY.md §Search). */ | |
| + | /** Standard text-search input (≥52px tall, 18px text - ACCESSIBILITY.md §Search). */ | |
| export function FilterInput({ | ||
| name, | ||
| value, |
| @@ -1,5 +1,6 @@ | ||
| import Link from "next/link"; | ||
| import { CustodyMark } from "@/components/custody-mark"; | ||
| + | import { OwlBust } from "@/components/owl"; | |
| export function SiteFooter() { | ||
| const year = new Date().getFullYear(); | ||
| @@ -17,6 +18,10 @@ export function SiteFooter() { | ||
| <p className="prose-lead mt-4 italic text-ash"> | ||
| Record the action, not the argument. | ||
| </p> | ||
| + | <div className="mt-6 flex items-center gap-3"> | |
| + | <OwlBust size={40} className="shrink-0 opacity-90" title="Archie, the Docket archivist" /> | |
| + | <span className="label text-ash-2">Docket - the public record</span> | |
| + | </div> | |
| </div> | ||
| <nav aria-label="Footer" className="grid grid-cols-2 gap-x-10 gap-y-2 sm:grid-cols-3"> |
| @@ -8,17 +8,18 @@ import { DisplayToggle } from "@/components/display-toggle"; | ||
| import { clsx } from "@/lib/cx"; | ||
| /** | ||
| - | * Site header - single-row, treetrace/zionboggan aesthetic (DESIGN_V3). | |
| + | * Site header - single-row, treetrace/zionboggan aesthetic (DESIGN_V3). | |
| * | ||
| * Layout: wordmark (left) | nav links (center, desktop only) | search + toggle + hamburger (right) | ||
| * Type: JetBrains Mono uppercase tracked nav links (≥14px) | ||
| * Active: 2px brass underline, no background fill | ||
| * Mobile: slide-in panel from right (like treetrace tt-mobile pattern) | ||
| - | * Sticky + backdrop-blur - stays in layout flow (no padding-top hack needed) | |
| + | * Sticky + backdrop-blur - stays in layout flow (no padding-top hack needed) | |
| */ | ||
| const NAV: Array<{ href: string; label: string }> = [ | ||
| { href: "/", label: "Home" }, | ||
| { href: "/bills", label: "Bills" }, | ||
| + | { href: "/laws", label: "Laws" }, | |
| { href: "/votes", label: "Votes" }, | ||
| { href: "/members", label: "Reps" }, | ||
| { href: "/spending", label: "Spending" }, | ||
| @@ -72,15 +73,15 @@ export function SiteHeader() { | ||
| aria-label="Docket - home" | ||
| style={{ display: "inline-flex", alignItems: "center", gap: "10px" }} | ||
| > | ||
| - | <CustodyMark size={26} showTick={false} /> | |
| + | <CustodyMark size={28} /> | |
| <Wordmark className="font-display text-paper" /> | ||
| </Link> | ||
| - | {/* Desktop nav - center */} | |
| + | {/* Desktop nav - center */} | |
| <nav | ||
| aria-label="Primary" | ||
| className="hidden md:flex" | ||
| - | style={{ marginLeft: "16px", gap: "2px" }} | |
| + | style={{ marginLeft: "12px", gap: "0px" }} | |
| > | ||
| {NAV.map((item) => { | ||
| const active = isActive(pathname, item.href); | ||
| @@ -94,10 +95,11 @@ export function SiteHeader() { | ||
| fontFamily: "var(--font-mono), ui-monospace, monospace", | ||
| fontSize: "11.5px", | ||
| fontWeight: 400, | ||
| - | letterSpacing: "0.16em", | |
| + | letterSpacing: "0.13em", | |
| textTransform: "uppercase", | ||
| color: active ? "var(--dk-paper)" : "var(--dk-ash)", | ||
| - | padding: "8px 12px", | |
| + | padding: "8px 7px", | |
| + | whiteSpace: "nowrap", | |
| borderRadius: "6px", | ||
| position: "relative", | ||
| transition: "color 0.2s", | ||
| @@ -110,8 +112,8 @@ export function SiteHeader() { | ||
| aria-hidden="true" | ||
| style={{ | ||
| position: "absolute", | ||
| - | left: "12px", | |
| - | right: "12px", | |
| + | left: "7px", | |
| + | right: "7px", | |
| bottom: "2px", | ||
| height: "2px", | ||
| background: "var(--dk-signal)", | ||
| @@ -153,12 +155,12 @@ export function SiteHeader() { | ||
| }} | ||
| > | ||
| <SearchIcon /> | ||
| - | <span className="hidden sm:inline">Search</span> | |
| + | <span className="hidden lg:inline">Search</span> | |
| </Link> | ||
| <DisplayToggle /> | ||
| - | {/* Hamburger (mobile only - md:hidden hides on >=768px) */} | |
| + | {/* Hamburger (mobile only - md:hidden hides on >=768px) */} | |
| <button | ||
| type="button" | ||
| onClick={() => setOpen((v) => !v)} | ||
| @@ -167,6 +169,7 @@ export function SiteHeader() { | ||
| aria-controls="primary-mobile-nav" | ||
| className="focus-ring md:hidden" | ||
| style={{ | ||
| + | display: "flex", | |
| alignItems: "center", | ||
| justifyContent: "center", | ||
| flexDirection: "column", |
| @@ -4,7 +4,7 @@ import { isAllowedSourceHost } from "@/lib/neutrality"; | ||
| /** | ||
| * Visible "Source" link on every fact (neutrality rule #1). | ||
| * Hosts must be in the allowlist; otherwise we refuse to render the link | ||
| - | * as a "Source" and show a guard marker instead - the build-time lint also | |
| + | * as a "Source" and show a guard marker instead - the build-time lint also | |
| * rejects these, so this is a defense-in-depth. | ||
| */ | ||
| export function SourceLink({ | ||
| @@ -18,7 +18,7 @@ export function SourceLink({ | ||
| }) { | ||
| const allowed = isAllowedSourceHost(url); | ||
| const host = safeHost(url); | ||
| - | // Only ever emit http(s) hrefs - guard against javascript:/data: schemes | |
| + | // Only ever emit http(s) hrefs - guard against javascript:/data: schemes | |
| // even if a bad URL slipped through ingest (defense-in-depth). | ||
| const safeHref = /^https?:\/\//i.test(url) ? url : "#"; | ||
| return ( | ||
| @@ -29,7 +29,9 @@ export function SourceLink({ | ||
| className={clsx( | ||
| // a11y #3/#7: every Source link is a >=40px underlined target, never | ||
| // color alone. `label` keeps the mono look; min-h keeps it tappable. | ||
| - | "focus-ring label inline-flex min-h-[40px] items-center gap-1 rounded-[3px] border border-line px-2 py-1 underline underline-offset-4 transition-colors", | |
| + | // flex-wrap + max-w-full let the host wrap to a second line inside a | |
| + | // narrow table cell instead of overflowing and being clipped (fix #1). | |
| + | "focus-ring label inline-flex min-h-[40px] max-w-full flex-wrap items-center gap-x-1 gap-y-0.5 rounded-[3px] border border-line px-2 py-1 underline underline-offset-4 transition-colors", | |
| allowed | ||
| ? "text-ash hover:text-paper hover:border-paper" | ||
| : "text-nay", | ||
| @@ -38,8 +40,12 @@ export function SourceLink({ | ||
| title={allowed ? `Official record · ${host}` : "Source host not allowlisted"} | ||
| > | ||
| <span aria-hidden="true">↗</span> | ||
| - | <span>{label}</span> | |
| - | {allowed && host ? <span className="text-ash-2">· {host}</span> : null} | |
| + | <span className="whitespace-nowrap">{label}</span> | |
| + | {allowed && host ? ( | |
| + | <span className="text-[11px] tracking-normal text-ash-2 [overflow-wrap:anywhere]"> | |
| + | · {host} | |
| + | </span> | |
| + | ) : null} | |
| </a> | ||
| ); | ||
| } |
| @@ -30,13 +30,17 @@ function AwardsSkeleton() { | ||
| } | ||
| function AwardsTable({ awards }: { awards: Award[] }) { | ||
| + | // Award action dates are frequently absent in the ingest; drop the Date | |
| + | // column entirely when not a single award carries one, rather than show a | |
| + | // dead " - " column (fix #5). | |
| + | const hasDate = awards.some((a) => a.actionDate && a.actionDate.trim()); | |
| const columns: Column[] = [ | ||
| { key: "agency", header: "Agency", cardTitle: true }, | ||
| { key: "recipient", header: "Recipient" }, | ||
| { key: "program", header: "Program", hideOnCard: true }, | ||
| - | { key: "date", header: "Date" }, | |
| - | { key: "amount", header: "Amount", align: "right" }, | |
| - | { key: "source", header: "Source", align: "right" }, | |
| + | ...(hasDate ? [{ key: "date", header: "Date" } as Column] : []), | |
| + | { key: "amount", header: "Amount", align: "right" as const }, | |
| + | { key: "source", header: "Source", align: "right" as const }, | |
| ]; | ||
| const rows: Row[] = awards.map((a) => ({ | ||
| key: a.id, | ||
| @@ -44,17 +48,26 @@ function AwardsTable({ awards }: { awards: Award[] }) { | ||
| agency: <span className="text-paper">{a.agency}</span>, | ||
| recipient: <span className="text-ash">{a.recipient}</span>, | ||
| program: <span className="text-ash">{a.program}</span>, | ||
| - | date: <span className="tabular-nums text-ash-2">{formatDate(a.actionDate)}</span>, | |
| + | ...(hasDate | |
| + | ? { | |
| + | date: ( | |
| + | <span className="tabular-nums text-ash-2"> | |
| + | {formatDate(a.actionDate)} | |
| + | </span> | |
| + | ), | |
| + | } | |
| + | : {}), | |
| amount: <span className="tabular-nums text-paper">{a.amountText}</span>, | ||
| source: <SourceLink url={a.sourceUrl} />, | ||
| }, | ||
| })); | ||
| return ( | ||
| <DataTable | ||
| - | caption="Federal awards: agency, recipient, program, date, amount, source." | |
| + | caption="Federal awards: agency, recipient, program, amount, source." | |
| cardLabel="Federal awards" | ||
| columns={columns} | ||
| rows={rows} | ||
| + | pageSize={50} | |
| /> | ||
| ); | ||
| } | ||
| @@ -116,9 +129,9 @@ export function SpendingAwardsBrowser() { | ||
| defaultValue={sortParam} | ||
| aria-label="Sort awards" | ||
| options={[ | ||
| - | { value: "", label: "Largest amount first" }, | |
| - | { value: "date", label: "Most recent first" }, | |
| - | { value: "agency", label: "By agency (A-Z)" }, | |
| + | { value: "", label: "Largest first" }, | |
| + | { value: "date", label: "Most recent" }, | |
| + | { value: "agency", label: "By agency" }, | |
| ]} | ||
| /> | ||
| </FilterField> | ||
| @@ -132,7 +145,7 @@ export function SpendingAwardsBrowser() { | ||
| <SectionHeader | ||
| eyebrow="Federal awards" | ||
| title="Contracts, grants, and assistance" | ||
| - | description={`${awards.length} award${awards.length === 1 ? "" : "s"}${isFiltered ? " match" : ""} from USAspending.gov, totaling ${formatUsd(totalAwards, { compact: true })}. Largest first.`} | |
| + | description={`${awards.length} award${awards.length === 1 ? "" : "s"}${isFiltered ? " match" : ""} from USAspending.gov, largest first. The ${awards.length} shown here total ${formatUsd(totalAwards, { compact: true })}.`} | |
| /> | ||
| {awards.length === 0 ? ( | ||
| <Card className="p-8 text-center"> |
| @@ -3,7 +3,7 @@ import { clsx } from "@/lib/cx"; | ||
| export type StatusVariant = "passed" | "failed" | "held" | "pending" | "neutral"; | ||
| const VARIANT_TOKEN: Record<StatusVariant, { color: string; label?: string }> = { | ||
| - | // muted semantic tones only - never vivid (neutrality §Viz) | |
| + | // muted semantic tones only - never vivid (neutrality §Viz) | |
| passed: { color: "var(--dk-yea)" }, | ||
| failed: { color: "var(--dk-nay)" }, | ||
| held: { color: "var(--dk-hold)" }, | ||
| @@ -13,7 +13,7 @@ const VARIANT_TOKEN: Record<StatusVariant, { color: string; label?: string }> = | ||
| /** | ||
| * Mono status pill. Color comes from the muted semantic tokens. | ||
| - | * No brass here - brass is rationed to one signal per view (the live-dot). | |
| + | * No brass here - brass is rationed to one signal per view (the live-dot). | |
| */ | ||
| export function StatusPill({ | ||
| variant = "neutral", |
| @@ -28,7 +28,7 @@ export function Card({ | ||
| ); | ||
| } | ||
| - | /** Page section header - mono label + optional display headline. */ | |
| + | /** Page section header - mono label + optional display headline. */ | |
| export function SectionHeader({ | ||
| eyebrow, | ||
| title, | ||
| @@ -58,7 +58,7 @@ export function SectionHeader({ | ||
| ); | ||
| } | ||
| - | /** Page header block used at the top of each route. */ | |
| + | /** Page header block used at the top of each route - V3 style. */ | |
| export function PageHeader({ | ||
| eyebrow, | ||
| title, | ||
| @@ -69,15 +69,70 @@ export function PageHeader({ | ||
| lead?: string; | ||
| }) { | ||
| return ( | ||
| - | <header className="border-b border-line pb-8"> | |
| - | <p className="label mb-3">{eyebrow}</p> | |
| + | <header | |
| + | style={{ | |
| + | borderBottom: "1px solid var(--dk-line)", | |
| + | paddingBottom: "clamp(28px, 5vw, 48px)", | |
| + | marginBottom: "clamp(28px, 5vw, 48px)", | |
| + | }} | |
| + | > | |
| + | {/* Brass dot kicker */} | |
| + | <p | |
| + | className="label" | |
| + | style={{ | |
| + | color: "var(--dk-signal)", | |
| + | marginBottom: "20px", | |
| + | display: "flex", | |
| + | alignItems: "center", | |
| + | gap: "8px", | |
| + | fontSize: "12px", | |
| + | letterSpacing: "0.14em", | |
| + | }} | |
| + | > | |
| + | <span | |
| + | aria-hidden="true" | |
| + | style={{ | |
| + | width: "5px", | |
| + | height: "5px", | |
| + | borderRadius: "50%", | |
| + | background: "var(--dk-signal)", | |
| + | display: "inline-block", | |
| + | flexShrink: 0, | |
| + | }} | |
| + | /> | |
| + | {eyebrow} | |
| + | </p> | |
| + | {/* HUGE Archivo uppercase headline */} | |
| <h1 | ||
| - | className="display text-paper" | |
| - | style={{ fontSize: "clamp(1.875rem, 7vw, 3rem)", lineHeight: 1 }} | |
| + | className="font-display text-paper" | |
| + | style={{ | |
| + | fontSize: "clamp(2.2rem, 7vw, 3.75rem)", | |
| + | fontWeight: 900, | |
| + | lineHeight: 1.0, | |
| + | letterSpacing: "-0.01em", | |
| + | wordSpacing: "0.06em", | |
| + | textTransform: "uppercase", | |
| + | maxWidth: "18ch", | |
| + | marginBottom: lead ? "24px" : 0, | |
| + | }} | |
| > | ||
| {title} | ||
| </h1> | ||
| - | {lead ? <p className="prose-lead mt-4 max-w-2xl text-ash">{lead}</p> : null} | |
| + | {/* Fraunces italic lead */} | |
| + | {lead ? ( | |
| + | <p | |
| + | className="font-prose" | |
| + | style={{ | |
| + | fontStyle: "italic", | |
| + | fontSize: "clamp(1rem, 1.8vw, 1.125rem)", | |
| + | lineHeight: 1.55, | |
| + | color: "var(--dk-ash)", | |
| + | maxWidth: "52ch", | |
| + | }} | |
| + | > | |
| + | {lead} | |
| + | </p> | |
| + | ) : null} | |
| </header> | ||
| ); | ||
| } |
| @@ -1,12 +1,12 @@ | ||
| import { clsx } from "@/lib/cx"; | ||
| /** | ||
| - | * Vote tally - muted yea / nay visualization (neutrality §Viz). | |
| + | * Vote tally - muted yea / nay visualization (neutrality §Viz). | |
| * Yea uses --dk-yea (muted sage), Nay uses --dk-nay (muted clay). | ||
| * Equal visual weight; a thin hold/present sliver is rendered in ash when | ||
| * present > 0. Bars are proportional, never oversaturated. | ||
| * | ||
| - | * Default ordering: yea then nay then present, by-count (not by party) - | |
| + | * Default ordering: yea then nay then present, by-count (not by party) - | |
| * grouping by party primes partisan reading and is forbidden. | ||
| */ | ||
| export function VoteTally({ |
| @@ -6,6 +6,7 @@ import Link from "next/link"; | ||
| import { Card, SectionHeader } from "@/components/ui"; | ||
| import { DataTable, type Column, type Row } from "@/components/data-table"; | ||
| import { SourceLink } from "@/components/source-link"; | ||
| + | import { ArchiveLink } from "@/components/record-archive"; | |
| import { StatusPill, classifyStatus } from "@/components/status-pill"; | ||
| import { | ||
| SearchFilterBar, | ||
| @@ -40,11 +41,11 @@ function VotesSkeleton() { | ||
| function VotesTable({ votes }: { votes: Vote[] }) { | ||
| const columns: Column[] = [ | ||
| - | { key: "date", header: "Date" }, | |
| - | { key: "chamber", header: "Chamber", hideOnCard: true }, | |
| + | { key: "date", header: "Date", width: "6.5rem" }, | |
| + | { key: "chamber", header: "Chamber", hideOnCard: true, width: "6rem" }, | |
| { key: "what", header: "What was voted on", cardTitle: true }, | ||
| - | { key: "tally", header: "Tally", align: "right" }, | |
| - | { key: "record", header: "Record", align: "right" }, | |
| + | { key: "tally", header: "Tally", align: "right", width: "6rem" }, | |
| + | { key: "record", header: "Record", align: "right", width: "13rem" }, | |
| ]; | ||
| const rows: Row[] = votes.map((v) => { | ||
| const variant = classifyStatus(v.result); | ||
| @@ -87,7 +88,12 @@ function VotesTable({ votes }: { votes: Vote[] }) { | ||
| {v.recorded ? `${v.yea}-${v.nay}` : "-"} | ||
| </span> | ||
| ), | ||
| - | record: <SourceLink url={v.sourceUrl} />, | |
| + | record: ( | |
| + | <span className="inline-flex flex-col items-start gap-1.5 sm:items-end"> | |
| + | <SourceLink url={v.sourceUrl} /> | |
| + | {v.archiveUrl ? <ArchiveLink url={v.archiveUrl} label="Archived" /> : null} | |
| + | </span> | |
| + | ), | |
| }, | ||
| }; | ||
| }); | ||
| @@ -97,6 +103,7 @@ function VotesTable({ votes }: { votes: Vote[] }) { | ||
| cardLabel="Recorded votes" | ||
| columns={columns} | ||
| rows={rows} | ||
| + | pageSize={50} | |
| /> | ||
| ); | ||
| } | ||
| @@ -143,7 +150,7 @@ export function VotesBrowser() { | ||
| return ( | ||
| <div> | ||
| - | {/* Search/filter bar - client-side via GET form. */} | |
| + | {/* Search/filter bar - client-side via GET form. */} | |
| <div className="mt-8"> | ||
| <form method="get" action="/votes" className="contents"> | ||
| <SearchFilterBar | ||
| @@ -179,8 +186,8 @@ export function VotesBrowser() { | ||
| aria-label="Filter by vote result" | ||
| options={[ | ||
| { value: "", label: "All results" }, | ||
| - | { value: "passed", label: "Passed (Yea majority)" }, | |
| - | { value: "failed", label: "Failed (Nay majority)" }, | |
| + | { value: "passed", label: "Passed" }, | |
| + | { value: "failed", label: "Failed" }, | |
| ]} | ||
| /> | ||
| </FilterField> |
| @@ -1,6 +1,6 @@ | ||
| - | # Docket - Accessibility & Plain-Language Spec | |
| + | # Docket - Accessibility & Plain-Language Spec | |
| - | Hard requirement (PLAN.md §1c): Docket must be usable by **any** citizen - explicitly an 80-year-old with low vision, low tech-literacy, keyboard-only or imprecise motor control. The forensic dark brand is the *default*, not a barrier. The brand wins by being **usable**, not just sharp. | |
| + | Hard requirement (PLAN.md §1c): Docket must be usable by **any** citizen - explicitly an 80-year-old with low vision, low tech-literacy, keyboard-only or imprecise motor control. The forensic dark brand is the *default*, not a barrier. The brand wins by being **usable**, not just sharp. | |
| ## The 8 rules (enforce in code / CI) | ||
| | # | Rule | Number | | ||
| @@ -15,7 +15,7 @@ Hard requirement (PLAN.md §1c): Docket must be usable by **any** citizen - ex | ||
| | 8 | Text-size / mode toggle | Always-visible, labeled, 44×44px. Scales text 1.5-2×, no layout break. | | ||
| ## Three display modes (one toggle, persisted) | ||
| - | 1. **Default - Dark forensic (brand):** ink `#121316` base, paper text, brass rationed, mono. Enforce: every pair ≥4.5:1, body 18px, line-height 1.6, accents muted (no vibrating saturated colors on dark). | |
| + | 1. **Default - Dark forensic (brand):** ink `#121316` base, paper text, brass rationed, mono. Enforce: every pair ≥4.5:1, body 18px, line-height 1.6, accents muted (no vibrating saturated colors on dark). | |
| 2. **Readable / Light:** parchment `#F6F4EF` bg, ink text, body 16px, calm saturated accents. The "I just want to read it" mode. | ||
| 3. **High-contrast "Plain Mode":** system sans (not mono), ≥20px, max contrast (black on `#FFFFCC` or inverted), no accent color, uniform weight. One click from anywhere. | ||
| @@ -37,10 +37,10 @@ When does it take effect? | ||
| [ View the full bill on Congress.gov ] ← source link, rule #1 | ||
| ``` | ||
| - | Banned (jargon + spin): "notwithstanding," "shall/may" legalese, undefined acronyms, double negatives, and any "good/bad/burdensome/finally-closes-a-loophole" framing. **Describe what it does, never whether it's good** - this is where accessibility meets the neutrality doctrine (NEUTRALITY.md §2, §5). | |
| + | Banned (jargon + spin): "notwithstanding," "shall/may" legalese, undefined acronyms, double negatives, and any "good/bad/burdensome/finally-closes-a-loophole" framing. **Describe what it does, never whether it's good** - this is where accessibility meets the neutrality doctrine (NEUTRALITY.md §2, §5). | |
| ## Search / lookup for non-experts (the 80-year-old path) | ||
| - | - "Look up my state / my rep / my topic" as labeled drop-downs with type-ahead - not a bare search box. | |
| + | - "Look up my state / my rep / my topic" as labeled drop-downs with type-ahead - not a bare search box. | |
| - Suggested pre-built filters as big clickable chips ([My State] · [Topic] · [This Year]); no "did you mean?" cognitive-load prompts. | ||
| - Search field ≥50px tall, ≥18px font. | ||
| - Persistent visible **Back**; breadcrumbs with whitespace separators, not `///`. |
| @@ -1,6 +1,6 @@ | ||
| - | # Docket - Internal API Contract | |
| + | # Docket - Internal API Contract | |
| - | Single-box deployment (CT247): Next.js app + SQLite (local, `/opt/docket/data/docket.db`) populated by the Python ingest. Frontend reads its OWN Next.js route handlers (`app/api/*`) via `better-sqlite3`. Backend agents write to the SQLite schema below; frontend consumes these JSON shapes. This is the contract - build to it on both sides. | |
| + | Single-box deployment (CT247): Next.js app + SQLite (local, `/opt/docket/data/docket.db`) populated by the Python ingest. Frontend reads its OWN Next.js route handlers (`app/api/*`) via `better-sqlite3`. Backend agents write to the SQLite schema below; frontend consumes these JSON shapes. This is the contract - build to it on both sides. | |
| ## Conventions | ||
| - All responses JSON. Every item carries `sourceUrl` (allowlisted official `.gov`/state-leg host; aggregator → underlying official record). | ||
| @@ -30,20 +30,20 @@ Detail: `{ id, number, title, sponsor{name,party,state}, cosponsors, status, tim | ||
| ### `GET /api/spending/*` | ||
| `/api/spending/awards` (list), `/api/spending/fiscal` (debt/deficit/outlays), `/api/spending/apportionments` ← impound tracker: `{ agency, account, appropriated, apportioned, status:"released|delayed|withheld", date, sourceUrl }` | ||
| - | ### `GET /api/executive` - Federal Register: EOs, rules, notices | |
| - | ### `GET /api/lobbying?bill=` - LDA.gov filings | |
| + | ### `GET /api/executive` - Federal Register: EOs, rules, notices | |
| + | ### `GET /api/lobbying?bill=` - LDA.gov filings | |
| ### `GET /api/states` · `GET /api/states/:code` | ||
| `/api/states` returns coverage + confidence per state: `{ code, name, confidence:"ok|stale|missing", lastScrape, missingTypes[] }`. Detail = state bills/votes. | ||
| - | ### `GET /api/members/:id` - a rep's record (votes, sponsorships, riders authored) | |
| - | ### `GET /api/search?q=` - global lookup across bills/members/states/awards | |
| + | ### `GET /api/members/:id` - a rep's record (votes, sponsorships, riders authored) | |
| + | ### `GET /api/search?q=` - global lookup across bills/members/states/awards | |
| ## Backend → DB contract | ||
| - | Ingest writes the tables in ARCHITECTURE.md "Data model" + "Full-spectrum additions". Each poller module is self-contained, does `CREATE TABLE IF NOT EXISTS`, is idempotent (dedupe on source+externalId+updateDate), and stores an allowlisted `source_url` on every row. Pollers do NOT modify `run.py` - the overseer wires registration. | |
| + | Ingest writes the tables in ARCHITECTURE.md "Data model" + "Full-spectrum additions". Each poller module is self-contained, does `CREATE TABLE IF NOT EXISTS`, is idempotent (dedupe on source+externalId+updateDate), and stores an allowlisted `source_url` on every row. Pollers do NOT modify `run.py` - the overseer wires registration. | |
| ## Keyless data paths (no api.data.gov key needed) | ||
| - | - **GovInfo BULK DATA** `https://www.govinfo.gov/bulkdata/` - BILLS, BILLSTATUS, BILLSUM, CREC (no key). Primary path for bills, status, full text, versions (diff inputs), and Congressional Record. | |
| - | - **Federal Register** `federalregister.gov/api/v1` - no key. | |
| - | - **OpenOMB** `openomb.org` - no key (link source to underlying `.max.gov`). | |
| - | - **USAspending / Treasury Fiscal** - no key (already live). | |
| - | - **House/Senate vote XML** - no key (already live for Senate). | |
| + | - **GovInfo BULK DATA** `https://www.govinfo.gov/bulkdata/` - BILLS, BILLSTATUS, BILLSUM, CREC (no key). Primary path for bills, status, full text, versions (diff inputs), and Congressional Record. | |
| + | - **Federal Register** `federalregister.gov/api/v1` - no key. | |
| + | - **OpenOMB** `openomb.org` - no key (link source to underlying `.max.gov`). | |
| + | - **USAspending / Treasury Fiscal** - no key (already live). | |
| + | - **House/Senate vote XML** - no key (already live for Senate). | |
| Key-gated (build code now, run when key present): Congress.gov JSON, GovInfo JSON API (DOCKET_CONGRESS_KEY); OpenStates (DOCKET_OPENSTATES_KEY); LegiScan (DOCKET_LEGISCAN_KEY). |
| @@ -1,8 +1,8 @@ | ||
| - | # Docket - Architecture | |
| + | # Docket - Architecture | |
| ## Shape | ||
| ``` | ||
| - | POLLERS (Codex) API + WEB (GLM) | |
| + | POLLERS (Python) API + WEB (Next.js) | |
| Congress.gov ─┐ ┌─ Next.js app (PWA, dark-first) | ||
| GovInfo ──────┤ │ /register (feed) /bill/[id] | ||
| USAspending ──┤→ normalize → Postgres ──→│ /spending /states | ||
| @@ -14,33 +14,33 @@ | ||
| ``` | ||
| ## Services | ||
| - | 1. **ingest** (Codex-owned, Python) - one poller module per source (`pollers/congress.py`, `usaspending.py`, `treasury.py`, `govinfo.py`, `votes_house.py`, `votes_senate.py`, `openstates.py`), each idempotent, rate-budget aware, with Congress.gov scraper fallback. A scheduler (cron/APScheduler) runs them at the cadences in `DATA_SOURCES.md`. | |
| - | 2. **change-detection** - diffs each poll against stored `updateDate`/hash, writes normalized `actions` + emits `events`. | |
| - | 3. **summaries** (Phase 2) - extraction-template pipeline; local Ollama (3060) draft; banned-words + Polarization-Index gate before publish. MVP ships structured fields only. | |
| - | 4. **web/api** (GLM-owned, Next.js) - read API over Postgres for the feed + entity pages; the brand-pack UI. | |
| - | 5. **alerts** - topic/agency/threshold subscriptions → ntfy + web-push. No party-based subscriptions. | |
| + | 1. **ingest** (Python) - one poller module per source (`pollers/congress.py`, `usaspending.py`, `treasury.py`, `govinfo.py`, `votes_house.py`, `votes_senate.py`, `openstates.py`), each idempotent, rate-budget aware, with Congress.gov scraper fallback. A scheduler (cron/APScheduler) runs them at the cadences in `DATA_SOURCES.md`. | |
| + | 2. **change-detection** - diffs each poll against stored `updateDate`/hash, writes normalized `actions` + emits `events`. | |
| + | 3. **summaries** (Phase 2) - extraction-template pipeline; banned-words + Polarization-Index gate before publish. MVP ships structured fields only. | |
| + | 4. **web/api** (Next.js) - read API over SQLite/Postgres for the feed + entity pages; the brand-pack UI. | |
| + | 5. **alerts** - topic/agency/threshold subscriptions → ntfy + web-push. No party-based subscriptions. | |
| ## Data model (Postgres, draft) | ||
| - `bills` (id, source, externalId, congress/session, number, title, sponsor_id, status, policy_area, updated_at, source_url) | ||
| - | - `actions` (id, bill_id?, source, action_type, action_date, chamber, text, dollar_amount?, affected_estimate?, source_url) - the spine of the feed | |
| + | - `actions` (id, bill_id?, source, action_type, action_date, chamber, text, dollar_amount?, affected_estimate?, source_url) - the spine of the feed | |
| - `votes` (id, bill_id, chamber, date, yea, nay, present, result, source_url) + `vote_positions` (vote_id, member_id, position) | ||
| - `members` (id, name, party, state, chamber, source_url) | ||
| - `awards` (id, source, externalId, agency, recipient, amount, action_date, program, source_url) | ||
| - | - `fiscal` (metric, date, value, source_url) - debt/deficit/outlays | |
| - | - `states` (code, name, openstates_coverage_level, last_seen) - powers the coverage map | |
| - | - `events` (id, ref_type, ref_id, action_date, dollar_amount, affected_estimate, action_type) - the ranked feed source (rule #3: ONLY these fields feed ranking) | |
| - | - `raw_events` (source, externalId, payload jsonb, fetched_at) - provenance | |
| + | - `fiscal` (metric, date, value, source_url) - debt/deficit/outlays | |
| + | - `states` (code, name, openstates_coverage_level, last_seen) - powers the coverage map | |
| + | - `events` (id, ref_type, ref_id, action_date, dollar_amount, affected_estimate, action_type) - the ranked feed source (rule #3: ONLY these fields feed ranking) | |
| + | - `raw_events` (source, externalId, payload jsonb, fetched_at) - provenance | |
| - `subscriptions` (user_id, kind[topic|agency|threshold|bill|state|member], value) | ||
| ### Full-spectrum additions (2026-06-26) | ||
| - `amendments` (id, congress, type[HAMDT|SAMDT], number, bill_id, sponsor_id, status, agreed_method[recorded|voice|uc|null], text?, source_url) | ||
| - | - `bill_versions` (id, bill_id, version_code[ih|rh|eh|eas|enr|…], govinfo_package_id, fetched_at, text_ref) - the diff inputs | |
| - | - `bill_diffs` (id, bill_id, from_version, to_version, section_id, change[added|removed|amended], snippet, rider_flag bool, source_url) - powers "what got slipped in" | |
| - | - `procedural_events` (id, bill_id?, kind[voice_vote|unanimous_consent|cloture|suspension|mtr|discharge], date, chamber, detail, members_jsonb?, source_url) - parsed from CREC; `members` null when no individual record exists | |
| - | - `lobbying` (id, registrant, client, covered_bill_id?, amount_range, period, source_url) - LDA.gov | |
| - | - `executive_actions` (id, type[eo|nprm|final_rule|notice], title, agency, fr_doc_number, date, source_url) - Federal Register | |
| - | - `apportionments` (id, agency, account, appropriated, apportioned, status[released|delayed|withheld], date, source_url→.max.gov) - the impound tracker | |
| - | - `state_coverage` (state, source[openstates|legiscan], data_type, last_scrape, confidence[ok|stale|missing]) - the honesty layer | |
| + | - `bill_versions` (id, bill_id, version_code[ih|rh|eh|eas|enr|…], govinfo_package_id, fetched_at, text_ref) - the diff inputs | |
| + | - `bill_diffs` (id, bill_id, from_version, to_version, section_id, change[added|removed|amended], snippet, rider_flag bool, source_url) - powers "what got slipped in" | |
| + | - `procedural_events` (id, bill_id?, kind[voice_vote|unanimous_consent|cloture|suspension|mtr|discharge], date, chamber, detail, members_jsonb?, source_url) - parsed from CREC; `members` null when no individual record exists | |
| + | - `lobbying` (id, registrant, client, covered_bill_id?, amount_range, period, source_url) - LDA.gov | |
| + | - `executive_actions` (id, type[eo|nprm|final_rule|notice], title, agency, fr_doc_number, date, source_url) - Federal Register | |
| + | - `apportionments` (id, agency, account, appropriated, apportioned, status[released|delayed|withheld], date, source_url→.max.gov) - the impound tracker | |
| + | - `state_coverage` (state, source[openstates|legiscan], data_type, last_scrape, confidence[ok|stale|missing]) - the honesty layer | |
| **Hard constraint:** every user-facing row exposes a non-null `source_url` whose host is allowlisted (neutrality rule #1, enforced by test). For aggregator-sourced rows (OpenStates/LegiScan/OpenOMB/Lambda), `source_url` points to the **underlying official record**, not the aggregator (DATA_SOURCES.md third-party rule). | ||
| @@ -48,19 +48,19 @@ | ||
| Per bill: `/related/BILLS-{id}` → fetch each version XML → section-parse + align (id + fuzzy heading) → diff → write `bill_diffs` (added/removed/amended), run rider keyword regex on appropriations-bill added sections → `rider_flag`. Surfaced as a "What changed / what was added" view + a per-section diff. Conference reports diffed against last engrossed version. Voice-vote/UC additions labeled "no individual record." | ||
| ## Ranking (neutrality rule #3) | ||
| - | `score(event) = f(actionDate, dollarAmount, affectedEstimate, actionType)` - no engagement/news/social inputs. Pure function, unit-tested to reference only those four fields. | |
| + | `score(event) = f(actionDate, dollarAmount, affectedEstimate, actionType)` - no engagement/news/social inputs. Pure function, unit-tested to reference only those four fields. | |
| ## Build/run hosts | ||
| - | - **Frontend (GLM):** build + dev in place on **CT247** (`/shared/projects/docket`, ext4 exec confirmed). `npm`/Next dev server runs there. | |
| - | - **Backend (Codex):** Python ingest on **CT241** (verify exec on its mount at dispatch; fall back to a venv in a CT-local dir if its share is noexec). | |
| - | - **Postgres:** dedicated CT or container (decide at Phase 1; SQLite acceptable for the Phase 0/1 spike). | |
| - | - **Canonical source tree:** `/shared/projects/docket` (RAG/cbm-indexed, git). Deploy target later: Cloudflare front + homelab CT API. | |
| + | - **Frontend:** Next.js dev server; `npm run dev` from the repo root. | |
| + | - **Backend:** Python ingest; `python -m ingest.run` from `services/ingest/`. | |
| + | - **Database:** SQLite for dev/staging; Postgres for production (Phase 1+). | |
| + | - **Deploy target:** Cloudflare front + self-hosted API backend. | |
| ## Phases (see PLAN.md §8 for detail) | ||
| - | 0. ✅ Brand + clickable shell (fake data) - GLM · DONE | |
| - | 1. Federal ingest spine → real feed (read-only) - DONE backend (USAspending/votes); **NEXT: wire read-API so frontend shows real data** | |
| + | 0. ✅ Brand + clickable shell (fake data) - DONE | |
| + | 1. Federal ingest spine → real feed (read-only) - DONE backend (USAspending/votes); **NEXT: wire read-API so frontend shows real data** | |
| 2. **Full bills/laws**: Congress.gov bills + GovInfo full text → bill pages + lookup/search (federal) | ||
| - | 3. **"What got slipped in"**: bill_versions + diff pipeline + rider flags - the signature feature | |
| + | 3. **"What got slipped in"**: bill_versions + diff pipeline + rider flags - the signature feature | |
| 4. **Closed-door layer**: CREC parsing (voice votes/UC/procedural) + Federal Register + lobbying + apportionment impound tracker | ||
| 5. **State granular**: OpenStates + LegiScan + per-state confidence scores | ||
| 6. **Accessibility modes**: Readable/Light + Plain Mode + plain-language summaries (docs/ACCESSIBILITY.md) |
| @@ -1,8 +1,8 @@ | ||
| - | # Docket - Brand & Design System | |
| + | # Docket - Brand & Design System | |
| - | Adapted from the TreeTrace "evidence/record" pattern, with a deliberate divergence: **the accent is brass, not cobalt** - because the brand color itself must be politically neutral (blue reads Democrat, red reads Republican; brass reads "official record / seal / ledger" and belongs to neither side). | |
| + | Adapted from the TreeTrace "evidence/record" pattern, with a deliberate divergence: **the accent is brass, not cobalt** - because the brand color itself must be politically neutral (blue reads Democrat, red reads Republican; brass reads "official record / seal / ledger" and belongs to neither side). | |
| - | > **Philosophy:** *Record the action, not the argument - every entry sealed to its source.* | |
| + | > **Philosophy:** *Record the action, not the argument - every entry sealed to its source.* | |
| Dark-first. Forensic-calm. Monospace data. Rationed accent. The UI should feel like an **official register being kept in real time**, not an op-ed and not a news app. | ||
| @@ -14,7 +14,7 @@ Define as CSS custom properties on `:root` and mirror into the Tailwind theme. | ||
| ```css | ||
| :root { | ||
| - | /* base - dark-first */ | |
| + | /* base - dark-first */ | |
| --dk-ink: #0B0C0E; /* page background, near-black */ | ||
| --dk-surface: #121316; /* cards, register chrome */ | ||
| --dk-graphite: #23262B; /* raised / hover */ | ||
| @@ -23,17 +23,17 @@ Define as CSS custom properties on `:root` and mirror into the Tailwind theme. | ||
| --dk-ash-2: #6E7480; /* tertiary text */ | ||
| --dk-line: rgba(155,160,166,.18); /* hairline borders */ | ||
| - | /* signal - brass, RATIONED (one per view) */ | |
| + | /* signal - brass, RATIONED (one per view) */ | |
| --dk-signal: #E0A53B; /* primary accent */ | ||
| --dk-signal-dim: #B07F25; /* hover / pressed */ | ||
| --dk-signal-tint: rgba(224,165,59,.08); /* faint active-row wash */ | ||
| - | /* semantic - MUTED, never vivid (neutrality) */ | |
| - | --dk-yea: #7FA88C; /* yea / passed / funded - muted sage, NOT vivid green */ | |
| - | --dk-nay: #B98A8A; /* nay / failed / cut - muted clay, NOT vivid red */ | |
| + | /* semantic - MUTED, never vivid (neutrality) */ | |
| + | --dk-yea: #7FA88C; /* yea / passed / funded - muted sage, NOT vivid green */ | |
| + | --dk-nay: #B98A8A; /* nay / failed / cut - muted clay, NOT vivid red */ | |
| --dk-hold: #C9A66B; /* stalled / pending / in-committee */ | ||
| - | /* party tones - desaturated, EQUAL visual weight, only when party is the datum */ | |
| + | /* party tones - desaturated, EQUAL visual weight, only when party is the datum */ | |
| --dk-party-d: #6E86A8; /* muted slate-blue */ | ||
| --dk-party-r: #A87E6E; /* muted clay-brown */ | ||
| --dk-party-i: #8A8F96; /* muted grey */ | ||
| @@ -43,7 +43,7 @@ Define as CSS custom properties on `:root` and mirror into the Tailwind theme. | ||
| **Light mode** (reads like an official document): `--dk-light-bg: #F6F4EF` (parchment), `--dk-light-text: #0B0C0E`, brass darkens to `#B07F25` for contrast. | ||
| ### Color rules (hard) | ||
| - | - **On solid brass, text/glyphs are INK `#0B0C0E`** (brass is light - dark text reads cleanly). *(Inverse of TreeTrace, where cobalt is dark and text is white.)* | |
| + | - **On solid brass, text/glyphs are INK `#0B0C0E`** (brass is light - dark text reads cleanly). *(Inverse of TreeTrace, where cobalt is dark and text is white.)* | |
| - Brass is **rationed**: one signal per view (a live dot, the active filter pill, a "just happened" flag). **Never** large brass fills or brass body text. | ||
| - **Never** use saturated red vs saturated blue for vote/party viz. Use the muted party/semantic tokens at equal weight. The eye must not read "teams." | ||
| - No gradients on logo/marks. Flat strokes and fills only. | ||
| @@ -53,16 +53,16 @@ Define as CSS custom properties on `:root` and mirror into the Tailwind theme. | ||
| ## 2. Typography | ||
| - **Display:** `Archivo` 800-900, uppercase, tracking −0.02 to −0.04em, line-height 0.85-0.92. Headlines, big numbers, vote tallies. | ||
| - | - **Data / labels / nav:** `JetBrains Mono`, uppercase, tracked. Bill numbers, dollar figures, timestamps, status pills, nav. This is the workhorse - it sells "this is a record." | |
| + | - **Data / labels / nav:** `JetBrains Mono`, uppercase, tracked. Bill numbers, dollar figures, timestamps, status pills, nav. This is the workhorse - it sells "this is a record." | |
| - **Prose:** `Fraunces` (+ italic) for the rare plain-English lead sentence / summary. One per surface, max. | ||
| - | - **Wordmark:** "Docket" - Archivo 800, sentence case. | |
| + | - **Wordmark:** "Docket" - Archivo 800, sentence case. | |
| --- | ||
| ## 3. Visual signature | ||
| - | - **Mark / logo:** a **custody-stamp seal** - a registration frame (outer rule + inner offset rule) around a `[D]` monogram with a small brass record-tick, reading as an *official stamp on the record*. Flat. **No eagle, no flag, no dome, no scales** - those cliches carry political valence or legal-tool connotations. Favicon: solid brass tile, ink "D" monogram (drop the inner rule below 24px). | |
| - | - **Hero - the live Register:** the signature element. A real-time stamped ledger of official actions materializing line by line, e.g.: | |
| + | - **Mark / logo:** a **custody-stamp seal** - a registration frame (outer rule + inner offset rule) around a `[D]` monogram with a small brass record-tick, reading as an *official stamp on the record*. Flat. **No eagle, no flag, no dome, no scales** - those cliches carry political valence or legal-tool connotations. Favicon: solid brass tile, ink "D" monogram (drop the inner rule below 24px). | |
| + | - **Hero - the live Register:** the signature element. A real-time stamped ledger of official actions materializing line by line, e.g.: | |
| ``` | ||
| 14:02 ET US HOUSE H.R. 22 FAILED 209-214 · | ||
| 13:47 ET US SENATE S. 1582 CLOTURE OK 62-38 · | ||
| @@ -83,5 +83,5 @@ Define as CSS custom properties on `:root` and mirror into the Tailwind theme. | ||
| ## 5. Voice (copy) | ||
| - Declarative, factual, no adjectives. "The House failed to pass H.R. 22, 209-214." Never "narrowly failed," "controversial," "key." | ||
| - | - Push copy: "The Senate voted on S. 1582 - cloture passed 62-38." Never "Politicians just voted to…" | |
| - | - See `docs/NEUTRALITY.md` §Copy for the banned-words list - it is enforced by a lint rule. | |
| + | - Push copy: "The Senate voted on S. 1582 - cloture passed 62-38." Never "Politicians just voted to…" | |
| + | - See `docs/NEUTRALITY.md` §Copy for the banned-words list - it is enforced by a lint rule. |
| @@ -1,10 +1,10 @@ | ||
| - | # Docket - Data Source Contracts | |
| + | # Docket - Data Source Contracts | |
| All primary sources are free + official. State (OpenStates) is the one paid tier. No source offers webhooks → **everything is polling + change-detection.** | ||
| ## Primary (the backbone) | ||
| - | ### Congress.gov API - federal bills, votes, members · PRIMARY | |
| + | ### Congress.gov API - federal bills, votes, members · PRIMARY | |
| - Base: `https://api.congress.gov/v3/` | ||
| - Auth: free api.data.gov key, `?api_key=` · **5,000 req/hr** · 250 records/call max | ||
| - Freshness: same-day for actions/status | ||
| @@ -12,62 +12,62 @@ All primary sources are free + official. State (OpenStates) is the one paid tier | ||
| - Covers: bills (101st+), amendments, summaries, members, committees, nominations; **House roll-call vote endpoints (beta, 118th+)**. | ||
| - ⚠️ **Reliability risk:** multi-day silent outage Aug 2025. **Fallback:** `unitedstates/congress` scraper (GovInfo bulk XML, LoC-independent). Ingest must tolerate 24-72h Congress.gov downtime. | ||
| - | ### GovInfo API - full bill text, Congressional Record, public laws · PRIMARY | |
| + | ### GovInfo API - full bill text, Congressional Record, public laws · PRIMARY | |
| - Base: `https://api.govinfo.gov/` · same api.data.gov key · **36,000 req/hr** (most generous) | ||
| - Delta poll: `GET /published/{start}/{end}` hourly. Collections: `BILLS`, `CREC`, `PLAW`, appropriations, `CHRG`, `FR`, `CFR`. | ||
| - Use for: full text, public-law text, appropriations docs. 1-2 business-day lag. | ||
| - | ### USAspending.gov API - federal spending (awards, outlays) · PRIMARY | |
| + | ### USAspending.gov API - federal spending (awards, outlays) · PRIMARY | |
| - Base: `https://api.usaspending.gov/api/v2/` · **no auth** | ||
| - Awards: `POST /search/spending_by_award/` (10k-result cap/call → use `/bulk_download/` for completeness). Agency: `/agency/`, accounts: `/federal_accounts/`. | ||
| - Change detection: filter by `action_date`, poll daily (agencies have 2-business-day reporting SLA). | ||
| - | ### Treasury Fiscal Data API - debt, deficit, cash flows · PRIMARY | |
| + | ### Treasury Fiscal Data API - debt, deficit, cash flows · PRIMARY | |
| - Base: `https://api.fiscaldata.treasury.gov/services/api/fiscal_service/` · **no auth** | ||
| - Datasets: Debt to the Penny (daily), Daily Treasury Statement (T+1), Monthly Treasury Statement (outlays by agency, ~3wk lag), Historical Debt. | ||
| - Poll daily ~6pm ET. | ||
| ## Vote-speed supplements | ||
| - | - **House Clerk XML** `https://clerk.house.gov/evs/` - votes within hours, no key. Poll 10-30 min during session. | |
| - | - **Senate roll-call XML** `https://www.senate.gov/legislative/LIS/roll_call_lists/` - no key. | |
| - | - **Voteview** `https://voteview.com/data` - bulk CSV + ideology scores. **Internal QA only** (never user-facing labels). | |
| + | - **House Clerk XML** `https://clerk.house.gov/evs/` - votes within hours, no key. Poll 10-30 min during session. | |
| + | - **Senate roll-call XML** `https://www.senate.gov/legislative/LIS/roll_call_lists/` - no key. | |
| + | - **Voteview** `https://voteview.com/data` - bulk CSV + ideology scores. **Internal QA only** (never user-facing labels). | |
| ## State · IN MVP (paid) | ||
| - | ### OpenStates API v3 - 50 states + DC/PR | |
| + | ### OpenStates API v3 - 50 states + DC/PR | |
| - Base: `https://v3.openstates.org/` · **API key required** | ||
| - Tiers: Free 500/day·10rps (too tight) → **Bronze 5k/day / Silver 50k/day (paid, contact for pricing)**. Production needs paid. | ||
| - JSON, standardized schema. Freshness varies by state. | ||
| - | - **Ship an honest per-state coverage map** - surface which states lag so the UI never implies completeness it doesn't have. | |
| + | - **Ship an honest per-state coverage map** - surface which states lag so the UI never implies completeness it doesn't have. | |
| ## CBO (cost estimates) · USEFUL, no API | ||
| - Scrape `cbo.gov/cost-estimates` by bill for per-bill scores; macro CSVs at `github.com/us-cbo`. Dollar figures only, no characterization. | ||
| --- | ||
| - | # FULL-SPECTRUM SOURCES (added 2026-06-26 - the real Docket scope) | |
| + | # FULL-SPECTRUM SOURCES (added 2026-06-26 - the real Docket scope) | |
| Spending was only one lens. These power the bills/laws/amendments/riders/closed-door pillars (PLAN.md §1b). | ||
| ## Amendments & "what got slipped in" · SIGNATURE FEATURE | ||
| - | ### Amendments data - Congress.gov API | |
| + | ### Amendments data - Congress.gov API | |
| - `GET /v3/amendment/{congress}/{type}/{number}` (+ `/actions`, `/text`, `/amendments`, `/cosponsors`). Types: `HAMDT`, `SAMDT`. Each carries `amendedBill`. Parent bill: `/bill/{c}/{type}/{num}/amendments`. | ||
| - | - ⚠️ Coverage: searchable Senate amendment **text only 117th Congress (2021)+**; House + older link to Congressional Record, not machine-readable. Manager's/en-bloc packages are **not** discrete objects anywhere - only inferable from sequence + CREC narrative. | |
| + | - ⚠️ Coverage: searchable Senate amendment **text only 117th Congress (2021)+**; House + older link to Congressional Record, not machine-readable. Manager's/en-bloc packages are **not** discrete objects anywhere - only inferable from sequence + CREC narrative. | |
| ### Bill-version text-diff (the engine of "what changed") | ||
| - **GovInfo BILLS** XML, 113th Congress (2013)+. Version chain: `IH`→`RH`→`EH`→`EAS`→`ENR` (House; `IS/RS/ES` Senate). PackageId `BILLS-{congress}{chamber}{number}{version}` (e.g. `BILLS-117hr3076ih`). | ||
| - | - List versions: `GET /related/BILLS-{id}` · fetch text: `GET /packages/{packageId}/xml`. | |
| + | - List versions: `GET /related/BILLS-{id}` · fetch text: `GET /packages/{packageId}/xml`. | |
| - **Diff recipe:** fetch consecutive versions → parse `<section>`/`<subsection>` w/ identifiers → align by id (+ fuzzy heading match, the hard part) → `difflib` within aligned sections → classify **added / removed / amended**. New section w/ no prior counterpart = inserted provision. | ||
| - **Rider detection** (R Street keyword signals on appropriations bills, regex over added sections): `Notwithstanding | Is Amended | Are Authorized | Provided, | Provided Further, | In General | Unless | Until | Determines`. Flags candidates for surfacing, not a verdict. | ||
| - | - **Conference reports** (GovInfo `CRPT`, 104th+): the biggest slipped-in vector. No structured diff trail - must diff the conference report text against the last engrossed version ourselves. Explanatory statements (modern omnibus) live in CREC, carry legal force. | |
| + | - **Conference reports** (GovInfo `CRPT`, 104th+): the biggest slipped-in vector. No structured diff trail - must diff the conference report text against the last engrossed version ourselves. Explanatory statements (modern omnibus) live in CREC, carry legal force. | |
| ### Honest gaps (state these in-product, never hide) | ||
| - | - Voice-vote / unanimous-consent additions: **no individual record exists** (~94% of Senate measures; most manager's amendments). Surface "passed by voice vote - no individual record." That absence is itself information. | |
| + | - Voice-vote / unanimous-consent additions: **no individual record exists** (~94% of Senate measures; most manager's amendments). Surface "passed by voice vote - no individual record." That absence is itself information. | |
| - Conference negotiation records, pre-117th Senate amendment text, "who wrote this rider": not public. | ||
| ## Closed-door / procedural · GovInfo CREC | ||
| - | - `CREC` collection (api.govinfo.gov, free key), sections HOUSE/SENATE/DIGEST/EXTENSIONS. Daily, ~1-day lag, bulk XML 1994+. The single best source to extract voice votes, UC agreements, cloture, suspension, motions-to-recommit, discharge-petition signatories - all via **text parsing** (no structured endpoint). | |
| - | - Caveat: "Extensions of Remarks" = statements never spoken on the floor; speeches can be revised after delivery. Not a verbatim transcript - don't inflate "floor activity." | |
| + | - `CREC` collection (api.govinfo.gov, free key), sections HOUSE/SENATE/DIGEST/EXTENSIONS. Daily, ~1-day lag, bulk XML 1994+. The single best source to extract voice votes, UC agreements, cloture, suspension, motions-to-recommit, discharge-petition signatories - all via **text parsing** (no structured endpoint). | |
| + | - Caveat: "Extensions of Remarks" = statements never spoken on the floor; speeches can be revised after delivery. Not a verbatim transcript - don't inflate "floor activity." | |
| - Committee markups/hearings: Congress.gov schedule (House 2017+, Senate 2019+) via docs.house.gov; transcripts discretionary + months-late (GovInfo `CHRG`). | ||
| ## Lobbying · LDA.gov · EASY WIN | ||
| @@ -75,16 +75,16 @@ Spending was only one lens. These power the bills/laws/amendments/riders/closed- | ||
| ## Executive branch (where policy is made between laws) | ||
| - **Federal Register API** `federalregister.gov/api/v1` · **no key**, JSON/CSV, same-day. Executive Orders, proposed/final rules (NPRM), notices. EASY WIN. | ||
| - | - **regulations.gov API** (GSA key) - public comment dockets on rules (citizen-engagement layer). | |
| - | - **OpenOMB** `openomb.org` (JSON, daily) - OMB **apportionments** (executive withholding/release of appropriated funds), FY2022+, mirrors statutory `apportionment-public.max.gov`. Live 2025-26 controversy. → **link source to the official `.max.gov` record, not openomb.org** (see third-party rule below). | |
| + | - **regulations.gov API** (GSA key) - public comment dockets on rules (citizen-engagement layer). | |
| + | - **OpenOMB** `openomb.org` (JSON, daily) - OMB **apportionments** (executive withholding/release of appropriated funds), FY2022+, mirrors statutory `apportionment-public.max.gov`. Live 2025-26 controversy. → **link source to the official `.max.gov` record, not openomb.org** (see third-party rule below). | |
| ## State granular · OpenStates + LegiScan | ||
| - | - **LegiScan API** - 50 states + Congress, free tier 30k q/mo, better bill-text + amendment completeness than OpenStates. Use as completeness supplement. | |
| - | - **OpenStates v3** - richer standardized vote/people schema, fully open. Both lag 24-48h; worst states historically MA/NE/KY/AL; ballot measures + interim committee work poorly covered by both. | |
| + | - **LegiScan API** - 50 states + Congress, free tier 30k q/mo, better bill-text + amendment completeness than OpenStates. Use as completeness supplement. | |
| + | - **OpenStates v3** - richer standardized vote/people schema, fully open. Both lag 24-48h; worst states historically MA/NE/KY/AL; ballot measures + interim committee work poorly covered by both. | |
| - **Per-state data-confidence score** (required): flag stale scrapes + known-missing data types (votes, committee actions, amendment text). Honesty about gaps is a differentiator. | ||
| ## Member trades · STOCK Act · HARD | ||
| - | - No official API. Official PDFs at `efdsearch.senate.gov` + `disclosures-clerk.house.gov`. Third-party normalized JSON via Lambda Finance (free) - but **display the official disclosure as source**. Amounts are ranges; the trade→disclosure gap is the metric. | |
| + | - No official API. Official PDFs at `efdsearch.senate.gov` + `disclosures-clerk.house.gov`. Third-party normalized JSON via Lambda Finance (free) - but **display the official disclosure as source**. Amounts are ranges; the trade→disclosure gap is the metric. | |
| --- | ||
| @@ -1,6 +1,6 @@ | ||
| - | # Docket - Design V2 (ground-up, elder-first) - LOCKED 2026-06-27 | |
| + | # Docket - Design V2 (ground-up, elder-first) - LOCKED 2026-06-27 | |
| - | The dark "forensic terminal" look is RETIRED as the design. New direction (Zion's pick): **"Large-print public service"** + **brass accent**. Built for an 80-year-old first; a high-schooler citing it second. Precise spec below - implement EXACTLY; do not reintroduce monospace, dense tables, dark default, or a 10-item nav. | |
| + | The dark "forensic terminal" look is RETIRED as the design. New direction (Zion's pick): **"Large-print public service"** + **brass accent**. Built for an 80-year-old first; a high-schooler citing it second. Precise spec below - implement EXACTLY; do not reintroduce monospace, dense tables, dark default, or a 10-item nav. | |
| ## Non-negotiables (the failures to fix) | ||
| - **NO monospace anywhere in the UI** (it reads techy/intimidating). Data figures use tabular-lining numerals in the UI sans, not a mono font. | ||
| @@ -9,33 +9,33 @@ The dark "forensic terminal" look is RETIRED as the design. New direction (Zion' | ||
| - **Nav consolidated** from ~10 items to a few plain ones (below). | ||
| - Plain language everywhere (≤8th-grade), plain dates ("June 26, 2026"), acronyms expanded. | ||
| - | ## SOURCE THE AESTHETIC FROM ZION'S OWN SITES - never a generated/AI design pack | |
| + | ## SOURCE THE AESTHETIC FROM ZION'S OWN SITES - never a generated/AI design pack | |
| The look must come from Zion's real, de-fingerprinted sites (zionboggan.com + treetrace.dev), NOT the ui-ux-pro-max pack, NOT Atkinson Hyperlegible / Public Sans / generic cards (that IS the fingerprint he's rejecting). Extracted from his live CSS: | ||
| - | - **His type trio (use it):** **Archivo** (bold display + headings + uppercase labels) · **Fraunces** (serif - leads, long reading, optional headlines) · **JetBrains Mono** (DATA ONLY - tallies, bill numbers, dates; never prose/nav). | |
| + | - **His type trio (use it):** **Archivo** (bold display + headings + uppercase labels) · **Fraunces** (serif - leads, long reading, optional headlines) · **JetBrains Mono** (DATA ONLY - tallies, bill numbers, dates; never prose/nav). | |
| - zionboggan.com = light, white/near-black + Tango palette (#204A87 #4E9A06 #8F5902 #A40000). treetrace.dev = dark ink #0B0C0E/#121316 + paper #F2EFE9 + cobalt #2E5BFF. | ||
| - | ## Color - Docket default (light, his palette, neutral, AAA target) | |
| + | ## Color - Docket default (light, his palette, neutral, AAA target) | |
| ``` | ||
| --bg: #FFFFFF /* white page (max contrast / familiar) */ | ||
| --surface: #F7F4ED /* his warm paper tone for cards/sections */ | ||
| --text: #0B0C0E /* his ink near-black, ~18:1 on white */ | ||
| --text-soft: #44423D /* secondary, >=7:1 */ | ||
| --border: #D8D2C4 /* warm visible border */ | ||
| - | --accent: #8F5902 /* HIS amber/ochre (Tango) - links + small text, ~6:1 on white; neutral, brass-adjacent */ | |
| + | --accent: #8F5902 /* HIS amber/ochre (Tango) - links + small text, ~6:1 on white; neutral, brass-adjacent */ | |
| --accent-strong: #CE5C00 /* his deeper ochre for large fills (dark text on it) */ | ||
| --accent-wash: #FBF3DF /* faint amber for active/selected */ | ||
| --focus: #0B0C0E /* 3px focus ring */ | ||
| - | --yea: #4E7A06 /* his Tango green, slightly muted - PASS, always with a text label */ | |
| - | --nay: #A40000 /* his Tango red, used SMALL for FAIL only - never as a dominant/party color */ | |
| + | --yea: #4E7A06 /* his Tango green, slightly muted - PASS, always with a text label */ | |
| + | --nay: #A40000 /* his Tango red, used SMALL for FAIL only - never as a dominant/party color */ | |
| ``` | ||
| Dark mode = his treetrace treatment (ink #0B0C0E/#121316 + cobalt #2E5BFF + paper text). Plain = max-contrast. Both stay as toggles; light is DEFAULT. | ||
| - | ## Type - his trio (mono is DATA-ONLY, not banned, just constrained) | |
| + | ## Type - his trio (mono is DATA-ONLY, not banned, just constrained) | |
| - **Headings / display / labels:** **Archivo** (700-900, can be uppercase for kicker labels). | ||
| - | - **Leads / long-form reading:** **Fraunces** (serif) - gives the credible "research publication" feel. | |
| + | - **Leads / long-form reading:** **Fraunces** (serif) - gives the credible "research publication" feel. | |
| - **Data tokens only:** **JetBrains Mono** for vote tallies, bill numbers, dollar amounts, dates (tabular). NOT for nav, body prose, or buttons. | ||
| - **Sizes (elder-first scale):** body/lead **20px** (line-height 1.55-1.65); min text anywhere **16px**; H1 **34-42px**; H2 26-28px; H3 20-22px. | ||
| - | - The elder-friendliness comes from SCALE + SIMPLICITY + WHITESPACE applied to HIS aesthetic - not from swapping his fonts for a generic legibility font. | |
| + | - The elder-friendliness comes from SCALE + SIMPLICITY + WHITESPACE applied to HIS aesthetic - not from swapping his fonts for a generic legibility font. | |
| ## Layout | ||
| - **Single column**, max reading width ~720px (text) / ~860px (lists); generous whitespace (section spacing 32-48px). | ||
| @@ -46,26 +46,26 @@ Dark mode = his treetrace treatment (ink #0B0C0E/#121316 + cobalt #2E5BFF + pape | ||
| ## Navigation / IA (the "mapping" fix) | ||
| - **Top bar (every page):** large **Docket** wordmark (links Home) · a prominent **Search** (big, labeled, always reachable) · a **Menu**. | ||
| - | - **Primary nav - 6 plain items, big labels, active item clearly highlighted:** | |
| + | - **Primary nav - 6 plain items, big labels, active item clearly highlighted:** | |
| `Home` · `Bills` · `Votes` · `Representatives` · `Spending` · `States` | ||
| (Rename: Members → **Representatives**. The Register feed = "What's happening", surfaced on Home. Calendar / Schedule / Executive / Federal Register move under Home cards or a clearly-labeled "More", NOT the primary bar.) | ||
| - | - **Mobile:** a clearly-LABELED "Menu" button (text + icon, >=44px) opening a big, simple full-screen menu with large rows - not a tiny ambiguous hamburger. | |
| + | - **Mobile:** a clearly-LABELED "Menu" button (text + icon, >=44px) opening a big, simple full-screen menu with large rows - not a tiny ambiguous hamburger. | |
| - **Mode toggle:** small, out of the way (e.g. in the menu/footer): Standard (this large-print, default) / Dark / Plain. | ||
| ## Per-page pattern (apply to ALL) | ||
| 1. Breadcrumb + big Back. | ||
| 2. Big H1 page title + one plain-language sentence under it. | ||
| 3. (List pages) prominent Search + a few simple Sort/Filter controls (big, labeled). | ||
| - | 4. Content as **big simple cards** - each record shows its key facts in large text, the one most-important fact first (e.g. a vote: "H.R. 22 - [title]" big, then "Passed, 414-12" then date + Source). NO multi-column dense tables. | |
| + | 4. Content as **big simple cards** - each record shows its key facts in large text, the one most-important fact first (e.g. a vote: "H.R. 22 - [title]" big, then "Passed, 414-12" then date + Source). NO multi-column dense tables. | |
| 5. Source link (rule #1) + **Cite this** on records. | ||
| - Home = welcoming hub: big headline, plain "what this is", a prominent Search, and 6-8 big cards to the sections + a "How to cite" note. | ||
| ## Keep (don't lose) | ||
| - | All data, neutrality gate (no adjectives, source on every fact, no comments, muted/never-saturated party colors), the Cite-this feature, search+sort, the perf/latency work (KEEP `revalidate`/ISR caching - do NOT reintroduce `force-dynamic`), responsive (no overflow at 390px). | |
| + | All data, neutrality gate (no adjectives, source on every fact, no comments, muted/never-saturated party colors), the Cite-this feature, search+sort, the perf/latency work (KEEP `revalidate`/ISR caching - do NOT reintroduce `force-dynamic`), responsive (no overflow at 390px). | |
| - | ## Build order (controlled - verify the LOOK before propagating) | |
| + | ## Build order (controlled - verify the LOOK before propagating) | |
| 1. Design system: tokens above + Public Sans/Atkinson fonts + set light large-print as DEFAULT (Dark/Plain stay toggles). New simplified header/nav + mobile menu. Breadcrumb + Back components. Big-card + big-button + big-search components. | ||
| 2. Rebuild HOME + ONE list page (Bills) + ONE detail page (a bill) as the TEMPLATE in this language. | ||
| - | 3. **STOP - screenshot home + the two template pages; overseer shows Zion for approval.** | |
| + | 3. **STOP - screenshot home + the two template pages; overseer shows Zion for approval.** | |
| 4. After approval: propagate the pattern to every remaining page (Votes, Members→Representatives, Spending, States, Calendar, Executive, Search, etc.). | ||
| 5. QA crawl CLEAN (no overflow/#418) + all 3 modes work + AAA contrast on default. |
| @@ -1,17 +1,17 @@ | ||
| - | # Docket - Design V3: MATCH ZION'S LIVE SITES (LOCKED 2026-06-27) | |
| + | # Docket - Design V3: MATCH ZION'S LIVE SITES (LOCKED 2026-06-27) | |
| - | The light V2 (Atkinson, then Archivo/Fraunces light + serif body) is REJECTED - it reads as a generic AI look. Replicate the theme of Zion's actual live sites. DO NOT invent or use any generated/AI design pack. | |
| + | The light V2 (Atkinson, then Archivo/Fraunces light + serif body) is REJECTED - it reads as a generic AI look. Replicate the theme of Zion's actual live sites. DO NOT invent or use any generated/AI design pack. | |
| ## GROUND TRUTH (study these, replicate them) | ||
| - Reference screenshots (captured from the live sites): `_ops/qa-view/REF-treetrace-desktop.png`, `_ops/qa-view/REF-zionboggan-desktop.png`. | ||
| - Their live CSS (curl from CT247 to read the real values): treetrace.dev `/assets/css/{tokens,shell,glass}.css`; zionboggan.com `/assets/css/{main,flare,sections}.css`. | ||
| - What both share (the aesthetic to match): | ||
| - | - **DARK** near-black backgrounds (ink ~#0B0C0E, raised surface ~#121316/#1A1C1F). | |
| - | - **Archivo 800-900, UPPERCASE, big, tight tracking** for display headlines (e.g. "RECORD THE PATH", "ZION BOGGAN"). | |
| - | - **JetBrains Mono, uppercase, tracked** for kicker labels, nav, tags, data, terminal cards. | |
| - | - A **serif** (Fraunces) for paragraph/body prose (warm, readable) - verify against their CSS and match their body role. | |
| - | - **ONE bold accent per view** (treetrace cobalt #2E5BFF, zionboggan red ~#A40000) - used sparingly (buttons, kickers, highlights), bordered/outline buttons. | |
| - | - Composition: small mono kicker → HUGE Archivo headline → serif paragraph → outline buttons + tag row. Generous whitespace; it BREATHES (do not crowd). | |
| + | - **DARK** near-black backgrounds (ink ~#0B0C0E, raised surface ~#121316/#1A1C1F). | |
| + | - **Archivo 800-900, UPPERCASE, big, tight tracking** for display headlines (e.g. "RECORD THE PATH", "ZION BOGGAN"). | |
| + | - **JetBrains Mono, uppercase, tracked** for kicker labels, nav, tags, data, terminal cards. | |
| + | - A **serif** (Fraunces) for paragraph/body prose (warm, readable) - verify against their CSS and match their body role. | |
| + | - **ONE bold accent per view** (treetrace cobalt #2E5BFF, zionboggan red ~#A40000) - used sparingly (buttons, kickers, highlights), bordered/outline buttons. | |
| + | - Composition: small mono kicker → HUGE Archivo headline → serif paragraph → outline buttons + tag row. Generous whitespace; it BREATHES (do not crowd). | |
| ## Docket V3 tokens (his treetrace dark treatment + NEUTRAL accent) | ||
| ``` | ||
| @@ -21,15 +21,15 @@ The light V2 (Atkinson, then Archivo/Fraunces light + serif body) is REJECTED | ||
| --text: #F2EFE9 /* warm paper */ | ||
| --text-soft: #9BA0A6 /* ash secondary */ | ||
| --border: rgba(155,160,166,.18) | ||
| - | --accent: #E0A53B /* BRASS - neutral (cobalt/red are partisan-coded for a gov site); rationed, one signal per view */ | |
| + | --accent: #E0A53B /* BRASS - neutral (cobalt/red are partisan-coded for a gov site); rationed, one signal per view */ | |
| --accent-dim:#B07F25 | ||
| --accent-wash:rgba(224,165,59,.10) | ||
| --yea: #7FA88C /* muted sage, labeled */ --nay: #B98A8A /* muted clay, labeled */ | ||
| ``` | ||
| Keep Light + Plain as optional toggles; **DEFAULT = this dark theme** (his look). | ||
| - | ## Fonts (his trio - match his ROLES exactly) | |
| - | - Archivo (display, 800-900, uppercase headlines/labels). JetBrains Mono (kickers, nav, tags, data tokens - readable size >=14px, NOT tiny). Fraunces (serif paragraph/body prose, comfortable >=17px). Drop Atkinson/Public-Sans entirely. | |
| + | ## Fonts (his trio - match his ROLES exactly) | |
| + | - Archivo (display, 800-900, uppercase headlines/labels). JetBrains Mono (kickers, nav, tags, data tokens - readable size >=14px, NOT tiny). Fraunces (serif paragraph/body prose, comfortable >=17px). Drop Atkinson/Public-Sans entirely. | |
| ## FIX what was wrong (within the dark aesthetic) | ||
| 1. **Crowding** → generous whitespace + clear vertical rhythm; fewer elements per screen; let headlines breathe like his sites do. | ||
| @@ -44,6 +44,6 @@ All functional work: perf/ISR (no force-dynamic), search+sort+filters, Cite this | ||
| ## Build order | ||
| 1. Read his live CSS + the REF screenshots. Set the dark theme tokens + font roles to match. | ||
| 2. Rebuild HOME + /bills + /bill/[id] as the TEMPLATE in his theme (dark, Archivo-uppercase, mono labels, brass, breathing whitespace, big text, simple nav). | ||
| - | 3. STOP - screenshot (dark default) → overseer shows Zion for approval. | |
| + | 3. STOP - screenshot (dark default) → overseer shows Zion for approval. | |
| 4. After approval: propagate to all pages + the federal/state bill distinction. | ||
| 5. QA crawl CLEAN; purge Cloudflare cache so it goes live. |
| @@ -1,156 +0,0 @@ | ||
| - | # Docket - Frontend Notes | |
| - | ||
| - | ## Design Version History | |
| - | ||
| - | | Version | Date | Status | Description | | |
| - | |---------|------------|----------|-------------| | |
| - | | V1 | 2026-06 | Retired | Initial build | | |
| - | | V2 | 2026-06 | Rejected | Light/Atkinson/serif look - "AI fingerprint", crowded | | |
| - | | V3 | 2026-06-27 | CURRENT | Matches Zion's live sites (treetrace.dev + zionboggan.com) - dark ink, Archivo uppercase, brass accent | | |
| - | ||
| - | --- | |
| - | ||
| - | ## V3 Design System (LOCKED 2026-06-27) | |
| - | ||
| - | Source of truth: `docs/DESIGN_V3.md`. Do NOT deviate from these without owner approval. | |
| - | ||
| - | ### Color tokens (globals.css `:root`) | |
| - | ||
| - | | Token | Value | Role | | |
| - | |-------|-------|------| | |
| - | | `--dk-ink` | `#0B0C0E` | Page background (near-black, matches tt-ink) | | |
| - | | `--dk-surface` | `#121316` | Cards, chrome (tt-surface) | | |
| - | | `--dk-graphite` | `#1A1C20` | Raised / hover | | |
| - | | `--dk-paper` | `#F2EFE9` | Primary text (warm bone, not pure white) | | |
| - | | `--dk-ash` | `#9BA0A6` | Secondary text | | |
| - | | `--dk-ash-2` | `#6E7480` | Tertiary text | | |
| - | | `--dk-line` | `rgba(155,160,166,.18)` | Hairline borders | | |
| - | | `--dk-signal` | `#E0A53B` | Brass accent - RATIONED, one signal per view | | |
| - | | `--dk-signal-dim` | `#B07F25` | Hover/pressed brass | | |
| - | | `--dk-signal-tint` | `rgba(224,165,59,.10)` | Faint active wash | | |
| - | | `--dk-yea` | `#7FA88C` | Muted sage (always labeled) | | |
| - | | `--dk-nay` | `#B98A8A` | Muted clay (always labeled) | | |
| - | ||
| - | **Why brass instead of cobalt/red:** cobalt (#2E5BFF) is TreeTrace's brand identity; red is partisan-coded. Brass is neutral for a government site. | |
| - | ||
| - | ### Font trio (matching both live sites exactly) | |
| - | ||
| - | | Family | Role | CSS var | | |
| - | |--------|------|---------| | |
| - | | Archivo 800-900 | Display headlines, UPPERCASE - the anchor | `--font-display` | | |
| - | | JetBrains Mono | Kickers, nav, tags, data labels - >=14px | `--font-mono` | | |
| - | | Fraunces 400 (+ italic) | Paragraph prose - >=17px | `--font-prose` | | |
| - | ||
| - | Atkinson Hyperlegible and Public Sans are REMOVED entirely. | |
| - | ||
| - | ### Type scale | |
| - | - Hero h1: `clamp(2.8rem, 9vw, 5.5rem)`, weight 900, uppercase, tracking `-0.03em` | |
| - | - Page h1 (bills/detail): `clamp(2.2rem, 7vw, 3.75rem)`, same | |
| - | - Section h2: `clamp(1.2rem, 3.5vw, 1.5rem)`, uppercase | |
| - | - Labels/kickers (`.label` class): JetBrains Mono, 14px, uppercase, `0.12em` tracking | |
| - | - Lead paragraph: Fraunces italic, `clamp(1rem, 2vw, 1.25rem)`, `1.55` line-height | |
| - | - Body prose: Fraunces, 17-18px | |
| - | ||
| - | ### Composition pattern | |
| - | Every page section follows: | |
| - | 1. Small mono kicker (brass, 11-14px) - identifies the section | |
| - | 2. HUGE Archivo uppercase headline - the anchor | |
| - | 3. Fraunces italic lead paragraph - warm, readable | |
| - | 4. Outline/brass buttons + tags | |
| - | ||
| - | ### Whitespace | |
| - | Generous - matching his sites. Hero padding: `clamp(56px, 10vw, 120px)` top. Section gaps: `clamp(32px, 6vw, 56px)`. Cards: `clamp(20px, 3vw, 28px)` internal padding. | |
| - | ||
| - | --- | |
| - | ||
| - | ## Default Theme: DARK | |
| - | ||
| - | Changed in V3. Dark ink (#0B0C0E) is the DEFAULT for all new visitors. Light and Plain are optional toggles persisted in localStorage. | |
| - | ||
| - | - `layout.tsx` html class: no theme class = dark (:root tokens active) | |
| - | - Inline script default: `localStorage.getItem('docket-display') || 'dark'` | |
| - | - DisplayToggle order: Dark (default) → Light → Plain | |
| - | - `themeColor` meta: `#0B0C0E` | |
| - | ||
| - | --- | |
| - | ||
| - | ## Navigation (site-header.tsx) | |
| - | ||
| - | Single-row fixed-height (64px) header with backdrop blur - matches treetrace.dev pattern: | |
| - | - Left: custody-mark seal + "Docket" wordmark (Archivo 800) | |
| - | - Center: 6 JetBrains Mono nav links, uppercase, 11.5px, 0.16em tracked | |
| - | - Right: Search button + Display toggle (Dark/Light/Plain) + hamburger (mobile only) | |
| - | ||
| - | Active state: 2px brass underline at bottom of link (`box-shadow`), no background fill. | |
| - | ||
| - | Mobile: slide-in panel from right (same pattern as tt-mobile), brass left-border on active item. | |
| - | ||
| - | **Do NOT go back to the two-row layout (wordmark+search row + nav row below).** | |
| - | ||
| - | --- | |
| - | ||
| - | ## Pages Built in V3 | |
| - | ||
| - | The V3 redesign covers the TEMPLATE pages. Propagate to other pages only after owner approval. | |
| - | ||
| - | | Page | File | Status | | |
| - | |------|------|--------| | |
| - | | Home | `app/page.tsx` | V3 complete | | |
| - | | Bills list | `app/bills/page.tsx` | V3 complete | | |
| - | | Bill detail | `app/bill/[id]/page.tsx` | V3 complete | | |
| - | | Site header | `components/site-header.tsx` | V3 complete | | |
| - | | Display toggle | `components/display-toggle.tsx` | V3 complete (dark default) | | |
| - | | All other pages | - | Pending owner approval | | |
| - | ||
| - | --- | |
| - | ||
| - | ## What Was Fixed (V2 → V3) | |
| - | ||
| - | 1. **Crowding** → Generous `clamp()` padding throughout; fewer elements per screen; headlines breathe like his sites | |
| - | 2. **Font size** → Body >=17px, labels >=14px (was 11px), hero h1 scales to 5.5rem | |
| - | 3. **Nav** → Single row, mono uppercase, brass active underline; no more two-row layout | |
| - | 4. **Accent** → Brass #E0A53B (neutral for gov); removed partisan cobalt and Atkinson/Public-Sans | |
| - | 5. **Mobile** → Slide-in panel (no fullscreen takeover); zero overflow at 390px confirmed | |
| - | 6. **Cards** → Clean border-radius 10px, 1px border (not 2px), more internal padding | |
| - | 7. **Default** → Dark is the default (not light); users can toggle | |
| - | ||
| - | --- | |
| - | ||
| - | ## QA | |
| - | ||
| - | Run from CT247: | |
| - | ```bash | |
| - | bash /shared/projects/docket/_ops/rebuild-qa.sh | |
| - | ``` | |
| - | Expected: "QA CLEAN - no overflow, no hydration errors across N routes" | |
| - | ||
| - | Screenshots (v3): | |
| - | ```bash | |
| - | cd /opt/docket && node /shared/projects/docket/_ops/screenshot-v3.js | |
| - | ``` | |
| - | Output: `_ops/qa-view/v3-{home,bills,bill}-{mobile,desktop}.png` | |
| - | ||
| - | --- | |
| - | ||
| - | ## Functional Work (Preserved from V2 - DO NOT LOSE) | |
| - | ||
| - | - ISR/revalidate - no force-dynamic (`revalidate = 300` on /bills, `revalidate = 3600` on home) | |
| - | - Search + sort + filters on /bills | |
| - | - Cite this (APA/MLA/Chicago) on bill detail pages | |
| - | - Vote → bill display links | |
| - | - State data | |
| - | - Member photos (MemberAvatar) | |
| - | - Neutrality gate: no adjectives, no editorial weighting, muted/equal-weight party colors | |
| - | - Federal vs. state bill distinction (see STATE_INGEST_FIX_NOTES.md) | |
| - | - Responsive cards - NOT tables (zero overflow at 390px) | |
| - | - Accessibility: skip-link, focus-visible ring, >=44px tap targets, color-not-alone, reduced-motion | |
| - | ||
| - | --- | |
| - | ||
| - | ## Build Gotchas | |
| - | ||
| - | - `services/` files contain data logic - edit those for data, not UI | |
| - | - `middleware.ts` - do not touch; it handles auth gating | |
| - | - `lib/feed.ts` + `lib/format.ts` - date/data utilities; no UI changes needed there | |
| - | - The site runs from `/opt/docket/` on CT247; shared source is `/shared/projects/docket/`; `rebuild-qa.sh` syncs and rebuilds | |
| - | - node_modules, .next, data, _ops/qa are NOT synced from share (they stay in /opt/docket) |
| @@ -1,39 +1,39 @@ | ||
| - | # Docket - Ingestion Strategy (efficiency & automation) | |
| + | # Docket - Ingestion Strategy (efficiency & automation) | |
| Roadmap for ingest v2. Today's pollers are naive (full pull each run). This is how we make 50-state + federal ingestion fast, cheap, automated, and self-healing. Ranked by impact-to-effort. | ||
| ## Top 10 (impact-to-effort) | ||
| - | 1. **Watermark change-detection** - GovInfo `collections/{col}/{lastModStart}/{lastModEnd}?offsetMark=*`; Congress.gov `fromDateTime` + **`updateDateIncludingText`** (catches text-only edits `updateDate` misses). Store `(source, last_run_at)` in `poller_watermarks`; update atomically only on successful upsert batch. ~10 lines, kills re-pulling. | |
| - | 2. **LegiScan weekly ZIP bulk** - `getDatasetList`/`getDataset`: one download = all 50 states + Congress. Public API caps at 30k/mo (~one busy state). Bulk for backfill + weekly; API only for intra-week deltas. | |
| - | 3. **OpenStates / Plural Postgres dump** (`open.pluralpolicy.com/downloads/`) - monthly all-50-state bootstrap, zero scraping; v3 API for incrementals. | |
| - | 4. **SHA-256 content-hash gate** - hash raw payload before parse; skip parse/upsert if unchanged. `INSERT … ON CONFLICT(source,external_id) DO UPDATE … WHERE hash != EXCLUDED.hash`. Applies everywhere. | |
| - | 5. **RSS/Atom feed registry** - ~20 states publish official feeds (KS, WA, HI, TX, WI, WV…). `feedparser` + last-seen GUID = sub-minute, zero-budget change detection. | |
| - | 6. **Session-calendar priority polling** - `sessions` table (active/interim/dormant); poll active hourly, recess daily, dormant weekly. ~70% volume cut off-season. | |
| - | 7. **ARQ + aiometer async workers** - Redis-backed asyncio queue (better fit than Celery here); `aiometer` for per-domain concurrency + rps caps; `tenacity` backoff on 429. Distribute across **CT215 + CT220** via worker hash-ring. | |
| - | 8. **LLM selector self-healing (3060)** - on field-extraction failure, queue raw HTML + schema → local `mistral-nemo:12b` proposes CSS/XPath → validate against live HTML → write to `scraper_selectors` table (no redeploy). ~98% accuracy across layout changes (2025 research). | |
| - | 9. **Embedding action-vocab normalization** - 50 states use different action strings → embed with `nomic-embed-text` (Ollama/3060) → cluster (HDBSCAN) → map to one canonical taxonomy (INTRODUCED/REFERRED/PASSED/SIGNED/VETOED…). One-time batch, re-run on new strings. | |
| - | 10. **Near-duplicate bill clustering** - embed bill summaries → store vectors in the existing **Qdrant** → cosine >0.92 flags model legislation (ALEC etc.) spreading across states. Unique Docket feature; zero marginal API cost. | |
| + | 1. **Watermark change-detection** - GovInfo `collections/{col}/{lastModStart}/{lastModEnd}?offsetMark=*`; Congress.gov `fromDateTime` + **`updateDateIncludingText`** (catches text-only edits `updateDate` misses). Store `(source, last_run_at)` in `poller_watermarks`; update atomically only on successful upsert batch. ~10 lines, kills re-pulling. | |
| + | 2. **LegiScan weekly ZIP bulk** - `getDatasetList`/`getDataset`: one download = all 50 states + Congress. Public API caps at 30k/mo (~one busy state). Bulk for backfill + weekly; API only for intra-week deltas. | |
| + | 3. **OpenStates / Plural Postgres dump** (`open.pluralpolicy.com/downloads/`) - monthly all-50-state bootstrap, zero scraping; v3 API for incrementals. | |
| + | 4. **SHA-256 content-hash gate** - hash raw payload before parse; skip parse/upsert if unchanged. `INSERT … ON CONFLICT(source,external_id) DO UPDATE … WHERE hash != EXCLUDED.hash`. Applies everywhere. | |
| + | 5. **RSS/Atom feed registry** - ~20 states publish official feeds (KS, WA, HI, TX, WI, WV…). `feedparser` + last-seen GUID = sub-minute, zero-budget change detection. | |
| + | 6. **Session-calendar priority polling** - `sessions` table (active/interim/dormant); poll active hourly, recess daily, dormant weekly. ~70% volume cut off-season. | |
| + | 7. **ARQ + aiometer async workers** - Redis-backed asyncio queue (better fit than Celery here); `aiometer` for per-domain concurrency + rps caps; `tenacity` backoff on 429. Distribute across **CT215 + CT220** via worker hash-ring. | |
| + | 8. **LLM selector self-healing (3060)** - on field-extraction failure, queue raw HTML + schema → local `mistral-nemo:12b` proposes CSS/XPath → validate against live HTML → write to `scraper_selectors` table (no redeploy). ~98% accuracy across layout changes (2025 research). | |
| + | 9. **Embedding action-vocab normalization** - 50 states use different action strings → embed with `nomic-embed-text` (Ollama/3060) → cluster (HDBSCAN) → map to one canonical taxonomy (INTRODUCED/REFERRED/PASSED/SIGNED/VETOED…). One-time batch, re-run on new strings. | |
| + | 10. **Near-duplicate bill clustering** - embed bill summaries → store vectors in the existing **Qdrant** → cosine >0.92 flags model legislation (ALEC etc.) spreading across states. Unique Docket feature; zero marginal API cost. | |
| ## Recommended architecture | ||
| - | - **Source registry YAML** - each source: `type` (api|bulk|rss|scrape), `change_detection` (watermark|rss|hash|head), `poll_interval_active/dormant`, `rate_limit_rps`, `js_required`, `priority`. Scheduler is data-driven; new/changed sources need no code. | |
| + | - **Source registry YAML** - each source: `type` (api|bulk|rss|scrape), `change_detection` (watermark|rss|hash|head), `poll_interval_active/dormant`, `rate_limit_rps`, `js_required`, `priority`. Scheduler is data-driven; new/changed sources need no code. | |
| - **Three-tier ingest:** (1) bulk bootstrap once per session/congress; (2) delta API polling via watermarks (hourly active / daily dormant); (3) persistent RSS watchers emitting change events immediately. | ||
| - **ARQ worker pool (CT215+CT220)**, partitioned by source hash-ring, Redis broker; workers write raw payloads → `raw_ingest` staging. | ||
| - | - **Hash-gated parse stage** - second worker computes `content_hash`, skips unchanged, else runs source parser → `*_staging` → idempotent upsert + emit `bill_versions` diff on text change. | |
| - | - **Dead-letter + alerting** - failures → `scrape_failures` (raw error + HTML snapshot + retry count); Discord webhook after 3; 6-hour retry worker. | |
| - | - **LLM repair sidecar** - separate process on the 3060 (Ollama), consumes a Redis `selector_repair_queue`; main scrapers read selectors from DB each run. | |
| + | - **Hash-gated parse stage** - second worker computes `content_hash`, skips unchanged, else runs source parser → `*_staging` → idempotent upsert + emit `bill_versions` diff on text change. | |
| + | - **Dead-letter + alerting** - failures → `scrape_failures` (raw error + HTML snapshot + retry count); Discord webhook after 3; 6-hour retry worker. | |
| + | - **LLM repair sidecar** - separate process on the 3060 (Ollama), consumes a Redis `selector_repair_queue`; main scrapers read selectors from DB each run. | |
| ## 50-state scraping | ||
| - | - **Fork `openstates/openstates-scrapers`** (~200 scrapers on `scrapelib` + `spatula` + `pupa`) - battle-tested session mgmt/robots/retry. Don't start from scratch. | |
| - | - JS-rendered states (~15: FL, IL…): `scrapy-playwright`. But **probe first** - ~40% of "JS" state sites call an undocumented JSON/GraphQL endpoint that's cleaner + more stable than the DOM. Catalog these as a priority tier. | |
| - | - **LLM-as-parser** only for the messy tail (PDFs, unstructured HTML) with `Outlines` constraint-decoded JSON + Pydantic validation. `lxml`/`trafilatura` beats LLM on structured HTML by 1000× - never put the LLM in the hot path. | |
| + | - **Fork `openstates/openstates-scrapers`** (~200 scrapers on `scrapelib` + `spatula` + `pupa`) - battle-tested session mgmt/robots/retry. Don't start from scratch. | |
| + | - JS-rendered states (~15: FL, IL…): `scrapy-playwright`. But **probe first** - ~40% of "JS" state sites call an undocumented JSON/GraphQL endpoint that's cleaner + more stable than the DOM. Catalog these as a priority tier. | |
| + | - **LLM-as-parser** only for the messy tail (PDFs, unstructured HTML) with `Outlines` constraint-decoded JSON + Pydantic validation. `lxml`/`trafilatura` beats LLM on structured HTML by 1000× - never put the LLM in the hot path. | |
| ## Creative / wildcards | ||
| - **Wayback CDX** (`web.archive.org/cdx/search/cdx`) for historical backfill of states that purge old sessions. | ||
| - | - **GovTrack/`unitedstates/congress` git mirrors** - bill versions as commits; `git log --diff-filter=M` gives changed-bills-since-last-sync without API. | |
| - | - **Undocumented state JSON endpoints** - check the network tab before writing any scraper. | |
| + | - **GovTrack/`unitedstates/congress` git mirrors** - bill versions as commits; `git log --diff-filter=M` gives changed-bills-since-last-sync without API. | |
| + | - **Undocumented state JSON endpoints** - check the network tab before writing any scraper. | |
| ## 3060 LLM: value vs trap | ||
| - **Use:** selector self-healing, action-vocab embedding/normalization, PDF/messy-HTML extraction (long tail), near-dup clustering. All batch/exception-path. | ||
| - **Trap:** LLM as primary parser for structured HTML; LLM in the real-time hot path (creates permanent backlog at 100k+ bills/session); trusting LLM output without schema validation; re-parsing the Congressional Record (use `unitedstates/congressional-record`). | ||
| - | *Adopt incrementally: watermarks + hash-gate + bulk-first first (items 1-4) - biggest win, least effort - then the registry + workers, then the LLM sidecar.* | |
| + | *Adopt incrementally: watermarks + hash-gate + bulk-first first (items 1-4) - biggest win, least effort - then the registry + workers, then the LLM sidecar.* |
| @@ -1,6 +1,6 @@ | ||
| - | # Docket - Neutrality Doctrine (enforceable) | |
| + | # Docket - Neutrality Doctrine (enforceable) | |
| - | Neutrality is the product. This file turns "be non-partisan" into **engineering constraints, lint rules, and tests** that ship in the codebase. Every PR is checked against it. Opus (overseer) owns this gate; GLM and Codex must build to it. | |
| + | Neutrality is the product. This file turns "be non-partisan" into **engineering constraints, lint rules, and tests** that ship in the codebase. Every PR is checked against it. | |
| --- | ||
| @@ -13,10 +13,10 @@ Every user-facing fact MUST carry a `sourceUrl` pointing to an official governme | ||
| ### 2. No adjectives | ||
| Generated/display text contains **no qualitative modifiers**. Numbers, status, dates, names only. | ||
| - | - **Banned-words lint** (`lint/neutrality.ts`) - fails the build if any appear in generated copy, summaries, or push text: | |
| + | - **Banned-words lint** (`lint/neutrality.ts`) - fails the build if any appear in generated copy, summaries, or push text: | |
| `slashed, gutted, slammed, blasted, massive, tiny, huge, sweeping, radical, extreme, controversial, popular, unpopular, landmark, historic, devastating, crippling, generous, stingy, bloated, wasteful, common-sense, partisan, bipartisan(as praise), key, crucial, vital, narrowly, barely, overwhelmingly, finally, surprisingly` | ||
| (List lives in code; extend over time.) | ||
| - | - "Reduced by $X" / "added $Y" - yes. "Cut" / "slashed" - no. | |
| + | - "Reduced by $X" / "added $Y" - yes. "Cut" / "slashed" - no. | |
| ### 3. Surface by official action, never by virality | ||
| Feed ranking = **(a) date of official legislative/spending action → (b) dollar magnitude → (c) people affected (per CBO/agency estimate)**. | ||
| @@ -24,32 +24,32 @@ Feed ranking = **(a) date of official legislative/spending action → (b) dollar | ||
| - **Enforcement:** the ranking function takes ONLY `{actionDate, dollarAmount, affectedEstimate, actionType}`. A test asserts no engagement/news field is referenced. Topic selection is proportional to official-action volume + dollar magnitude, not editorial judgment. | ||
| ### 4. No comment sections, ever | ||
| - | No free-text user comments on bills, votes, or spending - anywhere. The single largest bias-injection surface. | |
| + | No free-text user comments on bills, votes, or spending - anywhere. The single largest bias-injection surface. | |
| - Permitted engagement: a single **"Was this summary clear?" yes/no** on summary *clarity* (never on the bill's merit). No reactions, votes, ratings, or emoji on legislation itself. | ||
| - No social-share buttons on individual bills (sharing imports outside political discourse). Share-as-image (factual card) only, later. | ||
| ### 5. LLM summaries = extraction templates, not free-form | ||
| When prose summaries land (Phase 2+; MVP is structured-fields-only), they are produced by **field extraction into a fixed template**, not open-ended generation. | ||
| - | - Template: (1) official title + number, (2) sponsor name/party/state - factual, uncharacterized, (3) what the text authorizes (extracted clauses, plain language), (4) CBO score if any (dollar figure only), (5) current status. | |
| + | - Template: (1) official title + number, (2) sponsor name/party/state - factual, uncharacterized, (3) what the text authorizes (extracted clauses, plain language), (4) CBO score if any (dollar figure only), (5) current status. | |
| - Every summary passes the banned-words lint + a **Polarization-Index check** before publish (generate D-leaning and R-leaning variants, assert the default sits statistically between them). High-salience bills (100+ media mentions) get human review before push. | ||
| - LLM runs locally on the 3060 (Ollama) for drafting; extraction is preferred over generation wherever a field can be pulled deterministically. | ||
| --- | ||
| ## Visualization neutrality | ||
| - | - Party affiliation and vote splits ARE factual data and may be shown - but with **muted, equal-weight tones** (`--dk-party-d/r/i`, `--dk-yea/nay`). Never saturated red-vs-blue. Equal saturation, equal area, alphabetical or by-count order (not party order). | |
| - | - Default sort of any member/vote list: **by vote, then alphabetical** - not grouped by party (grouping primes partisan reading). | |
| + | - Party affiliation and vote splits ARE factual data and may be shown - but with **muted, equal-weight tones** (`--dk-party-d/r/i`, `--dk-yea/nay`). Never saturated red-vs-blue. Equal saturation, equal area, alphabetical or by-count order (not party order). | |
| + | - Default sort of any member/vote list: **by vote, then alphabetical** - not grouped by party (grouping primes partisan reading). | |
| ## Trust surfaces (must ship) | ||
| - | - **/how-we-pick** - plain explanation of the ranking algorithm (rule #3), public. | |
| - | - **/who-pays** - funding/ownership transparency. | |
| + | - **/how-we-pick** - plain explanation of the ranking algorithm (rule #3), public. | |
| + | - **/who-pays** - funding/ownership transparency. | |
| - **Source link on every fact** (rule #1). | ||
| - No political-campaign or advocacy ads, ever. | ||
| ## Standing risks (monitor, not one-time) | ||
| - | 1. **LLM bias drift** - re-run the Polarization-Index suite on every model change; it's CI, not a launch checklist item. | |
| - | 2. **Topic-selection soft bias** - audit feed composition periodically against official-action volume; if it skews, the ranking inputs are wrong, not the world. | |
| - | 3. **Partisan co-option** - bipartisan advisory board + dual-side launch outreach; track which communities adopt. | |
| + | 1. **LLM bias drift** - re-run the Polarization-Index suite on every model change; it's CI, not a launch checklist item. | |
| + | 2. **Topic-selection soft bias** - audit feed composition periodically against official-action volume; if it skews, the ranking inputs are wrong, not the world. | |
| + | 3. **Partisan co-option** - bipartisan advisory board + dual-side launch outreach; track which communities adopt. | |
| ## CI gate (definition of done for any user-facing text feature) | ||
| - [ ] banned-words lint passes |
| @@ -1,59 +0,0 @@ | ||
| - | # Docket - Build Orchestration | |
| - | ||
| - | Overseer method. Opus 4.8 coordinates; GLM-5.2 and Codex do the heavy app code. Pull out all the stops, don't settle. | |
| - | ||
| - | ## Recursive fan-out (everyone can spawn) | |
| - | This is a **tree, not a flat hand-off.** Every node may spawn its own sub-agents, matched to task by model strength - depth is allowed wherever it raises quality or parallelism. | |
| - | ||
| - | ``` | |
| - | Opus 4.8 (overseer) | |
| - | ├── GLM-5.2 (frontend lead, CT247) ──► spawns opencode sub-agents (GLM or any opencode-configured model) | |
| - | │ ├── component-builder, brand-QA, a11y-pass, perf-pass … | |
| - | ├── Codex (backend lead, CT241) ──────► spawns its own GPT sub-agents (codex sub-tasks) | |
| - | │ ├── per-poller agent (one per source), schema agent, test agent … | |
| - | └── Opus ──► spawns Claude sub-agents (Explore/Plan/general-purpose) for review, neutrality-audit, integration | |
| - | ``` | |
| - | ||
| - | Rules for spawning: | |
| - | - **Match the model to the task.** Cheap mechanical work → cheapest capable model; hard reasoning/architecture/neutrality calls → strongest. Don't send a scalpel job to a sledgehammer or vice-versa. | |
| - | - A lead that splits its wave into independent sub-tasks SHOULD spawn (e.g. Codex spawns one sub-agent per poller; GLM spawns one per page/component) - parallel + isolated context. | |
| - | - Every spawned sub-agent inherits the same carry-on contract below (RAG-first, spec pointers, neutrality gate, DoD). | |
| - | - Keep each level's concurrency small (≤3-4) and have the parent review children before merging up. The tree must converge, not sprawl - parents own correctness of their subtree. | |
| - | - Depth is a tool, not a goal: spawn when it adds parallelism or specialization, not to add layers. | |
| - | ||
| - | ## Division of labor | |
| - | | Agent | Host / driver | Owns | | |
| - | |---|---|---| | |
| - | | **Opus 4.8** (overseer) | CT215 (this) | Architecture, brand system, the **neutrality gate**, data-source contracts, task specs, integration, review/synthesis, final correctness sign-off | | |
| - | | **GLM-5.2** | CT247 `agent3` - `opencode run -m zai/glm-5.2` | **Frontend**: Next.js app, brand-pack implementation (tokens→Tailwind, components), the live Register hero, feed, bill/spending/states pages, trust pages, PWA | | |
| - | | ~~**Codex**~~ | CT241 - **BENCHED 2026-06-26** (auth dead; Zion will redo). Do not route work here until re-login confirmed. | (was backend) | | |
| - | | **Claude builders** | CT215 (Opus spawns) → run/test on CT241/CT247 | **Backend** while Codex is out: ingest pollers, change-detection, DB schema, read API | | |
| - | ||
| - | ## How to drive the fleet (from CT215 over SSH) | |
| - | ```bash | |
| - | # GLM (frontend) - runs in the repo, full MCP (RAG/serena/memory) attached | |
| - | ssh root@REDACTED-IP "pct exec 247 -- bash -lc 'cd /shared/projects/docket && opencode run -m zai/glm-5.2 \"<task>\"'" | |
| - | ||
| - | # Codex (backend) | |
| - | ssh root@REDACTED-IP "pct exec 241 -- bash -lc 'cd /shared/projects/docket && codex exec \"<task>\"'" | |
| - | ``` | |
| - | Run long builds in the background; capture logs to `_ops/logs/`. Verify GLM frontend output from Zion's **real browser** (Brave, worky profile, hard-refresh) - not just a curl. | |
| - | ||
| - | ## Every dispatched task MUST carry | |
| - | 1. **RAG-first + targeted-discovery** rule (query `homelab_context` first; read only the specific file needed; no full-tree reads). | |
| - | 2. A pointer to the relevant spec: `PLAN.md`, `docs/BRAND.md`, `docs/NEUTRALITY.md`, `docs/DATA_SOURCES.md`, `docs/ARCHITECTURE.md`. | |
| - | 3. The **neutrality gate** for any user-facing text/UI: government-source-only, no adjectives, no comment sections, muted (never red/blue-team) viz, ranking by official-action only. | |
| - | 4. A crisp definition of done + where to write output. | |
| - | ||
| - | ## Cadence | |
| - | - Small waves (≤3-4 concurrent agent runs). | |
| - | - Opus reviews every wave's output against the spec + neutrality gate before the next wave. | |
| - | - Memory-graph: record each significant decision/handoff so the next agent inherits state. | |
| - | ||
| - | ## Wave 0 (kickoff) | |
| - | - **GLM:** scaffold Next.js + Tailwind w/ Docket tokens; build the static shell of all pages with the live-Register hero on fake data; wire the brand system from `docs/BRAND.md`. | |
| - | - **Codex:** scaffold the ingest service (poller skeletons + DB schema from `docs/ARCHITECTURE.md`), with Congress.gov + USAspending pollers returning real data into SQLite (Phase-0 store). | |
| - | - **Opus:** review both, wire the neutrality lint (`lint/neutrality.ts`), integrate, report. | |
| - | ||
| - | ## Public-output gate (when anything ships) | |
| - | No AI tells, no em-dashes, no model footers; strip ` |
| @@ -1,4 +1,4 @@ | ||
| - | # Docket - Member Photo Coverage Notes | |
| + | # Docket - Member Photo Coverage Notes | |
| Generated: 2026-06-26 | ||
| --- | ||
| @@ -21,7 +21,7 @@ Generated: 2026-06-26 | ||
| (CC0, github.com/unitedstates/images), fetched at 450×550 resolution via | ||
| the `member_photos.py` poller. | ||
| - | ### Current Congress gap (12 members without photos - pre-fix) | |
| + | ### Current Congress gap (12 members without photos - pre-fix) | |
| These were newly elected members not yet in the unitedstates/images dataset: | ||
| | Bioguide | Name | State/Chamber | | ||
| @@ -43,7 +43,7 @@ These were newly elected members not yet in the unitedstates/images dataset: | ||
| ## 2. New Sources Added | ||
| - | ### Source 1 - bioguide.congress.gov (official .gov, public domain) | |
| + | ### Source 1 - bioguide.congress.gov (official .gov, public domain) | |
| - **URL pattern:** `https://bioguide.congress.gov/photo/{bioguide_id}.jpg` | ||
| - **Status:** Official U.S. government photographs. Public domain. Served by | ||
| @@ -53,14 +53,14 @@ These were newly elected members not yet in the unitedstates/images dataset: | ||
| `member_photos_extra.py` uses `requests.get(stream=True)` + immediate | ||
| `.close()` to check existence without downloading the image body. | ||
| - **Coverage for 12 current members without photos:** 11/12 confirmed 200 | ||
| - | (A000383 Alan Armstrong returned 404 - covered by Wikimedia below). | |
| + | (A000383 Alan Armstrong returned 404 - covered by Wikimedia below). | |
| - **Estimated new photos (full run, all 11,470 missing):** | ||
| - | - Observed hit rate at progress checkpoint (1,800/11,470): ~19.7% | |
| - | - Extrapolated: **~2,250-2,300 new photos** from bioguide alone | |
| - | - The rate declines moving from recent to older historical members as | |
| + | - Observed hit rate at progress checkpoint (1,800/11,470): ~19.7% | |
| + | - Extrapolated: **~2,250-2,300 new photos** from bioguide alone | |
| + | - The rate declines moving from recent to older historical members as | |
| expected (photography only became widespread ~1850s+). | ||
| - | ### Source 2 - Wikimedia Commons via Wikipedia API (CC / PD portraits) | |
| + | ### Source 2 - Wikimedia Commons via Wikipedia API (CC / PD portraits) | |
| - **API:** `https://en.wikipedia.org/w/api.php?action=query&prop=pageimages` | ||
| - **Image CDN:** `https://upload.wikimedia.org/wikipedia/commons/...` | ||
| @@ -72,16 +72,16 @@ These were newly elected members not yet in the unitedstates/images dataset: | ||
| typically CC-BY-SA or CC-BY. Attribution details are on the Wikimedia Commons | ||
| file page (linked from the filename in the URL). | ||
| - **Sample hit rate (50 random members without bioguide photos):** ~54% | ||
| - | - notably higher than bioguide for the historical-but-notable cohort | |
| + | - notably higher than bioguide for the historical-but-notable cohort | |
| (Bella Abzug, James Abdnor, and similar members with Wikipedia pages). | ||
| - **Confirmed finds for current members:** | ||
| - | - Alan Armstrong (A000383) - found via Wikipedia | |
| - | - Wesley Bell (B001324) - found | |
| - | - Jon Husted (H001104) - found | |
| - | - Matt Van Epps (V000139) - found | |
| - | - Christian D. Menefee (M001245) - found | |
| - | - Clay Fuller (F000485) - found | |
| - | - Analilia Mejia (M001246) - found | |
| + | - Alan Armstrong (A000383) - found via Wikipedia | |
| + | - Wesley Bell (B001324) - found | |
| + | - Jon Husted (H001104) - found | |
| + | - Matt Van Epps (V000139) - found | |
| + | - Christian D. Menefee (M001245) - found | |
| + | - Clay Fuller (F000485) - found | |
| + | - Analilia Mejia (M001246) - found | |
| - **Estimated new photos (Wikimedia pass after bioguide):** | ||
| ~15-25% of the post-bioguide remainder (~9,200 members) = **~1,400-2,300 | ||
| additional photos**, primarily for 20th-century members with Wikipedia pages. | ||
| @@ -108,7 +108,7 @@ unrepresented). Based on hit rates: | ||
| These are genuine pre-photography or obscure historical members for whom no | ||
| image has ever been digitized or uploaded to any of these three sources. | ||
| - | That cohort is the honest floor - no further scraping will change it | |
| + | That cohort is the honest floor - no further scraping will change it | |
| without a digitization project. | ||
| --- | ||
| @@ -131,7 +131,7 @@ DOCKET_DB_PATH=/opt/docket/data/docket.db \ | ||
| | `--delay SECONDS` | Request rate (default 0.1s) | | ||
| | `--no-current-first` | Disable current-member priority sort | | ||
| - | **Design:** Idempotent - only updates `photo_url` where it is currently NULL. | |
| + | **Design:** Idempotent - only updates `photo_url` where it is currently NULL. | |
| Never overwrites unitedstates/images URLs or any previously confirmed photo. | ||
| Commits every 200 members. Current congress members are front-loaded (processed | ||
| first) to ensure the visible /members roster looks complete before historical | ||
| @@ -160,7 +160,7 @@ To render photos from the new sources, extend to: | ||
| "img-src 'self' data: https://unitedstates.github.io https://bioguide.congress.gov https://upload.wikimedia.org" | ||
| ``` | ||
| - | **Do not edit next.config.js in this PR** - this note is for the frontend agent | |
| + | **Do not edit next.config.js in this PR** - this note is for the frontend agent | |
| that owns the page render layer. | ||
| --- | ||
| @@ -168,15 +168,15 @@ that owns the page render layer. | ||
| ## 5. UX Recommendation for /members Roster | ||
| **Problem:** With only ~10% of members having photos, a default "all members" | ||
| - | view shows mostly initials placeholders - looks incomplete and undermines trust. | |
| + | view shows mostly initials placeholders - looks incomplete and undermines trust. | |
| **Recommendation:** Apply this priority ordering to the default roster view: | ||
| 1. **Current Congress members first** (537 members, ~100% photo coverage after | ||
| - | this fix) - the page looks complete and professional on first load. | |
| - | 2. **Historical members with a photo** - the ~4,000+ additional historical | |
| + | this fix) - the page looks complete and professional on first load. | |
| + | 2. **Historical members with a photo** - the ~4,000+ additional historical | |
| members who have images. | ||
| - | 3. **Historical members without a photo** - reachable via a "Show all historical | |
| + | 3. **Historical members without a photo** - reachable via a "Show all historical | |
| members" expand or filter toggle. | ||
| This gives the impression of completeness (which reflects reality for the | ||
| @@ -184,9 +184,9 @@ current Congress) while making historical depth discoverable rather than | ||
| burying current members in a sea of placeholders. | ||
| **Filter/toggle copy:** "Showing current Congress. Toggle to see all X,XXX | ||
| - | historical members." - factual, no adjectives. | |
| + | historical members." - factual, no adjectives. | |
| - | **Alternative:** A tab layout - "Current" / "Historical" - with photo-only | |
| + | **Alternative:** A tab layout - "Current" / "Historical" - with photo-only | |
| toggle inside each tab. | ||
| --- |
| @@ -1,4 +1,4 @@ | ||
| - | # Docket - UX/QoL Redesign Spec (2026-06-27) | |
| + | # Docket - UX/QoL Redesign Spec (2026-06-27) | |
| "Back to the drawing board" for usability. Audience = an **elderly citizen** (low tech-literacy, low vision) AND a **high-school/college student citing Docket as an educational source**. Decisions below are LOCKED by Zion. Keep strict neutrality (no lean, no adjectives) and keep all data/sourcing. | ||
| @@ -9,14 +9,14 @@ | ||
| ## Theme system (3 modes; default changes) | ||
| Mode toggle stays, but the DEFAULT becomes **Reference**: | ||
| - | - **Reference (NEW DEFAULT - light):** background `#F8FAFC`, surface `#FFFFFF`, text `#0F172A`/`#020617`, muted `#475569`, border `#E2E8F0`. **Accent = brass `#B07F25`** (the brand thread; brass darkened for AA on light - NEVER use blue/red as accent, neutrality). Body font **Atkinson Hyperlegible** (engineered for low vision). Headings **Fraunces** (keep, scholarly/citable). Mono **JetBrains Mono ONLY for true data tokens** (vote tallies, bill numbers, dates) - not for prose. Body ≥18px, line-height 1.6, generous whitespace, single-column, max-w ~720-860px for reading. | |
| - | - **Dark (forensic - now optional):** the existing dark brand (ink/brass/mono). Preserved for power users. | |
| + | - **Reference (NEW DEFAULT - light):** background `#F8FAFC`, surface `#FFFFFF`, text `#0F172A`/`#020617`, muted `#475569`, border `#E2E8F0`. **Accent = brass `#B07F25`** (the brand thread; brass darkened for AA on light - NEVER use blue/red as accent, neutrality). Body font **Atkinson Hyperlegible** (engineered for low vision). Headings **Fraunces** (keep, scholarly/citable). Mono **JetBrains Mono ONLY for true data tokens** (vote tallies, bill numbers, dates) - not for prose. Body ≥18px, line-height 1.6, generous whitespace, single-column, max-w ~720-860px for reading. | |
| + | - **Dark (forensic - now optional):** the existing dark brand (ink/brass/mono). Preserved for power users. | |
| - **Plain (high-contrast):** existing max-contrast mode (system font, ≥20px) for max accessibility. | ||
| All three: WCAG AA min (AAA target), ≥44px tap targets, visible focus rings, underlined links, no hover-only, prefers-reduced-motion respected. | ||
| ## IA / declutter (each page does ONE job) | ||
| - | - **Register (home):** the ranked feed, period. Remove the crowding side cards (composition chart, awards card, topic chips, changes feed) from the main view - move them to a compact, collapsed "More" area or their own pages. Lead with a short plain-language explainer + the feed. One primary action. | |
| - | - **Every list page (Register, Bills, Votes, Members, Spending, Executive, States):** a prominent labeled **search/filter bar at the top** (mobile + desktop) - search box ≥44px tall, plus the relevant dropdown filters. Results as the existing responsive cards. | |
| + | - **Register (home):** the ranked feed, period. Remove the crowding side cards (composition chart, awards card, topic chips, changes feed) from the main view - move them to a compact, collapsed "More" area or their own pages. Lead with a short plain-language explainer + the feed. One primary action. | |
| + | - **Every list page (Register, Bills, Votes, Members, Spending, Executive, States):** a prominent labeled **search/filter bar at the top** (mobile + desktop) - search box ≥44px tall, plus the relevant dropdown filters. Results as the existing responsive cards. | |
| - **Detail pages (bill, vote, member):** clear heading → key facts → source → "Cite this". No crowding. | ||
| - Consistent header on every page: wordmark, a clear page-nav (hamburger on mobile / simple bar on desktop) with active-page indicator, the mode toggle as a small distinct control, and a global search entry. | ||
| @@ -26,14 +26,14 @@ All three: WCAG AA min (AAA target), ≥44px tap targets, visible focus rings, u | ||
| - **/votes:** add search/filter (by chamber, result, date/keyword). | ||
| - Reuse a shared SearchFilterBar component. Server-side filtering where possible (perf), client-side for already-loaded sets. | ||
| - | ## "Cite this" (item #2 - first-class) | |
| + | ## "Cite this" (item #2 - first-class) | |
| - A `CiteThis` component on every record (bill, vote, member, award) and every page. | ||
| - | - Generates a ready citation in **APA, MLA, and Chicago**, plus a **permalink** and the **accessed date** (today). Example (APA): *Docket. (2026). H.R. 22 - [title] [Data set]. U.S. Congress. Retrieved Month D, YYYY, from https://usdocket.org/bill/...* - but ALWAYS point the underlying source to the official .gov record too (neutrality rule #1). | |
| + | - Generates a ready citation in **APA, MLA, and Chicago**, plus a **permalink** and the **accessed date** (today). Example (APA): *Docket. (2026). H.R. 22 - [title] [Data set]. U.S. Congress. Retrieved Month D, YYYY, from https://usdocket.org/bill/...* - but ALWAYS point the underlying source to the official .gov record too (neutrality rule #1). | |
| - Copy-to-clipboard buttons. Neutral, factual, no adjectives. | ||
| - This is the differentiator for the student/educator audience. | ||
| ## Labels / clarity (the "51" problem) | ||
| - | - **States page:** show **"50 states"** as the count. **DC and the 5 territories are their own clearly-labeled rows/section - DC is a federal district, NOT a state; never produce a "51 states" number.** Coverage framed plainly. | |
| + | - **States page:** show **"50 states"** as the count. **DC and the 5 territories are their own clearly-labeled rows/section - DC is a federal district, NOT a state; never produce a "51 states" number.** Coverage framed plainly. | |
| - Plain dates ("June 26, 2026"), plain status words, expand acronyms on first use, ≤8th-grade reading level for any explanatory copy (docs/ACCESSIBILITY.md). | ||
| ## Out of scope / keep |
| @@ -1,5 +1,5 @@ | ||
| /** | ||
| - | * Docket read DB - defensive SQLite access (docs/API.md). | |
| + | * Docket read DB - defensive SQLite access (docs/API.md). | |
| * | ||
| * The DB at process.env.DOCKET_DB_PATH (default /opt/docket/data/docket.db) is | ||
| * written by the Python ingest. Some tables do not exist yet while other agents | ||
| @@ -21,7 +21,7 @@ let _db: Database.Database | null = null; | ||
| * the DB is in WAL mode (so ingest can write while we read without blocking), | ||
| * and a read-only SQLite handle cannot operate on a WAL database because WAL | ||
| * requires writing the shared-memory index. This layer only ever issues | ||
| - | * SELECTs, so it behaves as a WAL *reader* - it never writes data. A | |
| + | * SELECTs, so it behaves as a WAL *reader* - it never writes data. A | |
| * busy_timeout means a read waits briefly for a checkpoint instead of the | ||
| * synchronous driver hanging the Node event loop. | ||
| */ | ||
| @@ -43,7 +43,7 @@ function db(): Database.Database { | ||
| /** | ||
| * Run a SELECT and map rows. Any error (missing table/column, corrupt read) | ||
| - | * is swallowed and [] is returned - the API contract is "always respond". | |
| + | * is swallowed and [] is returned - the API contract is "always respond". | |
| */ | ||
| export function safeAll<T>( | ||
| sql: string, |
| @@ -1,10 +1,10 @@ | ||
| /** | ||
| - | * Docket read model - maps SQLite ingest rows to the JSON shapes in | |
| + | * Docket read model - maps SQLite ingest rows to the JSON shapes in | |
| * docs/API.md. Shared by the /api route handlers AND the server-rendered | ||
| * pages (pages call these directly instead of fetch-to-self). | ||
| * | ||
| * Ranking (neutrality rule #3) uses ONLY { actionDate, dollarAmount, | ||
| - | * affectedEstimate, actionType } - no engagement/news/social fields. | |
| + | * affectedEstimate, actionType } - no engagement/news/social fields. | |
| * | ||
| * Server-only. Imports better-sqlite3 (kept external via next.config.js). | ||
| */ | ||
| @@ -92,9 +92,9 @@ export type Apportionment = { | ||
| export type Vote = { | ||
| id: string; | ||
| billId?: number | null; | ||
| - | /** Bill number (e.g. "H.R. 22") - populated when votes.bill_id is linked. */ | |
| + | /** Bill number (e.g. "H.R. 22") - populated when votes.bill_id is linked. */ | |
| billNumber?: string; | ||
| - | /** Bill title - populated when votes.bill_id is linked. */ | |
| + | /** Bill title - populated when votes.bill_id is linked. */ | |
| billTitle?: string; | ||
| /** | ||
| * Raw bill/doc reference from the vote record (e.g. "H.R. 6938", "PN123", | ||
| @@ -112,6 +112,9 @@ export type Vote = { | ||
| recorded: boolean; | ||
| sourceUrl: string; | ||
| href?: string; | ||
| + | /** Docket's preserved copy of the roll-call XML on archive.usdocket.org. | |
| + | * Present only when we have archived the record; undefined otherwise. */ | |
| + | archiveUrl?: string; | |
| }; | ||
| export type ExecutiveAction = { | ||
| @@ -189,12 +192,12 @@ export type MemberVote = { | ||
| }; | ||
| /** | ||
| - | * A year bucket of a member's votes - the "year by year how they voted" | |
| + | * A year bucket of a member's votes - the "year by year how they voted" | |
| * view. `year` is null when the underlying vote row is undated; such a | ||
| * bucket sorts last and is labelled "Undated" by the UI. | ||
| */ | ||
| /** Year-summary row for the member year-by-year panel header. | ||
| - | * Votes are NOT embedded here - they are fetched lazily by the | |
| + | * Votes are NOT embedded here - they are fetched lazily by the | |
| * MemberVoteYear client component when a panel is first opened. */ | ||
| export type MemberYearGroup = { | ||
| year: number | null; | ||
| @@ -300,6 +303,15 @@ export type BillDetail = { | ||
| subjects: string[]; | ||
| fullTextUrl: string; | ||
| summaryPlain: string; | ||
| + | /** Plain-language summary text (Congressional Research Service via GovInfo). | |
| + | * "" when none has been ingested yet - most bills have no summary. */ | |
| + | summary: string; | |
| + | /** Provenance label for the summary, e.g. "CRS via GovInfo". */ | |
| + | summarySource: string; | |
| + | /** Official government PDF of the bill text (govinfo.gov). "" when none. */ | |
| + | pdfUrl: string; | |
| + | /** Docket's preserved copy of the bill PDF (archive.usdocket.org). "" when none. */ | |
| + | archivePdfUrl: string; | |
| }; | ||
| /* ------------------------------------------------------------------ */ | ||
| @@ -321,7 +333,7 @@ export type StateDetail = StateCoverage & { | ||
| }; | ||
| /* ------------------------------------------------------------------ */ | ||
| - | /* Ranking (rule #3) - pure, references only the 4 permitted fields. */ | |
| + | /* Ranking (rule #3) - pure, references only the 4 permitted fields. */ | |
| /* ------------------------------------------------------------------ */ | ||
| type Rankable = { | ||
| @@ -336,7 +348,7 @@ type Rankable = { | ||
| * magnitude, then by people affected, then action type. Empty/missing dates | ||
| * sink to the bottom (undated). Returns a sort comparator. | ||
| * | ||
| - | * Intentionally reads ONLY the four ranking fields - no engagement inputs. | |
| + | * Intentionally reads ONLY the four ranking fields - no engagement inputs. | |
| */ | ||
| export function rankByRule(): (a: Rankable, b: Rankable) => number { | ||
| return (a, b) => { | ||
| @@ -367,20 +379,20 @@ function dateKey(iso: string): number { | ||
| } | ||
| /* ------------------------------------------------------------------ */ | ||
| - | /* FEED - mixed, ranked. Pulls each source defensively. */ | |
| + | /* FEED - mixed, ranked. Pulls each source defensively. */ | |
| /* ------------------------------------------------------------------ */ | ||
| /** | ||
| * Get the ranked, diversified feed. Each source is capped at `perSource` | ||
| * rows in SQL so we never pull tens-of-thousands of rows just to slice 150. | ||
| - | * Results are cached for CACHE_TTL_MS (5 min) - staleness is acceptable for | |
| + | * Results are cached for CACHE_TTL_MS (5 min) - staleness is acceptable for | |
| * this public read-only data site. | ||
| */ | ||
| export function getFeedItems(perSource = 400): FeedItem[] { | ||
| return cached(`feed:${perSource}`, () => { | ||
| const items: FeedItem[] = []; | ||
| - | // Awards (USAspending) - top by dollar amount. | |
| + | // Awards (USAspending) - top by dollar amount. | |
| items.push( | ||
| ...safeAll<FeedItem>( | ||
| `SELECT id, agency, recipient, amount, action_date, program, source_url | ||
| @@ -392,7 +404,7 @@ export function getFeedItems(perSource = 400): FeedItem[] { | ||
| ) | ||
| ); | ||
| - | // Votes (House/Senate roll calls) - most recent. | |
| + | // Votes (House/Senate roll calls) - most recent. | |
| items.push( | ||
| ...safeAll<FeedItem>( | ||
| `SELECT id, bill_id, chamber, date, yea, nay, present, result, source_url | ||
| @@ -404,7 +416,7 @@ export function getFeedItems(perSource = 400): FeedItem[] { | ||
| ) | ||
| ); | ||
| - | // Executive actions (Federal Register) - most recent. | |
| + | // Executive actions (Federal Register) - most recent. | |
| items.push( | ||
| ...safeAll<FeedItem>( | ||
| `SELECT id, type, title, agency, fr_doc_number, date, source_url | ||
| @@ -416,6 +428,18 @@ export function getFeedItems(perSource = 400): FeedItem[] { | ||
| ) | ||
| ); | ||
| + | // Bills (Congress.gov) - most recently acted on (latest action / updated_at). | |
| + | items.push( | |
| + | ...safeAll<FeedItem>( | |
| + | `SELECT id, number, title, status, updated_at, source_url | |
| + | FROM bills | |
| + | ORDER BY updated_at DESC | |
| + | LIMIT :lim`, | |
| + | { lim: perSource }, | |
| + | (r) => toFeedBill(r) | |
| + | ) | |
| + | ); | |
| + | ||
| return diversifyByKind(items.sort(rankItems)); | ||
| }); | ||
| } | ||
| @@ -424,9 +448,11 @@ export function getFeedItems(perSource = 400): FeedItem[] { | ||
| export function getFeedKindCounts(): Map<string, number> { | ||
| return cached("feed:kindcounts", () => { | ||
| const map = new Map<string, number>(); | ||
| + | const billsCount = safeCount("bills"); | |
| const awardsCount = safeCount("awards"); | ||
| const votesCount = safeCount("votes"); | ||
| const execCount = safeCount("executive_actions"); | ||
| + | if (billsCount) map.set("bill", billsCount); | |
| if (awardsCount) map.set("award", awardsCount); | ||
| if (votesCount) map.set("vote", votesCount); | ||
| if (execCount) map.set("executive", execCount); | ||
| @@ -441,7 +467,7 @@ export function getFeedKindCounts(): Map<string, number> { | ||
| * vote / executive / …) so the visible top of the Register isn't a wall of | ||
| * same-source, same-day Federal Register rows. Buckets stay in rank order and | ||
| * items never leave their date bucket, so the official-action ranking is | ||
| - | * preserved - only the interleaving within a day changes. | |
| + | * preserved - only the interleaving within a day changes. | |
| */ | ||
| function diversifyByKind(items: FeedItem[]): FeedItem[] { | ||
| if (items.length <= 1) return items; | ||
| @@ -493,7 +519,7 @@ function diversifyByKind(items: FeedItem[]): FeedItem[] { | ||
| /** | ||
| * Project a feed item onto ONLY the four ranking fields (rule #3), then | ||
| * compare. The underlying comparator (rankByRule) is pure and references | ||
| - | * nothing else - no engagement/news/social input. | |
| + | * nothing else - no engagement/news/social input. | |
| */ | ||
| function toRankable(i: FeedItem): Rankable { | ||
| return { | ||
| @@ -564,6 +590,26 @@ function toFeedExecutive(r: Record<string, unknown>): FeedItem { | ||
| }; | ||
| } | ||
| + | function toFeedBill(r: Record<string, unknown>): FeedItem { | |
| + | const billId = num(r.id); | |
| + | const number = str(r.number) || "Bill"; | |
| + | const status = str(r.status); | |
| + | return { | |
| + | id: `bill:${r.id}`, | |
| + | kind: "bill", | |
| + | actionDate: str(r.updated_at), | |
| + | chamber: "CONGRESS", | |
| + | title: number, | |
| + | summary: str(r.title) || undefined, | |
| + | actionType: status ? status.toUpperCase() : "BILL", | |
| + | amount: null, | |
| + | amountText: undefined, | |
| + | jurisdiction: "US", | |
| + | sourceUrl: ensureSource(r.source_url), | |
| + | href: `/bill/${billId}`, | |
| + | }; | |
| + | } | |
| + | ||
| function execTypeLabel(t: string): string { | ||
| switch (t.toLowerCase()) { | ||
| case "eo": | ||
| @@ -712,7 +758,7 @@ export function getVoteCount(chamber?: string): number { | ||
| return safeCount("votes"); | ||
| } | ||
| - | /** Filter shape for votes list - pushed into SQL for performance (35k+ rows). */ | |
| + | /** Filter shape for votes list - pushed into SQL for performance (35k+ rows). */ | |
| export type VoteFilter = { | ||
| chamber?: string; // "senate" | "house" | ||
| q?: string; // keyword search on result + question text | ||
| @@ -746,7 +792,7 @@ function buildVotesWhere(filter: VoteFilter): { where: string; params: Record<st | ||
| return { where, params }; | ||
| } | ||
| - | /** ORDER BY clause for vote list sort - never references non-ranking fields. */ | |
| + | /** ORDER BY clause for vote list sort - never references non-ranking fields. */ | |
| function buildVotesOrder(sort?: VoteFilter["sort"]): string { | ||
| switch (sort) { | ||
| case "chamber": return "v.chamber, v.date DESC, v.id DESC"; | ||
| @@ -755,7 +801,7 @@ function buildVotesOrder(sort?: VoteFilter["sort"]): string { | ||
| } | ||
| } | ||
| - | /** Count votes matching the given filter - cheap SELECT COUNT(*). */ | |
| + | /** Count votes matching the given filter - cheap SELECT COUNT(*). */ | |
| export function getVoteCountFiltered(filter: VoteFilter = {}): number { | ||
| const { where, params } = buildVotesWhere(filter); | ||
| // Use `v` alias to match the WHERE clauses produced by buildVotesWhere. | ||
| @@ -766,7 +812,7 @@ export function getVoteCountFiltered(filter: VoteFilter = {}): number { | ||
| return row?.c ?? 0; | ||
| } | ||
| - | /** Paginated votes list - delegates to getVotesFiltered with optional chamber. */ | |
| + | /** Paginated votes list - delegates to getVotesFiltered with optional chamber. */ | |
| export function getVotes(limit = 100, offset = 0, chamber?: string): Vote[] { | ||
| return getVotesFiltered(chamber ? { chamber } : {}, limit, offset); | ||
| } | ||
| @@ -774,14 +820,14 @@ export function getVotes(limit = 100, offset = 0, chamber?: string): Vote[] { | ||
| /** | ||
| * Filtered, paginated votes list. | ||
| * LEFT JOINs bills so each row carries the linked bill's number + title | ||
| - | * (gracefully null when votes.bill_id is not set - no bill is shown). | |
| + | * (gracefully null when votes.bill_id is not set - no bill is shown). | |
| */ | ||
| export function getVotesFiltered(filter: VoteFilter = {}, limit = 100, offset = 0): Vote[] { | ||
| const { where, params } = buildVotesWhere(filter); | ||
| const orderBy = buildVotesOrder(filter.sort); | ||
| return safeAll<Vote>( | ||
| `SELECT v.id, v.bill_id, v.bill_text_ref, v.chamber, v.date, | ||
| - | v.yea, v.nay, v.present, v.result, v.source_url, | |
| + | v.yea, v.nay, v.present, v.result, v.source_url, v.archive_url, | |
| b.number AS bill_number, b.title AS bill_title | ||
| FROM votes v | ||
| LEFT JOIN bills b ON b.id = v.bill_id${where} | ||
| @@ -801,7 +847,7 @@ export function getVoteById(id: number): VoteDetail | null { | ||
| const r = safeGet<Record<string, unknown>>( | ||
| `SELECT v.id, v.bill_id, v.bill_text_ref, v.chamber, v.date, | ||
| v.yea, v.nay, v.present, v.not_voting, v.roll_number, | ||
| - | v.question, v.recorded, v.result, v.source_url, | |
| + | v.question, v.recorded, v.result, v.source_url, v.archive_url, | |
| b.number AS bill_number, b.title AS bill_title | ||
| FROM votes v | ||
| LEFT JOIN bills b ON b.id = v.bill_id | ||
| @@ -850,6 +896,7 @@ export function getVoteById(id: number): VoteDetail | null { | ||
| recorded, | ||
| sourceUrl: ensureSource(r.source_url), | ||
| href: undefined, | ||
| + | archiveUrl: str(r.archive_url) || undefined, | |
| question: str(r.question) || undefined, | ||
| rollNumber: nullableNum(r.roll_number) ?? undefined, | ||
| notVoting: nullableNum(r.not_voting) ?? undefined, | ||
| @@ -878,6 +925,7 @@ function toVote(r: Record<string, unknown>): Vote { | ||
| recorded, | ||
| sourceUrl: ensureSource(r.source_url), | ||
| href: `/votes/${voteId}`, | ||
| + | archiveUrl: str(r.archive_url) || undefined, | |
| }; | ||
| } | ||
| @@ -905,7 +953,7 @@ export function getExecutive(): ExecutiveAction[] { | ||
| } | ||
| /* ------------------------------------------------------------------ */ | ||
| - | /* BILLS - list, detail, diff */ | |
| + | /* BILLS - list, detail, diff */ | |
| /* ------------------------------------------------------------------ */ | ||
| export type BillListItem = { | ||
| @@ -1032,10 +1080,133 @@ export function getBillCongresses(): number[] { | ||
| ); | ||
| } | ||
| + | /* ------------------------------------------------------------------ */ | |
| + | /* LAWS - enacted bills (Public Laws) */ | |
| + | /* ------------------------------------------------------------------ */ | |
| + | ||
| + | export type LawListItem = { | |
| + | /** Bill id token, e.g. "bill:14496". */ | |
| + | id: string; | |
| + | /** Link to the enacting bill detail page, e.g. "/bill/14496". */ | |
| + | billHref: string; | |
| + | /** Bill number that became law, e.g. "S.J.Res. 80". */ | |
| + | billNumber: string; | |
| + | /** Parsed Public Law number, e.g. "119-47". */ | |
| + | plNumber: string; | |
| + | /** Congress that enacted it (parsed from the PL number), e.g. 119. */ | |
| + | plCongress: number; | |
| + | /** Sequence number within the congress (parsed), e.g. 47. */ | |
| + | plSequence: number; | |
| + | title: string; | |
| + | congress: number; | |
| + | status: string; | |
| + | /** First lines of the plain-language summary, when present ("" otherwise). */ | |
| + | summary: string; | |
| + | summarySource: string; | |
| + | sourceUrl: string; | |
| + | /** Official government PDF (govinfo.gov), "" when none. */ | |
| + | pdfUrl: string; | |
| + | /** Docket's preserved copy (archive.usdocket.org), "" when none. */ | |
| + | archivePdfUrl: string; | |
| + | }; | |
| + | ||
| + | /** Pull the "119-47" out of "Became Public Law No: 119-47." */ | |
| + | function parsePlNumber(status: string): { plNumber: string; congress: number; seq: number } { | |
| + | const m = /Public Law No:\s*(\d+)\s*-\s*(\d+)/i.exec(status || ""); | |
| + | if (!m) return { plNumber: "", congress: 0, seq: 0 }; | |
| + | return { plNumber: `${m[1]}-${m[2]}`, congress: parseInt(m[1], 10), seq: parseInt(m[2], 10) }; | |
| + | } | |
| + | ||
| + | /** Count of enacted Private Laws (a separate, much smaller class). */ | |
| + | export function getPrivateLawsCount(): number { | |
| + | const row = safeGet<{ c: number }>( | |
| + | `SELECT COUNT(*) AS c FROM bills WHERE status LIKE 'Became Private Law No:%'` | |
| + | ); | |
| + | return row?.c ?? 0; | |
| + | } | |
| + | ||
| + | /** | |
| + | * Enacted Public Laws - bills whose status records "Became Public Law No: N-M". | |
| + | * Sorted most-recent-first (congress desc, then PL sequence desc). The PL number | |
| + | * is parsed from the status text, which SQLite cannot sort on numerically, so we | |
| + | * fetch the matching rows (a small set - under ~200 across all congresses) and | |
| + | * sort + paginate in JS. NULL-safe: summary/pdf may be absent. | |
| + | */ | |
| + | function getPublicLawRows(q?: string): LawListItem[] { | |
| + | const params: Record<string, unknown> = {}; | |
| + | let where = "WHERE b.status LIKE 'Became Public Law No:%'"; | |
| + | if (q && q.trim()) { | |
| + | const like = `%${q.trim().replace(/[%_\\]/g, (m) => `\\${m}`)}%`; | |
| + | where += | |
| + | " AND (b.number LIKE :q ESCAPE '\\' OR b.title LIKE :q ESCAPE '\\' OR b.status LIKE :q ESCAPE '\\')"; | |
| + | params.q = like; | |
| + | } | |
| + | const rows = safeAll<Record<string, unknown>>( | |
| + | `SELECT b.id, b.congress, b.number, b.title, b.status, b.source_url, | |
| + | b.summary, b.summary_source, b.pdf_url, b.archive_pdf_url | |
| + | FROM bills b ${where}`, | |
| + | params | |
| + | ); | |
| + | return rows.map((r) => { | |
| + | const status = str(r.status); | |
| + | const parsed = parsePlNumber(status); | |
| + | return { | |
| + | id: `bill:${r.id}`, | |
| + | billHref: `/bill/${r.id}`, | |
| + | billNumber: str(r.number), | |
| + | plNumber: parsed.plNumber, | |
| + | plCongress: parsed.congress || num(r.congress), | |
| + | plSequence: parsed.seq, | |
| + | title: str(r.title), | |
| + | congress: num(r.congress), | |
| + | status, | |
| + | summary: str(r.summary), | |
| + | summarySource: str(r.summary_source), | |
| + | sourceUrl: ensureSource(r.source_url), | |
| + | pdfUrl: str(r.pdf_url), | |
| + | archivePdfUrl: str(r.archive_pdf_url), | |
| + | }; | |
| + | }); | |
| + | } | |
| + | ||
| + | function sortLaws(items: LawListItem[], sort?: "recent" | "oldest" | "number"): LawListItem[] { | |
| + | const byRecent = (a: LawListItem, b: LawListItem) => | |
| + | b.plCongress - a.plCongress || b.plSequence - a.plSequence; | |
| + | switch (sort) { | |
| + | case "oldest": | |
| + | return [...items].sort((a, b) => -byRecent(a, b)); | |
| + | case "number": | |
| + | // Ascending PL number within most-recent congress first. | |
| + | return [...items].sort( | |
| + | (a, b) => b.plCongress - a.plCongress || a.plSequence - b.plSequence | |
| + | ); | |
| + | default: | |
| + | return [...items].sort(byRecent); | |
| + | } | |
| + | } | |
| + | ||
| + | export type LawSort = "recent" | "oldest" | "number"; | |
| + | ||
| + | /** Paginated, sorted, optionally-filtered list of enacted Public Laws. */ | |
| + | export function getPublicLaws( | |
| + | opts: { q?: string; sort?: LawSort } = {}, | |
| + | limit = 200, | |
| + | offset = 0 | |
| + | ): LawListItem[] { | |
| + | const sorted = sortLaws(getPublicLawRows(opts.q), opts.sort); | |
| + | return sorted.slice(offset, offset + limit); | |
| + | } | |
| + | ||
| + | /** Total enacted Public Laws matching the (optional) query. */ | |
| + | export function getPublicLawsCount(q?: string): number { | |
| + | return getPublicLawRows(q).length; | |
| + | } | |
| + | ||
| export function getBillDetail(idNum: number): BillDetail | null { | ||
| const r = safeGet<Record<string, unknown>>( | ||
| `SELECT b.id, b.congress, b.number, b.title, b.status, b.policy_area, | ||
| b.updated_at, b.source_url, b.sponsor_id, | ||
| + | b.summary, b.summary_source, b.pdf_url, b.archive_pdf_url, | |
| (SELECT e.sponsor_name FROM bills_ext e WHERE e.bill_id = b.id) AS sponsor_name | ||
| FROM bills b WHERE b.id = :id`, | ||
| { id: idNum } | ||
| @@ -1128,12 +1299,18 @@ export function getBillDetail(idNum: number): BillDetail | null { | ||
| timeline, | ||
| subjects: getBillSubjects(idNum), | ||
| fullTextUrl: ensureSource(r.source_url), | ||
| - | summaryPlain: "", | |
| + | summaryPlain: str(r.summary), | |
| + | summary: str(r.summary), | |
| + | summarySource: str(r.summary_source), | |
| + | // PDF urls are NOT run through ensureSource: absent must stay empty so the | |
| + | // page can branch on "no PDF" rather than show a stub official link. | |
| + | pdfUrl: str(r.pdf_url), | |
| + | archivePdfUrl: str(r.archive_pdf_url), | |
| }; | ||
| } | ||
| /** | ||
| - | * Section-level diff between two versions - the "what got slipped in" view. | |
| + | * Section-level diff between two versions - the "what got slipped in" view. | |
| * (docs/API.md signature feature.) Returns the changes for the first matching | ||
| * version pair, or all changes for the bill when from/to are omitted. | ||
| */ | ||
| @@ -1186,11 +1363,11 @@ export type MemberFilter = { | ||
| party?: PartyCode; | ||
| }; | ||
| - | /** All members - cached so the members page and per-state filtering don't re-query. */ | |
| + | /** All members - cached so the members page and per-state filtering don't re-query. */ | |
| function getAllMembersRaw(): MemberSummary[] { | ||
| return cached("members:all", () => | ||
| safeAll<MemberSummary>( | ||
| - | `SELECT id, bioguide_id, full_name, party, state, district, chamber, photo_url, source_url | |
| + | `SELECT id, bioguide_id, full_name, party, state, district, chamber, photo_url, source_url, is_current | |
| FROM members | ||
| ORDER BY chamber, state, full_name`, | ||
| {}, | ||
| @@ -1201,7 +1378,7 @@ function getAllMembersRaw(): MemberSummary[] { | ||
| /** | ||
| * Current Congress members only (IDs 1-600, the 119th sitting House + Senate). | ||
| - | * This is the default SSR seed for the members page - ~537 members instead of | |
| + | * This is the default SSR seed for the members page - ~537 members instead of | |
| * 12.7k. Historical members are fetched lazily via /api/members when the user | ||
| * opens the disclosure toggle. Keeps the SSR HTML payload under 1MB. | ||
| */ | ||
| @@ -1287,7 +1464,7 @@ function _getMemberImpl(rawId: string): MemberDetail | null { | ||
| const numeric = parseInt(cleaned, 10); | ||
| const byId = Number.isFinite(numeric) | ||
| ? safeGet<Record<string, unknown>>( | ||
| - | `SELECT id, bioguide_id, full_name, party, state, district, chamber, photo_url, source_url | |
| + | `SELECT id, bioguide_id, full_name, party, state, district, chamber, photo_url, source_url, is_current | |
| FROM members WHERE id = :id`, | ||
| { id: numeric } | ||
| ) | ||
| @@ -1295,7 +1472,7 @@ function _getMemberImpl(rawId: string): MemberDetail | null { | ||
| const row = | ||
| byId ?? | ||
| safeGet<Record<string, unknown>>( | ||
| - | `SELECT id, bioguide_id, full_name, party, state, district, chamber, photo_url, source_url | |
| + | `SELECT id, bioguide_id, full_name, party, state, district, chamber, photo_url, source_url, is_current | |
| FROM members WHERE bioguide_id = :bg`, | ||
| { bg: cleaned } | ||
| ); | ||
| @@ -1366,7 +1543,7 @@ function _getMemberImpl(rawId: string): MemberDetail | null { | ||
| { mid: summary.numericId } | ||
| )?.c ?? votes.length; | ||
| - | // Year-by-year summary - cheap GROUP BY, no vote rows serialised into SSR. | |
| + | // Year-by-year summary - cheap GROUP BY, no vote rows serialised into SSR. | |
| // Votes for each year are fetched lazily by the MemberVoteYear client component. | ||
| const votesByYear = safeAll<MemberYearGroup>( | ||
| `SELECT v.year, COUNT(*) AS count | ||
| @@ -1450,16 +1627,19 @@ function toMemberSummary(r: Record<string, unknown>): MemberSummary { | ||
| photoUrl: str(r.photo_url), | ||
| sourceUrl: ensureSource(r.source_url), | ||
| href: `/members/${idNum}`, | ||
| - | // IDs 1-537 are the 119th-Congress batch (sitting House + Senate). | |
| - | // Historical members begin at ID 2149. Threshold 600 gives safe headroom. | |
| - | isCurrent: idNum <= 600, | |
| + | // Authoritative current-member flag: the members.is_current column is set | |
| + | // to 1 for every bioguide present in unitedstates/congress-legislators | |
| + | // legislators-current.json (the 119th-Congress sitting House + Senate), | |
| + | // backfilled by services/ingest/backfill_is_current.py and maintained by | |
| + | // the members poller. Defaults to false when the column/value is absent. | |
| + | isCurrent: num(r.is_current) === 1, | |
| }; | ||
| } | ||
| function getMemberSummaryByBioguide(bioguide: string): MemberSummary | null { | ||
| if (!bioguide) return null; | ||
| const r = safeGet<Record<string, unknown>>( | ||
| - | `SELECT id, bioguide_id, full_name, party, state, district, chamber, photo_url, source_url | |
| + | `SELECT id, bioguide_id, full_name, party, state, district, chamber, photo_url, source_url, is_current | |
| FROM members WHERE bioguide_id = :bg`, | ||
| { bg: bioguide } | ||
| ); | ||
| @@ -1568,8 +1748,11 @@ export function getStates(): StateCoverage[] { | ||
| .sort((a, b) => a.code.localeCompare(b.code)); | ||
| } | ||
| - | export function getStateDetail(code: string): StateDetail { | |
| + | export function getStateDetail(code: string): StateDetail | null { | |
| const upper = (code || "").toUpperCase(); | ||
| + | // Unknown / invalid code → null so the route can 404. Only the known | |
| + | // state + territory set (STATE_NAMES) is valid. | |
| + | if (!Object.prototype.hasOwnProperty.call(STATE_NAMES, upper)) return null; | |
| const states = getStates(); | ||
| const coverage = | ||
| states.find((s) => s.code === upper) ?? { | ||
| @@ -1580,7 +1763,11 @@ export function getStateDetail(code: string): StateDetail { | ||
| missingTypes: ["bills", "votes", "sponsors"], | ||
| }; | ||
| - | const members = getMembers({ state: upper }); | |
| + | // Delegation = the CURRENT (sitting) members only. getMembers returns every | |
| + | // member ever associated with the state (CA has 438 historical reps); the | |
| + | // state page is a "who represents this state now" view, so filter to the | |
| + | // 119th-Congress batch (isCurrent). WY → 3, CA → 53. | |
| + | const members = getMembers({ state: upper }).filter((m) => m.isCurrent); | |
| const bills = safeAll<BillListItem>( | ||
| `SELECT b.id, b.congress, b.number, b.title, b.status, b.policy_area, | ||
| b.updated_at, b.source_url | ||
| @@ -1670,7 +1857,7 @@ export function globalSearch(q: string): SearchResults { | ||
| ); | ||
| const members = safeAll<MemberSummary>( | ||
| - | `SELECT id, bioguide_id, full_name, party, state, district, chamber, photo_url, source_url | |
| + | `SELECT id, bioguide_id, full_name, party, state, district, chamber, photo_url, source_url, is_current | |
| FROM members | ||
| WHERE full_name LIKE :q ESCAPE '\\' OR bioguide_id LIKE :q ESCAPE '\\' | ||
| OR state LIKE :q ESCAPE '\\' | ||
| @@ -1708,7 +1895,7 @@ export function globalSearch(q: string): SearchResults { | ||
| } | ||
| /* ------------------------------------------------------------------ */ | ||
| - | /* CALENDAR - what takes effect when (calendar_events) */ | |
| + | /* CALENDAR - what takes effect when (calendar_events) */ | |
| /* ------------------------------------------------------------------ */ | ||
| export type CalendarKind = | ||
| @@ -1735,7 +1922,7 @@ export type CalendarEvent = { | ||
| }; | ||
| /** | ||
| - | * Per-day kind counts for the calendar grid - a lightweight alternative to | |
| + | * Per-day kind counts for the calendar grid - a lightweight alternative to | |
| * fetching the full CalendarEvent list. Returns one row per (date, kind) | ||
| * combination so the grid can render event-type dots without serializing | ||
| * 2,700+ full event objects into the page's RSC props. | ||
| @@ -1760,7 +1947,7 @@ export function getCalendarDaySummary(from: string, to: string): CalendarDayKind | ||
| /** | ||
| * Events in an inclusive date range (YYYY-MM-DD), optionally filtered by | ||
| - | * kind. Pulls from calendar_events - the parsed "takes effect / enacted / | |
| + | * kind. Pulls from calendar_events - the parsed "takes effect / enacted / | |
| * vote / floor / committee" dates. Each row carries an allowlisted | ||
| * source_url (neutrality #1). | ||
| */ | ||
| @@ -1822,7 +2009,7 @@ function calendarHref(refType: string, refId: number | null): string | null { | ||
| } | ||
| /* ------------------------------------------------------------------ */ | ||
| - | /* MEETINGS - congressional committee + floor schedule (meetings table)*/ | |
| + | /* MEETINGS - congressional committee + floor schedule (meetings table)*/ | |
| /* ------------------------------------------------------------------ */ | ||
| export type MeetingType = | ||
| @@ -1984,7 +2171,7 @@ export function getBillSubjects(billId: number, max = 8): string[] { | ||
| } | ||
| /* ------------------------------------------------------------------ */ | ||
| - | /* CHANGE EVENTS - the watcher feed (added / updated / removed) */ | |
| + | /* CHANGE EVENTS - the watcher feed (added / updated / removed) */ | |
| /* ------------------------------------------------------------------ */ | ||
| export type ChangeKind = "added" | "updated" | "removed"; | ||
| @@ -2001,7 +2188,7 @@ export type ChangeEvent = { | ||
| href: string | null; | ||
| }; | ||
| - | /** Recent change_events - what the ingest added / updated / removed. */ | |
| + | /** Recent change_events - what the ingest added / updated / removed. */ | |
| export function getChangeEvents(limit = 25): ChangeEvent[] { | ||
| return safeAll<ChangeEvent>( | ||
| `SELECT id, ts, source, ref_type, ref_id, change, summary, source_url |
| @@ -1,5 +1,5 @@ | ||
| /** | ||
| - | * Pure formatting helpers - no DB, safe for client + server. | |
| + | * Pure formatting helpers - no DB, safe for client + server. | |
| * (Previously lived in lib/mock-data.ts; centralized here so API + UI share one implementation.) | ||
| */ | ||
| @@ -21,6 +21,64 @@ export function formatUsd(n: number, opts?: { compact?: boolean }): string { | ||
| }); | ||
| } | ||
| + | /** Federal bill/resolution type prefixes → their canonical display form. */ | |
| + | const FEDERAL_BILL_TYPES: Record<string, string> = { | |
| + | HR: "H.R.", | |
| + | S: "S.", | |
| + | HRES: "H.Res.", | |
| + | SRES: "S.Res.", | |
| + | HJRES: "H.J.Res.", | |
| + | SJRES: "S.J.Res.", | |
| + | HCONRES: "H.Con.Res.", | |
| + | SCONRES: "S.Con.Res.", | |
| + | HAMDT: "H.Amdt.", | |
| + | SAMDT: "S.Amdt.", | |
| + | }; | |
| + | ||
| + | /** Normalize a bill identifier to the site standard. | |
| + | * "HR277" → "H.R. 277", "SJRES14" → "S.J.Res. 14". Already-spaced/dotted | |
| + | * identifiers ("H.R. 76") and non-federal/state numbers ("HB 12", "A 1") | |
| + | * pass through unchanged. */ | |
| + | export function formatBillNumber(raw: string | null | undefined): string { | |
| + | if (!raw) return raw ?? ""; | |
| + | const s = raw.trim(); | |
| + | // Already formatted (has a space or a dot) - leave it alone. | |
| + | if (/[.\s]/.test(s)) return s; | |
| + | const m = s.match(/^([A-Za-z]+)(\d+)$/); | |
| + | if (!m) return s; | |
| + | const pretty = FEDERAL_BILL_TYPES[m[1].toUpperCase()]; | |
| + | return pretty ? `${pretty} ${m[2]}` : s; | |
| + | } | |
| + | ||
| + | /** Ordinal suffix for an integer: 1→"st", 2→"nd", 3→"rd", 4→"th", with the | |
| + | * 11/12/13 exception (all "th"). */ | |
| + | export function ordinalSuffix(n: number): string { | |
| + | const abs = Math.abs(Math.trunc(n)); | |
| + | const mod100 = abs % 100; | |
| + | if (mod100 >= 11 && mod100 <= 13) return "th"; | |
| + | switch (abs % 10) { | |
| + | case 1: | |
| + | return "st"; | |
| + | case 2: | |
| + | return "nd"; | |
| + | case 3: | |
| + | return "rd"; | |
| + | default: | |
| + | return "th"; | |
| + | } | |
| + | } | |
| + | ||
| + | /** Number plus its ordinal suffix: 119→"119th", 1→"1st", 103→"103rd". */ | |
| + | export function ordinal(n: number): string { | |
| + | return `${n}${ordinalSuffix(n)}`; | |
| + | } | |
| + | ||
| + | /** "119th Congress" for a real federal congress number, or "" for non-federal | |
| + | * records (state bills carry congress 0 - never render "0th Congress"). */ | |
| + | export function congressLabel(congress: number | null | undefined): string { | |
| + | return congress && congress > 0 ? `${ordinal(congress)} Congress` : ""; | |
| + | } | |
| + | ||
| export function formatDate(iso: string): string { | ||
| if (!iso) return "\u2014"; | ||
| const d = new Date(iso + (iso.length === 10 ? "T00:00:00Z" : "")); | ||
| @@ -34,7 +92,7 @@ export function formatDate(iso: string): string { | ||
| } | ||
| /** Compact "Mon DD" label (UTC, deterministic) for tight UI slots like the | ||
| - | * live hero's date column - no year, no time-of-day theater. */ | |
| + | * live hero's date column - no year, no time-of-day theater. */ | |
| export function formatDateShort(iso: string): string { | ||
| if (!iso) return "\u2014"; | ||
| const d = new Date(iso.length === 10 ? iso + "T00:00:00Z" : iso); | ||
| @@ -48,7 +106,7 @@ export function formatDateShort(iso: string): string { | ||
| /** ET wall-clock label like "14:02 ET" from an ISO string, or "" if there is | ||
| * no real time component. Date-only values (e.g. Federal Register documents, | ||
| - | * which are dated but not timed) return "" so callers fall back to the DATE - | |
| + | * which are dated but not timed) return "" so callers fall back to the DATE - | |
| * otherwise a date-only string parses as UTC midnight and every row would | ||
| * wrongly read "20:00 ET". Only emit a time when the source actually has one. */ | ||
| export function formatEt(iso: string): string { |
| @@ -1,5 +1,5 @@ | ||
| /** | ||
| - | * MOCK DATA - Phase 0 static shell. No real fetching. Every record carries | |
| + | * MOCK DATA - Phase 0 static shell. No real fetching. Every record carries | |
| * a sourceUrl pointing to an allowlisted government host (neutrality rule #1). | ||
| * | ||
| * Copy is factual only (neutrality rule #2): numbers, status, dates, names. |
| @@ -1,5 +1,5 @@ | ||
| /** | ||
| - | * Docket neutrality helpers - encode rules from docs/NEUTRALITY.md. | |
| + | * Docket neutrality helpers - encode rules from docs/NEUTRALITY.md. | |
| * | ||
| * Rule #1: government-source-only. Every fact's sourceUrl host must be in | ||
| * the allowlist below. The build-time lint (lint/neutrality.mjs) also checks |
| @@ -1,11 +1,11 @@ | ||
| /** | ||
| * Resolve what a vote was about, in priority order: | ||
| - | * 1. Linked bill (billNumber + billTitle) - votes.bill_id is set | |
| + | * 1. Linked bill (billNumber + billTitle) - votes.bill_id is set | |
| * 2. PN-prefix bill_text_ref → Presidential Nomination | ||
| * 3. TREATYDOC-prefix bill_text_ref → Treaty document | ||
| * 4. Other bill_text_ref (historical ref, no link in DB) + question | ||
| * 5. Question text | ||
| - | * 6. Result text (last resort - never leads with just "Passed") | |
| + | * 6. Result text (last resort - never leads with just "Passed") | |
| * | ||
| * Returns { primary, secondary, kind } where `primary` is the bold/linked | ||
| * identifier and `secondary` is additional context (question / result). | ||
| @@ -14,6 +14,8 @@ | ||
| * logic is consistent across surfaces. | ||
| */ | ||
| + | import { formatBillNumber } from "@/lib/format"; | |
| + | ||
| export type VoteSubjectKind = | ||
| | "bill" | ||
| | "nomination" | ||
| @@ -46,7 +48,7 @@ export function resolveVoteSubject(v: VoteSubjectInput): VoteSubject { | ||
| if (v.billNumber) { | ||
| const title = v.billTitle ? ` - ${v.billTitle}` : ""; | ||
| return { | ||
| - | primary: `${v.billNumber}${title}`, | |
| + | primary: `${formatBillNumber(v.billNumber)}${title}`, | |
| secondary: question || result, | ||
| kind: "bill", | ||
| }; | ||
| @@ -73,7 +75,7 @@ export function resolveVoteSubject(v: VoteSubjectInput): VoteSubject { | ||
| // 4. Historical bill reference (bill in record but not yet in our DB) | ||
| if (ref) { | ||
| return { | ||
| - | primary: ref, | |
| + | primary: formatBillNumber(ref), | |
| secondary: question || result, | ||
| kind: "ref", | ||
| }; | ||
| @@ -88,7 +90,7 @@ export function resolveVoteSubject(v: VoteSubjectInput): VoteSubject { | ||
| }; | ||
| } | ||
| - | // 6. Result text as last resort - still better than blank | |
| + | // 6. Result text as last resort - still better than blank | |
| return { | ||
| primary: result || "Roll-call vote", | ||
| secondary: "", |
| @@ -64,7 +64,7 @@ function checkAllowed(url, file) { | ||
| console.log("Docket neutrality lint"); | ||
| console.log("----------------------"); | ||
| - | // Rule #2 - banned words across all copy-bearing files | |
| + | // Rule #2 - banned words across all copy-bearing files | |
| const bannedRe = new RegExp(`\\b(${BANNED.join("|")})\\b`, "gi"); | ||
| console.log("\n#2 no adjectives"); | ||
| for (const rel of SCANS) { | ||
| @@ -87,7 +87,7 @@ for (const rel of SCANS) { | ||
| console.log(` - ${rel}: scanned`); | ||
| } | ||
| - | // Rule #1 - every sourceUrl allowlisted (scan mock data + any string ending in a gov host) | |
| + | // Rule #1 - every sourceUrl allowlisted (scan mock data + any string ending in a gov host) | |
| console.log("\n#1 government sources"); | ||
| const dataFile = path.join(ROOT, "lib/mock-data.ts"); | ||
| if (fs.existsSync(dataFile)) { |
| @@ -2,7 +2,7 @@ import { NextResponse, type NextRequest } from "next/server"; | ||
| /** | ||
| * Lightweight per-IP rate limiter for the public read-API. | ||
| - | * Fixed-window counter in process memory - Docket runs as a single Node | |
| + | * Fixed-window counter in process memory - Docket runs as a single Node | |
| * server (`next start`), so the map persists across requests. This is a | ||
| * basic abuse/DoS guard, not a billing meter; a reverse proxy (Caddy/CF) | ||
| * should sit in front in production for L3/L4 protection. | ||
| @@ -19,7 +19,7 @@ const buckets: Map<string, Bucket> = (g.__docketBuckets ??= new Map()); | ||
| function clientIp(req: NextRequest): string { | ||
| // Trust the CDN's own client-IP header first (Cloudflare). Raw | ||
| // X-Forwarded-For is client-spoofable, so it's the last resort and only | ||
| - | // a backstop - production rate-limiting/DoS protection lives at the | |
| + | // a backstop - production rate-limiting/DoS protection lives at the | |
| // CDN/reverse-proxy edge, not here. | ||
| const cf = req.headers.get("cf-connecting-ip"); | ||
| if (cf) return cf.trim(); |
| @@ -8,7 +8,11 @@ const csp = [ | ||
| "base-uri 'self'", | ||
| "form-action 'self'", | ||
| "frame-ancestors 'none'", | ||
| - | "object-src 'none'", | |
| + | // Embedded bill-text PDFs: Docket's preserved copy (archive.usdocket.org) and | |
| + | // the official govinfo.gov source. Both frame-src (for <iframe>) and object-src | |
| + | // (for <object>) are allowed so the on-page PDF viewer can render either. | |
| + | "frame-src 'self' https://archive.usdocket.org https://www.govinfo.gov https://govinfo.gov", | |
| + | "object-src 'self' https://archive.usdocket.org https://www.govinfo.gov https://govinfo.gov", | |
| // Next.js injects a small inline hydration bootstrap; 'unsafe-inline' kept | ||
| // for scripts/styles (no nonce pipeline). No third-party script origins. | ||
| // In dev only, React/Next need 'unsafe-eval' for HMR + refresh; never in prod. | ||
| @@ -36,7 +40,7 @@ const nextConfig = { | ||
| reactStrictMode: true, | ||
| poweredByHeader: false, // don't advertise Next.js version | ||
| experimental: { | ||
| - | // better-sqlite3 is a native Node addon - never let webpack bundle it. | |
| + | // better-sqlite3 is a native Node addon - never let webpack bundle it. | |
| serverComponentsExternalPackages: ["better-sqlite3"], | ||
| }, | ||
| async headers() { | ||
| @@ -48,7 +52,7 @@ const nextConfig = { | ||
| // will NOT emit Cache-Control: private, no-store. This header ensures CF | ||
| // caches the rendered HTML at the edge for 5 minutes. | ||
| { | ||
| - | source: "/(votes|bills|bill|members|spending|executive|search|states)(.*)?", | |
| + | source: "/(votes|bills|bill|laws|members|spending|executive|search|states)(.*)?", | |
| headers: [ | ||
| { | ||
| key: "Cache-Control", |
| @@ -1,6 +1,5 @@ | ||
| <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32" width="32" height="32"> | ||
| - | <rect width="32" height="32" rx="5" fill="#E0A53B"/> | |
| - | <text x="16" y="17" text-anchor="middle" dominant-baseline="central" | |
| - | font-family="Arial, Helvetica, sans-serif" font-weight="800" | |
| - | font-size="20" fill="#0B0C0E">D</text> | |
| + | <rect width="32" height="32" rx="6" fill="#E0A53B"/> | |
| + | <path d="M9 7 H19 L23 11 V25 H9 Z" fill="#0B0C0E"/> | |
| + | <path d="M19 7 L23 11 H19 Z" fill="#E0A53B"/> | |
| </svg> |
| @@ -1,4 +1,4 @@ | ||
| - | # Bills Pipeline - Build Notes | |
| + | # Bills Pipeline - Build Notes | |
| ## Architecture | ||
| @@ -13,10 +13,10 @@ Five new modules under `services/ingest/ingest/`: | ||
| | `amendments.py` | Parses `<amendments>` from BILLSTATUS XML; re-fetches for backfill | | ||
| Schema additions (all `CREATE TABLE IF NOT EXISTS`): | ||
| - | - `bills_ext` - extended bill metadata (bill_type, sponsor_name/party/state) | |
| - | - `bill_versions` - version rows with govinfo_package_id + text_ref (XML URL) | |
| - | - `bill_diffs` - per-section diff rows with change type + snippet + rider_flag | |
| - | - `amendments` - amendment rows with agreed_method + congress.gov source_url | |
| + | - `bills_ext` - extended bill metadata (bill_type, sponsor_name/party/state) | |
| + | - `bill_versions` - version rows with govinfo_package_id + text_ref (XML URL) | |
| + | - `bill_diffs` - per-section diff rows with change type + snippet + rider_flag | |
| + | - `amendments` - amendment rows with agreed_method + congress.gov source_url | |
| --- | ||
| @@ -42,7 +42,7 @@ https://www.govinfo.gov/bulkdata/BILLSTATUS/{congress}/{type}/BILLSTATUS-{congre | ||
| ### Directory listing returns HTML error | ||
| `GET /bulkdata/BILLSTATUS/119/hr/` returns a "Bulkdata Service Error" HTML page. Use the sitemap instead. | ||
| - | ### Bill text (BILLS collection) - no key needed | |
| + | ### Bill text (BILLS collection) - no key needed | |
| Full bill XML is accessible at: | ||
| ``` | ||
| https://www.govinfo.gov/content/pkg/BILLS-{congress}{type}{number}{version}/xml/BILLS-{congress}{type}{number}{version}.xml | ||
| @@ -63,7 +63,7 @@ Different XML schemas are used for different version types: | ||
| - Introduced (`ih`, `rh`) → long hex IDs: `H71241E5177324574B4C749197110EAB0` | ||
| - Enrolled (`enr`) → short USLM IDs: `d244e72` | ||
| - | When IDs change between versions, the aligner falls back to fuzzy header matching (threshold 0.6). For short bills where all sections have the same content (e.g., Born-Alive Act H.R. 21 ih→eh), this produces 0 diff rows - correct, the text is identical. | |
| + | When IDs change between versions, the aligner falls back to fuzzy header matching (threshold 0.6). For short bills where all sections have the same content (e.g., Born-Alive Act H.R. 21 ih→eh), this produces 0 diff rows - correct, the text is identical. | |
| For bills where the enrolled version switches from House HTML schema to USLM schema (common for enrolled bills), the aligner may incorrectly classify "removed" + "added" instead of "amended." This is a known GovInfo data quality issue. The enrolled→public-law pairs are most affected. | ||
| @@ -98,24 +98,24 @@ The BILLSTATUS XML sometimes includes a textVersion with type like `-119publ61` | ||
| **Rider flags (3 flagged diff rows):** | ||
| - | 1. **H.R. 43** (`rds→enr`) - Alaska Native Claims Settlement Act amendment: | |
| + | 1. **H.R. 43** (`rds→enr`) - Alaska Native Claims Settlement Act amendment: | |
| > "Section 14(c) of the Alaska Native Claims Settlement Act (43 U.S.C. 1613(c)) is amended..." | ||
| Signal: `Is Amended` | ||
| - | 2. **S. 5 - Laken Riley Act** (`es→enr`): | |
| + | 2. **S. 5 - Laken Riley Act** (`es→enr`): | |
| > "Section 236(c) of the Immigration and Nationality Act (8 U.S.C. 1226(c)) is amended..." | ||
| Signal: `Is Amended` | ||
| - | 3. **S. 5 - Laken Riley Act** (`es→enr`, second section): | |
| + | 3. **S. 5 - Laken Riley Act** (`es→enr`, second section): | |
| > "Section 235(b) of the Immigration and Nationality Act (8 U.S.C. 1225(b)) is amended..." | ||
| Signal: `Is Amended` | ||
| **Real amended section example (H.R. 41, ih→rh):** | ||
| - | H.R. 41 - Unrecognized Southeast Alaska Native Communities Recognition and Compensation Act: | |
| + | H.R. 41 - Unrecognized Southeast Alaska Native Communities Recognition and Compensation Act: | |
| Section H4DAD6E8E44444A988CA8A1864ADE20A6 was amended between the introduced version and the committee-reported version. The unified diff shows rewording of the Urban Corporation definition and conveyance authorization. | ||
| **Real amended section example (S. 5, pcs→es):** | ||
| - | S. 5 - Laken Riley Act: | |
| + | S. 5 - Laken Riley Act: | |
| Section H793DBCBA2A07463588AC4F4FDBEDECAF was amended between the placed-on-calendar and engrossed versions, showing Senate floor amendments to the detention mandate language (INA 236(c)). | ||
| --- | ||
| @@ -150,7 +150,7 @@ python -m pytest tests/test_bills_diff.py -v | ||
| | Variable | Purpose | | ||
| |---|---| | ||
| | `DOCKET_DB_PATH` | SQLite path (default `/var/tmp/docket/docket.db`) | | ||
| - | | `DOCKET_CONGRESS_KEY` | api.data.gov key - unlocks Congress.gov API path in `amendments.py` | | |
| + | | `DOCKET_CONGRESS_KEY` | api.data.gov key - unlocks Congress.gov API path in `amendments.py` | | |
| ## Version Chain Order | ||
| @@ -1,4 +1,4 @@ | ||
| - | # Calendar Ingest - Notes | |
| + | # Calendar Ingest - Notes | |
| Module: `ingest/calendar.py` | ||
| Table: `calendar_events` | ||
| @@ -15,7 +15,7 @@ DOCKET_DB_PATH=/opt/docket/data/docket.db \ | ||
| Builds `calendar_events` by aggregating dated events from every other ingest | ||
| table. One run takes ~20 seconds (mostly FR API calls for effective_on | ||
| - | enrichment). Idempotent - safe to re-run. | |
| + | enrichment). Idempotent - safe to re-run. | |
| ### Event kinds and sources | ||
| @@ -62,10 +62,10 @@ Source URL: https://www.federalregister.gov/documents/2026/06/11/2026-11800/... | ||
| ## FR enrichment flow | ||
| - | 1. `fetch_fr_rules_with_effective(conn, days_back=90)` - fetches 90 days of | |
| + | 1. `fetch_fr_rules_with_effective(conn, days_back=90)` - fetches 90 days of | |
| RULE-type documents with `effective_on` field included in the request; | ||
| upserts into `executive_actions`. | ||
| - | 2. `enrich_fr_effective_on(conn)` - for any remaining `executive_actions` rows | |
| + | 2. `enrich_fr_effective_on(conn)` - for any remaining `executive_actions` rows | |
| with `effective_on IS NULL`, batch-fetches from FR API in groups of 50. | ||
| Notices/NPRMs/EOs typically return null (no effective date); that is correct. | ||
| @@ -134,7 +134,7 @@ ORDER BY date, kind | ||
| LIMIT :limit OFFSET :offset | ||
| ``` | ||
| - | This table is designed for date-range queries - the `date` index makes those | |
| + | This table is designed for date-range queries - the `date` index makes those | |
| fast even at 40k+ rows. | ||
| ## Neutrality |
| @@ -1,12 +1,12 @@ | ||
| - | # Categorize Module - Build Notes | |
| + | # Categorize Module - Build Notes | |
| ## What was built | ||
| - | `ingest/categorize.py` - unified taxonomy backfill for bills and spending awards. | |
| + | `ingest/categorize.py` - unified taxonomy backfill for bills and spending awards. | |
| ### Files created | ||
| - | - `ingest/categorize.py` - the module | |
| - | - `tests/test_categorize.py` - 57 tests | |
| + | - `ingest/categorize.py` - the module | |
| + | - `tests/test_categorize.py` - 57 tests | |
| ### Schema changes (all additive) | ||
| | Change | Table | Type | | ||
| @@ -23,7 +23,7 @@ | ||
| - Total bills: **7,190** | ||
| - Bills with `policy_area` (already stored by bills_bulk.py): **7,179 / 7,190** (99.8%) | ||
| - Distinct policy areas in use: **31 of 32** canonical areas | ||
| - | - 11 bills have no policy_area (XML had no `<policyArea>` element - not an error) | |
| + | - 11 bills have no policy_area (XML had no `<policyArea>` element - not an error) | |
| ### Bill subjects (initial backfill of 50 bills) | ||
| - 50 bills fetched → **279 subject rows**, 170 distinct subjects | ||
| @@ -102,13 +102,13 @@ Key mappings: | ||
| 57 tests, 0 failures, 0.47s. | ||
| Test classes: | ||
| - | - `TestAgencyToCategory` - 25 tests: exact match, prefix match, unknown/null handling, all map values are valid labels | |
| - | - `TestPolicyAreas` - 6 tests: count, no duplicates, slug format, key areas present, slug/label consistency | |
| - | - `TestParseSubjects` - 7 tests: single/multiple subjects, empty, missing element, malformed XML, whitespace, alternate root element | |
| - | - `TestEnsureSchema` - 4 tests: creates both tables, adds column, idempotent | |
| - | - `TestSeedCategories` - 4 tests: seeds all areas, idempotent, picks up extra bill areas, Health present | |
| - | - `TestBackfillAwards` - 6 tests: known agency, unknown→NULL, no-overwrite, multiple, count, idempotent | |
| - | - `TestCategoryCounts` - 4 tests: returns list, expected keys, bill counted, award counted | |
| + | - `TestAgencyToCategory` - 25 tests: exact match, prefix match, unknown/null handling, all map values are valid labels | |
| + | - `TestPolicyAreas` - 6 tests: count, no duplicates, slug format, key areas present, slug/label consistency | |
| + | - `TestParseSubjects` - 7 tests: single/multiple subjects, empty, missing element, malformed XML, whitespace, alternate root element | |
| + | - `TestEnsureSchema` - 4 tests: creates both tables, adds column, idempotent | |
| + | - `TestSeedCategories` - 4 tests: seeds all areas, idempotent, picks up extra bill areas, Health present | |
| + | - `TestBackfillAwards` - 6 tests: known agency, unknown→NULL, no-overwrite, multiple, count, idempotent | |
| + | - `TestCategoryCounts` - 4 tests: returns list, expected keys, bill counted, award counted | |
| --- | ||
| @@ -133,7 +133,7 @@ DOCKET_DB_PATH=/opt/docket/data/docket.db \ | ||
| ## Neutrality | ||
| - | - `policy_area` values come from Congress.gov's own taxonomy (Congressional Research Service) - no editorial classification | |
| + | - `policy_area` values come from Congress.gov's own taxonomy (Congressional Research Service) - no editorial classification | |
| - `legislativeSubjects` come from BILLSTATUS XML verbatim | ||
| - Agency→category map keys are official agency names from USAspending data; values align to the same CRS taxonomy | ||
| - No "good/bad" groupings; no partisan framing; categories are factual government classifications |
| @@ -1,4 +1,4 @@ | ||
| - | # Closed-Door + Executive Layer - Build Notes | |
| + | # Closed-Door + Executive Layer - Build Notes | |
| _Built 2026-06-26. All pollers verified live on CT247._ | ||
| ## What was built | ||
| @@ -20,14 +20,14 @@ write; `is_new_event()` / `mark_seen()` enforce idempotency. | ||
| --- | ||
| - | ## Live run results (CT247 - 2026-06-26) | |
| + | ## Live run results (CT247 - 2026-06-26) | |
| | Table | Rows | Notes | | ||
| |-------|------|-------| | ||
| | `executive_actions` | **500** | 7 days of FR docs, all 4 types (PRESDOCU/RULE/PRORULE/NOTICE) | | ||
| | `apportionments` | **20** | 20 records from FY2025 Corps of Engineers slice | | ||
| | `procedural_events` | **188** | 5 session days from CREC-2026-06-16 through 06-23 | | ||
| - | | `lobbying` | 0 | Graceful skip - `DOCKET_LDA_KEY` not set | | |
| + | | `lobbying` | 0 | Graceful skip - `DOCKET_LDA_KEY` not set | | |
| --- | ||
| @@ -60,7 +60,7 @@ Status logic: apportioned/appropriated ≥ 95% → released; > 0 but < 95% → d | ||
| The FY2025 Corps of Engineers records all show "released" (normal operating year). | ||
| The impound-tracker signal emerges when `status = withheld` (OMB released $0 against | ||
| - | an appropriation) - those rows are the headline feature for Docket's "buried data" pillar. | |
| + | an appropriation) - those rows are the headline feature for Docket's "buried data" pillar. | |
| --- | ||
| @@ -77,7 +77,7 @@ source_url: https://www.govinfo.gov/app/details/CREC-2026-06-23 | ||
| ``` | ||
| CREC-2026-06-23 yielded: 80 events (cloture × 13, unanimous_consent × 63, suspension × 4). | ||
| - | No voice-vote matches on that day - the Senate was on cloture votes, not voice-vote-only measures. | |
| + | No voice-vote matches on that day - the Senate was on cloture votes, not voice-vote-only measures. | |
| --- | ||
| @@ -87,7 +87,7 @@ No voice-vote matches on that day - the Senate was on cloture votes, not voice | ||
| 191 passed in 20.80s | ||
| ``` | ||
| - | - `test_crec_voice_vote.py` - 62 tests: voice-vote regex, UC, cloture, suspension, MTR, | |
| + | - `test_crec_voice_vote.py` - 62 tests: voice-vote regex, UC, cloture, suspension, MTR, | |
| discharge, XML/HTML extractor, chamber inference, snippet extraction, `_detect_events()` | ||
| end-to-end, source-URL .gov enforcement, idempotency. | ||
| @@ -95,27 +95,27 @@ No voice-vote matches on that day - the Senate was on cloture votes, not voice | ||
| ## Technical gotchas discovered during build | ||
| - | ### Federal Register API - array param encoding | |
| + | ### Federal Register API - array param encoding | |
| Must use a list of `(key, value)` tuples (not a dict) for `fields[]` and | ||
| `conditions[type][]`. The field `presidential_document_type` does not exist | ||
| - | in the API - causes HTTP 400. Use `abstract` instead. | |
| + | in the API - causes HTTP 400. Use `abstract` instead. | |
| - | ### GovInfo CREC - correct URL | |
| + | ### GovInfo CREC - correct URL | |
| Bulk data path (`/bulkdata/CREC/`) returns HTML error pages, not ZIPs. | ||
| The working URL is: | ||
| ``` | ||
| https://www.govinfo.gov/content/pkg/CREC-YYYY-MM-DD/zip/CREC-YYYY-MM-DD.zip | ||
| ``` | ||
| - | Non-session days (weekends, recesses) return HTML (not 404) - "bad ZIP" errors | |
| + | Non-session days (weekends, recesses) return HTML (not 404) - "bad ZIP" errors | |
| are expected and handled gracefully. Default look-back is 14 days to cover 5+ | ||
| session days. | ||
| - | ### CREC ZIP contents - HTML, not XML | |
| + | ### CREC ZIP contents - HTML, not XML | |
| The ZIP contains `.htm` files (not standalone XML). Text extraction strips HTML | ||
| tags with regex + `html.unescape()`. The `_extract_text_from_xml()` function | ||
| handles both correctly. | ||
| - | ### max.gov apportionments - no REST API | |
| + | ### max.gov apportionments - no REST API | |
| `apportionment-public.max.gov` has no JSON API endpoint. The site serves | ||
| individual JSON files per apportionment record (one per TAFS account + FY + | ||
| iteration), listed in an HTML index page. The poller: | ||
| @@ -126,11 +126,11 @@ iteration), listed in an HTML index page. The poller: | ||
| FY2025 has 6,172 JSON links; FY2026 will grow through the year. A scheduled | ||
| hourly/daily run can work through the full set over time. | ||
| - | openomb.org (the aggregator) returns HTTP 403 for automated requests - use | |
| + | openomb.org (the aggregator) returns HTTP 403 for automated requests - use | |
| max.gov directly, which is also the correct source_url target per the | ||
| neutrality rule. | ||
| - | ### Neutrality rule - source_url enforcement | |
| + | ### Neutrality rule - source_url enforcement | |
| - `federal_register.py`: html_url from FR API → federalregister.gov (.gov ✓) | ||
| - `openomb.py`: direct max.gov JSON file URL (.max.gov = .gov ✓) | ||
| - `crec.py`: govinfo.gov app/details page (.gov ✓) | ||
| @@ -161,7 +161,7 @@ DOCKET_DB_PATH=/opt/docket/data/docket.db DOCKET_LDA_KEY=<your_key> \ | ||
| ## LDA key | ||
| Register free at https://lda.gov/api/. Without a key the lobbying poller logs | ||
| - | a clear message and returns 0 - no crash, no partial state. | |
| + | a clear message and returns 0 - no crash, no partial state. | |
| ## Recommended poll cadences | ||
| @@ -1,4 +1,4 @@ | ||
| - | # Docket Ingest Service - Notes | |
| + | # Docket Ingest Service - Notes | |
| ## What Was Built | ||
| @@ -13,19 +13,19 @@ ingest/ | ||
| ├── run.py # CLI entrypoint (python -m ingest.run --once) | ||
| ├── db.py # SQLite schema + connection (Postgres target in prod) | ||
| ├── change_detection.py # Dedupe on (source, externalId, updateDate) | ||
| - | ├── ranking.py # Feed ranking - only 4 permitted fields (rule #3) | |
| + | ├── ranking.py # Feed ranking - only 4 permitted fields (rule #3) | |
| ├── source_validator.py # .gov allowlist enforcement (rule #1) | ||
| └── pollers/ | ||
| - | ├── usaspending.py # FULLY LIVE - no auth, pulls real awards | |
| - | ├── congress.py # Skeleton - needs DOCKET_CONGRESS_KEY | |
| - | ├── govinfo.py # Skeleton - needs DOCKET_CONGRESS_KEY (same key) | |
| - | ├── treasury.py # Skeleton - no auth (endpoints need verification) | |
| - | ├── votes_house.py # Live - House Clerk XML, no auth | |
| - | ├── votes_senate.py # Live - Senate roll-call XML, no auth | |
| - | └── openstates.py # Skeleton - needs DOCKET_OPENSTATES_KEY (paid tier) | |
| + | ├── usaspending.py # FULLY LIVE - no auth, pulls real awards | |
| + | ├── congress.py # Skeleton - needs DOCKET_CONGRESS_KEY | |
| + | ├── govinfo.py # Skeleton - needs DOCKET_CONGRESS_KEY (same key) | |
| + | ├── treasury.py # Skeleton - no auth (endpoints need verification) | |
| + | ├── votes_house.py # Live - House Clerk XML, no auth | |
| + | ├── votes_senate.py # Live - Senate roll-call XML, no auth | |
| + | └── openstates.py # Skeleton - needs DOCKET_OPENSTATES_KEY (paid tier) | |
| tests/ | ||
| - | ├── test_ranking.py # 16 tests - neutrality rule #3 enforcement | |
| - | └── test_source_url.py # 19 tests - neutrality rule #1 enforcement | |
| + | ├── test_ranking.py # 16 tests - neutrality rule #3 enforcement | |
| + | └── test_source_url.py # 19 tests - neutrality rule #1 enforcement | |
| ``` | ||
| ## How to Run | ||
| @@ -33,7 +33,7 @@ tests/ | ||
| ### Setup (one-time on CT241) | ||
| ```bash | ||
| - | # Venv must be local - the CIFS share (/shared/projects) does not support | |
| + | # Venv must be local - the CIFS share (/shared/projects) does not support | |
| # SQLite file locking or venv symlinks | ||
| python3 -m venv /tmp/docket-venv | ||
| /tmp/docket-venv/bin/pip install -r /shared/projects/docket/services/ingest/requirements.txt | ||
| @@ -83,7 +83,7 @@ votes_senate → 17 new Senate vote rows | ||
| congress → 0 (no API key) | ||
| govinfo → 0 (no API key) | ||
| treasury → 0 (endpoint paths need verification) | ||
| - | votes_house → 0 (no votes found for 2026 - correct, House not in session) | |
| + | votes_house → 0 (no votes found for 2026 - correct, House not in session) | |
| openstates → 0 (no API key) | ||
| ``` | ||
| @@ -95,7 +95,7 @@ external_id: CONT_AWD_DEAC3608GO28308_8900_-NONE-_-NONE- | ||
| agency: Department of Energy | ||
| recipient: ALLIANCE FOR ENERGY INNOVATION, LLC | ||
| amount: $10,788,357,674.94 | ||
| - | action_date: (null - this award type reports null action_date from USAspending) | |
| + | action_date: (null - this award type reports null action_date from USAspending) | |
| program: other | ||
| source_url: https://www.usaspending.gov/award/CONT_AWD_DEAC3608GO28308_8900_-NONE-_-NONE- | ||
| ``` | ||
| @@ -109,32 +109,32 @@ Override with: `export DOCKET_DB_PATH=/path/to/docket.db` | ||
| - **Env var:** `DOCKET_CONGRESS_KEY` | ||
| - **Get it free:** https://api.data.gov/signup | ||
| - Rate limit: 5,000 req/hr (Congress.gov) / 36,000 req/hr (GovInfo) | ||
| - | - Without this key the congress and govinfo pollers skip gracefully - no crash. | |
| + | - Without this key the congress and govinfo pollers skip gracefully - no crash. | |
| ### OpenStates (state bills) | ||
| - **Env var:** `DOCKET_OPENSTATES_KEY` | ||
| - | - **Free tier:** 500 req/day - too tight for production. | |
| - | - **Required tier:** Bronze (5k/day) or Silver (50k/day) - contact OpenStates for pricing. | |
| - | - Without this key the openstates poller skips gracefully - no crash. | |
| + | - **Free tier:** 500 req/day - too tight for production. | |
| + | - **Required tier:** Bronze (5k/day) or Silver (50k/day) - contact OpenStates for pricing. | |
| + | - Without this key the openstates poller skips gracefully - no crash. | |
| ## Known Issues / Follow-ups | ||
| - | 1. **Treasury endpoints return 404** - the fiscaldata.treasury.gov API paths may | |
| + | 1. **Treasury endpoints return 404** - the fiscaldata.treasury.gov API paths may | |
| have changed. Needs a quick endpoint audit against their API docs before the | ||
| Treasury poller will be useful. | ||
| - | 2. **USAspending `loans` group returns 400** - the loan award codes (07, 08) seem | |
| + | 2. **USAspending `loans` group returns 400** - the loan award codes (07, 08) seem | |
| to require a different query structure. The other 4 groups work correctly. | ||
| - | 3. **`Action Date` is null for many USAspending awards** - the `/search/spending_by_award/` | |
| + | 3. **`Action Date` is null for many USAspending awards** - the `/search/spending_by_award/` | |
| endpoint returns null for Action Date on many records. For completeness, use the | ||
| `/bulk_download/` endpoint or per-award detail endpoint to get richer metadata. | ||
| - | 4. **SQLite on CIFS does not work** - `/shared/projects` on CT241 is a CIFS mount, | |
| + | 4. **SQLite on CIFS does not work** - `/shared/projects` on CT241 is a CIFS mount, | |
| which does not support SQLite's file-locking mechanism. DB defaults to local | ||
| `/var/tmp/docket/docket.db`. For production, migrate to Postgres on a dedicated CT. | ||
| - | 5. **Venv on CIFS does not work** - same CIFS limitation; venv must live in | |
| + | 5. **Venv on CIFS does not work** - same CIFS limitation; venv must live in | |
| `/tmp/docket-venv` or another local path on CT241. | ||
| ## Neutrality Enforcement (architectural summary) |
| @@ -1,4 +1,4 @@ | ||
| - | # Meetings Ingest - Notes | |
| + | # Meetings Ingest - Notes | |
| Module: `ingest/pollers/meetings.py` | ||
| Table: `meetings` | ||
| @@ -12,9 +12,9 @@ DOCKET_DB_PATH=/opt/docket/data/docket.db \ | ||
| ``` | ||
| Options: | ||
| - | - `--days-back N` - days of past House committee meetings to scan (default 30) | |
| - | - `--days-ahead N` - days of future House committee meetings to scan (default 90) | |
| - | - `--no-detail` - skip per-event detail fetches (faster; meeting type omitted for new events) | |
| + | - `--days-back N` - days of past House committee meetings to scan (default 30) | |
| + | - `--days-ahead N` - days of future House committee meetings to scan (default 90) | |
| + | - `--no-detail` - skip per-event detail fetches (faster; meeting type omitted for new events) | |
| ## What it does | ||
| @@ -32,7 +32,7 @@ Ingests congressional committee + floor activity from three keyless sources. | ||
| | docs.house.gov | House | other | 11 | | ||
| | **TOTAL** | | | **166** | | ||
| - | ### Source 1 - Senate committee meetings XML | |
| + | ### Source 1 - Senate committee meetings XML | |
| **URL:** `https://www.senate.gov/general/committee_schedules/hearings.xml` | ||
| **Status:** Working. Returns clean structured XML with all upcoming meetings. | ||
| @@ -44,7 +44,7 @@ The XML provides a `last_update_iso_8601` field used for change-detection in the | ||
| **Note:** On days with no Senate committee meetings, the XML returns a single placeholder entry with no `<identifier>` element. This is correctly skipped. | ||
| - | ### Source 2 - House committee meetings (HTML) | |
| + | ### Source 2 - House committee meetings (HTML) | |
| **Calendar day index:** `https://docs.house.gov/Committee/Calendar/ByDay.aspx?DayID=MMDDYYYY` | ||
| **Event detail:** `https://docs.house.gov/Committee/Calendar/ByEvent.aspx?EventID=N` | ||
| @@ -54,13 +54,13 @@ Two-phase approach: | ||
| 1. **Phase 1 (parallel):** Fetch all `ByDay.aspx` pages for the date window using `ThreadPoolExecutor` (8 workers). Extracts `EventID`, title, committee, time, location, status (Rescheduled/Canceled/Postponed). | ||
| 2. **Phase 2 (parallel):** For new `EventID`s only, fetch the event detail page (4 workers) to extract meeting type (`Hearing:` / `Markup:` / `Business Meeting:` prefix in `<h1>`). | ||
| - | Change-detection key: `status|date` - re-processes a meeting if its status or scheduled date changes. | |
| + | Change-detection key: `status|date` - re-processes a meeting if its status or scheduled date changes. | |
| **Source URL per row:** `https://docs.house.gov/Committee/Calendar/ByEvent.aspx?EventID={N}` (docs.house.gov is .gov ✓) | ||
| - | **Noted:** Individual meeting XML download on ByEvent.aspx requires JavaScript postback (`__doPostBack`) - inaccessible without a browser. The HTML scrape provides all fields needed. | |
| + | **Noted:** Individual meeting XML download on ByEvent.aspx requires JavaScript postback (`__doPostBack`) - inaccessible without a browser. The HTML scrape provides all fields needed. | |
| - | ### Source 3 - House floor schedule XML | |
| + | ### Source 3 - House floor schedule XML | |
| **URL pattern:** `https://docs.house.gov/billsthisweek/YYYYMMDD/YYYYMMDD.xml` (Monday date) | ||
| **Status:** Working. Scans current week + 4 prior weeks (weeks with no XML return 404 and are skipped). | ||
| @@ -68,8 +68,8 @@ Change-detection key: `status|date` - re-processes a meeting if its status or | ||
| **Type:** always `floor` | ||
| **Source URL per row:** `https://docs.house.gov/floor/Default.aspx?date=YYYY-MM-DD` | ||
| - | **Not a source:** Congress.gov committee schedule - key-gated. The `docs.house.gov` committee repository is the keyless equivalent. | |
| - | **Not a source:** Federal Register Sunshine Act notices - FR poller (`executive_actions`) owns those; committee meetings here focus on congressional activity only. | |
| + | **Not a source:** Congress.gov committee schedule - key-gated. The `docs.house.gov` committee repository is the keyless equivalent. | |
| + | **Not a source:** Federal Register Sunshine Act notices - FR poller (`executive_actions`) owns those; committee meetings here focus on congressional activity only. | |
| ## Schema | ||
| @@ -99,7 +99,7 @@ CREATE INDEX idx_meetings_committee ON meetings(committee); | ||
| ## 3 Real example meetings (from production run 2026-06-26) | ||
| - | **Example 1 - Senate committee (upcoming)** | |
| + | **Example 1 - Senate committee (upcoming)** | |
| ``` | ||
| Chamber: Senate | ||
| Committee: Personnel Subcommittee (Armed Services) | ||
| @@ -110,7 +110,7 @@ Location: SR-222 | ||
| Source: https://www.senate.gov/committees/hearings_meetings.htm | ||
| ``` | ||
| - | **Example 2 - House committee hearing (upcoming)** | |
| + | **Example 2 - House committee hearing (upcoming)** | |
| ``` | ||
| Chamber: House | ||
| Committee: Europe Subcommittee (Committee on Foreign Affairs) | ||
| @@ -121,7 +121,7 @@ Location: 2172 RHOB | ||
| Source: https://docs.house.gov/Committee/Calendar/ByEvent.aspx?EventID=119256 | ||
| ``` | ||
| - | **Example 3 - House committee hearing (past)** | |
| + | **Example 3 - House committee hearing (past)** | |
| ``` | ||
| Chamber: House | ||
| Committee: Select Committee on the Strategic Competition Between the United States and the Chinese Communist Party | ||
| @@ -137,7 +137,7 @@ Source: https://docs.house.gov/Committee/Calendar/ByEvent.aspx?EventID=... | ||
| | Source | Status | Reason | | ||
| |---|---|---| | ||
| | Congress.gov committee schedule API | Key-gated | Requires `DOCKET_CONGRESS_KEY` | | ||
| - | | docs.house.gov Meeting XML download | JS-only | `__doPostBack` - no direct URL | | |
| + | | docs.house.gov Meeting XML download | JS-only | `__doPostBack` - no direct URL | | |
| | Individual Senate committee pages | No structured feed | Per-committee HTML only | | ||
| | CHRG (hearing transcripts) | Key-gated (GovInfo API) | Available in bulk with key | | ||
| @@ -156,7 +156,7 @@ ORDER BY datetime_iso NULLS LAST, date, committee | ||
| LIMIT :limit OFFSET :offset | ||
| ``` | ||
| - | This supports the target UX: "Mon 2026-08-25, 4:55 PM - 4 committee meetings on X, Y, Z." | |
| + | This supports the target UX: "Mon 2026-08-25, 4:55 PM - 4 committee meetings on X, Y, Z." | |
| The `date` index makes date-range queries fast. The `committee` index supports committee-specific filtering. | ||
| ## Neutrality |
| @@ -1,4 +1,4 @@ | ||
| - | # State-Level Ingest - Notes | |
| + | # State-Level Ingest - Notes | |
| ## What Was Built | ||
| @@ -10,10 +10,10 @@ jurisdictions (50 states + DC + PR). | ||
| | File | Change | | ||
| |------|--------| | ||
| | `ingest/source_validator.py` | Added `STATE_LEG_HOSTS` + `_is_state_leg_pattern()` + updated `is_allowlisted()` | | ||
| - | | `ingest/pollers/openstates.py` | Full rewrite - was a stub; now fetches bills + votes + sponsors for all 52 jurisdictions | | |
| - | | `ingest/pollers/legiscan.py` | **New** - LegiScan API poller (completeness supplement) | | |
| - | | `ingest/state_coverage.py` | **New** - honesty layer; computes + writes state_coverage rows | | |
| - | | `tests/test_state_coverage.py` | **New** - 55 tests for coverage logic + state-host allowlist | | |
| + | | `ingest/pollers/openstates.py` | Full rewrite - was a stub; now fetches bills + votes + sponsors for all 52 jurisdictions | | |
| + | | `ingest/pollers/legiscan.py` | **New** - LegiScan API poller (completeness supplement) | | |
| + | | `ingest/state_coverage.py` | **New** - honesty layer; computes + writes state_coverage rows | | |
| + | | `tests/test_state_coverage.py` | **New** - 55 tests for coverage logic + state-host allowlist | | |
| --- | ||
| @@ -45,7 +45,7 @@ missing (never seen) : 52 | ||
| ```bash | ||
| export DOCKET_OPENSTATES_KEY=<your_key> | ||
| - | # MVP subset (12 jurisdictions - conserves 500/day free quota) | |
| + | # MVP subset (12 jurisdictions - conserves 500/day free quota) | |
| /opt/docket/services/ingest/.venv/bin/python -m ingest.run --source openstates | ||
| # All 52 jurisdictions (needs paid tier) | ||
| @@ -57,10 +57,10 @@ DOCKET_OPENSTATES_ALL=1 \ | ||
| ```bash | ||
| export DOCKET_LEGISCAN_KEY=<your_key> | ||
| - | # MVP subset (12 states - ~200 API calls) | |
| + | # MVP subset (12 states - ~200 API calls) | |
| /opt/docket/services/ingest/.venv/bin/python -m ingest.run --source legiscan | ||
| - | # All 52 jurisdictions (needs ~2k API calls - monitor monthly quota) | |
| + | # All 52 jurisdictions (needs ~2k API calls - monitor monthly quota) | |
| DOCKET_LEGISCAN_ALL=1 \ | ||
| /opt/docket/services/ingest/.venv/bin/python -m ingest.run --source legiscan | ||
| ``` | ||
| @@ -83,14 +83,14 @@ cd /shared/projects/docket/services/ingest | ||
| - **Env var:** `DOCKET_OPENSTATES_KEY` | ||
| - **Caveat:** 500 req/day is too tight for all 52 jurisdictions. MVP polls 12 | ||
| jurisdictions (the 10 most-populated states + DC + PR) to stay within quota. | ||
| - | Production requires Bronze (5k/day) or Silver (50k/day) - contact | |
| + | Production requires Bronze (5k/day) or Silver (50k/day) - contact | |
| openstates@openstates.org for pricing. | ||
| ### LegiScan | ||
| - **URL:** https://legiscan.com/legiscan | ||
| - **Free tier:** ~30,000 requests/month (~1,000/day) | ||
| - **Env var:** `DOCKET_LEGISCAN_KEY` | ||
| - | - **Efficiency:** The `getMasterListRaw` sync strategy minimises API calls - | |
| + | - **Efficiency:** The `getMasterListRaw` sync strategy minimises API calls - | |
| only bills whose `change_hash` changed are fetched via `getBill`. Cap: | ||
| `MAX_BILL_FETCH_PER_STATE=50` per state per run (configurable in code). | ||
| @@ -197,17 +197,17 @@ The `state_coverage` table directly backs the `/api/states` endpoint: | ||
| --- | ||
| - | ## Known Gaps (surface in-product - never hide) | |
| + | ## Known Gaps (surface in-product - never hide) | |
| 1. **Free quota insufficient for prod**: OpenStates 500 req/day covers ~12 states/day; | ||
| LegiScan 30k/month covers all states if running daily (tight). | ||
| 2. **Historical coverage**: pollers default to `days_back=1`. First run on a state | ||
| - | won't have historical bill history - only recent changes. | |
| + | won't have historical bill history - only recent changes. | |
| 3. **Worst-lagging states**: MA, NE, KY, AL historically 24-48h behind in both | ||
| OpenStates and LegiScan (known in DATA_SOURCES.md). | ||
| 4. **Ballot measures / interim committees**: poorly covered by both APIs. | ||
| 5. **Amendment text**: LegiScan has better amendment completeness than OpenStates, | ||
| but pre-2021 coverage is spotty in both. | ||
| - | 6. **Vote positions** (`vote_positions` table): not yet populated - yea/nay counts | |
| - | only. Individual member positions require `getVote` (LegiScan) per vote - costs | |
| + | 6. **Vote positions** (`vote_positions` table): not yet populated - yea/nay counts | |
| + | only. Individual member positions require `getVote` (LegiScan) per vote - costs | |
| extra API calls. Phase 2 work. |
| @@ -1,4 +1,4 @@ | ||
| - | # Docket - Votes & Members Build Notes | |
| + | # Docket - Votes & Members Build Notes | |
| Verified 2026-06-26 on CT247 (venv `/opt/docket/services/ingest/.venv`, DB `/opt/docket/data/docket.db`). | ||
| @@ -11,7 +11,7 @@ Verified 2026-06-26 on CT247 (venv `/opt/docket/services/ingest/.venv`, DB `/opt | ||
| | `ingest/pollers/votes_enrich.py` | **new** | Senate positions backfill from existing vote source_urls | | ||
| | `tests/test_votes_house_parser.py` | **new** | 39 tests: XML parser, position enum, schema, both pivots | | ||
| - | ## Schema additions (additive - all CREATE TABLE IF NOT EXISTS / ALTER TABLE) | |
| + | ## Schema additions (additive - all CREATE TABLE IF NOT EXISTS / ALTER TABLE) | |
| ```sql | ||
| -- members table (new) | ||
| @@ -62,7 +62,7 @@ ALTER TABLE votes ADD COLUMN bill_text_ref TEXT; | ||
| ## Running the pollers | ||
| ```bash | ||
| - | # Members roster (run first - needed for member_id foreign keys) | |
| + | # Members roster (run first - needed for member_id foreign keys) | |
| python -m ingest.pollers.members --once | ||
| # House votes (probes current year sequentially, stops at 404) | ||
| @@ -75,12 +75,12 @@ python -m ingest.run --source votes_senate | ||
| python -m ingest.pollers.votes_enrich --once | ||
| ``` | ||
| - | ## Worked example - both pivot directions | |
| + | ## Worked example - both pivot directions | |
| ### Pivot 1: "Who voted how on House Roll #62?" | ||
| ``` | ||
| - | Roll #62 (2026-02-11) - On Agreeing to the Resolution, as Amended | |
| + | Roll #62 (2026-02-11) - On Agreeing to the Resolution, as Amended | |
| H RES 1057 | Passed | Y:216 N:215 P:0 NV:1 | ||
| 432 positions recorded | ||
| Source: https://clerk.house.gov/evs/2026/roll062.xml | ||
| @@ -125,7 +125,7 @@ ORDER BY v.date DESC, v.roll_number DESC; | ||
| ## Key design decisions | ||
| - | **Members source**: `unitedstates/congress-legislators` `gh-pages` branch JSON (not `main` - YAML only). Fetch URL is `raw.githubusercontent.com`; stored `source_url` is always `bioguide.congress.gov/search/bio/{id}` (allowlisted .gov). | |
| + | **Members source**: `unitedstates/congress-legislators` `gh-pages` branch JSON (not `main` - YAML only). Fetch URL is `raw.githubusercontent.com`; stored `source_url` is always `bioguide.congress.gov/search/bio/{id}` (allowlisted .gov). | |
| **House XML date format**: The Clerk XML uses `"D-Mon-YYYY"` (e.g. `"8-Jan-2026"`), not `MM/DD/YYYY` as some docs suggest. The parser tries `%d-%b-%Y` first, then fallbacks. | ||
| @@ -133,22 +133,22 @@ ORDER BY v.date DESC, v.roll_number DESC; | ||
| **Bioguide join**: House XML `<legislator name-id="...">` carries the bioguide directly. Senate XML uses `lis_member_id`; the enrich module maps it via `members.lis_member_id`. | ||
| - | **Unmatched members**: Delegates (DC, territories, etc.) appear in House roll-call XMLs but not in `legislators-current.json`. Their positions are stored with `member_bioguide` populated and `member_id = NULL` - data preserved, nothing fabricated. | |
| + | **Unmatched members**: Delegates (DC, territories, etc.) appear in House roll-call XMLs but not in `legislators-current.json`. Their positions are stored with `member_bioguide` populated and `member_id = NULL` - data preserved, nothing fabricated. | |
| **Honesty on voice/UC votes**: `parse_house_vote_xml()` returns `None` for non-recorded vote types (`VOICE VOTE`, etc.). Those votes are not in this table. The closed-door agent (CREC) owns them. | ||
| - | **Senate backfill approach**: `votes_enrich.py` re-fetches the `source_url` already stored on existing `votes` rows - no external mapping needed. Idempotent: only processes rows with no existing positions. | |
| + | **Senate backfill approach**: `votes_enrich.py` re-fetches the `source_url` already stored on existing `votes` rows - no external mapping needed. Idempotent: only processes rows with no existing positions. | |
| ## Tests (39 total, 230 total suite) | ||
| ``` | ||
| - | tests/test_votes_house_parser.py - 39 tests | |
| + | tests/test_votes_house_parser.py - 39 tests | |
| - | TestNormalizePosition (6) - position enum mapping incl. Aye/No variants | |
| - | TestParseLegisNum (9) - bill type normalization, all 8 prefixes | |
| - | TestParseHouseVoteXml (9) - fields, counts, bioguides, voice-vote honesty | |
| - | TestParseSenateVotePositions (9) - LIS IDs, all 4 positions, malformed XML | |
| - | TestSchemaAndInsertion (6) - table creation, column presence, enum enforcement, | |
| + | TestNormalizePosition (6) - position enum mapping incl. Aye/No variants | |
| + | TestParseLegisNum (9) - bill type normalization, all 8 prefixes | |
| + | TestParseHouseVoteXml (9) - fields, counts, bioguides, voice-vote honesty | |
| + | TestParseSenateVotePositions (9) - LIS IDs, all 4 positions, malformed XML | |
| + | TestSchemaAndInsertion (6) - table creation, column presence, enum enforcement, | |
| both pivot directions via JOIN | ||
| ``` | ||
| @@ -1,14 +1,14 @@ | ||
| - | # Voteview Ingest - Implementation Notes | |
| + | # Voteview Ingest - Implementation Notes | |
| ## What this poller does | ||
| `pollers/voteview.py` ingests complete per-member voting records from the | ||
| Voteview bulk-CSV dataset (Lewis et al., voteview.com). It populates: | ||
| - | - `votes` - one row per recorded roll call, with metadata, year index, and | |
| + | - `votes` - one row per recorded roll call, with metadata, year index, and | |
| an official .gov `source_url` (neutrality rule #1) | ||
| - | - `vote_positions` - one row per member per roll call, with canonical position | |
| - | - `voteview_crosswalk` - cached ICPSR → bioguide mapping | |
| + | - `vote_positions` - one row per member per roll call, with canonical position | |
| + | - `voteview_crosswalk` - cached ICPSR → bioguide mapping | |
| Voteview is the **ingestion pipe only**. No Voteview URLs appear anywhere in | ||
| user-facing data; every `source_url` points to the official government record. | ||
| @@ -35,7 +35,7 @@ Voteview assigns each member vote a `cast_code` (0-9) per their codebook | ||
| **Paired / announced votes (codes 2-5):** The member was physically absent but | ||
| announced their intent; the chamber counts them as Yea/Nay. We map them to | ||
| - | Yea/Nay for the same reason - they reflect a declared position. | |
| + | Yea/Nay for the same reason - they reflect a declared position. | |
| **Codes 7 and 8 are both Present (abstention)**, not Nay. Some secondary | ||
| sources conflate 7/8 with Nay; this is incorrect per the Voteview codebook. | ||
| @@ -76,7 +76,7 @@ This happens for: | ||
| ## Official source URLs (neutrality rule #1) | ||
| - | Voteview is an academic aggregator - it is never the user-facing source. | |
| + | Voteview is an academic aggregator - it is never the user-facing source. | |
| | Chamber | URL pattern | | ||
| |---------|-------------| | ||
| @@ -117,7 +117,7 @@ name appears in the official tally. This is the complete universe of votes | ||
| where per-member records exist in the public record. | ||
| **What Voteview does NOT cover:** | ||
| - | - Voice votes - passage is recorded but no per-member positions exist. This is | |
| + | - Voice votes - passage is recorded but no per-member positions exist. This is | |
| ~94% of Senate measures and a large fraction of House suspension-of-the-rules | ||
| votes. The product note "no individual record exists" must be displayed for | ||
| these (handled by `votes_house.py` + `votes_senate.py` which set | ||
| @@ -172,7 +172,7 @@ Voteview covers all US congresses from 1789. Rough row counts for a full | ||
| | Full archive | 1-119 | ~70,000 | ~20,000,000 | | ||
| Full archive runtime estimate: 4-6 hours (dominated by CSV downloads and | ||
| - | batch DB inserts). The ingest is fully resumable - the `seen` table prevents | |
| + | batch DB inserts). The ingest is fully resumable - the `seen` table prevents | |
| re-processing any rollcall already ingested. | ||
| SQLite comfortably handles 20M rows in `vote_positions` with proper indexes; |
| @@ -1,4 +1,4 @@ | ||
| - | # Docket Watcher - Notes | |
| + | # Docket Watcher - Notes | |
| ## What it does | ||
| @@ -13,7 +13,7 @@ Results are stored in `change_events` and are the foundation of Docket's | ||
| ## Run modes | ||
| ```bash | ||
| - | # Single cycle - designed for systemd timer (or manual testing) | |
| + | # Single cycle - designed for systemd timer (or manual testing) | |
| python -m ingest.watcher --once | ||
| # Continuous loop with 30-minute pause | ||
| @@ -61,8 +61,8 @@ The watcher snapshots the `seen` table (which tracks `(source, external_id, | ||
| update_date)` for every item processed by `mark_seen`) before and after the | ||
| pipeline run: | ||
| - | - **ADDED** - new `(source, external_id)` appeared in the post-run snapshot. | |
| - | - **UPDATED** - same `(source, external_id)`, but `update_date` changed. | |
| + | - **ADDED** - new `(source, external_id)` appeared in the post-run snapshot. | |
| + | - **UPDATED** - same `(source, external_id)`, but `update_date` changed. | |
| This covers every poller that calls `mark_seen()` (federal_register, openomb, | ||
| usaspending, treasury, congress, govinfo, bills_bulk, bill_versions, votes_senate, | ||
| @@ -71,7 +71,7 @@ votes_house, crec, lobbying, openstates, legiscan). | ||
| The `members` poller does a blind upsert without `mark_seen`, so its ADDED events | ||
| come from the manifest path instead (see below). | ||
| - | ## REMOVED detection - full-set sources only | |
| + | ## REMOVED detection - full-set sources only | |
| **Why only full-set?** Paginated or time-windowed sources (federal_register fetches | ||
| last-7-days, usaspending returns pages of awards, etc.) cannot distinguish "item | ||
| @@ -93,13 +93,13 @@ The next cycle diffs the new set against this manifest. | ||
| Two guards prevent false REMOVED events: | ||
| - | 1. **Non-empty guard** - if `_query_current_set()` returns an empty dict for a | |
| + | 1. **Non-empty guard** - if `_query_current_set()` returns an empty dict for a | |
| full-set source, the watcher writes a WARNING log and skips removal detection | ||
| for that source. This handles: network failures, table not yet created, source | ||
| returning unexpectedly empty HTML, etc. The existing manifest is preserved | ||
| unchanged. | ||
| - | 2. **First-run guard** - if there is no prior manifest entry for a source (first | |
| + | 2. **First-run guard** - if there is no prior manifest entry for a source (first | |
| ever cycle), no items are flagged as removed. Without a baseline, there is | ||
| nothing to diff against. | ||
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Amendments poller - populate the `amendments` table. | |
| + | Amendments poller - populate the `amendments` table. | |
| Data strategy (bulk / no key): | ||
| 1. Primary: parse amendment references from BILLSTATUS XML already stored | ||
| @@ -17,10 +17,10 @@ Schema (created by bills_bulk.ensure_schema): | ||
| agreed_method, text, source_url) | ||
| agreed_method values: | ||
| - | 'recorded' - roll-call vote (individual record exists) | |
| - | 'voice' - voice vote (no individual record) | |
| - | 'uc' - unanimous consent | |
| - | NULL - unknown / not public | |
| + | 'recorded' - roll-call vote (individual record exists) | |
| + | 'voice' - voice vote (no individual record) | |
| + | 'uc' - unanimous consent | |
| + | NULL - unknown / not public | |
| """ | ||
| import argparse | ||
| @@ -173,7 +173,7 @@ def _ingest_from_billstatus(conn: sqlite3.Connection, congress: int) -> int: | ||
| ( | ||
| congress, amdt_type, amdt_num, bill_id, | ||
| status, agreed, | ||
| - | None, # text=NULL - not machine-readable at this source | |
| + | None, # text=NULL - not machine-readable at this source | |
| amdt_url, | ||
| ), | ||
| ) |
| @@ -1,15 +1,15 @@ | ||
| """ | ||
| - | Calendar aggregator - builds `calendar_events` from all ingest tables. | |
| + | Calendar aggregator - builds `calendar_events` from all ingest tables. | |
| Each event carries a date and a kind: | ||
| - | effective - Federal Register final rules with a structured effective_on date | |
| - | enacted - Bills that became public law (actions.BecameLaw rows) | |
| - | vote - Roll-call vote dates (votes table) | |
| - | introduced - Bill introduction dates (actions.IntroReferral) | |
| - | committee - Committee referral/action dates (actions.Committee) | |
| - | floor - Floor action dates (actions.Floor) | |
| - | apportionment - Apportionment dates (apportionments table) | |
| - | effective_parsed - Effective dates parsed from bill/action text via regex (confidence 0.5) | |
| + | effective - Federal Register final rules with a structured effective_on date | |
| + | enacted - Bills that became public law (actions.BecameLaw rows) | |
| + | vote - Roll-call vote dates (votes table) | |
| + | introduced - Bill introduction dates (actions.IntroReferral) | |
| + | committee - Committee referral/action dates (actions.Committee) | |
| + | floor - Floor action dates (actions.Floor) | |
| + | apportionment - Apportionment dates (apportionments table) | |
| + | effective_parsed - Effective dates parsed from bill/action text via regex (confidence 0.5) | |
| Motivating use-case: "Bill X goes into effect August 5" appears on Aug 5 in the | ||
| calendar, sourced from the Federal Register `effective_on` field. | ||
| @@ -76,7 +76,7 @@ EFFECTIVE_DATE_PATTERNS: list[re.Pattern] = [ | ||
| re.compile(r"effective\s+on\s+" + _DATE_FRAG, re.IGNORECASE), | ||
| # "effective date is <date>" / "effective date shall be <date>" | ||
| re.compile(r"effective\s+date\s+(?:is|shall\s+be)\s+" + _DATE_FRAG, re.IGNORECASE), | ||
| - | # "effective <date>" (no "on") - lowest specificity, listed last | |
| + | # "effective <date>" (no "on") - lowest specificity, listed last | |
| re.compile(r"\beffective\s+" + _DATE_FRAG, re.IGNORECASE), | ||
| ] | ||
| @@ -170,7 +170,7 @@ def ensure_schema(conn: sqlite3.Connection) -> None: | ||
| conn.commit() | ||
| # Add effective_on to executive_actions if it was created by the FR poller | ||
| - | # without that column (SQLite ALTER TABLE has no IF NOT EXISTS - use try/except). | |
| + | # without that column (SQLite ALTER TABLE has no IF NOT EXISTS - use try/except). | |
| try: | ||
| conn.execute( | ||
| "ALTER TABLE executive_actions ADD COLUMN effective_on TEXT" | ||
| @@ -182,7 +182,7 @@ def ensure_schema(conn: sqlite3.Connection) -> None: | ||
| # --------------------------------------------------------------------------- | ||
| - | # FR enrichment - populate effective_on for stored final_rule rows | |
| + | # FR enrichment - populate effective_on for stored final_rule rows | |
| # --------------------------------------------------------------------------- | ||
| def enrich_fr_effective_on(conn: sqlite3.Connection) -> int: | ||
| @@ -251,7 +251,7 @@ def enrich_fr_effective_on(conn: sqlite3.Connection) -> int: | ||
| # --------------------------------------------------------------------------- | ||
| - | # Fresh FR fetch - pull recent final_rule docs with effective_on | |
| + | # Fresh FR fetch - pull recent final_rule docs with effective_on | |
| # --------------------------------------------------------------------------- | ||
| def fetch_fr_rules_with_effective( | ||
| @@ -303,7 +303,7 @@ def fetch_fr_rules_with_effective( | ||
| for doc in results: | ||
| dn = doc.get("document_number") or "" | ||
| - | eff = doc.get("effective_on") # may be null - that's fine | |
| + | eff = doc.get("effective_on") # may be null - that's fine | |
| pub_date = doc.get("publication_date") or "" | ||
| title = doc.get("title") or "" | ||
| html_url = doc.get("html_url") or "" | ||
| @@ -387,7 +387,7 @@ def _upsert_event( | ||
| def _aggregate_effective(conn: sqlite3.Connection) -> int: | ||
| """ | ||
| - | effective - Federal Register final rules with a structured effective_on date. | |
| + | effective - Federal Register final rules with a structured effective_on date. | |
| This is the canonical "goes into effect on X" source. Only rows where | ||
| effective_on IS NOT NULL are included. | ||
| @@ -425,7 +425,7 @@ def _aggregate_effective(conn: sqlite3.Connection) -> int: | ||
| def _aggregate_enacted(conn: sqlite3.Connection) -> int: | ||
| """ | ||
| - | enacted - Bills that became public law, sourced from actions.BecameLaw rows. | |
| + | enacted - Bills that became public law, sourced from actions.BecameLaw rows. | |
| """ | ||
| rows = conn.execute( | ||
| """ | ||
| @@ -459,7 +459,7 @@ def _aggregate_enacted(conn: sqlite3.Connection) -> int: | ||
| def _aggregate_votes(conn: sqlite3.Connection) -> int: | ||
| """ | ||
| - | vote - Roll-call vote dates from the votes table. | |
| + | vote - Roll-call vote dates from the votes table. | |
| Only references columns in the base schema (init_schema). Extended columns | ||
| (question, description) may or may not exist depending on which pollers | ||
| @@ -572,7 +572,7 @@ def _aggregate_apportionments(conn: sqlite3.Connection) -> int: | ||
| "SELECT name FROM sqlite_master WHERE type='table' AND name='apportionments'" | ||
| ).fetchone() | ||
| if tbl is None: | ||
| - | logger.info("calendar: apportionments table not found - skipping") | |
| + | logger.info("calendar: apportionments table not found - skipping") | |
| return 0 | ||
| rows = conn.execute( | ||
| @@ -594,7 +594,7 @@ def _aggregate_apportionments(conn: sqlite3.Connection) -> int: | ||
| title_parts.append(r["account"]) | ||
| if r["status"]: | ||
| title_parts.append(f"({r['status']})") | ||
| - | title = " - ".join(title_parts) if title_parts else "Apportionment" | |
| + | title = " - ".join(title_parts) if title_parts else "Apportionment" | |
| _upsert_event( | ||
| conn, | ||
| date=r["date"], | ||
| @@ -620,7 +620,7 @@ def _aggregate_effective_parsed(conn: sqlite3.Connection) -> int: | ||
| Confidence: 0.5 (regex inference, not a structured field). | ||
| """ | ||
| - | # Focus on action texts that accompany enacted bills - they sometimes carry | |
| + | # Focus on action texts that accompany enacted bills - they sometimes carry | |
| # the effective date in legislative boilerplate. | ||
| rows = conn.execute( | ||
| """ | ||
| @@ -674,18 +674,18 @@ def run(db_path: Optional[str] = None) -> dict[str, int]: | ||
| init_schema(conn) | ||
| ensure_schema(conn) | ||
| - | # Step 1: ensure executive_actions has up-to-date effective_on data | |
| + | # Update executive_actions with current effective_on data | |
| fetch_fr_rules_with_effective(conn, days_back=90) | ||
| enrich_fr_effective_on(conn) | ||
| - | # Step 2: aggregate all event kinds | |
| + | # Aggregate all event kinds | |
| counts: dict[str, int] = {} | ||
| counts["effective"] = _aggregate_effective(conn) | ||
| counts["enacted"] = _aggregate_enacted(conn) | ||
| counts["vote"] = _aggregate_votes(conn) | ||
| - | # Bill action types - one call handles all three | |
| + | # Bill action types - one call handles all three | |
| bill_action_total = _aggregate_bill_actions(conn) | ||
| # Disaggregate for reporting | ||
| for kind in ("introduced", "committee", "floor"): | ||
| @@ -701,7 +701,7 @@ def run(db_path: Optional[str] = None) -> dict[str, int]: | ||
| total = sum(counts.values()) | ||
| logger.info( | ||
| - | "calendar: run complete - %d total calendar_events across %d kinds", | |
| + | "calendar: run complete - %d total calendar_events across %d kinds", | |
| total, | ||
| len(counts), | ||
| ) | ||
| @@ -722,7 +722,7 @@ def main() -> None: | ||
| ) | ||
| ap = argparse.ArgumentParser( | ||
| - | description="Docket calendar aggregator - builds calendar_events table" | |
| + | description="Docket calendar aggregator - builds calendar_events table" | |
| ) | ||
| ap.add_argument( | ||
| "--once", |
| @@ -1,22 +1,22 @@ | ||
| """ | ||
| - | ingest/categorize.py - Unified taxonomy backfill for bills and spending awards. | |
| + | ingest/categorize.py - Unified taxonomy backfill for bills and spending awards. | |
| Creates / extends: | ||
| - | categories (slug, label, kind) - ~32 standard Congressional policy areas | |
| - | bill_subjects (bill_id, subject) - granular legislative subjects from BILLSTATUS XML | |
| - | awards.category (TEXT column) - deterministic agency→policy-area mapping | |
| + | categories (slug, label, kind) - ~32 standard Congressional policy areas | |
| + | bill_subjects (bill_id, subject) - granular legislative subjects from BILLSTATUS XML | |
| + | awards.category (TEXT column) - deterministic agency→policy-area mapping | |
| run(db_path) backfills all three over whatever is already in the DB. | ||
| Subjects are fetched on demand from GovInfo BULKDATA BILLSTATUS XML (no API key | ||
| - | required) - the same source used by bills_bulk.py. Bills already processed are | |
| + | required) - the same source used by bills_bulk.py. Bills already processed are | |
| skipped on subsequent runs (idempotent). | ||
| CLI: | ||
| python -m ingest.categorize --once [--limit-subjects N] [--delay SECS] | ||
| All changes are additive: no existing rows or columns are dropped. | ||
| - | Categories come straight from Congress.gov's own taxonomy - no editorial groupings. | |
| + | Categories come straight from Congress.gov's own taxonomy - no editorial groupings. | |
| """ | ||
| from __future__ import annotations | ||
| @@ -208,11 +208,11 @@ def agency_to_category(agency: str) -> Optional[str]: | ||
| # Exact | ||
| if agency in AGENCY_CATEGORY_MAP: | ||
| return AGENCY_CATEGORY_MAP[agency] | ||
| - | # Prefix match - e.g. "Department of Defense - Navy" → "Armed Forces …" | |
| + | # Prefix match - e.g. "Department of Defense - Navy" → "Armed Forces …" | |
| for key, cat in AGENCY_CATEGORY_MAP.items(): | ||
| if agency.startswith(key): | ||
| return cat | ||
| - | # Substring / contained - e.g. "HHS Office of Inspector General" | |
| + | # Substring / contained - e.g. "HHS Office of Inspector General" | |
| agency_lower = agency.lower() | ||
| for key, cat in AGENCY_CATEGORY_MAP.items(): | ||
| if key.lower() in agency_lower: | ||
| @@ -225,7 +225,7 @@ def agency_to_category(agency: str) -> Optional[str]: | ||
| # --------------------------------------------------------------------------- | ||
| def _ensure_schema(conn: sqlite3.Connection) -> None: | ||
| - | """Create new tables and add missing columns - all additive.""" | |
| + | """Create new tables and add missing columns - all additive.""" | |
| conn.executescript(""" | ||
| CREATE TABLE IF NOT EXISTS categories ( | ||
| @@ -244,7 +244,7 @@ def _ensure_schema(conn: sqlite3.Connection) -> None: | ||
| """) | ||
| conn.commit() | ||
| - | # awards.category - guarded ALTER (column may already exist from a prior run) | |
| + | # awards.category - guarded ALTER (column may already exist from a prior run) | |
| existing_cols = {row[1] for row in conn.execute("PRAGMA table_info(awards)")} | ||
| if "category" not in existing_cols: | ||
| try: | ||
| @@ -300,7 +300,7 @@ def _seed_categories(conn: sqlite3.Connection) -> int: | ||
| # --------------------------------------------------------------------------- | ||
| - | # Bill subjects backfill - re-fetch BILLSTATUS XML from GovInfo | |
| + | # Bill subjects backfill - re-fetch BILLSTATUS XML from GovInfo | |
| # --------------------------------------------------------------------------- | ||
| _BULKDATA_BASE = "https://www.govinfo.gov/bulkdata/BILLSTATUS" | ||
| @@ -431,7 +431,7 @@ def _backfill_subjects( | ||
| else: | ||
| # Insert a placeholder subject to mark this bill as "processed" even if it | ||
| # has no subjects (prevents re-fetching on every run). | ||
| - | # Use empty string - UNIQUE(bill_id, subject) allows one empty row per bill. | |
| + | # Use empty string - UNIQUE(bill_id, subject) allows one empty row per bill. | |
| try: | ||
| conn.execute( | ||
| "INSERT OR IGNORE INTO bill_subjects (bill_id, subject) VALUES (?, '')", |
| @@ -1,14 +1,15 @@ | ||
| """ | ||
| SQLite schema and connection helpers for the Docket ingest service. | ||
| - | Phase 0/1 spike - Postgres is the production target (see ARCHITECTURE.md). | |
| + | Phase 0/1 spike - Postgres is the production target (see ARCHITECTURE.md). | |
| """ | ||
| import os | ||
| import sqlite3 | ||
| - | # Default to a local path - SQLite does not work on CIFS/NFS mounts. | |
| - | # Override with DOCKET_DB_PATH env var, or set to a local dir (e.g. /var/tmp/docket). | |
| - | _DEFAULT_LOCAL = os.path.join("/var", "tmp", "docket", "docket.db") | |
| + | # Default to the deployed local ext4 path - SQLite does not work on CIFS/NFS | |
| + | # mounts. This matches lib/db.ts (the frontend read model) so both sides point | |
| + | # at the same DB by default. Override with the DOCKET_DB_PATH env var. | |
| + | _DEFAULT_LOCAL = os.path.join("/opt", "docket", "data", "docket.db") | |
| _SHARE_RELATIVE = os.path.join(os.path.dirname(__file__), "..", "data", "docket.db") | ||
| # Use env override if set; else use the local path. | ||
| DB_PATH = os.environ.get("DOCKET_DB_PATH", _DEFAULT_LOCAL) | ||
| @@ -21,7 +22,7 @@ def get_connection(path: str = DB_PATH) -> sqlite3.Connection: | ||
| conn.row_factory = sqlite3.Row | ||
| # WAL mode is not supported on CIFS/NFS mounts; use default DELETE journal. | ||
| conn.execute("PRAGMA foreign_keys=ON") | ||
| - | conn.execute("PRAGMA busy_timeout=10000") | |
| + | conn.execute("PRAGMA busy_timeout=60000") | |
| return conn | ||
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Bill text diff engine - the "what got slipped in" feature. | |
| + | Bill text diff engine - the "what got slipped in" feature. | |
| Given two consecutive bill versions, this module: | ||
| 1. Fetches their XML from GovInfo (no key needed via /content/pkg/). | ||
| @@ -9,8 +9,8 @@ Given two consecutive bill versions, this module: | ||
| 5. Writes bill_diffs rows. | ||
| Entry point: | ||
| - | diff_bill(bill_id, conn) - diffs the consecutive version chain for a bill. | |
| - | diff_versions(bill_id, from_code, to_code, conn) - diffs two specific versions. | |
| + | diff_bill(bill_id, conn) - diffs the consecutive version chain for a bill. | |
| + | diff_versions(bill_id, from_code, to_code, conn) - diffs two specific versions. | |
| """ | ||
| import difflib |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | link_votes.py - Retroactive vote-to-bill linkage for the Docket ingest service. | |
| + | link_votes.py - Retroactive vote-to-bill linkage for the Docket ingest service. | |
| Problem | ||
| ------- | ||
| @@ -36,7 +36,7 @@ Supported reference formats | ||
| Unresolvable references | ||
| ----------------------- | ||
| - | Presidential Nomination refs (PN\d+) are Senate confirmation votes - they have | |
| + | Presidential Nomination refs (PN\d+) are Senate confirmation votes - they have | |
| no corresponding bill row, so they are correctly left with bill_id = NULL. | ||
| Votes with congress = NULL cannot be disambiguated and are also skipped. | ||
| Historical votes (congress 101-118) cannot be linked until bills for those | ||
| @@ -49,7 +49,7 @@ ingested will pick up any newly matchable votes without double-processing. | ||
| Usage | ||
| ----- | ||
| - | # Via module (preferred - picks up DOCKET_DB_PATH env): | |
| + | # Via module (preferred - picks up DOCKET_DB_PATH env): | |
| python -m ingest.link_votes | ||
| # With an explicit DB path: | ||
| @@ -98,7 +98,7 @@ _COMPACT_PATTERNS: list[tuple[re.Pattern, str]] = [ | ||
| (re.compile(r"^S(\d+)$"), "s"), | ||
| ] | ||
| - | # Presidential Nomination refs - Senate confirmation votes, no bill linkage. | |
| + | # Presidential Nomination refs - Senate confirmation votes, no bill linkage. | |
| _PN_PATTERN = re.compile(r"^PN\d+", re.IGNORECASE) | ||
| @@ -121,7 +121,7 @@ def parse_bill_ref(raw: str) -> tuple[Optional[str], Optional[str]]: | ||
| s = raw.strip().upper() | ||
| - | # Skip Presidential Nominations - they have no bill entry. | |
| + | # Skip Presidential Nominations - they have no bill entry. | |
| if _PN_PATTERN.match(s): | ||
| return None, None | ||
| @@ -162,15 +162,15 @@ def link_votes( | ||
| Returns | ||
| ------- | ||
| dict with keys: | ||
| - | before_linked - votes.bill_id NOT NULL before this run | |
| - | candidate_rows - votes with bill_text_ref and bill_id NULL | |
| - | matched - rows successfully linked | |
| - | skipped_no_congress - rows skipped because congress is NULL | |
| - | skipped_pn - rows skipped because ref is a PN nomination | |
| - | skipped_parse_fail - refs that could not be parsed (not PN) | |
| - | skipped_no_bill - parsed ok but no matching bill in the DB | |
| - | after_linked - votes.bill_id NOT NULL after this run | |
| - | examples - up to 5 dict with {vote_id, question, | |
| + | before_linked - votes.bill_id NOT NULL before this run | |
| + | candidate_rows - votes with bill_text_ref and bill_id NULL | |
| + | matched - rows successfully linked | |
| + | skipped_no_congress - rows skipped because congress is NULL | |
| + | skipped_pn - rows skipped because ref is a PN nomination | |
| + | skipped_parse_fail - refs that could not be parsed (not PN) | |
| + | skipped_no_bill - parsed ok but no matching bill in the DB | |
| + | after_linked - votes.bill_id NOT NULL after this run | |
| + | examples - up to 5 dict with {vote_id, question, | |
| bill_text_ref, external_id, bill_title} | ||
| """ | ||
| before_linked = conn.execute( |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | GovInfo BILLS bulk poller - supplementary text-version discovery. | |
| + | GovInfo BILLS bulk poller - supplementary text-version discovery. | |
| For bills already in the DB that have fewer than the expected versions, | ||
| this poller probes GovInfo's /content/pkg/ content path (no key) for |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | GovInfo BULKDATA BILLSTATUS poller - federal bills, status, sponsors, actions. | |
| + | GovInfo BULKDATA BILLSTATUS poller - federal bills, status, sponsors, actions. | |
| Enumerates bills via the GovInfo sitemap (no API key required), fetches | ||
| individual BILLSTATUS XML files, and stores results in: |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Congress.gov API poller - federal bills and actions. | |
| + | Congress.gov API poller - federal bills and actions. | |
| Endpoint: GET https://api.congress.gov/v3/bill | ||
| Auth: free api.data.gov key in env DOCKET_CONGRESS_KEY |
| @@ -1,7 +1,7 @@ | ||
| """ | ||
| - | Congressional Record (CREC) bulk-data poller - procedural event detector. | |
| + | Congressional Record (CREC) bulk-data poller - procedural event detector. | |
| - | Source: GovInfo Bulk Data - https://www.govinfo.gov/bulkdata/CREC/ | |
| + | Source: GovInfo Bulk Data - https://www.govinfo.gov/bulkdata/CREC/ | |
| Auth: none (bulk data is public) | ||
| Strategy: | ||
| @@ -9,22 +9,22 @@ Strategy: | ||
| 2. Extract all XML files from the ZIP in-memory. | ||
| 3. Parse text content from each XML section. | ||
| 4. Apply regex patterns to detect procedural events: | ||
| - | voice_vote - "the ayes have it" / "the noes have it" | |
| - | unanimous_consent - "without objection" / "so ordered" | |
| - | cloture - "cloture motion", "invoke cloture" | |
| - | suspension - "suspend the rules" | |
| - | mtr - "motion to recommit" | |
| - | discharge - "discharge petition" | |
| + | voice_vote - "the ayes have it" / "the noes have it" | |
| + | unanimous_consent - "without objection" / "so ordered" | |
| + | cloture - "cloture motion", "invoke cloture" | |
| + | suspension - "suspend the rules" | |
| + | mtr - "motion to recommit" | |
| + | discharge - "discharge petition" | |
| 5. Write detected events to procedural_events. | ||
| Honest gaps (per DATA_SOURCES.md + NEUTRALITY.md): | ||
| - Voice votes and UC agreements have NO individual member record. | ||
| The `members_jsonb` column is NULL for these rows; the UI must label them | ||
| - | "no individual record exists" - the absence is itself information. | |
| + | "no individual record exists" - the absence is itself information. | |
| - CREC text can be revised after delivery and is not verbatim; label as | ||
| "Congressional Record" not "floor transcript." | ||
| - | Writes to: procedural_events table (created if absent - does not touch db.py core) | |
| + | Writes to: procedural_events table (created if absent - does not touch db.py core) | |
| """ | ||
| import argparse | ||
| @@ -48,7 +48,7 @@ from ingest.source_validator import validate_source_url | ||
| logger = logging.getLogger(__name__) | ||
| SOURCE = "crec" | ||
| - | # GovInfo content/pkg endpoint - the correct path for the CREC ZIP. | |
| + | # GovInfo content/pkg endpoint - the correct path for the CREC ZIP. | |
| # The bulkdata URL path returns HTML, not ZIPs. | ||
| CONTENT_BASE = "https://www.govinfo.gov/content/pkg" | ||
| @@ -179,7 +179,7 @@ def _extract_text_from_xml(xml_bytes: bytes) -> str: | ||
| GovInfo CREC ZIPs contain .htm files (not standalone XML). | ||
| We strip HTML tags with regex and unescape entities; this is intentionally | ||
| - | simple - procedural-phrase detection needs only the word sequence, not | |
| + | simple - procedural-phrase detection needs only the word sequence, not | |
| structure. Falls back gracefully on parse errors. | ||
| """ | ||
| try: |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Federal Register API poller - Executive Orders, proposed/final rules, notices. | |
| + | Federal Register API poller - Executive Orders, proposed/final rules, notices. | |
| Endpoint: GET https://www.federalregister.gov/api/v1/documents.json | ||
| Auth: none (fully open) | ||
| @@ -7,12 +7,12 @@ Change-detection: (source='federal_register', externalId=document_number, update | ||
| Source URL: html_url field from the API (federalregister.gov/.gov) | ||
| Types fetched: | ||
| - | PRESDOCU - Presidential Documents (Executive Orders, proclamations, memos) | |
| - | RULE - Final rules | |
| - | PRORULE - Proposed rules (NPRM) | |
| - | NOTICE - Notices | |
| + | PRESDOCU - Presidential Documents (Executive Orders, proclamations, memos) | |
| + | RULE - Final rules | |
| + | PRORULE - Proposed rules (NPRM) | |
| + | NOTICE - Notices | |
| - | Writes to: executive_actions table (created if absent - does not touch db.py core) | |
| + | Writes to: executive_actions table (created if absent - does not touch db.py core) | |
| """ | ||
| import argparse | ||
| @@ -150,7 +150,7 @@ def poll(days_back: int = 7, db_path: Optional[str] = None) -> int: | ||
| logger.debug("federal_register: already seen %s @ %s", doc_number, pub_date) | ||
| continue | ||
| - | # Validate source URL - federalregister.gov is .gov ✓ | |
| + | # Validate source URL - federalregister.gov is .gov ✓ | |
| try: | ||
| validate_source_url(html_url) | ||
| except Exception as exc: |
| @@ -1,9 +1,9 @@ | ||
| """ | ||
| - | GovInfo API poller - full bill text, Congressional Record, public laws. | |
| + | GovInfo API poller - full bill text, Congressional Record, public laws. | |
| Endpoint: GET https://api.govinfo.gov/published/{start}/{end} | ||
| Auth: same api.data.gov key as Congress.gov (DOCKET_CONGRESS_KEY) | ||
| - | Rate: 36,000 req/hr - most generous of the four primary sources. | |
| + | Rate: 36,000 req/hr - most generous of the four primary sources. | |
| Lag: 1-2 business days behind the legislative action. | ||
| If DOCKET_CONGRESS_KEY is not set, this poller skips gracefully. |
| @@ -1,13 +1,13 @@ | ||
| """ | ||
| - | LegiScan API poller - 50-state + DC + PR legislative data. | |
| + | LegiScan API poller - 50-state + DC + PR legislative data. | |
| Base: https://api.legiscan.com/?key={KEY}&op={OPERATION} | ||
| Auth: free API key required (set DOCKET_LEGISCAN_KEY). | ||
| Free tier: ~30k requests/month (~1k/day). | ||
| - | If absent, logs clearly and returns 0 - no crash. | |
| + | If absent, logs clearly and returns 0 - no crash. | |
| Free key signup: https://legiscan.com/legiscan | ||
| - | Per DATA_SOURCES.md: LegiScan is the completeness supplement - richer bill-text | |
| + | Per DATA_SOURCES.md: LegiScan is the completeness supplement - richer bill-text | |
| and amendment coverage than OpenStates. The user-facing source_url points to | ||
| the official state legislature record (bill.state_link) when available; when the | ||
| masterlist only provides the legiscan.com bill page, that aggregator URL is used | ||
| @@ -16,9 +16,9 @@ is present in the masterlist payload). | ||
| Sync strategy (efficient for free tier): | ||
| 1. getMasterList per state → all bills in current session with title, url, | ||
| - | status, last_action - ~2 API calls per state, ~104 for all 52 jurisdictions. | |
| + | status, last_action - ~2 API calls per state, ~104 for all 52 jurisdictions. | |
| 2. Deduplicate by change_hash stored in the seen table. | ||
| - | 3. No per-bill getBill calls - saves quota for future enrichment runs. | |
| + | 3. No per-bill getBill calls - saves quota for future enrichment runs. | |
| State coverage: writes scrape_log rows via ingest.state_coverage.log_scrape(). | ||
| """ | ||
| @@ -194,7 +194,7 @@ def _store_bill_from_master(conn, state: str, item: dict) -> Optional[int]: | ||
| if state_link and is_allowlisted(state_link): | ||
| source_url = state_link | ||
| elif url: | ||
| - | source_url = url # legiscan.com aggregator URL - acceptable fallback | |
| + | source_url = url # legiscan.com aggregator URL - acceptable fallback | |
| else: | ||
| logger.debug("legiscan %s bill %s: no URL in masterlist - skipping", state, bill_id) | ||
| return None | ||
| @@ -247,13 +247,13 @@ def poll( | ||
| Fetch current-session bills from LegiScan for each state. | ||
| Strategy: getSessionList (1 call) + getMasterList (1 call) per state. | ||
| - | All bills in the current session are stored from the masterlist payload - | |
| + | All bills in the current session are stored from the masterlist payload - | |
| no per-bill getBill calls needed for basic coverage. | ||
| Total API calls: ~2 per state. For 52 jurisdictions: ~104 calls total, | ||
| well within the 30k/month free-tier budget. | ||
| - | If DOCKET_LEGISCAN_KEY is not set, logs and returns 0 - no crash. | |
| + | If DOCKET_LEGISCAN_KEY is not set, logs and returns 0 - no crash. | |
| Set DOCKET_LEGISCAN_ALL=1 to poll all 52 jurisdictions (default: MVP_STATES). | ||
| Parameters |
| @@ -9,7 +9,7 @@ Note: lda.senate.gov retires 2026-06-30 → this poller targets lda.gov/api only | ||
| Each filing links to the official lda.gov record; source_url = lda.gov (✓.gov). | ||
| - | Writes to: lobbying table (created if absent - does not touch db.py core) | |
| + | Writes to: lobbying table (created if absent - does not touch db.py core) | |
| """ | ||
| import argparse | ||
| @@ -33,7 +33,7 @@ BASE_URL = "https://lda.gov/api/v1" | ||
| LDA_KEY_ENV = "DOCKET_LDA_KEY" | ||
| - | # Amount-range bucketing - LDA reports income/expenses as dollar figures; | |
| + | # Amount-range bucketing - LDA reports income/expenses as dollar figures; | |
| # we store as a human-readable range string for the lobbying table. | ||
| _AMOUNT_RANGES = [ | ||
| ( 500_000, "≥$500k"), |
| @@ -1,7 +1,7 @@ | ||
| """ | ||
| Congressional meetings/schedule poller. | ||
| - | Writes to: meetings table (created if absent - does not touch db.py core). | |
| + | Writes to: meetings table (created if absent - does not touch db.py core). | |
| Idempotent on (source, external_id). | ||
| Sources | ||
| @@ -20,7 +20,7 @@ Sources | ||
| Weekly XML: https://docs.house.gov/billsthisweek/YYYYMMDD/YYYYMMDD.xml | ||
| Auth: none. Bills scheduled for House floor consideration by week. | ||
| - | Neutrality: factual meeting data only - title, committee, date, time, location, type. | |
| + | Neutrality: factual meeting data only - title, committee, date, time, location, type. | |
| No adjectives, no editorial weighting. Source URLs all .gov. | ||
| Run with: | ||
| @@ -52,7 +52,7 @@ SOURCE_SENATE = "senate_committee" | ||
| SOURCE_HOUSE_CMT = "house_committee" | ||
| SOURCE_HOUSE_FLR = "house_floor" | ||
| - | # ── Endpoint URLs (all .gov - allowlisted) ──────────────────────────────────── | |
| + | # ── Endpoint URLs (all .gov - allowlisted) ──────────────────────────────────── | |
| SENATE_XML_URL = "https://www.senate.gov/general/committee_schedules/hearings.xml" | ||
| SENATE_PAGE_URL = "https://www.senate.gov/committees/hearings_meetings.htm" | ||
| @@ -380,7 +380,7 @@ def _parse_house_day_page(html: str, day: date) -> List[dict]: | ||
| continue | ||
| event_id = eid_m.group(1) | ||
| - | # Title (from anchor title= attribute - already HTML-decoded by server) | |
| + | # Title (from anchor title= attribute - already HTML-decoded by server) | |
| title_m = re.search(r'<a[^>]+title="([^"]+)"', row) | ||
| title = html_module.unescape(title_m.group(1).strip()) if title_m else "" | ||
| @@ -460,7 +460,7 @@ def poll_house_committee(conn, days_back: int = 30, days_ahead: int = 90, | ||
| today + timedelta(days=i) for i in range(days_ahead + 1) | ||
| ] | ||
| - | # Phase 1 - collect all (EventID, basic_data) from day pages | |
| + | # Phase 1 - collect all (EventID, basic_data) from day pages | |
| all_partials: List[dict] = [] | ||
| def _fetch_day(d: date) -> Tuple[date, List[dict]]: | ||
| @@ -494,7 +494,7 @@ def poll_house_committee(conn, days_back: int = 30, days_ahead: int = 90, | ||
| logger.info("house_cmt: %d unique event(s) after dedup", len(by_id)) | ||
| - | # Phase 2 - for new events without stored type, fetch event-detail pages | |
| + | # Phase 2 - for new events without stored type, fetch event-detail pages | |
| needs_detail: List[str] = [] | ||
| if fetch_detail: | ||
| for eid in by_id: |
| @@ -1,21 +1,21 @@ | ||
| """ | ||
| - | Member photo URL poller - unitedstates/images project. | |
| + | Member photo URL poller - unitedstates/images project. | |
| For every member in the `members` table with a known bioguide_id, performs | ||
| a HEAD request to confirm the official headshot exists on GitHub Pages, then | ||
| writes photo_url if the server returns HTTP 200. photo_url is left NULL for | ||
| - | members with no confirmed image - the frontend renders an initials placeholder. | |
| + | members with no confirmed image - the frontend renders an initials placeholder. | |
| Image URL pattern (public domain / CC0): | ||
| https://unitedstates.github.io/images/congress/450x550/{bioguide}.jpg | ||
| Available sizes: | ||
| - | 225x275 - thumbnail | |
| - | 450x550 - standard (this poller uses this size) | |
| - | original - full resolution | |
| + | 225x275 - thumbnail | |
| + | 450x550 - standard (this poller uses this size) | |
| + | original - full resolution | |
| Source attribution: | ||
| - | github.com/unitedstates/images - CC0 / public domain. | |
| + | github.com/unitedstates/images - CC0 / public domain. | |
| These are official congressional headshots released without restriction. | ||
| Images are NOT downloaded or hosted locally; photo_url stores only the | ||
| external URL. Local caching/mirroring is a future hardening step. | ||
| @@ -70,7 +70,7 @@ def head_check(url: str, session: requests.Session, timeout: int = 10) -> bool: | ||
| Return True if a HEAD request to `url` returns HTTP 200. | ||
| Follows redirects (allow_redirects=True). Any network error or non-200 | ||
| - | status returns False - the caller treats this as "no image available." | |
| + | status returns False - the caller treats this as "no image available." | |
| """ | ||
| try: | ||
| resp = session.head(url, timeout=timeout, allow_redirects=True) | ||
| @@ -92,7 +92,7 @@ def poll( | ||
| Idempotent: re-running safely overwrites photo_url with the same verified | ||
| URL. Members whose HEAD check returns non-200 are left at NULL (first run) | ||
| - | or retain their existing value (re-run) - we never clear a previously | |
| + | or retain their existing value (re-run) - we never clear a previously | |
| confirmed URL unless `reset=True`. | ||
| Parameters | ||
| @@ -104,9 +104,9 @@ def poll( | ||
| Returns | ||
| ------- | ||
| dict with keys: checked, updated, skipped | ||
| - | checked - total members evaluated | |
| - | updated - photo_url set (HTTP 200 confirmed) | |
| - | skipped - no image found (HTTP 404 / other non-200) | |
| + | checked - total members evaluated | |
| + | updated - photo_url set (HTTP 200 confirmed) | |
| + | skipped - no image found (HTTP 404 / other non-200) | |
| """ | ||
| conn = get_connection(db_path) if db_path else get_connection() | ||
| init_schema(conn) |
| @@ -1,17 +1,17 @@ | ||
| """ | ||
| - | Member photo URL poller - supplemental sources. | |
| + | Member photo URL poller - supplemental sources. | |
| Fills photo_url for members where the primary unitedstates/images check | ||
| (member_photos.py) found no image. Tries two additional sources, in order: | ||
| - | 1. bioguide.congress.gov - the official Congressional Biographical Directory. | |
| + | 1. bioguide.congress.gov - the official Congressional Biographical Directory. | |
| Image URL: https://bioguide.congress.gov/photo/{bioguide_id}.jpg | ||
| This is a .gov domain (official government source, public domain). | ||
| IMPORTANT: bioguide.congress.gov returns HTTP 403 on HEAD requests; this | ||
| poller uses GET with stream=True to check existence without downloading the | ||
| image body. | ||
| - | 2. Wikimedia Commons via Wikipedia API - public-domain / CC-licensed portraits. | |
| + | 2. Wikimedia Commons via Wikipedia API - public-domain / CC-licensed portraits. | |
| Uses the Wikipedia pageimages API to find the page image for the member. | ||
| Multiple title variants are tried in order; a safety guard requires the | ||
| returned page title to contain the member's last name. | ||
| @@ -129,7 +129,7 @@ def check_bioguide(bioguide_id: str, session: requests.Session, timeout: int = 1 | ||
| url = bioguide_photo_url(bioguide_id) | ||
| try: | ||
| resp = session.get(url, timeout=timeout, allow_redirects=True, stream=True) | ||
| - | resp.close() # immediately release connection - we only wanted status | |
| + | resp.close() # immediately release connection - we only wanted status | |
| if resp.status_code == 200: | ||
| return url | ||
| except requests.RequestException: | ||
| @@ -163,7 +163,7 @@ def check_wikimedia( | ||
| return result | ||
| # --- Step 2: opensearch fallback --- | ||
| - | # E.g. "Armstrong Congress senator" - the last name + "Congress" narrows | |
| + | # E.g. "Armstrong Congress senator" - the last name + "Congress" narrows | |
| # the search to the political person without needing the exact title. | ||
| parts = full_name.split() | ||
| last = _extract_last_name(full_name) |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Member roster poller - unitedstates/congress-legislators. | |
| + | Member roster poller - unitedstates/congress-legislators. | |
| Fetches `legislators-current.json` (and optionally `legislators-historical.json`) | ||
| from the keyless GitHub dataset and populates the `members` table with full | ||
| @@ -8,7 +8,7 @@ bioguide.congress.gov (a .gov domain, allowlisted by rule #1). | ||
| Also owns the shared schema additions used by votes_house.py and | ||
| votes_enrich.py: | ||
| - | • members table (including photo_url column - see member_photos.py) | |
| + | • members table (including photo_url column - see member_photos.py) | |
| • vote_positions table (+ indexes) | ||
| • Extended columns on the votes table (congress, session, roll_number, …) | ||
| @@ -17,8 +17,8 @@ Run once (CLI): | ||
| python -m ingest.pollers.members --once --historical # also ingests historical roster | ||
| Imported by: | ||
| - | votes_house.poll() - calls ensure_votes_schema() before inserting | |
| - | votes_enrich.poll() - same | |
| + | votes_house.poll() - calls ensure_votes_schema() before inserting | |
| + | votes_enrich.poll() - same | |
| """ | ||
| import argparse | ||
| @@ -44,7 +44,7 @@ LEGISLATORS_HISTORICAL_JSON_URL = ( | ||
| "/gh-pages/legislators-historical.json" | ||
| ) | ||
| - | # Official bioguide page - ends in .gov, always allowlisted. | |
| + | # Official bioguide page - ends in .gov, always allowlisted. | |
| BIOGUIDE_BASE = "https://bioguide.congress.gov/search/bio" | ||
| @@ -69,7 +69,8 @@ def ensure_votes_schema(conn) -> None: | ||
| govtrack_id INTEGER, | ||
| icpsr_id INTEGER, | ||
| source_url TEXT NOT NULL, | ||
| - | photo_url TEXT | |
| + | photo_url TEXT, | |
| + | is_current INTEGER NOT NULL DEFAULT 0 | |
| ); | ||
| CREATE INDEX IF NOT EXISTS idx_members_lis | ||
| @@ -109,13 +110,22 @@ def ensure_votes_schema(conn) -> None: | ||
| try: | ||
| conn.execute(f"ALTER TABLE votes ADD COLUMN {col} {col_type}") | ||
| except Exception: | ||
| - | pass # column already present - ignore | |
| + | pass # column already present - ignore | |
| - | # Additive column on members - photo_url set by member_photos.py. | |
| + | # Additive column on members - photo_url set by member_photos.py. | |
| try: | ||
| conn.execute("ALTER TABLE members ADD COLUMN photo_url TEXT") | ||
| except Exception: | ||
| - | pass # column already present - ignore | |
| + | pass # column already present - ignore | |
| + | ||
| + | # Additive column on members - is_current marks the sitting (119th) | |
| + | # Congress. Set to 1 by poll() for every bioguide in legislators-current.json. | |
| + | try: | |
| + | conn.execute( | |
| + | "ALTER TABLE members ADD COLUMN is_current INTEGER NOT NULL DEFAULT 0" | |
| + | ) | |
| + | except Exception: | |
| + | pass # column already present - ignore | |
| conn.commit() | ||
| @@ -174,25 +184,35 @@ def _parse_legislators(data: list) -> list[dict]: | ||
| # ─── Shared upsert helper ───────────────────────────────────────────────────── | ||
| - | def _upsert_members(conn, members: list[dict]) -> int: | |
| + | def _upsert_members(conn, members: list[dict], mark_current: bool = False) -> int: | |
| """ | ||
| Upsert a list of parsed member dicts into the members table. | ||
| photo_url is intentionally excluded from the SET clause so that | ||
| member_photos.py can own that column independently without conflicts. | ||
| + | mark_current | |
| + | When True (the current-roster poll), is_current is set to 1 for every | |
| + | upserted member - on both insert and conflict-update - so the | |
| + | sitting 119th Congress is authoritatively flagged. When False (the | |
| + | historical poll), is_current is left untouched on conflict and inserts | |
| + | default to 0, so a member who is also current is never flipped off. | |
| + | ||
| Returns the number of rows written. | ||
| """ | ||
| + | is_current = 1 if mark_current else 0 | |
| + | conflict_set_current = "is_current = 1," if mark_current else "" | |
| upserted = 0 | ||
| for m in members: | ||
| - | validate_source_url(m["source_url"]) # gate - must not raise for bioguide.congress.gov | |
| + | validate_source_url(m["source_url"]) # gate - must not raise for bioguide.congress.gov | |
| conn.execute( | ||
| - | """ | |
| + | f""" | |
| INSERT INTO members | ||
| (bioguide_id, full_name, party, state, district, chamber, | ||
| - | lis_member_id, govtrack_id, icpsr_id, source_url) | |
| - | VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) | |
| + | lis_member_id, govtrack_id, icpsr_id, source_url, is_current) | |
| + | VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) | |
| ON CONFLICT(bioguide_id) DO UPDATE SET | ||
| + | {conflict_set_current} | |
| full_name = excluded.full_name, | ||
| party = excluded.party, | ||
| state = excluded.state, | ||
| @@ -214,6 +234,7 @@ def _upsert_members(conn, members: list[dict]) -> int: | ||
| m["govtrack_id"], | ||
| m["icpsr_id"], | ||
| m["source_url"], | ||
| + | is_current, | |
| ), | ||
| ) | ||
| upserted += 1 | ||
| @@ -258,7 +279,11 @@ def poll(db_path: Optional[str] = None) -> int: | ||
| members = _parse_legislators(data) | ||
| logger.info("members: parsed %d legislator(s) from current roster", len(members)) | ||
| - | upserted = _upsert_members(conn, members) | |
| + | # Authoritative reset: clear is_current for everyone, then mark the roster | |
| + | # rows below. A member who has left Congress is dropped from the current | |
| + | # file and so correctly falls back to is_current = 0. | |
| + | conn.execute("UPDATE members SET is_current = 0") | |
| + | upserted = _upsert_members(conn, members, mark_current=True) | |
| conn.commit() | ||
| conn.close() | ||
| logger.info("members: upserted %d member(s)", upserted) | ||
| @@ -272,7 +297,7 @@ def poll_historical(db_path: Optional[str] = None) -> int: | ||
| The bioguide_id key is shared with the current roster, so current members | ||
| already present are simply refreshed (not duplicated). photo_url is left | ||
| - | untouched - member_photos.py owns that column. | |
| + | untouched - member_photos.py owns that column. | |
| Returns | ||
| ------- | ||
| @@ -308,7 +333,9 @@ def poll_historical(db_path: Optional[str] = None) -> int: | ||
| members = _parse_legislators(data) | ||
| logger.info("members: parsed %d historical legislator(s)", len(members)) | ||
| - | upserted = _upsert_members(conn, members) | |
| + | # Historical roster: do NOT mark is_current. On conflict the column is left | |
| + | # untouched so a member who is also in the current roster keeps is_current=1. | |
| + | upserted = _upsert_members(conn, members, mark_current=False) | |
| conn.commit() | ||
| conn.close() | ||
| logger.info("members: upserted %d historical member(s)", upserted) |
| @@ -1,7 +1,7 @@ | ||
| """ | ||
| - | OMB Apportionment poller - the impound tracker (flagship "buried data" feature). | |
| + | OMB Apportionment poller - the impound tracker (flagship "buried data" feature). | |
| - | Data source: apportionment-public.max.gov (official OMB portal - .gov ✓) | |
| + | Data source: apportionment-public.max.gov (official OMB portal - .gov ✓) | |
| max.gov serves individual JSON files per apportionment record (one per TAFS | ||
| account + iteration). There is no bulk REST API; this poller: | ||
| @@ -23,7 +23,7 @@ Honest gap: this reflects the most recent approved apportionment, not any | ||
| informal hold or pending review. "Withheld" here means "not yet apportioned | ||
| by OMB" which includes genuine impoundments and normal timing gaps. | ||
| - | Writes to: apportionments table (created if absent - does not touch db.py core) | |
| + | Writes to: apportionments table (created if absent - does not touch db.py core) | |
| """ | ||
| import argparse | ||
| @@ -219,7 +219,7 @@ def poll(fy: Optional[int] = None, db_path: Optional[str] = None, | ||
| init_schema(conn) | ||
| _ensure_tables(conn) | ||
| - | # Step 1: get the file index | |
| + | # Fetch the file index | |
| html = _fetch_index_html() | ||
| if not html: | ||
| conn.close() | ||
| @@ -237,7 +237,7 @@ def poll(fy: Optional[int] = None, db_path: Optional[str] = None, | ||
| conn.close() | ||
| return 0 | ||
| - | # Step 2: download and store records (cap at max_records) | |
| + | # Download and store records (cap at max_records) | |
| stored = 0 | ||
| processed = 0 | ||
| @@ -1,9 +1,9 @@ | ||
| """ | ||
| - | OpenStates API v3 poller - 50 states + DC + PR legislative data. | |
| + | OpenStates API v3 poller - 50 states + DC + PR legislative data. | |
| Base: https://v3.openstates.org/ | ||
| Auth: API key required (set DOCKET_OPENSTATES_KEY). | ||
| - | If absent, logs clearly and returns 0 - no crash. | |
| + | If absent, logs clearly and returns 0 - no crash. | |
| Tiers: Free 500/day (too tight for prod) → Bronze/Silver paid tiers. | ||
| See https://openstates.org/data/ for pricing. | ||
| @@ -55,7 +55,7 @@ MVP_JURISDICTIONS = [ | ||
| "dc", "pr", | ||
| ] | ||
| - | # Requests per second - free tier allows 10 rps; stay well under it. | |
| + | # Requests per second - free tier allows 10 rps; stay well under it. | |
| _RATE_SLEEP = 0.15 # seconds between requests | ||
| @@ -103,7 +103,7 @@ def _init_bills_jurisdiction(conn) -> None: | ||
| conn.execute("ALTER TABLE bills ADD COLUMN jurisdiction TEXT") | ||
| conn.commit() | ||
| except Exception: | ||
| - | pass # Column already exists - safe to ignore | |
| + | pass # Column already exists - safe to ignore | |
| def _store_bill(conn, bill: dict, state_code: str, api_key: str) -> Optional[int]: |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Treasury Fiscal Data API poller - debt, deficit, daily cash flows. | |
| + | Treasury Fiscal Data API poller - debt, deficit, daily cash flows. | |
| Endpoints used: | ||
| Debt to the Penny : /v1/accounting/od/debt_to_penny |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | USAspending.gov poller - federal awards (contracts, grants, loans, direct). | |
| + | USAspending.gov poller - federal awards (contracts, grants, loans, direct). | |
| Endpoint: POST https://api.usaspending.gov/api/v2/search/spending_by_award/ | ||
| Auth: none | ||
| @@ -7,15 +7,24 @@ Change-detection: (source, external_id, action_date) | ||
| Source URL: https://www.usaspending.gov/award/{generated_internal_id} | ||
| API constraint: award_type_codes must belong to exactly one group per request. | ||
| - | Groups: contracts [A,B,C,D], grants [02-05], loans [07,08], direct [06], | |
| - | idvs [IDV_A, IDV_B, IDV_B_A, IDV_B_B, IDV_B_C], other [10, 06]. | |
| - | We issue one request per group and merge the results. | |
| + | Groups: contracts [A,B,C,D], grants [02-05], loans [07,08], direct [06,10], | |
| + | idvs [IDV_A, IDV_B, IDV_B_A, IDV_B_B, IDV_B_C]. | |
| + | We issue one request per group (paged) and merge the results. | |
| + | ||
| + | Date note: when results are sorted by the aggregated "Award Amount", the API | |
| + | returns award-SUMMARY rows in which "Action Date" (and "Award Type" for | |
| + | contracts) come back null. The populated, meaningful date fields on a summary | |
| + | row are the period-of-performance "Start Date" and the "Base Obligation Date". | |
| + | We record the period-of-performance start as the award's action_date (falling | |
| + | back to base obligation / last modified) so every award carries a real date. | |
| """ | ||
| + | import argparse | |
| import json | ||
| import logging | ||
| import sqlite3 | ||
| - | from datetime import datetime, timedelta | |
| + | import sys | |
| + | from datetime import date, datetime | |
| from typing import Optional | ||
| import requests | ||
| @@ -30,56 +39,95 @@ SOURCE = "usaspending" | ||
| BASE_URL = "https://api.usaspending.gov/api/v2" | ||
| AWARD_DETAIL_BASE = "https://www.usaspending.gov/award" | ||
| - | # Each group must be queried in a separate API request. | |
| - | AWARD_TYPE_GROUPS = { | |
| - | "contracts": ["A", "B", "C", "D"], | |
| - | "grants": ["02", "03", "04", "05"], | |
| - | "loans": ["07", "08"], | |
| - | "direct": ["06", "10"], | |
| - | "idvs": ["IDV_A", "IDV_B", "IDV_B_A", "IDV_B_B", "IDV_B_C"], | |
| + | # Each award-type group is queried in a separate API request. Loans have a | |
| + | # distinct field mapping (no "Award Amount"/"Start Date"/"Action Date"): they | |
| + | # expose "Loan Value" and "Issued Date" instead, so they get their own config. | |
| + | _BASE_FIELDS = [ | |
| + | "Award ID", "Recipient Name", "Awarding Agency", "Description", | |
| + | "Last Modified Date", "Base Obligation Date", "generated_internal_id", | |
| + | ] | |
| + | _STD_FIELDS = _BASE_FIELDS + [ | |
| + | "Award Amount", "Award Type", "Action Date", "Start Date", "End Date", | |
| + | ] | |
| + | _LOAN_FIELDS = _BASE_FIELDS + ["Loan Value", "Subsidy Cost", "Issued Date"] | |
| + | ||
| + | _STD_DATES = ["Start Date", "Base Obligation Date", "Action Date", | |
| + | "Last Modified Date", "End Date"] | |
| + | _LOAN_DATES = ["Issued Date", "Base Obligation Date", "Last Modified Date"] | |
| + | ||
| + | # program label is derived from the request group because per-row "Award Type" | |
| + | # is null on amount-sorted contract summary rows. | |
| + | GROUP_CONFIG = { | |
| + | "contracts": {"codes": ["A", "B", "C", "D"], "label": "contract", | |
| + | "fields": _STD_FIELDS, "sort": "Award Amount", | |
| + | "amount": "Award Amount", "dates": _STD_DATES}, | |
| + | "grants": {"codes": ["02", "03", "04", "05"], "label": "grant", | |
| + | "fields": _STD_FIELDS, "sort": "Award Amount", | |
| + | "amount": "Award Amount", "dates": _STD_DATES}, | |
| + | "loans": {"codes": ["07", "08"], "label": "loan", | |
| + | "fields": _LOAN_FIELDS, "sort": "Loan Value", | |
| + | "amount": "Loan Value", "dates": _LOAN_DATES}, | |
| + | "direct": {"codes": ["06", "10"], "label": "direct", | |
| + | "fields": _STD_FIELDS, "sort": "Award Amount", | |
| + | "amount": "Award Amount", "dates": _STD_DATES}, | |
| + | "idvs": {"codes": ["IDV_A", "IDV_B", "IDV_B_A", "IDV_B_B", "IDV_B_C"], | |
| + | "label": "idv", "fields": _STD_FIELDS, "sort": "Award Amount", | |
| + | "amount": "Award Amount", "dates": _STD_DATES}, | |
| } | ||
| - | AWARD_TYPE_LABEL = { | |
| - | "A": "contract", "B": "contract", "C": "contract", "D": "contract", | |
| - | "02": "grant", "03": "grant", "04": "grant", "05": "grant", | |
| - | "07": "loan", "08": "loan", | |
| - | "06": "direct", "10": "direct", | |
| - | "IDV_A": "idv", "IDV_B": "idv", "IDV_B_A": "idv", "IDV_B_B": "idv", "IDV_B_C": "idv", | |
| - | } | |
| - | REQUESTED_FIELDS = [ | |
| - | "Award ID", | |
| - | "Recipient Name", | |
| - | "Award Amount", | |
| - | "Award Type", | |
| - | "Awarding Agency", | |
| - | "Action Date", | |
| - | "Description", | |
| - | "generated_internal_id", | |
| - | ] | |
| + | def _fiscal_year_periods(today: Optional[date] = None) -> list: | |
| + | """Return [prior_fy, current_fy] time-period ranges (Oct 1 - Sep 30). | |
| - | ||
| - | def _build_payload( | |
| - | start_date: str, | |
| - | end_date: str, | |
| - | type_codes: list, | |
| - | page: int = 1, | |
| - | ) -> dict: | |
| + | A federal fiscal year FY{N} runs Oct 1 (year N-1) through Sep 30 (year N). | |
| + | We request awards active in the current FY or the prior FY so the page shows | |
| + | a substantive, recent cross-section of federal spending. | |
| + | """ | |
| + | today = today or datetime.utcnow().date() | |
| + | fy = today.year + 1 if today.month >= 10 else today.year | |
| + | cur_start = date(fy - 1, 10, 1) | |
| + | prior_start = date(fy - 2, 10, 1) | |
| + | prior_end = date(fy - 1, 9, 30) | |
| + | return [ | |
| + | {"start_date": prior_start.isoformat(), "end_date": prior_end.isoformat()}, | |
| + | {"start_date": cur_start.isoformat(), "end_date": today.isoformat()}, | |
| + | ] | |
| + | ||
| + | ||
| + | def _build_payload(time_period: list, cfg: dict, page: int) -> dict: | |
| return { | ||
| "filters": { | ||
| - | "time_period": [{"start_date": start_date, "end_date": end_date}], | |
| - | "award_type_codes": type_codes, | |
| + | "time_period": time_period, | |
| + | "award_type_codes": cfg["codes"], | |
| }, | ||
| - | "fields": REQUESTED_FIELDS, | |
| - | "sort": "Award Amount", | |
| + | "fields": cfg["fields"], | |
| + | "sort": cfg["sort"], | |
| "order": "desc", | ||
| "limit": 100, | ||
| "page": page, | ||
| } | ||
| - | def _store_award(conn: sqlite3.Connection, award: dict, source_url: str) -> Optional[int]: | |
| - | """Insert or replace an award row; return the row id.""" | |
| + | def _award_date(award: dict, date_keys: list) -> str: | |
| + | """Best available date for an award-summary row (YYYY-MM-DD). | |
| + | ||
| + | Action Date is null on amount-sorted summary rows; fall back through the | |
| + | period-of-performance start, base obligation, then last-modified dates. | |
| + | """ | |
| + | for key in date_keys: | |
| + | val = award.get(key) | |
| + | if val: | |
| + | # Some fields carry a time component ("2026-02-10 17:52:09"). | |
| + | return str(val).split(" ")[0].split("T")[0] | |
| + | return "" | |
| + | ||
| + | ||
| + | def _store_award(conn: sqlite3.Connection, award: dict, source_url: str) -> tuple: | |
| + | """Insert or update an award row. Returns (row_id, was_new).""" | |
| + | existing = conn.execute( | |
| + | "SELECT id FROM awards WHERE source=? AND external_id=?", | |
| + | (SOURCE, award["external_id"]), | |
| + | ).fetchone() | |
| conn.execute( | ||
| """ | ||
| INSERT INTO awards (source, external_id, agency, recipient, amount, | ||
| @@ -99,139 +147,161 @@ def _store_award(conn: sqlite3.Connection, award: dict, source_url: str) -> Opti | ||
| award["external_id"], | ||
| award.get("Awarding Agency") or "", | ||
| award.get("Recipient Name") or "", | ||
| - | award.get("Award Amount") or 0.0, | |
| - | award.get("Action Date") or "", | |
| - | AWARD_TYPE_LABEL.get(award.get("Award Type") or "", "other"), | |
| + | award["_amount"], | |
| + | award["_action_date"], | |
| + | award["_program"], | |
| source_url, | ||
| ), | ||
| ) | ||
| + | if existing: | |
| + | return existing["id"], False | |
| row = conn.execute( | ||
| "SELECT id FROM awards WHERE source=? AND external_id=?", | ||
| (SOURCE, award["external_id"]), | ||
| ).fetchone() | ||
| - | return row["id"] if row else None | |
| + | return (row["id"] if row else None), True | |
| - | def _fetch_group( | |
| - | group_name: str, | |
| - | type_codes: list, | |
| - | start_str: str, | |
| - | end_str: str, | |
| - | ) -> list: | |
| - | """Fetch up to 100 awards for one award-type group.""" | |
| + | def _fetch_page(group_name: str, cfg: dict, time_period: list, | |
| + | page: int, session: requests.Session) -> tuple: | |
| + | """Fetch one page for a group. Returns (results, has_next).""" | |
| url = f"{BASE_URL}/search/spending_by_award/" | ||
| - | payload = _build_payload(start_str, end_str, type_codes, page=1) | |
| + | payload = _build_payload(time_period, cfg, page) | |
| try: | ||
| - | resp = requests.post(url, json=payload, timeout=60) | |
| + | resp = session.post(url, json=payload, timeout=90) | |
| resp.raise_for_status() | ||
| except requests.RequestException as exc: | ||
| - | logger.error("USAspending %s request failed: %s", group_name, exc) | |
| - | return [] | |
| + | logger.error("USAspending %s page %d failed: %s", group_name, page, exc) | |
| + | return [], False | |
| data = resp.json() | ||
| results = data.get("results", []) | ||
| - | logger.info("USAspending %s: %d award(s)", group_name, len(results)) | |
| - | return results | |
| + | has_next = bool(data.get("page_metadata", {}).get("hasNext")) | |
| + | return results, has_next | |
| - | def poll(days_back: int = 7, db_path: Optional[str] = None) -> int: | |
| + | def poll(pages: int = 10, db_path: Optional[str] = None) -> int: | |
| """ | ||
| - | Pull recent awards from USAspending.gov (one request per award-type group). | |
| + | Pull top awards by amount across the current + prior fiscal year. | |
| Parameters | ||
| ---------- | ||
| - | days_back : Number of days of history to fetch (default 7). | |
| - | db_path : Override the default SQLite path (useful in tests). | |
| + | pages : Max API pages (100 awards each) to pull per award-type group. | |
| + | db_path : Override the default SQLite path (useful in tests). | |
| Returns | ||
| ------- | ||
| - | Number of new award rows stored. | |
| + | Number of award rows stored or updated. | |
| """ | ||
| - | end_date = datetime.utcnow().date() | |
| - | start_date = end_date - timedelta(days=days_back) | |
| - | start_str = start_date.isoformat() | |
| - | end_str = end_date.isoformat() | |
| - | ||
| - | logger.info("USAspending: fetching awards %s → %s", start_str, end_str) | |
| + | time_period = _fiscal_year_periods() | |
| + | logger.info( | |
| + | "USAspending: fetching top awards over %s and %s (<=%d pages/group)", | |
| + | time_period[0], time_period[1], pages, | |
| + | ) | |
| conn = get_connection(db_path) if db_path else get_connection() | ||
| init_schema(conn) | ||
| - | stored = 0 | |
| + | session = requests.Session() | |
| + | session.headers["User-Agent"] = "docket-ingest/1.0 (+https://usdocket.org)" | |
| - | for group_name, type_codes in AWARD_TYPE_GROUPS.items(): | |
| - | results = _fetch_group(group_name, type_codes, start_str, end_str) | |
| + | stored = 0 | |
| - | for award in results: | |
| - | internal_id = ( | |
| - | award.get("generated_internal_id") | |
| - | or award.get("Award ID") | |
| - | or "" | |
| + | for group_name, cfg in GROUP_CONFIG.items(): | |
| + | program_label = cfg["label"] | |
| + | group_stored = 0 | |
| + | for page in range(1, pages + 1): | |
| + | # Network fetch happens OUTSIDE any open write transaction. | |
| + | results, has_next = _fetch_page( | |
| + | group_name, cfg, time_period, page, session | |
| ) | ||
| - | if not internal_id: | |
| - | logger.debug("USAspending: skipping award with no ID") | |
| - | continue | |
| - | ||
| - | award["external_id"] = internal_id | |
| - | action_date = award.get("Action Date") or "" | |
| - | ||
| - | if not is_new_event(conn, SOURCE, internal_id, action_date): | |
| - | logger.debug( | |
| - | "USAspending: already seen %s @ %s", internal_id, action_date | |
| + | if not results: | |
| + | break | |
| + | ||
| + | page_written = 0 | |
| + | for award in results: | |
| + | internal_id = ( | |
| + | award.get("generated_internal_id") | |
| + | or award.get("Award ID") | |
| + | or "" | |
| ) | ||
| - | continue | |
| - | ||
| - | source_url = f"{AWARD_DETAIL_BASE}/{internal_id}" | |
| - | try: | |
| - | validate_source_url(source_url) | |
| - | except Exception as exc: | |
| - | logger.error( | |
| - | "USAspending: invalid source_url %s: %s", source_url, exc | |
| - | ) | |
| - | continue | |
| - | ||
| - | # Raw event | |
| - | conn.execute( | |
| - | """ | |
| - | INSERT INTO raw_events (source, external_id, payload, fetched_at) | |
| - | VALUES (?, ?, ?, ?) | |
| - | ON CONFLICT(source, external_id) DO UPDATE SET | |
| - | payload = excluded.payload, | |
| - | fetched_at = excluded.fetched_at | |
| - | """, | |
| - | ( | |
| - | SOURCE, | |
| - | internal_id, | |
| - | json.dumps(award), | |
| - | datetime.utcnow().isoformat(), | |
| - | ), | |
| - | ) | |
| - | ||
| - | # Normalized award | |
| - | award_id = _store_award(conn, award, source_url) | |
| + | if not internal_id: | |
| + | continue | |
| + | ||
| + | award["external_id"] = internal_id | |
| + | action_date = _award_date(award, cfg["dates"]) | |
| + | award["_action_date"] = action_date | |
| + | award["_amount"] = award.get(cfg["amount"]) or 0.0 | |
| + | award["_program"] = program_label | |
| + | ||
| + | if not is_new_event(conn, SOURCE, internal_id, action_date): | |
| + | continue | |
| + | ||
| + | source_url = f"{AWARD_DETAIL_BASE}/{internal_id}" | |
| + | try: | |
| + | validate_source_url(source_url) | |
| + | except Exception as exc: | |
| + | logger.error("USAspending: invalid source_url %s: %s", | |
| + | source_url, exc) | |
| + | continue | |
| - | # Feed event - only the 4 permitted ranking fields | |
| - | if award_id is not None: | |
| conn.execute( | ||
| """ | ||
| - | INSERT INTO events | |
| - | (ref_type, ref_id, action_date, dollar_amount, | |
| - | affected_estimate, action_type) | |
| - | VALUES (?, ?, ?, ?, ?, ?) | |
| + | INSERT INTO raw_events (source, external_id, payload, fetched_at) | |
| + | VALUES (?, ?, ?, ?) | |
| + | ON CONFLICT(source, external_id) DO UPDATE SET | |
| + | payload = excluded.payload, | |
| + | fetched_at = excluded.fetched_at | |
| """, | ||
| - | ( | |
| - | "award", | |
| - | award_id, | |
| - | action_date, | |
| - | award.get("Award Amount") or 0.0, | |
| - | None, | |
| - | AWARD_TYPE_LABEL.get(award.get("Award Type") or "", "other"), | |
| - | ), | |
| + | (SOURCE, internal_id, json.dumps(award), | |
| + | datetime.utcnow().isoformat()), | |
| ) | ||
| - | mark_seen(conn, SOURCE, internal_id, action_date) | |
| - | stored += 1 | |
| + | award_id, was_new = _store_award(conn, award, source_url) | |
| + | ||
| + | # Feed event - only on first sighting, to avoid duplicate rows | |
| + | # on re-runs. Carries only the permitted ranking fields. | |
| + | if award_id is not None and was_new: | |
| + | conn.execute( | |
| + | """ | |
| + | INSERT INTO events | |
| + | (ref_type, ref_id, action_date, dollar_amount, | |
| + | affected_estimate, action_type) | |
| + | VALUES (?, ?, ?, ?, ?, ?) | |
| + | """, | |
| + | ("award", award_id, action_date, | |
| + | award["_amount"], None, program_label), | |
| + | ) | |
| + | ||
| + | mark_seen(conn, SOURCE, internal_id, action_date) | |
| + | page_written += 1 | |
| + | ||
| + | # Commit per page - small batches, no long-held transaction. | |
| + | conn.commit() | |
| + | stored += page_written | |
| + | group_stored += page_written | |
| + | if not has_next: | |
| + | break | |
| + | ||
| + | logger.info("USAspending %s: %d award(s) stored/updated", | |
| + | group_name, group_stored) | |
| - | conn.commit() | |
| conn.close() | ||
| - | logger.info("USAspending: stored %d new award(s) total", stored) | |
| + | logger.info("USAspending: stored/updated %d award(s) total", stored) | |
| return stored | ||
| + | ||
| + | ||
| + | if __name__ == "__main__": | |
| + | logging.basicConfig( | |
| + | level=logging.INFO, | |
| + | format="%(asctime)s [%(levelname)s] %(name)s: %(message)s", | |
| + | stream=sys.stdout, | |
| + | ) | |
| + | parser = argparse.ArgumentParser(description="USAspending awards poller") | |
| + | parser.add_argument("--once", action="store_true", help="Run once and exit") | |
| + | parser.add_argument("--pages", type=int, default=10, | |
| + | help="Max API pages (100 awards each) per award-type group") | |
| + | args = parser.parse_args() | |
| + | ||
| + | count = poll(pages=args.pages) | |
| + | print(f"Stored/updated {count} award row(s).") | |
| + | sys.exit(0) |
| @@ -1,12 +1,12 @@ | ||
| """ | ||
| - | Votes enrichment - per-member positions for Senate roll-call votes. | |
| + | Votes enrichment - per-member positions for Senate roll-call votes. | |
| Senate completeness layer. The votes_senate.py poller captures aggregate | ||
| counts (yea/nay/present) but NOT per-member positions. This module: | ||
| 1. Scans the `votes` table for Senate rows that have NO entries in | ||
| vote_positions. | ||
| - | 2. Re-fetches each vote's source_url (already stored on the row - the | |
| + | 2. Re-fetches each vote's source_url (already stored on the row - the | |
| original senate.gov XML URL). | ||
| 3. Parses the per-member <member><lis_member_id>/<vote_cast> fields. | ||
| 4. Maps lis_member_id → bioguide_id via the `members` table (populated | ||
| @@ -158,7 +158,7 @@ def enrich_senate_positions( | ||
| vote_id = row["id"] | ||
| source_url = row["source_url"] | ||
| - | # The source_url IS the senate.gov XML - already allowlisted. | |
| + | # The source_url IS the senate.gov XML - already allowlisted. | |
| try: | ||
| validate_source_url(source_url) | ||
| except Exception as exc: |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | House Clerk XML roll-call vote poller - full per-member positions. | |
| + | House Clerk XML roll-call vote poller - full per-member positions. | |
| Source: https://clerk.house.gov/evs/{year}/ROLL_{NNN}.xml | ||
| No auth required. Posted within hours of the vote. |
| @@ -1,9 +1,13 @@ | ||
| """ | ||
| Senate roll-call XML poller. | ||
| - | Source: https://www.senate.gov/legislative/LIS/roll_call_lists/roll_call_vote_cfm.cfm | |
| - | and individual vote XML: | |
| - | https://www.senate.gov/legislative/LIS/roll_call_votes/vote{congress}{session}/vote_{congress}_{session}_{vote_num:05d}.xml | |
| + | Source (data): https://www.senate.gov/legislative/LIS/roll_call_votes/vote{c}{s}/vote_{c}_{s}_{n:05d}.xml | |
| + | Source (display): https://www.senate.gov/legislative/LIS/roll_call_votes/vote{c}{s}/vote_{c}_{s}_{n:05d}.htm | |
| + | ||
| + | The .htm page is what senate.gov links to from its roll-call index; it renders | |
| + | the vote details in a human-readable HTML page. The .xml URL is the machine- | |
| + | readable data we fetch and archive. We store the .htm URL as source_url so | |
| + | users get a readable page when they click "Official source." | |
| No auth required. Posted within hours of the vote. | ||
| @@ -28,16 +32,25 @@ VOTE_XML_BASE = "https://www.senate.gov/legislative/LIS/roll_call_votes" | ||
| # 119th Congress started Jan 2025; sessions typically 1 = odd year, 2 = even year. | ||
| CURRENT_CONGRESS = 119 | ||
| - | CURRENT_SESSION = 1 # update when 2nd session begins | |
| + | CURRENT_SESSION = 2 # 2nd session began Jan 2026 | |
| def _vote_url(congress: int, session: int, vote_num: int) -> str: | ||
| + | """XML data URL - used for fetching and archiving.""" | |
| return ( | ||
| f"{VOTE_XML_BASE}/vote{congress}{session}/" | ||
| f"vote_{congress}_{session}_{vote_num:05d}.xml" | ||
| ) | ||
| + | def _source_url(congress: int, session: int, vote_num: int) -> str: | |
| + | """Human-readable .htm page URL - stored as source_url in the DB.""" | |
| + | return ( | |
| + | f"{VOTE_XML_BASE}/vote{congress}{session}/" | |
| + | f"vote_{congress}_{session}_{vote_num:05d}.htm" | |
| + | ) | |
| + | ||
| + | ||
| def _parse_senate_vote(xml_text: str, source_url: str) -> Optional[dict]: | ||
| try: | ||
| root = ET.fromstring(xml_text) | ||
| @@ -112,38 +125,41 @@ def poll( | ||
| probe_num += 1 | ||
| continue | ||
| - | url = _vote_url(congress, session, probe_num) | |
| + | fetch_url = _vote_url(congress, session, probe_num) | |
| + | disp_url = _source_url(congress, session, probe_num) | |
| try: | ||
| - | resp = requests.get(url, timeout=15) | |
| + | resp = requests.get(fetch_url, timeout=15) | |
| except requests.RequestException as exc: | ||
| - | logger.error("votes_senate: request error %s: %s", url, exc) | |
| + | logger.error("votes_senate: request error %s: %s", fetch_url, exc) | |
| break | ||
| if resp.status_code == 404: | ||
| logger.info("votes_senate: no vote %s (probe stopped)", external_id) | ||
| break | ||
| if not resp.ok: | ||
| - | logger.warning("votes_senate: HTTP %d for %s", resp.status_code, url) | |
| + | logger.warning("votes_senate: HTTP %d for %s", resp.status_code, fetch_url) | |
| break | ||
| try: | ||
| - | validate_source_url(url) | |
| + | validate_source_url(disp_url) | |
| except Exception as exc: | ||
| - | logger.error("votes_senate: invalid source_url %s: %s", url, exc) | |
| + | logger.error("votes_senate: invalid source_url %s: %s", disp_url, exc) | |
| break | ||
| - | vote = _parse_senate_vote(resp.text, url) | |
| + | vote = _parse_senate_vote(resp.text, disp_url) | |
| if vote is None: | ||
| probe_num += 1 | ||
| continue | ||
| conn.execute( | ||
| """ | ||
| - | INSERT INTO votes (chamber, date, yea, nay, present, result, source_url) | |
| - | VALUES (?, ?, ?, ?, ?, ?, ?) | |
| + | INSERT INTO votes (chamber, date, yea, nay, present, result, source_url, | |
| + | congress, session, roll_number) | |
| + | VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) | |
| """, | ||
| ("senate", vote["action_date"], vote["yea"], vote["nay"], | ||
| - | vote["present"], vote["result"], vote["source_url"]), | |
| + | vote["present"], vote["result"], vote["source_url"], | |
| + | congress, session, probe_num), | |
| ) | ||
| vote_row = conn.execute("SELECT last_insert_rowid() AS id").fetchone() |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Voteview bulk-CSV poller - complete historical per-member voting index. | |
| + | Voteview bulk-CSV poller - complete historical per-member voting index. | |
| Ingestion pipe (NOT user-facing): https://voteview.com/static/data/out/ | ||
| User-facing source_url (neutrality rule #1, official .gov only): | ||
| @@ -30,7 +30,7 @@ Cast-code mapping (Voteview codebook): | ||
| NOTE: Voteview covers only RECORDED (roll-call) votes. Voice votes and | ||
| unanimous-consent proceedings have no per-member record anywhere in the | ||
| - | public record. This is structural - not a gap in this ingestion. | |
| + | public record. This is structural - not a gap in this ingestion. | |
| ICPSR → bioguide crosswalk: | ||
| Fetched from unitedstates/congress-legislators (current + historical). | ||
| @@ -103,16 +103,26 @@ _LEGISLATORS_URLS = [ | ||
| # ─── Official source URL builders (neutrality rule #1) ─────────────────────── | ||
| def _house_source_url(year: int, clerk_rollnumber: int) -> str: | ||
| - | """Official House Clerk vote page URL.""" | |
| - | return f"https://clerk.house.gov/Votes/{year}/roll{clerk_rollnumber:03d}" | |
| + | """Official House Clerk roll-call XML URL (evs/ path). | |
| + | ||
| + | The /Votes/{year}/roll{NNN} page URL returns 404 as of mid-2026 after the | |
| + | House Clerk reorganized their site. The evs/ XML URL is the canonical | |
| + | machine-readable record and resolves cleanly. | |
| + | """ | |
| + | return f"https://clerk.house.gov/evs/{year}/roll{clerk_rollnumber:03d}.xml" | |
| def _senate_source_url(congress: int, session: int, rollnumber: int) -> str: | ||
| - | """Official Senate roll-call vote XML URL.""" | |
| + | """Official Senate roll-call vote human-readable page URL (.htm). | |
| + | ||
| + | senate.gov links its roll-call index to the .htm pages (not .xml). | |
| + | The .xml URL still resolves but delivers raw XML; the .htm page | |
| + | renders the vote details in a human-readable layout. | |
| + | """ | |
| return ( | ||
| f"https://www.senate.gov/legislative/LIS/roll_call_votes/" | ||
| f"vote{congress}{session}/" | ||
| - | f"vote_{congress}_{session}_{rollnumber:05d}.xml" | |
| + | f"vote_{congress}_{session}_{rollnumber:05d}.htm" | |
| ) | ||
| @@ -122,10 +132,10 @@ def _senate_source_url(congress: int, session: int, rollnumber: int) -> str: | ||
| CAST_CODE_MAP: dict[int, str] = { | ||
| 0: "Not Voting", # Not a member of the chamber, or absent with no record | ||
| 1: "Yea", | ||
| - | 2: "Yea", # Paired Yea - absent but announced intent | |
| - | 3: "Yea", # Announced Yea - absent but announced intent | |
| - | 4: "Nay", # Announced Nay - absent but announced intent | |
| - | 5: "Nay", # Paired Nay - absent but announced intent | |
| + | 2: "Yea", # Paired Yea - absent but announced intent | |
| + | 3: "Yea", # Announced Yea - absent but announced intent | |
| + | 4: "Nay", # Announced Nay - absent but announced intent | |
| + | 5: "Nay", # Paired Nay - absent but announced intent | |
| 6: "Nay", | ||
| 7: "Present", # Abstention variant 1 | ||
| 8: "Present", # Abstention variant 2 | ||
| @@ -183,7 +193,7 @@ def ensure_voteview_schema(conn: sqlite3.Connection) -> None: | ||
| try: | ||
| conn.execute(f"ALTER TABLE votes ADD COLUMN {col} {col_type}") | ||
| except Exception: | ||
| - | pass # already present - safe to ignore | |
| + | pass # already present - safe to ignore | |
| # Year-range indexes (the whole point of storing year explicitly). | ||
| for ddl in [ | ||
| @@ -390,7 +400,7 @@ def augment_crosswalk_from_voteview_members( | ||
| break | ||
| if len(valid_bios) != 1: | ||
| - | # Ambiguous or no match - skip to avoid false crosswalk entries. | |
| + | # Ambiguous or no match - skip to avoid false crosswalk entries. | |
| continue | ||
| bioguide = valid_bios[0] | ||
| @@ -630,7 +640,7 @@ def _ingest_congress_chamber( | ||
| ?) | ||
| """, | ||
| ( | ||
| - | None, # bill_id - look up later if needed | |
| + | None, # bill_id - look up later if needed | |
| chamber_name, | ||
| date_str, | ||
| csv_yea, | ||
| @@ -727,9 +737,9 @@ def poll( | ||
| Returns | ||
| ------- | ||
| dict with keys: | ||
| - | rollcalls_total - total vote rows inserted | |
| - | positions_total - total vote_position rows inserted | |
| - | per_congress - list of per-(congress,chamber) dicts | |
| + | rollcalls_total - total vote rows inserted | |
| + | positions_total - total vote_position rows inserted | |
| + | per_congress - list of per-(congress,chamber) dicts | |
| """ | ||
| conn = get_connection(db_path) if db_path else get_connection() | ||
| init_schema(conn) | ||
| @@ -813,7 +823,7 @@ def votes_by_member_year( | ||
| """ | ||
| Return all recorded votes for *bioguide* in *year*, with full metadata. | ||
| - | Leverages idx_votes_year + idx_vote_positions_bioguide. | |
| + | Uses idx_votes_year + idx_vote_positions_bioguide. | |
| Returns a list of dicts, each with: | ||
| date, chamber, congress, roll_number, question, result, | ||
| @@ -862,7 +872,7 @@ def votes_in_year( | ||
| """ | ||
| Return all recorded votes that took place in *year*. | ||
| - | Leverages idx_votes_year. | |
| + | Uses idx_votes_year. | |
| Returns a list of dicts, each with: | ||
| id, date, chamber, congress, roll_number, question, result, | ||
| @@ -906,7 +916,7 @@ def completeness_report( | ||
| for congress in range(start_congress, end_congress + 1): | ||
| for chamber_prefix, chamber_name in [("H", "house"), ("S", "senate")]: | ||
| - | # Count from DB via `seen` table (most reliable - one entry per | |
| + | # Count from DB via `seen` table (most reliable - one entry per | |
| # processed rollcall, written atomically with mark_seen). | ||
| db_count = conn.execute( | ||
| "SELECT COUNT(*) FROM seen WHERE source=? AND external_id LIKE ?", |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Feed ranking - neutrality rule #3. | |
| + | Feed ranking - neutrality rule #3. | |
| score(event) = f(actionDate, dollarAmount, affectedEstimate, actionType) | ||
| @@ -19,10 +19,10 @@ from typing import Optional | ||
| # How many days back before recency weight decays to near-zero. | ||
| HALF_LIFE_DAYS = 14 | ||
| - | # Dollar amount used to normalise - $1 trillion (federal scale). | |
| + | # Dollar amount used to normalise - $1 trillion (federal scale). | |
| DOLLAR_SCALE = 1_000_000_000_000.0 | ||
| - | # Maximum people-affected normaliser - ~340 M (US population). | |
| + | # Maximum people-affected normaliser - ~340 M (US population). | |
| AFFECTED_SCALE = 340_000_000.0 | ||
| # Base weights for action types (higher = more significant in the feed). | ||
| @@ -115,7 +115,7 @@ def score( | ||
| Returns | ||
| ------- | ||
| - | float in [0.0, 1.0] - higher is more prominent in the feed. | |
| + | float in [0.0, 1.0] - higher is more prominent in the feed. | |
| """ | ||
| return ( | ||
| W_RECENCY * _recency_score(actionDate) |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Rider / policy-rider detection - R Street keyword signal scan. | |
| + | Rider / policy-rider detection - R Street keyword signal scan. | |
| Runs over ADDED sections in bill_diffs. Sets rider_flag=1 on any row | ||
| whose snippet contains the R Street keyword signals. | ||
| @@ -9,7 +9,7 @@ R Street signals (per DATA_SOURCES.md): | ||
| Provided Further, | In General | Unless | Until | Determines | ||
| IMPORTANT: rider_flag is a CANDIDATE flag, not a verdict. | ||
| - | The UI must label it "may contain a policy rider" - never "IS a rider." | |
| + | The UI must label it "may contain a policy rider" - never "IS a rider." | |
| Appropriations bills are the primary target (type: hjres, s/h appropriations). | ||
| The scan also runs on other bill types but the signal weight is higher for |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Docket ingest CLI - full-spectrum orchestrator. | |
| + | Docket ingest CLI - full-spectrum orchestrator. | |
| Usage | ||
| ----- | ||
| @@ -11,7 +11,7 @@ Each step is a module with its own tested `__main__` CLI (or a poll() function). | ||
| The orchestrator runs them in dependency order, best-effort: a failing step is | ||
| logged and the pipeline continues. Counts are emitted by each module's own logs. | ||
| Keys (DOCKET_CONGRESS_KEY / DOCKET_OPENSTATES_KEY / DOCKET_LEGISCAN_KEY / | ||
| - | DOCKET_LDA_KEY) are optional - modules that need one skip gracefully when absent. | |
| + | DOCKET_LDA_KEY) are optional - modules that need one skip gracefully when absent. | |
| DB path comes from env DOCKET_DB_PATH. | ||
| """ | ||
| @@ -49,6 +49,7 @@ PIPELINE = [ | ||
| ("votes_house", "ingest.pollers.votes_house", []), | ||
| ("votes_enrich", "ingest.pollers.votes_enrich", []), | ||
| ("link_votes", "ingest.link_votes", []), | ||
| + | ("archive_votes", "ingest.archive_votes", []), | |
| ("crec", "ingest.pollers.crec", []), | ||
| ("meetings", "ingest.pollers.meetings", []), | ||
| ("lobbying", "ingest.pollers.lobbying", []), | ||
| @@ -58,6 +59,9 @@ PIPELINE = [ | ||
| # post-processing (run after raw ingest) | ||
| ("categorize", "ingest.categorize", []), | ||
| ("calendar", "ingest.calendar", []), | ||
| + | # content enrichment (backfill; idempotent; run after bills_bulk) | |
| + | ("bill_summaries", "ingest.bill_summaries", []), | |
| + | ("archive_bills", "ingest.archive_bills", []), | |
| ] | ||
| # Heavy/optional, not in the default --once cycle: historical voting backfill. | ||
| # python -m ingest.pollers.voteview --once --start 101 --end 119 |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Source URL validation - neutrality rule #1. | |
| + | Source URL validation - neutrality rule #1. | |
| Every stored fact MUST carry a source_url whose host is an allowlisted .gov | ||
| domain. This module is the single enforcement point; call validate_source_url() | ||
| @@ -9,7 +9,7 @@ State legislature portals: many states use .gov subdomains (already covered by | ||
| the *.gov wildcard), but a minority use .us or .org TLDs. STATE_LEG_HOSTS | ||
| enumerates those non-.gov hosts explicitly, and a pattern rule catches the | ||
| common *.state.<XX>.us convention. Aggregator sources (OpenStates, LegiScan) | ||
| - | must resolve to the underlying official state record - not the aggregator. | |
| + | must resolve to the underlying official state record - not the aggregator. | |
| """ | ||
| from urllib.parse import urlparse | ||
| @@ -50,45 +50,45 @@ ALLOWLISTED_HOSTS = frozenset( | ||
| # Pattern *.state.<XX>.us is also handled in _is_state_leg_pattern(). | ||
| STATE_LEG_HOSTS = frozenset( | ||
| { | ||
| - | # Pennsylvania - legis.state.pa.us | |
| + | # Pennsylvania - legis.state.pa.us | |
| "legis.state.pa.us", | ||
| "www.legis.state.pa.us", | ||
| - | # Texas - capitol.state.tx.us and legis.state.tx.us | |
| + | # Texas - capitol.state.tx.us and legis.state.tx.us | |
| "capitol.state.tx.us", | ||
| "www.capitol.state.tx.us", | ||
| "legis.state.tx.us", | ||
| "www.legis.state.tx.us", | ||
| - | # Minnesota - leg.state.mn.us | |
| + | # Minnesota - leg.state.mn.us | |
| "leg.state.mn.us", | ||
| "www.leg.state.mn.us", | ||
| "house.leg.state.mn.us", | ||
| "www.house.leg.state.mn.us", | ||
| "senate.leg.state.mn.us", | ||
| "www.senate.leg.state.mn.us", | ||
| - | # Mississippi - ls.state.ms.us | |
| + | # Mississippi - ls.state.ms.us | |
| "ls.state.ms.us", | ||
| "www.ls.state.ms.us", | ||
| "billstatus.ls.state.ms.us", | ||
| - | # Nevada - leg.state.nv.us | |
| + | # Nevada - leg.state.nv.us | |
| "leg.state.nv.us", | ||
| "www.leg.state.nv.us", | ||
| - | # New Hampshire - gencourt.state.nh.us | |
| + | # New Hampshire - gencourt.state.nh.us | |
| "gencourt.state.nh.us", | ||
| "www.gencourt.state.nh.us", | ||
| - | # New Jersey - njleg.state.nj.us | |
| + | # New Jersey - njleg.state.nj.us | |
| "njleg.state.nj.us", | ||
| "www.njleg.state.nj.us", | ||
| - | # Alabama - alison.legislature.state.al.us | |
| + | # Alabama - alison.legislature.state.al.us | |
| "alison.legislature.state.al.us", | ||
| "legislature.state.al.us", | ||
| "www.legislature.state.al.us", | ||
| - | # Alaska - legis.state.ak.us (also legis.alaska.gov which is .gov) | |
| + | # Alaska - legis.state.ak.us (also legis.alaska.gov which is .gov) | |
| "legis.state.ak.us", | ||
| "www.legis.state.ak.us", | ||
| - | # Arkansas - arkleg.state.ar.us | |
| + | # Arkansas - arkleg.state.ar.us | |
| "arkleg.state.ar.us", | ||
| "www.arkleg.state.ar.us", | ||
| - | # Kansas - kslegislature.org (unique .org) | |
| + | # Kansas - kslegislature.org (unique .org) | |
| "kslegislature.org", | ||
| "www.kslegislature.org", | ||
| # Puerto Rico legislature (not .gov) | ||
| @@ -114,7 +114,7 @@ def _is_state_leg_pattern(host: str) -> bool: | ||
| legis.state.pa.us | ||
| www.leg.state.mn.us | ||
| njleg.state.nj.us | ||
| - | billstatus.ls.state.ms.us (depth > 4 - still matches) | |
| + | billstatus.ls.state.ms.us (depth > 4 - still matches) | |
| Examples that do NOT match: | ||
| govtrack.us (only 2 parts) | ||
| @@ -142,7 +142,7 @@ def is_allowlisted(url: str) -> bool: | ||
| Allowlist rules (in priority order): | ||
| 1. Host is in ALLOWLISTED_HOSTS (explicit federal list). | ||
| - | 2. Host ends with .gov (wildcard - covers all official .gov domains). | |
| + | 2. Host ends with .gov (wildcard - covers all official .gov domains). | |
| 3. Host is in STATE_LEG_HOSTS (explicit non-.gov state legislature list). | ||
| 4. Host matches the *.state.<XX>.us pattern (implicit state legislature rule). | ||
| """ |
| @@ -5,11 +5,11 @@ Computes and writes state_coverage rows for every US jurisdiction (50 states + | ||
| DC + PR = 52 total) covering both OpenStates and LegiScan sources. | ||
| Confidence levels: | ||
| - | ok - data scraped within the last STALE_HOURS hours | |
| - | stale - data present but older than STALE_HOURS | |
| - | missing - no data has ever been ingested for this (state, source, data_type) | |
| + | ok - data scraped within the last STALE_HOURS hours | |
| + | stale - data present but older than STALE_HOURS | |
| + | missing - no data has ever been ingested for this (state, source, data_type) | |
| - | Also provides log_scrape() - called by pollers after each successful run to | |
| + | Also provides log_scrape() - called by pollers after each successful run to | |
| record scrape timestamps without requiring a second full DB pass. | ||
| Run standalone: | ||
| @@ -95,7 +95,7 @@ def log_scrape( | ||
| source : 'openstates' or 'legiscan' | ||
| state : Uppercase 2-letter state code (e.g. 'CA') | ||
| data_type : 'bills', 'votes', or 'sponsors' | ||
| - | row_count : Number of rows stored in this run (0 is valid - means scraped but empty) | |
| + | row_count : Number of rows stored in this run (0 is valid - means scraped but empty) | |
| scraped_at: ISO datetime string; defaults to now UTC. | ||
| """ | ||
| _init_coverage_schema(conn) | ||
| @@ -111,7 +111,7 @@ def log_scrape( | ||
| """, | ||
| (source, state.upper(), data_type, scraped_at, row_count), | ||
| ) | ||
| - | # Don't commit here - caller (poller) commits its own transaction. | |
| + | # Don't commit here - caller (poller) commits its own transaction. | |
| # ── Confidence computation ───────────────────────────────────────────────────── | ||
| @@ -238,7 +238,7 @@ def compute(db_path: Optional[str] = None) -> list: | ||
| """ | ||
| Compute coverage for all 52 US jurisdictions and write state_coverage rows. | ||
| - | Works entirely from DB state - no API calls. All states start as 'missing' | |
| + | Works entirely from DB state - no API calls. All states start as 'missing' | |
| until a poller has actually ingested data for them. | ||
| Returns |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Docket ingest watcher - ADDED / UPDATED / REMOVED change detection. | |
| + | Docket ingest watcher - ADDED / UPDATED / REMOVED change detection. | |
| Run modes | ||
| --------- | ||
| @@ -22,16 +22,16 @@ ADDED / UPDATED detection | ||
| Members (which does a blind upsert without mark_seen) is covered separately | ||
| via the manifest path below. | ||
| - | REMOVED detection - full-set sources only | |
| + | REMOVED detection - full-set sources only | |
| ------------------------------------------ | ||
| Only sources that return the *complete* current set on every poll can | ||
| reliably detect removals. Two safety guards prevent false positives: | ||
| - | 1. Non-empty guard - if the current pull returned 0 items for a source, | |
| + | 1. Non-empty guard - if the current pull returned 0 items for a source, | |
| no removals are flagged. An empty result likely indicates a failed | ||
| or partial pull, not a genuine mass-removal. | ||
| - | 2. First-run guard - if there is no prior manifest entry for the source, | |
| + | 2. First-run guard - if there is no prior manifest entry for the source, | |
| no removals are flagged. Without a baseline, there is nothing to diff. | ||
| Append-only / paginated / time-windowed sources (federal_register, | ||
| @@ -110,7 +110,7 @@ def _ref_type(source: str) -> str: | ||
| # --------------------------------------------------------------------------- | ||
| - | # Schema (watcher-owned tables - additive, never touches existing tables) | |
| + | # Schema (watcher-owned tables - additive, never touches existing tables) | |
| # --------------------------------------------------------------------------- | ||
| def _ensure_watcher_tables(conn) -> None: | ||
| @@ -219,7 +219,7 @@ def _lookup_source_url(conn, source: str, external_id: str) -> str: | ||
| except Exception: | ||
| pass | ||
| - | # 3. Sentinel - indicates lookup failed; keeps source_url non-null | |
| + | # 3. Sentinel - indicates lookup failed; keeps source_url non-null | |
| return f"https://docket.internal/source/{source}/{external_id}" | ||
| @@ -231,7 +231,7 @@ def _get_next_cycle_id(conn) -> int: | ||
| # --------------------------------------------------------------------------- | ||
| - | # ADDED / UPDATED - seen-table diff | |
| + | # ADDED / UPDATED - seen-table diff | |
| # --------------------------------------------------------------------------- | ||
| def _classify_added_updated( | ||
| @@ -279,7 +279,7 @@ def _classify_added_updated( | ||
| # --------------------------------------------------------------------------- | ||
| - | # REMOVED / ADDED (manifest) - full-set sources only | |
| + | # REMOVED / ADDED (manifest) - full-set sources only | |
| # --------------------------------------------------------------------------- | ||
| def _query_current_set(conn, cfg: dict) -> dict[str, str]: |
| @@ -267,7 +267,7 @@ class TestRiderDetect: | ||
| ) | ||
| def test_in_general_triggers(self): | ||
| - | assert is_rider_signal("In General. - The Secretary shall...") | |
| + | assert is_rider_signal("In General. - The Secretary shall...") | |
| def test_clean_text_no_trigger(self): | ||
| assert not is_rider_signal( |
| @@ -2,16 +2,16 @@ | ||
| Tests for ingest/calendar.py | ||
| Coverage: | ||
| - | • parse_effective_dates() - all regex patterns, mixed text, no false positives | |
| - | • _normalize_date() - ISO passthrough, month-name parsing, bad input | |
| - | • ensure_schema() - table + index creation, effective_on column | |
| - | • _aggregate_effective() - pulls FR rules with effective_on from DB | |
| - | • _aggregate_enacted() - pulls BecameLaw actions from DB | |
| - | • _aggregate_votes() - pulls from votes table | |
| - | • _aggregate_bill_actions() - introduced / committee / floor | |
| - | • _aggregate_apportionments() - pulls from apportionments table | |
| - | • _aggregate_effective_parsed() - regex over action text | |
| - | • run() idempotency - double-run does not duplicate rows | |
| + | • parse_effective_dates() - all regex patterns, mixed text, no false positives | |
| + | • _normalize_date() - ISO passthrough, month-name parsing, bad input | |
| + | • ensure_schema() - table + index creation, effective_on column | |
| + | • _aggregate_effective() - pulls FR rules with effective_on from DB | |
| + | • _aggregate_enacted() - pulls BecameLaw actions from DB | |
| + | • _aggregate_votes() - pulls from votes table | |
| + | • _aggregate_bill_actions() - introduced / committee / floor | |
| + | • _aggregate_apportionments() - pulls from apportionments table | |
| + | • _aggregate_effective_parsed() - regex over action text | |
| + | • run() idempotency - double-run does not duplicate rows | |
| """ | ||
| import os | ||
| @@ -668,7 +668,7 @@ class TestAggregateEffectiveParsed: | ||
| # --------------------------------------------------------------------------- | ||
| - | # run() - idempotency | |
| + | # run() - idempotency | |
| # --------------------------------------------------------------------------- | ||
| class TestRunIdempotency: |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Tests for ingest.categorize - unified taxonomy module. | |
| + | Tests for ingest.categorize - unified taxonomy module. | |
| Covers: | ||
| - agency_to_category: deterministic mapping, prefix match, unknown agencies |
| @@ -208,7 +208,7 @@ class TestExtractTextFromXml: | ||
| assert "deep text" in _extract_text_from_xml(xml) | ||
| def test_html_tags_stripped(self): | ||
| - | # CREC files are .htm - tags must be stripped | |
| + | # CREC files are .htm - tags must be stripped | |
| htm = b"<p><b>Mr. JONES.</b> Without objection, so ordered.</p>" | ||
| text = _extract_text_from_xml(htm) | ||
| assert "without objection" in text.lower() | ||
| @@ -224,7 +224,7 @@ class TestExtractTextFromXml: | ||
| assert _extract_text_from_xml(b"") == "" | ||
| def test_malformed_html_decoded(self): | ||
| - | # Not valid XML - should still extract text | |
| + | # Not valid XML - should still extract text | |
| raw = b"<broken>The noes have it.<br>" | ||
| text = _extract_text_from_xml(raw) | ||
| assert "noes have it" in text | ||
| @@ -310,7 +310,7 @@ class TestDetectEvents: | ||
| assert any(e["kind"] == "discharge" for e in events) | ||
| def test_members_jsonb_is_none(self): | ||
| - | """Voice votes have no individual member record - members_jsonb must be null.""" | |
| + | """Voice votes have no individual member record - members_jsonb must be null.""" | |
| events = self._events("The ayes have it. The resolution is agreed to.") | ||
| vv = [e for e in events if e["kind"] == "voice_vote"] | ||
| assert len(vv) >= 1 | ||
| @@ -392,11 +392,11 @@ class TestIdempotency: | ||
| from ingest.change_detection import is_new_event, mark_seen | ||
| - | # First pass - mark all as seen | |
| + | # First pass - mark all as seen | |
| for evt in events: | ||
| mark_seen(temp_db, "crec", evt["external_id"], evt["update_date"]) | ||
| - | # Second pass - all should be "already seen" | |
| + | # Second pass - all should be "already seen" | |
| new = [ | ||
| e for e in events | ||
| if is_new_event(temp_db, "crec", e["external_id"], e["update_date"]) |
| @@ -2,16 +2,16 @@ | ||
| Tests for ingest/pollers/meetings.py | ||
| Coverage: | ||
| - | • _parse_senate_xml() - full XML, empty identifier skipped | |
| - | • _parse_house_day_page() - normal row, subcommittee row, rescheduled row | |
| - | • _parse_house_event_type() - hearing / markup / business_meeting / unknown | |
| - | • _parse_floor_xml() - typical floor XML structure | |
| - | • _build_datetime_iso() - 12h and 24h formats, missing inputs | |
| - | • _norm_senate_type() - all known types + unknown fallback | |
| - | • _norm_house_type() - prefix matching | |
| - | • _detect_status() - rescheduled / canceled / normal | |
| - | • _ensure_tables() - idempotent schema creation | |
| - | • _upsert_meeting() idempotency - double-upsert does not duplicate | |
| + | • _parse_senate_xml() - full XML, empty identifier skipped | |
| + | • _parse_house_day_page() - normal row, subcommittee row, rescheduled row | |
| + | • _parse_house_event_type() - hearing / markup / business_meeting / unknown | |
| + | • _parse_floor_xml() - typical floor XML structure | |
| + | • _build_datetime_iso() - 12h and 24h formats, missing inputs | |
| + | • _norm_senate_type() - all known types + unknown fallback | |
| + | • _norm_house_type() - prefix matching | |
| + | • _detect_status() - rescheduled / canceled / normal | |
| + | • _ensure_tables() - idempotent schema creation | |
| + | • _upsert_meeting() idempotency - double-upsert does not duplicate | |
| • poll_senate() with mocked HTTP | ||
| """ | ||
| @@ -2,10 +2,10 @@ | ||
| Tests for member_photos poller. | ||
| Coverage: | ||
| - | • build_photo_url() - URL format and bioguide substitution | |
| - | • head_check() - 200/404/exception handling (mocked network) | |
| - | • poll() - DB writes, idempotency, stats (mocked network) | |
| - | • Schema - photo_url column exists after ensure_votes_schema() | |
| + | • build_photo_url() - URL format and bioguide substitution | |
| + | • head_check() - 200/404/exception handling (mocked network) | |
| + | • poll() - DB writes, idempotency, stats (mocked network) | |
| + | • Schema - photo_url column exists after ensure_votes_schema() | |
| """ | ||
| import os | ||
| @@ -29,7 +29,7 @@ from ingest.pollers.member_photos import ( | ||
| class TestBuildPhotoUrl: | ||
| def test_known_bioguide(self): | ||
| - | """A000055 is Robert Aderholt - canonical test bioguide.""" | |
| + | """A000055 is Robert Aderholt - canonical test bioguide.""" | |
| url = build_photo_url("A000055") | ||
| assert url == f"{PHOTO_BASE}/A000055.jpg" | ||
| @@ -123,7 +123,7 @@ class TestSchema: | ||
| os.unlink(tmp) | ||
| def test_photo_url_is_nullable(self): | ||
| - | """photo_url must be nullable - many historical members have no image.""" | |
| + | """photo_url must be nullable - many historical members have no image.""" | |
| conn, tmp = self._make_db() | ||
| conn.execute( | ||
| "INSERT INTO members (bioguide_id, full_name, source_url) " | ||
| @@ -143,7 +143,7 @@ class TestSchema: | ||
| def test_ensure_votes_schema_idempotent_with_photo_url(self): | ||
| """Calling ensure_votes_schema twice should not raise.""" | ||
| conn, tmp = self._make_db() | ||
| - | ensure_votes_schema(conn) # second call - should be a no-op | |
| + | ensure_votes_schema(conn) # second call - should be a no-op | |
| info = {row[1] for row in conn.execute("PRAGMA table_info(members)")} | ||
| assert "photo_url" in info | ||
| conn.close() |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Tests for ingest.ranking - neutrality rule #3. | |
| + | Tests for ingest.ranking - neutrality rule #3. | |
| Key assertion: the ranking function references ONLY the four permitted fields | ||
| (actionDate, dollarAmount, affectedEstimate, actionType) and no |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Tests for ingest.source_validator - neutrality rule #1. | |
| + | Tests for ingest.source_validator - neutrality rule #1. | |
| Every stored fact MUST carry a source_url whose host is an allowlisted .gov domain. | ||
| """ | ||
| @@ -130,7 +130,7 @@ class TestDatabaseSourceUrlConstraint: | ||
| bad_url = "https://www.politico.com/award/XYZ" | ||
| with pytest.raises(SourceURLError): | ||
| validate_source_url(bad_url) | ||
| - | # If we get here the guard failed - forcibly fail | |
| + | # If we get here the guard failed - forcibly fail | |
| conn.execute( | ||
| "INSERT INTO awards (source, external_id, agency, recipient, amount, action_date, program, source_url) " | ||
| "VALUES (?, ?, ?, ?, ?, ?, ?, ?)", |
| @@ -2,7 +2,7 @@ | ||
| Tests for state coverage and the STATE_LEG_HOSTS allowlist. | ||
| Covers: | ||
| - | 1. STATE_LEG_HOSTS - explicit non-.gov state legislature hosts | |
| + | 1. STATE_LEG_HOSTS - explicit non-.gov state legislature hosts | |
| 2. *.state.<XX>.us pattern matching | ||
| 3. state_coverage.compute() populates all 52 jurisdictions as "missing" | ||
| when the DB is empty (keys absent scenario) | ||
| @@ -203,11 +203,11 @@ class TestIdParsing: | ||
| assert _infer_state_from_legiscan_id("legiscan-12345") == "" | ||
| - | # ── state_coverage.compute() - 52 jurisdictions ──────────────────────────────── | |
| + | # ── state_coverage.compute() - 52 jurisdictions ──────────────────────────────── | |
| class TestStateCoverageCompute: | ||
| """ | ||
| - | Core honesty layer tests. These run without any API keys - they verify | |
| + | Core honesty layer tests. These run without any API keys - they verify | |
| that compute() works off DB state alone. | ||
| """ | ||
| @@ -2,10 +2,10 @@ | ||
| Tests for votes_house XML parser and votes_enrich Senate parser. | ||
| Coverage: | ||
| - | • parse_house_vote_xml() - position enum, bill linkage, aggregate counts | |
| - | • _normalize_position() - canonical mapping including edge cases | |
| - | • _parse_legis_num() - bill-type normalization | |
| - | • parse_senate_vote_positions() - lis_member_id extraction, position mapping | |
| + | • parse_house_vote_xml() - position enum, bill linkage, aggregate counts | |
| + | • _normalize_position() - canonical mapping including edge cases | |
| + | • _parse_legis_num() - bill-type normalization | |
| + | • parse_senate_vote_positions() - lis_member_id extraction, position mapping | |
| • Integration: schema setup + position insertion via temp DB | ||
| """ | ||
| @@ -311,7 +311,7 @@ class TestParseHouseVoteXml: | ||
| assert p["position"] in valid, f"Invalid position: {p['position']!r}" | ||
| def test_voice_vote_returns_none(self): | ||
| - | """Voice votes must be skipped - no individual record exists.""" | |
| + | """Voice votes must be skipped - no individual record exists.""" | |
| result = parse_house_vote_xml(VOICE_VOTE_XML, self.SOURCE_URL) | ||
| assert result is None, "Voice vote should not produce position data" | ||
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Tests for voteview.py - cast_code mapping, crosswalk building, schema setup, | |
| + | Tests for voteview.py - cast_code mapping, crosswalk building, schema setup, | |
| source URL construction, and year-index helpers. | ||
| All network access is mocked; no live HTTP calls are made. | ||
| @@ -245,7 +245,7 @@ class TestBuildCrosswalk: | ||
| "terms": [{"type": "sen", "state": "CA"}], | ||
| }, | ||
| { | ||
| - | # Member with no ICPSR - should be skipped without error. | |
| + | # Member with no ICPSR - should be skipped without error. | |
| "id": {"bioguide": "C000003"}, | ||
| "name": {"official_full": "Carol Carter"}, | ||
| "terms": [{"type": "rep", "state": "TX"}], | ||
| @@ -313,7 +313,7 @@ class TestBuildCrosswalk: | ||
| def test_member_without_icpsr_not_in_crosswalk(self): | ||
| cw, conn, tmp = self._make_crosswalk_from_mock() | ||
| - | # C000003 has no ICPSR - no entry for it in the crosswalk. | |
| + | # C000003 has no ICPSR - no entry for it in the crosswalk. | |
| bioguides = set(cw.values()) | ||
| assert "C000003" not in bioguides or all( | ||
| v != "C000003" for v in cw.values() |
| @@ -1,5 +1,5 @@ | ||
| """ | ||
| - | Tests for ingest.watcher - ADDED / UPDATED / REMOVED classification. | |
| + | Tests for ingest.watcher - ADDED / UPDATED / REMOVED classification. | |
| All tests are purely synthetic (no network calls, no subprocess). | ||
| They exercise: | ||
| @@ -87,7 +87,7 @@ TS = "2026-06-26T00:00:00+00:00" | ||
| # --------------------------------------------------------------------------- | ||
| - | # _classify_added_updated - seen-table diff | |
| + | # _classify_added_updated - seen-table diff | |
| # --------------------------------------------------------------------------- | ||
| class TestClassifyAddedUpdated: | ||
| @@ -215,7 +215,7 @@ class TestClassifyAddedUpdated: | ||
| # --------------------------------------------------------------------------- | ||
| - | # _detect_full_set_changes - manifest diff + removal guards | |
| + | # _detect_full_set_changes - manifest diff + removal guards | |
| # --------------------------------------------------------------------------- | ||
| class TestFullSetChanges: | ||
| @@ -415,7 +415,7 @@ class TestFullSetChanges: | ||
| assert "A000001" in manifest | ||
| assert "A000002" not in manifest | ||
| - | # Cycle 3: still only A000001 - must NOT flag A000002 again | |
| + | # Cycle 3: still only A000001 - must NOT flag A000002 again | |
| events_before = conn.execute( | ||
| "SELECT COUNT(*) FROM change_events WHERE change='removed'" | ||
| ).fetchone()[0] |
| @@ -1,7 +1,7 @@ | ||
| import type { Config } from "tailwindcss"; | ||
| /** | ||
| - | * Docket Tailwind theme - every value is mapped to a CSS custom property | |
| + | * Docket Tailwind theme - every value is mapped to a CSS custom property | |
| * defined in app/globals.css. The token is the source of truth; Tailwind | ||
| * is the consumer. (docs/BRAND.md §1) | ||
| */ | ||
| @@ -29,11 +29,11 @@ const config: Config = { | ||
| dim: "var(--dk-signal-dim)", | ||
| tint: "var(--dk-signal-tint)", | ||
| }, | ||
| - | // semantic - muted, never vivid (neutrality) | |
| + | // semantic - muted, never vivid (neutrality) | |
| yea: "var(--dk-yea)", | ||
| nay: "var(--dk-nay)", | ||
| hold: "var(--dk-hold)", | ||
| - | // party - desaturated, equal weight | |
| + | // party - desaturated, equal weight | |
| party: { | ||
| d: "var(--dk-party-d)", | ||
| r: "var(--dk-party-r)", | ||
| @@ -44,7 +44,7 @@ const config: Config = { | ||
| display: ["var(--font-display)", "system-ui", "sans-serif"], | ||
| mono: ["var(--font-mono)", "ui-monospace", "monospace"], | ||
| prose: ["var(--font-prose)", "Georgia", "serif"], | ||
| - | // Atkinson Hyperlegible - body font for Reference (light) mode. | |
| + | // Atkinson Hyperlegible - body font for Reference (light) mode. | |
| body: ["var(--font-body)", "system-ui", "sans-serif"], | ||
| }, | ||
| letterSpacing: { |