Zion Boggan
repos/Docket/README.md
zionboggan.com ↗
234 lines · markdown
History for this file →
1
<p align="center">
2
  <img src="docs/banner.svg" alt="Docket" width="100%">
3
</p>
4
 
5
**The unfiltered public record of what government does.**
6
 
7
[![License](https://img.shields.io/badge/License-Apache_2.0-E0A53B.svg)](LICENSE)
8
[![Next.js](https://img.shields.io/badge/Next.js-14-black.svg)](https://nextjs.org)
9
[![Python](https://img.shields.io/badge/Python-3.11+-3776AB.svg)](https://python.org)
10
[![Status](https://img.shields.io/badge/Status-Active_Development-E0A53B.svg)](#)
11
 
12
---
13
 
14
Docket tracks federal and state legislative activity and makes it searchable for any citizen - bills, votes, amendments, riders, spending awards, executive actions, and the closed-door procedural moves that rarely make the news.
15
 
16
Every fact links directly to an official government record. No editorializing. No partisan framing. Just the record.
17
 
18
---
19
 
20
## Neutrality Statement
21
 
22
Docket is deliberately and structurally non-partisan. It presents the official public record as-is. To enforce that:
23
 
24
- **Government sources only.** Every displayed fact links to an official `.gov` or state legislature URL. No think tanks, no news sources, no secondhand data.
25
- **No adjectives in generated text.** Numbers, names, dates, and status only. "Reduced by $X" - yes. "Slashed" or "expanded" - no. This is enforced in code via a banned-words lint that fails the build.
26
- **Ranked by official action, not by virality.** Feed ranking uses date of legislative action, dollar magnitude, and affected-population estimates from CBO or agency data - never news coverage, social shares, or editorial judgment.
27
- **No comment sections, ever.** Comments are the single largest bias-injection surface. The only user feedback allowed is a yes/no on summary *clarity*, never on a bill's merit.
28
- **LLM summaries are field-extraction templates, not free-form generation.** Prose summaries pull structured fields from official bill text. High-salience bills are tested against a Polarization Index before they go live.
29
 
30
See [`docs/NEUTRALITY.md`](docs/NEUTRALITY.md) for the full enforceable doctrine, including CI gates.
31
 
32
---
33
 
34
## Screenshots
35
 
36
<table>
37
<tr>
38
<td width="50%">
39
 
40
**Activity feed - dark mode**
41
 
42
<img src="docs/screenshots/home-dark.png" alt="Docket home - dark mode" width="100%">
43
 
44
</td>
45
<td width="50%">
46
 
47
**Activity feed - light mode**
48
 
49
<img src="docs/screenshots/home-light.png" alt="Docket home - light mode" width="100%">
50
 
51
</td>
52
</tr>
53
<tr>
54
<td width="50%">
55
 
56
**Federal spending tracker**
57
 
58
<img src="docs/screenshots/spending.png" alt="Federal spending view" width="100%">
59
 
60
</td>
61
<td width="50%">
62
 
63
**Cross-type search**
64
 
65
<img src="docs/screenshots/search.png" alt="Search across bills, votes, spending" width="100%">
66
 
67
</td>
68
</tr>
69
</table>
70
 
71
---
72
 
73
## Features
74
 
75
- **Bills and laws** - every bill introduced, its full text, sponsor, committee path, and enacted law
76
- **Amendments and riders** - text-diff view showing what was added or removed between bill versions (Introduced to Enrolled), flagging provisions inserted in appropriations or conference reports
77
- **Votes - recorded and unrecorded** - roll calls with per-member positions, and surface notices when something passed by voice vote or unanimous consent (no individual record exists - that absence is itself information)
78
- **Executive actions** - Executive Orders, proposed rules, and final rules from the Federal Register
79
- **Federal spending** - contracts, grants, agency outlays, and appropriation apportionments from USAspending.gov and Treasury Fiscal Data
80
- **State coverage** - 50 states plus DC and Puerto Rico via OpenStates and LegiScan, with an honest per-state confidence map
81
- **Lobbying disclosures** - LD-1/LD-2 filings from LDA.gov showing who lobbied which bill
82
- **Congressional calendar and schedule** - committee markups, hearings, and floor session timelines
83
- **/how-we-pick** and **/who-pays** - public transparency pages explaining the ranking algorithm and project funding
84
 
85
---
86
 
87
## Data Sources
88
 
89
All primary sources are free and official. No secondhand sources appear in user-facing data.
90
 
91
| Source | Data | Auth |
92
|--------|------|------|
93
| [Congress.gov API](https://api.congress.gov) | Bills, amendments, members, votes | Free api.data.gov key |
94
| [GovInfo API](https://api.govinfo.gov) | Full bill text, Congressional Record, public laws | Same key |
95
| [USAspending.gov API](https://api.usaspending.gov) | Federal awards, agency outlays | None |
96
| [Treasury Fiscal Data](https://fiscaldata.treasury.gov) | Debt, deficit, daily cash flows | None |
97
| [House Clerk XML](https://clerk.house.gov/evs/) | Roll-call votes, posted within hours | None |
98
| [Senate roll-call XML](https://www.senate.gov/legislative/LIS/roll_call_lists/) | Senate roll calls | None |
99
| [Federal Register API](https://www.federalregister.gov/api/v1) | Executive Orders, rules, notices | None |
100
| [LDA.gov API](https://lda.gov/api) | Lobbying disclosures (LD-1/LD-2) | Free key |
101
| [OpenStates v3](https://v3.openstates.org) | 50-state bill and vote data | API key (paid tier for production) |
102
| [LegiScan](https://legiscan.com) | State coverage supplement | API key |
103
| [CBO](https://cbo.gov/cost-estimates) | Cost estimates per bill | None (scrape) |
104
 
105
When an aggregator normalizes data (OpenStates, LegiScan), the user-facing source link always points to the underlying official government record.
106
 
107
See [`docs/DATA_SOURCES.md`](docs/DATA_SOURCES.md) for the full contracts, rate limits, and change-detection strategy.
108
 
109
---
110
 
111
## How It Works
112
 
113
```
114
Government APIs  -->  Python ingest workers  -->  SQLite
115
(Congress.gov,        (services/ingest/)          (docket.db)
116
 GovInfo,                                               |
117
 USAspending,                                           v
118
 Federal Register,                             Next.js API routes
119
 OpenStates, ...)                              (app/api/*)
120
                                                        |
121
                                                        v
122
                                             Browser (dark + light)
123
```
124
 
125
The ingest layer polls each source on a configurable schedule, detects changes by watermark and content-hash, and writes normalized rows to SQLite. The Next.js frontend reads its own API routes backed by `better-sqlite3` - no separate API server needed for single-box deployments. Every row carries an allowlisted `sourceUrl` pointing to an official government endpoint; the build rejects any fact without one.
126
 
127
Detailed architecture: [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)  
128
API contract: [`docs/API.md`](docs/API.md)  
129
Ingest strategy: [`docs/INGESTION_STRATEGY.md`](docs/INGESTION_STRATEGY.md)
130
 
131
---
132
 
133
## Getting Started
134
 
135
### Prerequisites
136
 
137
- Node.js 20+
138
- Python 3.11+
139
- A free [api.data.gov](https://api.data.gov) key (covers Congress.gov and GovInfo)
140
 
141
### 1. Clone and install
142
 
143
```bash
144
git clone https://github.com/zionboggan/docket.git
145
cd docket
146
npm install
147
```
148
 
149
### 2. Set up environment variables
150
 
151
```bash
152
cp .env.example .env
153
# Fill in DOCKET_CONGRESS_KEY (api.data.gov key)
154
# Optional: DOCKET_OPENSTATES_KEY, DOCKET_LEGISCAN_KEY, DOCKET_LEGISCAN_KEY
155
```
156
 
157
### 3. Run ingest (first-time bootstrap)
158
 
159
```bash
160
cd services/ingest
161
python -m venv .venv && source .venv/bin/activate
162
pip install -r requirements.txt
163
 
164
# Seed with federal data (no keys needed for GovInfo bulk path)
165
DOCKET_DB_PATH=../../docket.db python -m ingest.run --source bills_bulk --once
166
DOCKET_DB_PATH=../../docket.db python -m ingest.run --source votes_house --once
167
```
168
 
169
### 4. Run the frontend
170
 
171
```bash
172
# from repo root
173
DOCKET_DB_PATH=./docket.db npm run dev
174
```
175
 
176
Open [http://localhost:3000](http://localhost:3000).
177
 
178
### 5. Run the neutrality lint
179
 
180
```bash
181
npm run neutrality
182
```
183
 
184
This checks all generated copy, summaries, and push text against the banned-words list and confirms every fact has an allowlisted source URL.
185
 
186
---
187
 
188
## Tech Stack
189
 
190
| Layer | Technology |
191
|-------|-----------|
192
| Frontend | Next.js 14 (App Router), React 18, Tailwind CSS |
193
| Database | SQLite via better-sqlite3 |
194
| Ingest | Python 3.11+, asyncio, ARQ (Redis-backed workers) |
195
| Fonts | Archivo (display), JetBrains Mono (data labels), Fraunces (prose) |
196
| Hosting | Cloudflare Pages (frontend edge) + self-hosted ingest workers |
197
| Push | ntfy (MVP), Web Push API |
198
 
199
---
200
 
201
## Roadmap
202
 
203
- [x] Federal bills, votes, members
204
- [x] Federal spending (USAspending + Treasury)
205
- [x] Executive actions (Federal Register)
206
- [x] Lobbying disclosures (LDA.gov)
207
- [x] State-level bills and votes (50 states + DC + PR)
208
- [x] Per-state data-confidence map
209
- [x] Congressional calendar and schedule
210
- [x] Bill version text-diff (rider/amendment detection)
211
- [x] Dark and light themes, mobile-first
212
- [x] Neutrality lint in CI
213
- [ ] Push alerts for topic follows (policy area, agency, or member)
214
- [ ] Per-member voting record page
215
- [ ] CBO cost-estimate integration per bill
216
- [ ] Personalized weekly digest
217
- [ ] Historical trend views
218
- [ ] Share-as-image (factual card, no comment import)
219
 
220
---
221
 
222
## Contributing
223
 
224
Contributions that strengthen neutrality, improve data coverage, or fix bugs are welcome.
225
 
226
Before opening a PR, run `npm run neutrality` and confirm the lint passes. Any copy that adds qualitative modifiers, assigns blame, or characterizes a bill's merit will not be merged.
227
 
228
See the neutrality doctrine at [`docs/NEUTRALITY.md`](docs/NEUTRALITY.md).
229
 
230
---
231
 
232
## License
233
 
234
Apache 2.0 - see [LICENSE](LICENSE).