● Zion Boggan
repos/Prediction Market Bot Postmortem/commits/0644838
zionboggan.com ↗
Code Commits Tags

add INVESTIGATION (+4 more)

0644838   Zion Boggan committed on May 17, 2026 (1 month ago)
LICENSE +21 -0
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
README.md +147 -0
@@ -0,0 +1,147 @@
+# prediction-market-bot-postmortem
+
+A post-mortem and the supporting evaluation framework for a Kalshi
+weather-market trading bot that lost money over its first two months of live
+trading, then was halted, audited, and retired.
+
+The bot itself - the live strategy, order routing, and Kalshi credentials - is
+not in this repository and won't be. What is in this repository is the part
+that turned out to be the actually-useful artifact: the framework that
+catches your own model bleeding money, and the writeup of the audit that
+caught it.
+
+## What you're looking at
+
+```
+docs/
+ INVESTIGATION.md The decisive audit. Era-split P&L,
+ payout-math derivation of the
+ impossible win rate, the three
+ cascading misdiagnoses that came
+ before the right answer.
+ hermes-v4-research-findings-and-fixes.md
+ Earlier research notes: variable
+ fees, ensemble blending, GFS run
+ timing, optimism-tax-on-longshots.
+ Mixed deployed/proposed.
+ PIVOT_SPEC.md The shadow-mode pivot spec that
+ did NOT get built - written to
+ gate any restart of the bot
+ behind a no-capital evaluation
+ window with a pre-committed
+ decision rule.
+eval/
+ c4_eval.py Pre-committed evaluation against the gate:
+ pulls shadow predictions, backfills outcomes
+ from the Kalshi API, applies a hard liquidity
+ filter, scores Brier + EV after fees, emits
+ a PASS/FAIL verdict.
+ empirical_analysis.py Walk-forward: empirical bracket-hit model
+ vs Gaussian. Brier, win rate, EV, total P&L.
+ Stdlib only.
+ effective_exposure.py "Effective exposure" module - discounts
+ near-certain unsettled positions so capital
+ isn't blocked by quasi-decided bets.
+ test_effective_exposure.py Eight scenario tests against the discount
+ logic, including the safety case of a bet
+ that flips from winning to losing.
+```
+
+## The headline lesson (from INVESTIGATION.md)
+
+The bot traded one specific Kalshi market type: single-degree (~2°F)
+temperature brackets, betting NO. It lost $160 in 97 trades pre-2026-04-21,
+then a clamp-on-overconfidence patch landed and the next 41 trades came in
+at roughly +$3 (essentially zero EV).
+
+The payout math:
+
+| Quantity | Value |
+|----------------------------------|------------|
+| Actual bracket hit rate | 62/138 = 44.9% |
+| Average win | +$3.32 |
+| Average loss | -$6.46 |
+| Realized reward:risk | 0.51 |
+| Break-even win rate at that ratio | 66.2% |
+| Bot's actual win rate | 54.3% |
+
+You cannot make money betting NO on near-coin-flip events when the payout
+structure demands a 66% win rate. **It is a market-selection problem, not a
+model problem.** Single-degree brackets sit below NWS/ensemble forecast
+resolution and Kalshi prices them efficiently.
+
+## Why the audit took three passes
+
+A bug had hidden the truth from three previous audits, including the first
+two passes of this one. The `INSERT INTO trades` statement omitted three
+diagnostic columns (`raw_ensemble_probability`, `model_count`, `models_used`),
+so every trade row showed `model_count = 1` and `raw_ensemble_probability =
+NULL`. Anyone (human or AI) inspecting the table concluded "the 31-member
+ensemble pipeline must be dead." It wasn't. The ensemble worked the whole
+time. The columns were simply never written.
+
+This bug never cost a cent in P&L. It did cost three audit cycles - one
+landed-and-deployed fix on a non-problem, and two false starts in the
+investigation itself.
+
+The honest record of that oscillation is preserved in the writeup.
+
+## Why this is the useful part
+
+The trading strategy was the point of the bot, but it had no edge and is
+not portable to anything. The evaluation framework, the gate-before-restart
+discipline, and the "eras + payout math" decomposition are portable. They
+work on any prediction market, any strategy, any bot.
+
+If you are about to ship a bot, the cheapest thing you can do is build
+`c4_eval.py` first, in shadow mode, with the gate criteria written down
+*before* you look at the numbers. Then you build the strategy.
+
+## Reproducible: the real trade dataset is committed
+
+The 138 settled trades from the bot's live run are checked in at
+`data/sample_trades.psv`. This is the actual data behind every number in
+INVESTIGATION.md. The columns:
+
+```
+id | ticker | market_title | nws_forecast | side | edge |
+ensemble_probability | pnl | opened_at | cost | count | avg_price |
+nws_probability | market_price | grok_probability | actual_outcome |
+correct | market_type
+```
+
+Public Kalshi market tickers; no account IDs, no API tokens, no Kalshi
+internal identifiers, no PII. `empirical_analysis.py` reads this file by
+default and reproduces the era-split walk-forward backtest:
+
+```bash
+python eval/empirical_analysis.py
+```
+
+Expected output includes the empirical-vs-Gaussian P(hit) table, Brier
+scores, and walk-forward P&L at four edge thresholds. The walk-forward
+P&L at `min_edge=0.25` reproduces the headline -$104 loss (the bot's
+live equivalent of this band was -$94, drift accounted for by data
+joins and the unclamped-Gaussian era).
+
+## Running the eval pieces
+
+`empirical_analysis.py` and the test suite are stdlib only; no install
+needed.
+
+```bash
+python eval/empirical_analysis.py
+
+export HERMES_DATASET=path/to/your/own.psv
+python eval/empirical_analysis.py
+
+# discount-logic tests
+python eval/test_effective_exposure.py
+```
+
+`c4_eval.py` expects a SQLite DB and a Kalshi API client; adapt it to
+your own data sources before running.
+
+## License
+
+MIT. See [LICENSE](LICENSE).
data/sample_trades.psv +139 -0
@@ -0,0 +1,139 @@
+id|ticker|market_title|nws_forecast|side|edge|ensemble_probability|pnl|opened_at|cost|count|avg_price|nws_probability|market_price|grok_probability|actual_outcome|correct|market_type
+85|KXHIGHNY-26MAR27-B62.5|Will the **high temp in NYC** be 62-63° on Mar 27, 2026?|54.0|no|0.22|0.01|1.5|2026-03-26 18:09:13|4.5|6.0|0.75|0.00834056197137967|0.75||no|1|high
+86|KXHIGHNY-26MAR27-B64.5|Will the **high temp in NYC** be 64-65° on Mar 27, 2026?|54.0|no|0.22|0.01|-4.44|2026-03-26 18:09:13|4.44|6.0|0.74|0.00219240663692188|0.74||yes|0|high
+87|KXHIGHNY-26MAR27-B66.5|Will the **high temp in NYC** be 66-67° on Mar 27, 2026?|53.0|no|0.13|0.01|0.6|2026-03-26 21:24:26|3.4|4.0|0.85|0.000174351356613367|0.85||no|1|high
+88|KXLOWTNYC-26MAR27-B39.5|Will the minimum temperature be 39-40° on Mar 27, 2026?|33.0|no|0.13|0.01|0.6|2026-03-26 21:24:31|3.4|4.0|0.85|0.0239496412732271|0.85||no|1|low
+89|KXHIGHCHI-26MAR27-B43.5|Will the high temp in Chicago be 43-44° on Mar 27, 2026?|42.0|no|0.166267134051702|0.103732865948298|-4.97|2026-03-26 23:00:15|4.97|7.0|0.71|0.0856172337081196|0.71||yes|0|high
+90|KXHIGHNY-26MAR28-B42.5|Will the **high temp in NYC** be 42-43° on Mar 28, 2026?|45.0|no|0.16482436230939|0.19517563769061|2.66|2026-03-28 12:06:14|4.34|7.0|0.62|0.0849209074997601|0.62||no|1|high
+91|KXHIGHNY-26MAR28-B44.5|Will the **high temp in NYC** be 44-45° on Mar 28, 2026?|45.0|no|0.278120420928126|0.151879579071874|-5.5|2026-03-28 12:06:14|5.5|10.0|0.55|0.104865577820644|0.55||yes|0|high
+92|KXHIGHCHI-26MAR28-B49.5|Will the high temp in Chicago be 49-50° on Mar 28, 2026?|49.0|no|0.396798880845161|0.0832011191548385|5.0|2026-03-28 12:06:16|5.0|10.0|0.5|0.090164059126656|0.5||no|1|high
+93|KXHIGHCHI-26MAR28-B51.5|Will the high temp in Chicago be 51-52° on Mar 28, 2026?|49.0|no|0.109567522975289|0.030432477024711|0.75|2026-03-28 12:06:23|4.25|5.0|0.85|0.0771998823536058|0.85||no|1|high
+94|KXHIGHNY-26MAR29-B54.5|Will the **high temp in NYC** be 54-55° on Mar 29, 2026?|55.0|no|0.133784914705815|0.296215085294184|-2.75|2026-03-29 12:32:42|2.75|5.0|0.55|0.104865577820644|0.55||yes|0|high
+95|KXHIGHNY-26MAR29-B56.5|Will the **high temp in NYC** be 56-57° on Mar 29, 2026?|55.0|no|0.291637617894306|0.0983623821056941|4.1|2026-03-29 12:32:43|5.9|10.0|0.59|0.0977447728379144|0.59||no|1|high
+96|KXHIGHCHI-26MAR29-B63.5|Will the high temp in Chicago be 63-64° on Mar 29, 2026?|64.0|no|0.23491661480027|0.0150833851997296|-5.11|2026-03-29 12:32:45|5.11|7.0|0.73|0.090164059126656|0.73||yes|0|high
+97|KXHIGHCHI-26MAR29-B65.5|Will the high temp in Chicago be 65-66° on Mar 29, 2026?|64.0|no|0.32|0.01|2.88|2026-03-29 12:32:45|5.12|8.0|0.64|0.0856172337081196|0.64||no|1|high
+98|KXHIGHCHI-26MAR30-B80.5|Will the high temp in Chicago be 80-81° on Mar 30, 2026?|77.0|no|0.2|0.01|-6.16|2026-03-30 12:05:01|6.16|8.0|0.77|0.0660997387714539|0.77||yes|0|high
+99|KXHIGHLAX-26MAR30-B67.5|Will the **high temp in LA** be 67-68° on Mar 30, 2026?|70.0|no|0.18|0.01|1.47|2026-03-30 12:05:22|5.53|7.0|0.79|0.0924695252639576|0.79||no|1|high
+100|KXLOWTLAX-26MAR30-B54.5|Will the minimum temperature be 54-55° on Mar 30, 2026?|58.0|no|0.18|0.01|1.54|2026-03-30 12:05:28|5.46|7.0|0.78|0.0682981424940058|0.78||no|1|low
+101|KXHIGHDEN-26MAR30-B78.5|Will the **high temp in Denver** be 78-79° on Mar 30, 2026?|81.0|no|0.25|0.01|1.68|2026-03-30 12:05:29|4.32|6.0|0.72|0.0640492342599768|0.72||no|1|high
+102|KXRAINNYC-26MAR31-T0|Will it **rain** in New York City on Tuesday?|76.0|yes|0.354193548387097|0.774193548387097|17.3|2026-03-30 21:22:26|11.7|29.0|0.41|0.56|0.4||yes|1|rain
+103|KXHIGHCHI-26MAR31-B76.5|Will the high temp in Chicago be 76-77° on Mar 31, 2026?|74.0|no|0.213557180619435|0.0164428193805655|0.25|2026-03-30 21:22:49|0.75|1.0|0.75|0.0771998823536058|0.75||no|1|high
+104|KXLOWTCHI-26MAR31-B43.5|Will the minimum temperature be 43-44° on Mar 31, 2026?|37.0|no|0.14|0.01|2.08|2026-03-30 21:58:08|10.92|13.0|0.84|0.0304168395818861|0.84||no|1|low
+105|KXLOWTDEN-26MAR31-B41.5|Will the minimum temperature be 41-42° on Mar 31, 2026?|39.0|no|0.237217464687146|0.0627825353128544|1.87|2026-03-31 11:15:57|4.13|6.0|0.69|0.0640492342599768|0.69||no|1|low
+106|KXLOWTLAX-26MAR31-B56.5|Will the minimum temperature be 56-57° on Mar 31, 2026?|59.0|no|0.47|0.01|7.51|2026-03-31 13:55:55|7.49|15.0|0.5|0.0924695252639576|0.5||no|1|low
+107|KXHIGHNY-26APR01-T79|Will the **high temp in NYC** be >79° on Apr 1, 2026?|76.0|no|0.217741935483871|0.032258064516129|-9.48|2026-03-31 14:05:51|9.48|12.0|0.79|0.212468741841681|0.79||yes|0|high
+108|KXHIGHNY-26APR01-B78.5|Will the **high temp in NYC** be 78-79° on Apr 1, 2026?|75.0|no|0.23201095143543|0.0279890485645704|8.12|2026-03-31 23:21:34|20.88|29.0|0.72|0.0687695257477211|0.72||no|1|high
+109|KXLOWTCHI-26APR01-B37.5|Will the minimum temperature be 37-38° on Apr 1, 2026?|42.0|no|0.167727975025115|0.192272024974885|-13.65|2026-03-31 23:21:42|13.65|22.0|0.62|0.0537415795566644|0.62||yes|0|low
+110|KXHIGHLAX-26APR01-B67.5|Will the **high temp in LA** be 67-68° on Apr 1, 2026?|65.0|no|0.147857762644122|0.312142237355878|-10.92|2026-04-01 11:47:01|10.92|21.0|0.52|0.0924695252639576|0.52||yes|0|high
+111|KXHIGHCHI-26APR01-B40.5|Will the high temp in Chicago be 40-41° on Apr 1, 2026?|43.0|no|0.208024091712523|0.0819759082874771|10.54|2026-04-01 12:46:52|23.46|34.0|0.69|0.0771998823536058|0.69||no|1|high
+112|KXLOWTLAX-26APR01-B59.5|Will the minimum temperature be 59-60° on Apr 1, 2026?|57.0|no|0.275708077009388|0.0942919229906117|-7.32|2026-04-01 12:46:55|7.32|12.0|0.61|0.0924695252639576|0.61||yes|0|low
+113|KXHIGHLAX-26APR02-B69.5|Will the **high temp in LA** be 69-70° on Apr 2, 2026?|67.0|no|0.205483870967742|0.0645161290322581|8.7|2026-04-02 06:02:54|21.3|30.0|0.71|0.0924695252639576|0.71||no|1|high
+114|KXLOWTDEN-26APR02-B37.5|Will the minimum temperature be 37-38° on Apr 2, 2026?|35.0|no|0.29|0.01|8.96|2026-04-02 09:02:59|19.04|28.0|0.68|0.0640492342599768|0.68||no|1|low
+115|KXHIGHCHI-26APR02-B70.5|Will the high temp in Chicago be 70-71° on Apr 2, 2026?|68.0|no|0.315483870967742|0.0645161290322581|-23.24|2026-04-02 15:08:19|23.24|39.0|0.6|0.0771998823536058|0.6||yes|0|high
+116|KXHIGHLAX-26APR03-T74|Will the **high temp in LA** be <74° on Apr 3, 2026?|76.0|no|0.307741935483871|0.032258064516129|0.36|2026-04-02 15:08:28|0.64|1.0|0.64|0.261636717020193|0.64||no|1|high
+117|KXLOWTLAX-26APR03-B56.5|Will the minimum temperature be 56-57° on Apr 3, 2026?|59.0|no|0.36|0.01|-20.74|2026-04-02 15:08:36|20.74|34.0|0.61|0.0924695252639576|0.61||yes|0|low
+118|KXHIGHMIA-26APR03-B84.5|Will the **high temp in Miami** be 84-85° on Apr 3, 2026?|82.0|no|0.25|0.01|-7.2|2026-04-02 15:08:45|7.2|10.0|0.72|0.0967817148298926|0.72||yes|0|high
+119|KXLOWTNYC-26APR02-B39.5|Will the minimum temperature be 39-40° on Apr 2, 2026?|42.0|no|0.23|0.01|5.27|2026-04-02 15:23:09|15.73|21.0|0.75|0.0849209074997601|0.75||no|1|low
+120|KXLOWTNYC-26APR03-B42.5|Will the minimum temperature be 42-43° on Apr 3, 2026?|59.0|no|0.213225806451613|0.0967741935483871|8.58|2026-04-02 15:38:08|17.42|26.0|0.67|7.36212680718662e-06|0.67||no|1|low
+121|KXLOWTLAX-26APR04-T56|Will the minimum temperature be >56° on Apr 4, 2026?|59.0|yes|0.30969696969697|0.96969696969697|-22.4|2026-04-03 14:14:12|22.4|35.0|0.64|0.830832808243765|0.64||no|0|low
+122|KXLOWTDEN-26APR04-T28|Will the minimum temperature be <28° on Apr 4, 2026?|33.0|no|0.54969696969697|0.0303030303030303|-20.4|2026-04-03 15:44:17|20.4|51.0|0.4|0.187663735381379|0.4||yes|0|low
+123|KXHIGHNY-26APR03-T65|Will the **high temp in NYC** be <65° on Apr 3, 2026?|70.0|no|0.37969696969697|0.0303030303030303|6.88|2026-04-03 17:14:08|9.12|16.0|0.57|0.0917908354065392|0.57||no|1|high
+124|KXLOWTNYC-26APR04-T46|Will the minimum temperature be <46° on Apr 4, 2026?|49.0|no|0.28969696969697|0.0303030303030303|-7.35|2026-04-03 21:40:53|7.35|11.0|0.67|0.212468741841681|0.67||yes|0|low
+125|KXLOWTCHI-26APR04-B41.5|Will the minimum temperature be 41-42° on Apr 4, 2026?|36.0|no|0.285483870967742|0.0645161290322581|-17.01|2026-04-03 23:56:04|17.01|27.0|0.63|0.0414904920402464|0.63||yes|0|low
+126|KXHIGHLAX-26APR05-T74|Will the **high temp in LA** be <74° on Apr 5, 2026?|76.0|no|0.31969696969697|0.0303030303030303|11.47|2026-04-04 14:18:00|19.53|31.0|0.63|0.261636717020193|0.63||no|1|high
+127|KXHIGHLAX-26APR04-T85|Will the **high temp in LA** be >85° on Apr 4, 2026?|80.0|no|0.23969696969697|0.0303030303030303|-17.85|2026-04-04 17:47:48|17.85|25.0|0.71|0.0552701748613679|0.71||yes|0|high
+128|KXLOWTLAX-26APR07-T58|Will the minimum temperature be >58° on Apr 7, 2026?|58.0|yes|0.221274509803922|0.661274509803922|-19.33|2026-04-06 14:32:42|19.33|45.0|0.43|0.5|0.43||no|0|low
+129|KXLOWTDEN-26APR07-T31|Will the minimum temperature be <31° on Apr 7, 2026?|41.0|no|0.235098039215686|0.0549019607843137|-0.69|2026-04-06 16:18:20|0.69|1.0|0.69|0.0381079907104944|0.69||yes|0|low
+130|KXHIGHMIA-26APR07-T81|Will the **high temp in Miami** be >81° on Apr 7, 2026?|81.0|no|0.108235294117647|0.471764705882353|-4.0|2026-04-07 09:20:59|4.0|10.0|0.4|0.5|0.4||yes|0|high
+131|KXLOWTCHI-26APR08-T35|Will the minimum temperature be >35° on Apr 8, 2026?|54.0|yes|0.375483870967742|0.935483870967742|14.26|2026-04-07 17:57:21|16.74|31.0|0.54|0.999992590890656|0.54||yes|1|low
+132|KXHIGHLAX-26APR07-B70.5|Will the **high temp in LA** be 70-71° on Apr 7, 2026?|78.0|no|0.227380777677251|0.312619222322749|-4.84|2026-04-07 17:57:29|4.84|11.0|0.44|0.00740257060944449|0.44||yes|0|high
+133|KXLOWTLAX-26APR08-B56.5|Will the minimum temperature be 56-57° on Apr 8, 2026?|59.0|no|0.24|0.01|2.7|2026-04-07 17:57:30|7.3|10.0|0.73|0.0924695252639576|0.73||no|1|low
+134|KXHIGHDEN-26APR08-T71|Will the **high temp in Denver** be <71° on Apr 8, 2026?|74.0|no|0.134193548387097|0.225806451612903|-6.82|2026-04-07 17:57:57|6.82|11.0|0.62|0.297389649341441|0.62||yes|0|high
+135|KXHIGHCHI-26APR08-B69.5|Will the high temp in Chicago be 69-70° on Apr 8, 2026?|67.0|no|0.23|0.01|2.34|2026-04-08 12:22:20|6.66|9.0|0.74|0.0771998823536058|0.74||no|1|high
+136|KXLOWTNYC-26APR09-T37|Will the minimum temperature be >37° on Apr 9, 2026?|42.0|yes|0.274193548387097|0.774193548387097|-6.72|2026-04-08 14:07:26|6.72|14.0|0.48|0.908209164593461|0.48||no|0|low
+137|KXHIGHLAX-26APR09-B74.5|Will the **high temp in LA** be 74-75° on Apr 9, 2026?|70.0|no|0.4|0.01|4.73|2026-04-08 14:07:39|6.27|11.0|0.57|0.0455988744008613|0.57||no|1|high
+138|KXHIGHMIA-26APR09-B80.5|Will the **high temp in Miami** be 80-81° on Apr 9, 2026?|78.0|no|0.214389283249648|0.0756107167503521|2.79|2026-04-09 11:17:31|6.21|9.0|0.69|0.0967817148298926|0.69||no|1|high
+139|KXLOWTCHI-26APR09-B52.5|Will the minimum temperature be 52-53° on Apr 9, 2026?|45.0|no|0.25|0.01|3.07|2026-04-09 12:37:21|7.93|11.0|0.72|0.0211741885923148|0.72||no|1|low
+140|KXHIGHMIA-26APR10-B82.5|Will the **high temp in Miami** be 82-83° on Apr 10, 2026?|80.0|no|0.26|0.01|-7.1|2026-04-09 14:17:31|7.1|10.0|0.71|0.0967817148298926|0.71||yes|0|high
+141|KXLOWTMIA-26APR10-B71.5|Will the minimum temperature be 71-72° on Apr 10, 2026?|69.0|no|0.161255102587|0.168744897413|-5.2|2026-04-09 14:43:07|5.2|8.0|0.65|0.0967817148298926|0.65||yes|0|low
+142|KXLOWTNYC-26APR10-B38.5|Will the minimum temperature be 38-39° on Apr 10, 2026?|49.0|no|0.24|0.01|2.43|2026-04-09 15:47:19|6.57|9.0|0.73|0.00219240663692188|0.73||no|1|low
+143|KXHIGHLAX-26APR09-B72.5|Will the **high temp in LA** be 72-73° on Apr 9, 2026?|70.0|no|0.0811542215503882|0.408845778449612|-1.47|2026-04-09 17:17:30|1.47|3.0|0.49|0.0924695252639576|0.49||yes|0|high
+144|KXLOWTNYC-26APR10-B40.5|Will the minimum temperature be 40-41° on Apr 10, 2026?|49.0|no|0.25|0.01|-6.48|2026-04-09 17:37:26|6.48|9.0|0.72|0.00834056197137978|0.72||yes|0|low
+145|KXLOWTCHI-26APR10-B40.5|Will the minimum temperature be 40-41° on Apr 10, 2026?|37.0|no|0.33|0.01|-0.64|2026-04-10 11:37:21|0.64|1.0|0.64|0.0660997387714539|0.64||yes|0|low
+146|KXHIGHLAX-26APR10-B69.5|Will the **high temp in LA** be 69-70° on Apr 10, 2026?|67.0|no|0.4|0.01|5.16|2026-04-10 11:37:25|6.84|12.0|0.57|0.0924695252639576|0.57||no|1|high
+147|KXLOWTCHI-26APR10-B42.5|Will the minimum temperature be 42-43° on Apr 10, 2026?|37.0|no|0.249066955840261|0.0109330441597386|0.28|2026-04-10 12:02:21|0.72|1.0|0.72|0.0414904920402464|0.72||no|1|low
+148|KXHIGHMIA-26APR11-B82.5|Will the **high temp in Miami** be 82-83° on Apr 11, 2026?|79.0|no|0.28|0.01|3.2|2026-04-10 14:02:26|6.8|10.0|0.68|0.0604168521504205|0.68||no|1|high
+149|KXLOWTCHI-26APR11-T38|Will the minimum temperature be >38° on Apr 11, 2026?|50.0|yes|0.47|0.99|-6.5|2026-04-10 14:07:31|6.5|13.0|0.5|0.996886706615925|0.5||no|0|low
+150|KXLOWTCHI-26APR11-B37.5|Will the minimum temperature be 37-38° on Apr 11, 2026?|50.0|no|0.23|0.01|-6.66|2026-04-10 16:07:21|6.66|9.0|0.74|0.0015928349936099|0.74||yes|0|low
+151|KXLOWTNYC-26APR11-B48.5|Will the minimum temperature be 48-49° on Apr 11, 2026?|43.0|no|0.299896577485005|0.0201034225149954|-5.32|2026-04-11 11:37:19|5.32|8.0|0.67|0.0365206605451712|0.67||yes|0|low
+152|KXHIGHLAX-26APR11-B68.5|Will the **high temp in LA** be 68-69° on Apr 11, 2026?|66.0|no|0.35|0.01|-3.11|2026-04-11 11:37:25|3.11|5.0|0.62|0.0924695252639576|0.62||yes|0|high
+153|KXLOWTCHI-26APR12-T53|Will the minimum temperature be >53° on Apr 12, 2026?|63.0|yes|0.447741935483871|0.967741935483871|7.0|2026-04-11 14:07:23|7.0|14.0|0.5|0.988686371221748|0.5||yes|1|low
+154|KXLOWTLAX-26APR12-B57.5|Will the minimum temperature be 57-58° on Apr 12, 2026?|52.0|no|0.38|0.01|-6.49|2026-04-11 14:12:26|6.49|11.0|0.59|0.0275190093415766|0.59||yes|0|low
+155|KXLOWTDEN-26APR12-T41|Will the minimum temperature be <41° on Apr 12, 2026?|45.0|no|0.27|0.01|-6.38|2026-04-11 14:12:30|6.38|9.0|0.71|0.239090656172175|0.71||yes|0|low
+156|KXHIGHNY-26APR12-B56.5|Will the **high temp in NYC** be 56-57° on Apr 12, 2026?|59.0|no|0.43|0.01|5.06|2026-04-12 11:37:18|5.94|11.0|0.54|0.0849209074997601|0.54||no|1|high
+157|KXLOWTNYC-26APR12-B43.5|Will the minimum temperature be 43-44° on Apr 12, 2026?|50.0|no|0.411355868020719|0.0286441319792813|-6.36|2026-04-12 11:37:19|6.36|12.0|0.53|0.023949641273227|0.53||yes|0|low
+158|KXHIGHNY-26APR13-B80.5|Will the **high temp in NYC** be 80-81° on Apr 13, 2026?|78.0|no|0.27574386196673|0.0142561380332697|2.79|2026-04-13 11:37:25|6.21|9.0|0.69|0.08492090749976|0.69||no|1|high
+159|KXHIGHCHI-26APR13-B80.5|Will the high temp in Chicago be 80-81° on Apr 13, 2026?|77.0|no|0.31|0.01|3.06|2026-04-13 11:37:50|5.94|9.0|0.66|0.0660997387714539|0.66||no|1|high
+160|KXHIGHLAX-26APR13-B66.5|Will the **high temp in LA** be 66-67° on Apr 13, 2026?|62.0|no|0.39|0.01|-5.8|2026-04-13 12:02:27|5.8|10.0|0.58|0.0527150013920873|0.58||yes|0|high
+161|KXLOWTDEN-26APR13-B46.5|Will the minimum temperature be 46-47° on Apr 13, 2026?|41.0|no|0.46|0.01|-1.53|2026-04-13 12:02:38|1.53|3.0|0.51|0.0439645192874191|0.51||yes|0|low
+162|KXHIGHLAX-26APR13-B64.5|Will the **high temp in LA** be 64-65° on Apr 13, 2026?|62.0|no|0.181150240640464|0.148849759359536|2.79|2026-04-13 12:07:29|5.21|8.0|0.65|0.0828736185354491|0.65||no|1|high
+163|KXHIGHLAX-26APR14-B69.5|Will the **high temp in LA** be 69-70° on Apr 14, 2026?|66.0|no|0.450070158522142|0.0299298414778583|-6.0|2026-04-14 11:37:38|6.0|12.0|0.5|0.0677865755256652|0.5||yes|0|high
+164|KXHIGHCHI-26APR14-B82.5|Will the high temp in Chicago be 82-83° on Apr 14, 2026?|80.0|no|0.29|0.01|-6.12|2026-04-14 12:07:21|6.12|9.0|0.68|0.0771998823536058|0.68||yes|0|high
+165|KXLOWTDEN-26APR14-B41.5|Will the minimum temperature be 41-42° on Apr 14, 2026?|37.0|no|0.32|0.01|2.73|2026-04-14 12:07:25|5.27|8.0|0.66|0.0514269207907956|0.66||no|1|low
+166|KXHIGHMIA-26APR14-B83.5|Will the **high temp in Miami** be 83-84° on Apr 14, 2026?|80.0|no|0.24|0.01|-5.84|2026-04-14 12:07:37|5.84|8.0|0.73|0.0604168521504205|0.73||yes|0|high
+167|KXHIGHNY-26APR15-B89.5|Will the **high temp in NYC** be 89-90° on Apr 15, 2026?|86.0|no|0.28|0.03|-3.35|2026-04-14 20:15:12|3.35|5.0|0.67|0.0687695257477211|0.67||yes|0|high
+168|KXHIGHNY-26APR16-B88.5|Will the **high temp in NYC** be 88-89° on Apr 16, 2026?|86.0|no|0.27|0.03|-3.4|2026-04-15 14:40:12|3.4|5.0|0.68|0.08492090749976|0.68||yes|0|high
+169|KXHIGHNY-26APR16-B90.5|Will the **high temp in NYC** be 90-91° on Apr 16, 2026?|87.0|no|0.28|0.03|1.65|2026-04-16 04:10:12|3.35|5.0|0.67|0.0687695257477211|0.67||no|1|high
+170|KXHIGHCHI-26APR16-B72.5|Will the high temp in Chicago be 72-73° on Apr 16, 2026?|70.0|no|0.238413566406438|0.0815864335935618|1.7|2026-04-16 04:20:13|3.3|5.0|0.66|0.0771998823536058|0.66||no|1|high
+171|KXHIGHCHI-26APR16-B74.5|Will the high temp in Chicago be 74-75° on Apr 16, 2026?|70.0|no|0.27|0.03|-3.4|2026-04-16 19:35:15|3.4|5.0|0.68|0.0537415795566644|0.68||yes|0|high
+172|KXHIGHCHI-26APR17-B81.5|Will the high temp in Chicago be 81-82° on Apr 17, 2026?||no|0.3|0.03|-3.25|2026-04-16 20:46:07|3.25|5.0|0.65||0.65||yes|0|high
+173|KXHIGHNY-26APR17-B81.5|Will the **high temp in NYC** be 81-82° on Apr 17, 2026?|78.0|no|0.38|0.03|-3.42|2026-04-17 16:55:12|3.42|6.0|0.57|0.0687695257477211|0.57||yes|0|high
+174|KXHIGHNY-26APR17-B79.5|Will the **high temp in NYC** be 79-80° on Apr 17, 2026?|77.0|no|0.267364189254234|0.0726358107457664|1.8|2026-04-17 17:45:12|3.2|5.0|0.64|0.08492090749976|0.64||no|1|high
+175|KXHIGHNY-26APR18-B67.5|Will the **high temp in NYC** be 67-68° on Apr 18, 2026?|65.0|no|0.28|0.03|1.29|2026-04-17 18:30:12|2.71|4.0|0.68|0.08492090749976|0.68||no|1|high
+176|KXHIGHCHI-26APR17-B83.5|Will the high temp in Chicago be 83-84° on Apr 17, 2026?|80.0|no|0.5|0.03|0.16|2026-04-17 18:30:15|0.84|1.0|0.84|0.0660997387714539|0.84||no|1|high
+177|KXHIGHNY-26APR18-B65.5|Will the **high temp in NYC** be 65-66° on Apr 18, 2026?|63.0|no|0.173357290310627|0.286642709689373|-2.4|2026-04-18 14:45:17|2.4|5.0|0.48|0.08492090749976|0.48||yes|0|high
+178|KXHIGHNY-26APR19-B54.5|Will the **high temp in NYC** be 54-55° on Apr 19, 2026?|59.0|no|0.259820351658981|0.0501796483410191|1.61|2026-04-18 20:55:12|3.39|5.0|0.68|0.0519083806874208|0.68||no|1|high
+179|KXHIGHCHI-26APR19-B48.5|Will the high temp in Chicago be 48-49° on Apr 19, 2026?|51.0|no|0.232294124525937|0.187705875474063|2.2|2026-04-18 23:40:14|2.8|5.0|0.56|0.0879019439316729|0.56||no|1|high
+180|KXHIGHCHI-26APR19-B52.5|Will the high temp in Chicago be 52-53° on Apr 19, 2026?|50.0|no|0.277362008840413|0.0426379911595868|1.32|2026-04-19 07:45:13|2.68|4.0|0.67|0.0879019439316729|0.67||no|1|high
+181|KXHIGHNY-26APR20-B51.5|Will the **high temp in NYC** be 51-52° on Apr 20, 2026?|54.0|no|0.174810673541855|0.225189326458145|-2.4|2026-04-19 19:15:16|2.4|4.0|0.6|0.0849209074997601|0.6||yes|0|high
+182|KXHIGHNY-26APR23-B74.5|Will the **high temp in NYC** be 74-75° on Apr 23, 2026?|70.0|no|0.23|0.1|1.8|2026-04-22 15:41:47|3.2|5.0|0.64|0.0504637189727674|0.64||no|1|high
+183|KXHIGHCHI-26APR22-B71.5|Will the high temp in Chicago be 71-72° on Apr 22, 2026?|68.0|no|0.23|0.1|1.71|2026-04-22 18:26:51|3.29|5.0|0.66|0.0689003414249409|0.66||no|1|high
+184|KXHIGHCHI-26APR23-B80.5|Will the high temp in Chicago be 80-81° on Apr 23, 2026?|83.0|no|0.25|0.1|1.85|2026-04-22 20:01:52|3.15|5.0|0.63|0.0906705242545767|0.63||no|1|high
+185|KXHIGHCHI-26APR23-B82.5|Will the high temp in Chicago be 82-83° on Apr 23, 2026?|80.0|no|0.34|0.1|-3.24|2026-04-23 06:01:50|3.24|6.0|0.54|0.0906705242545767|0.54||yes|0|high
+186|KXHIGHNY-26APR24-B66.5|Will the **high temp in NYC** be 66-67° on Apr 24, 2026?|63.0|no|0.23|0.1|1.75|2026-04-23 14:26:49|3.25|5.0|0.65|0.0691105468150992|0.65||no|1|high
+187|KXHIGHNY-26APR23-B72.5|Will the **high temp in NYC** be 72-73° on Apr 23, 2026?|75.0|no|0.25|0.1|-3.15|2026-04-23 17:41:50|3.15|5.0|0.63|0.0874918591596392|0.63||yes|0|high
+188|KXHIGHCHI-26APR24-B70.5|Will the high temp in Chicago be 70-71° on Apr 24, 2026?|74.0|no|0.344095879991743|0.135904120008257|-3.01|2026-04-24 18:06:50|3.01|7.0|0.43|0.0659547724761153|0.43||yes|0|high
+189|KXHIGHNY-26APR25-B51.5|Will the **high temp in NYC** be 51-52° on Apr 25, 2026?|49.0|no|0.238860573017407|0.111139426982593|1.85|2026-04-24 20:06:48|3.15|5.0|0.63|0.087279086620775|0.63||no|1|high
+190|KXHIGHCHI-26APR25-B58.5|Will the high temp in Chicago be 58-59° on Apr 25, 2026?|56.0|no|0.271754206810958|0.138245793189042|2.62|2026-04-25 15:31:51|3.38|6.0|0.56|0.0951155709755254|0.56||no|1|high
+191|KXHIGHNY-26APR27-B66.5|Will the **high temp in NYC** be 66-67° on Apr 27, 2026?|64.0|no|0.24|0.1|1.8|2026-04-26 14:21:49|3.2|5.0|0.64|0.0869439491709505|0.64||no|1|high
+192|KXHIGHNY-26APR27-B68.5|Will the **high temp in NYC** be 68-69° on Apr 27, 2026?|66.0|no|0.26|0.1|-3.14|2026-04-27 04:10:03|3.14|5.0|0.63|0.0738265575899699|0.63||yes|0|high
+193|KXHIGHNY-26APR28-B65.5|Will the **high temp in NYC** be 65-66° on Apr 28, 2026?|63.0|no|0.23|0.1|-3.4|2026-04-27 14:15:03|3.4|5.0|0.68|0.0738265575899699|0.68||yes|0|high
+194|KXHIGHCHI-26APR28-B62.5|Will the high temp in Chicago be 62-63° on Apr 28, 2026?|58.0|no|0.320504722056845|0.109495277943155|2.7|2026-04-28 05:20:04|3.3|6.0|0.55|0.050253517927551|0.55||no|1|high
+195|KXHIGHCHI-26APR28-B64.5|Will the high temp in Chicago be 64-65° on Apr 28, 2026?|58.0|no|0.23|0.1|-3.25|2026-04-28 09:55:05|3.25|5.0|0.65|0.00859717006412675|0.65||yes|0|high
+196|KXHIGHNY-26APR29-B60.5|Will the **high temp in NYC** be 60-61° on Apr 29, 2026?||no|0.362783291127722|0.107216708872278|2.94|2026-04-29 05:11:44|3.06|6.0|0.51||0.51||no|1|high
+197|KXHIGHCHI-26APR29-B58.5|Will the high temp in Chicago be 58-59° on Apr 29, 2026?|56.0|no|0.24|0.1|1.79|2026-04-29 05:51:32|3.21|5.0|0.64|0.087798415724998|0.64||no|1|high
+198|KXHIGHNY-26APR30-B61.5|Will the **high temp in NYC** be 61-62° on Apr 30, 2026?|64.0|no|0.26|0.1|1.85|2026-04-29 23:06:30|3.15|5.0|0.63|0.075975402558546|0.63||no|1|high
+199|KXHIGHCHI-26APR30-B53.5|Will the high temp in Chicago be 53-54° on Apr 30, 2026?|51.0|no|0.282933880608726|0.127066119391274|2.15|2026-04-30 11:01:26|2.85|5.0|0.57|0.0933711050047605|0.57||no|1|high
+200|KXHIGHCHI-26APR30-B55.5|Will the high temp in Chicago be 55-56° on Apr 30, 2026?|51.0|no|0.26|0.1|-3.1|2026-04-30 14:16:32|3.1|5.0|0.62|0.0426324257865244|0.62||yes|0|high
+201|KXHIGHNY-26MAY01-B66.5|Will the **high temp in NYC** be 66-67° on May 1, 2026?|64.0|no|0.24|0.1|1.8|2026-05-01 19:06:29|3.2|5.0|0.64|0.0876917179072645|0.64||no|1|high
+202|KXHIGHNY-26MAY02-B60.5|Will the **high temp in NYC** be 60-61° on May 2, 2026?|63.0|no|0.27|0.1|-3.66|2026-05-02 07:01:30|3.66|6.0|0.61|0.0876917179072645|0.61||yes|0|high
+203|KXHIGHNY-26MAY03-B59.5|Will the **high temp in NYC** be 59-60° on May 3, 2026?|62.0|no|0.29|0.1|-3.54|2026-05-02 14:01:29|3.54|6.0|0.59|0.0776968588129118|0.59||yes|0|high
+204|KXHIGHCHI-26MAY02-B57.5|Will the high temp in Chicago be 57-58° on May 2, 2026?|55.0|no|0.23|0.1|1.74|2026-05-02 14:41:31|3.26|5.0|0.65|0.092840815269698|0.65||no|1|high
+205|KXHIGHNY-26MAY02-B58.5|Will the **high temp in NYC** be 58-59° on May 2, 2026?|63.0|no|0.251179176919976|0.118820823080024|1.95|2026-05-02 17:06:31|3.05|5.0|0.61|0.0483645709408376|0.61||no|1|high
+206|KXHIGHCHI-26MAY02-B53.5|Will the high temp in Chicago be 53-54° on May 2, 2026?|56.0|no|0.244342077982103|0.125657922017897|1.95|2026-05-02 19:26:31|3.05|5.0|0.61|0.092840815269698|0.61||no|1|high
+207|KXHIGHCHI-26MAY03-B68.5|Will the high temp in Chicago be 68-69° on May 3, 2026?|65.0|no|0.23|0.1|1.75|2026-05-03 02:46:33|3.25|5.0|0.65|0.0678572277147726|0.65||no|1|high
+208|KXHIGHNY-26MAY05-B81.5|Will the **high temp in NYC** be 81-82° on May 5, 2026?|79.0|no|0.24|0.1|-3.2|2026-05-04 14:01:29|3.2|5.0|0.64|0.0771478932144392|0.64||yes|0|high
+209|KXHIGHCHI-26MAY05-B60.5|Will the high temp in Chicago be 60-61° on May 5, 2026?|55.0|no|0.235703731587291|0.104296268412709|-3.2|2026-05-04 14:01:35|3.2|5.0|0.64|0.0370791827116334|0.64||yes|0|high
+210|KXHIGHNY-26MAY05-B83.5|Will the **high temp in NYC** be 83-84° on May 5, 2026?|81.0|no|0.23|0.1|1.75|2026-05-04 20:11:29|3.25|5.0|0.65|0.0771478932144392|0.65||no|1|high
+211|KXHIGHNY-26MAY06-B69.5|Will the **high temp in NYC** be 69-70° on May 6, 2026?||no|0.27|0.1|1.95|2026-05-05 14:31:45|3.05|5.0|0.61||0.61||no|1|high
+212|KXHIGHCHI-26MAY06-B59.5|Will the high temp in Chicago be 59-60° on May 6, 2026?|57.0|no|0.25|0.1|1.93|2026-05-06 17:31:31|3.07|5.0|0.61|0.0924049955641063|0.61||no|1|high
+213|KXHIGHNY-26MAY07-B64.5|Will the **high temp in NYC** be 64-65° on May 7, 2026?|67.0|no|0.24|0.1|-3.2|2026-05-06 20:31:29|3.2|5.0|0.64|0.0788159481673856|0.64||yes|0|high
+214|KXHIGHNY-26MAY08-B64.5|Will the **high temp in NYC** be 64-65° on May 8, 2026?|67.0|no|0.23|0.1|1.75|2026-05-07 14:16:29|3.25|5.0|0.65|0.0788159481673856|0.65||no|1|high
+215|KXHIGHCHI-26MAY07-B60.5|Will the high temp in Chicago be 60-61° on May 7, 2026?||no|0.37|0.1|-3.6|2026-05-07 15:52:26|3.6|7.0|0.51||0.51||yes|0|high
+216|KXHIGHCHI-26MAY08-B65.5|Will the high temp in Chicago be 65-66° on May 8, 2026?||no|0.25|0.1|-3.15|2026-05-07 15:52:33|3.15|5.0|0.63||0.63||yes|0|high
+217|KXHIGHNY-26MAY09-B61.5|Will the **high temp in NYC** be 61-62° on May 9, 2026?|66.0|no|0.26|0.1|0.4|2026-05-08 14:11:41|0.6|1.0|0.6|0.0537199190421144|0.6||no|1|high
+218|KXHIGHNY-26MAY09-B63.5|Will the **high temp in NYC** be 63-64° on May 9, 2026?|66.0|no|0.23|0.1|1.75|2026-05-09 02:16:31|3.25|5.0|0.65|0.0775790935514575|0.65||no|1|high
+219|KXHIGHCHI-26MAY09-B74.5|Will the high temp in Chicago be 74-75° on May 9, 2026?||no|0.29|0.1|2.04|2026-05-09 10:01:47|2.96|5.0|0.59||0.59||no|1|high
+220|KXHIGHCHI-26MAY09-B72.5|Will the high temp in Chicago be 72-73° on May 9, 2026?|75.0|no|0.28|0.1|2.0|2026-05-09 15:16:35|3.0|5.0|0.6|0.0933639034926214|0.6||no|1|high
+221|KXHIGHNY-26MAY10-B72.5|Will the **high temp in NYC** be 72-73° on May 10, 2026?|77.0|no|0.36|0.2|-3.39|2026-05-10 14:31:24|3.39|8.0|0.42|0.0494269789647181|0.42||yes|0|high
+222|KXHIGHNY-26MAY13-B70.5|Will the **high temp in NYC** be 70-71° on May 13, 2026?|73.0|no|0.36285385984697|0.20714614015303|4.72|2026-05-13 16:01:24|3.28|8.0|0.41|0.0874975273121735|0.41||no|1|high
docs/INVESTIGATION.md +98 -0
@@ -0,0 +1,98 @@
+# Hermes Investigation - 2026-05-17
+
+**Context:** "Feels like the data is corrupted - worked for a bit then slowly bled me." Operator-initiated audit after a sustained drawdown.
+**Status:** Investigation only. No code changed. Auto-trading remains OFF (paused since 2026-05-14).
+**Confidence:** High. The final conclusion is consistent across four independent cuts of the data and required no further reversal.
+
+---
+
+## Bottom line up front
+
+1. **Your instinct was right that something was off - but it is not ongoing corruption or a dead pipeline.**
+2. **The money was bled before 2026-04-21** by an overconfident probability model betting NO on coin-flip markets with bad payout math.
+3. **The 2026-04-21 "MAE-σ floor" change stopped the bleed.** Trades since then are roughly breakeven. The −$157 total P&L and 48% drawdown you see are *old damage still showing on the cumulative chart*, not fresh losses.
+4. **Single-degree (2°F) temperature brackets have no exploitable edge.** They hit ~45% of the time and the payout structure needs a ~66% win rate. No probability model can fix a market with no edge.
+5. **A real bug exists but it never cost money:** three diagnostic columns are never written to the database. That logging gap caused *three separate audits* (2026-04-27, and my own first two passes today) to misdiagnose the problem. It is worth fixing for observability only.
+
+---
+
+## The numbers that settle it
+
+### Era split (the decisive cut)
+
+| Era | Trades | Avg ensemble prob | P&L | EV/trade |
+|---|---|---|---|---|
+| Pre-Apr-21 (unclamped Gaussian) | 97 | 0.01-0.14 | **−$160.72** | **−$1.66** |
+| Post-Apr-21 (MAE-σ floor active) | 41 | ~0.10 | **+$3.06** | **≈ $0.00** |
+
+Essentially **100% of the lifetime loss happened before 2026-04-21.** After the MAE-σ floor was added, the bot stopped bleeding.
+
+### Why the strategy can't win (the payout math)
+
+- Narrow-bracket actual hit rate: **62/138 = 44.9%** - these markets are near coin-flips.
+- Realized reward:risk: avg win **$3.32**, avg loss **−$6.46** → ratio **0.51**.
+- Break-even win rate at that ratio: **1 / (1 + 0.51) ≈ 66%**.
+- Bot's actual win rate: **54.3%**.
+
+You cannot make money betting NO on ~coin-flip events when the payout structure demands a 66% win rate. This is a **market-selection problem, not a model problem.** Single-degree temperature brackets sit below NWS/ensemble forecast resolution and Kalshi prices them efficiently.
+
+### What the model actually did wrong
+
+The Gaussian model (whether fed by NWS point forecast or the ensemble) was systematically **overconfident that the bracket would NOT hit** - it assigned 1-13% hit probability when reality was ~45%. Pre-Apr-21 this was unclamped, so it would say "1% chance" and bet NO at fake 99% confidence → catastrophic. The Apr-21 MAE-σ floor crudely clamped the floor to ~10%, which capped the overconfidence and stopped the catastrophic losses (but did not create a winning strategy - breakeven, not profit).
+
+---
+
+## The logging gap (real bug, zero P&L impact)
+
+`INSERT INTO trades` at `main.py:1585` lists its columns explicitly. Three columns that exist in the schema are **not in the INSERT** and are therefore never written:
+
+- `raw_ensemble_probability` → always NULL
+- `model_count` → always DEFAULT 1
+- `models_used` → always '' / NULL
+
+**Consequence:** anyone (human or AI) inspecting the trade table sees `model_count = 1` and `raw_ensemble_probability = NULL` on every row and concludes "the 31-member ensemble never ran / the pipeline is dead."
+
+This is false. Live testing on 2026-05-17 confirmed `fetch_ensemble_forecast()` returns a healthy 31-member ensemble (validate=True, ~1.7°F spread). The ensemble works. The columns are simply never recorded.
+
+**This single gap caused three misdiagnoses:**
+- 2026-04-27 audit: blamed a dead `OPENMETEO_PROXY` node, "fixed" it (the fix addressed a non-problem; that memory entry is now flagged invalid).
+- This session, pass 1: I repeated the same "ensemble pipeline died" error.
+- This session, pass 2: I then over-corrected to "the MAE-σ floor is destroying the signal and causing the bleed" - also wrong (the era split disproves it).
+
+The honest record of that oscillation is preserved in memory. The final era-split reconciliation is internally consistent and required no further reversal.
+
+---
+
+## Evidence trail (for review)
+
+1. `auto_config.enabled = 0` confirmed; last real trade 2026-05-13 (pre-pause). No trades since the 2026-05-14 fix. No ongoing bleeding.
+2. 138 settled trades, all single-degree temp brackets, ~100% NO side.
+3. Backtest by edge band and the rejected empirical model: **disregard these** - computed before the era split was understood; superseded.
+4. Live pipeline tests: ensemble OK (31 members), NWS OK, `OPENMETEO_PROXY = None` (the old trap is not active).
+5. Era-sliced P&L + clamp regime + ground-truth hit rate (the tables above): mutually consistent, no contradictions.
+
+Supporting files: `empirical_analysis.py`, `h_ds.psv`.
+
+---
+
+## Options (no action taken - your call)
+
+**A. Do nothing / stay paused (lowest risk).**
+Hermes is paused and not losing money. The strategy has no edge; "don't trade" is the correct play for a no-edge market. Cost: $0. Benefit: $0.
+
+**B. Fix the logging gap only.**
+Add the 3 missing columns to the INSERT so future audits aren't blind. ~10-line change, no behavioral effect, auto stays OFF. Recommended regardless of strategy decision - it stops the recurring misdiagnosis.
+
+**C. Pivot to markets that actually have edge (real project).**
+Abandon single-degree brackets. Target wider (≥3-5°F) brackets and above/below-threshold markets where |forecast − threshold| is large vs forecast error - the zones where NWS genuinely beats retail. **No historical data exists for these** (the bot only ever traded narrow brackets, and `market_history` is empty), so this *cannot be backtested* - it requires a shadow-mode data-collection window before any capital. Largest effort, only path with a plausible edge.
+
+**D. Retire Hermes.**
+If the appetite for a multi-week rebuild isn't there, the rational move for a no-edge bot is to stop. Funds stay safe.
+
+**Do NOT:** re-enable auto on bracket markets, or remove the MAE-σ floor. The floor is helping; removing it reproduces the pre-Apr-21 catastrophic bleed.
+
+---
+
+## Recommendation
+
+**B now (cheap, stops the misdiagnosis loop), then a deliberate choice between A/C/D - not under time pressure.** The one thing the data is unambiguous about: the current strategy (single-degree brackets) has no edge and should never be re-enabled as-is. Whether to invest in pivot (C) or retire (D) is a question of how much you want to spend chasing a weather-trading edge that, per the quant playbook, exists only in market types Hermes has never actually traded.
eval/empirical_analysis.py +165 -0
@@ -0,0 +1,165 @@
+"""Hermes root-cause analysis: empirical bracket-hit model vs Gaussian.
+
+Reads the joined historical dataset, reconstructs (NWS forecast, bracket bounds,
+hit/miss, market price), and:
+ 1. Examines the distance->hit relationship vs what the Gaussian predicts
+ 2. Builds an empirical P(hit | |forecast - bracket_mid|) lookup (Laplace-smoothed)
+ 3. Walk-forward backtests empirical vs Gaussian: Brier, WR, EV, total P&L
+No external deps (stdlib only).
+"""
+import csv, math, os, re, statistics
+from collections import defaultdict
+
+PSV = os.environ.get("HERMES_DATASET", "data/sample_trades.psv")
+COLS = ["id","ticker","market_title","nws_forecast","side","edge","ensemble_probability",
+ "pnl","opened_at","cost","count","avg_price","nws_probability","market_price",
+ "grok_probability","actual_outcome","correct","market_type"]
+
+BRACKET_RE = re.compile(r"be\s+\**\s*(-?\d+)\s*-\s*(-?\d+)\s*°")
+DEFAULT_MAE_FALLBACK = 3.0
+
+def f(x):
+ try: return float(x)
+ except (TypeError, ValueError): return None
+
+def load():
+ rows = []
+ with open(PSV) as fh:
+ for line in fh:
+ parts = line.rstrip("\n").split("|")
+ if len(parts) != len(COLS):
+ continue
+ r = dict(zip(COLS, parts))
+ m = BRACKET_RE.search(r["market_title"])
+ if not m:
+ continue
+ lo, hi = int(m.group(1)), int(m.group(2))
+ if lo > hi: lo, hi = hi, lo
+ r["lo"], r["hi"] = lo, hi
+ r["mid"] = (lo + hi) / 2.0
+ r["width"] = hi - lo
+ r["nws"] = f(r["nws_forecast"])
+ r["hit"] = 1 if r["actual_outcome"].strip().lower() == "yes" else 0
+ r["mkt_price"] = f(r["avg_price"])
+ r["mkt_p_hit"] = (1.0 - r["mkt_price"]) if r["mkt_price"] is not None else None
+ r["gauss_p_hit"] = f(r["nws_probability"])
+ r["pnl"] = f(r["pnl"]); r["cost"] = f(r["cost"]); r["cnt"] = f(r["count"])
+ rows.append(r)
+ return rows
+
+def gaussian_p_hit(nws, lo, hi, mae=DEFAULT_MAE_FALLBACK):
+ """Reproduce nws_implied_probability bracket branch."""
+ sigma = mae * math.sqrt(math.pi / 2.0)
+ z_lo = (lo - nws) / sigma
+ z_hi = (hi - nws) / sigma
+ cdf = lambda z: 0.5 * (1.0 + math.erf(z / math.sqrt(2)))
+ return max(0.0, cdf(z_hi) - cdf(z_lo))
+
+def absd_bin(ad):
+
+ for edge in (0.5, 1.5, 2.5, 3.5, 5.0, 8.0):
+ if ad < edge: return edge
+ return 99.0
+
+def build_empirical(train):
+ """P(hit | |d| bin) with Laplace smoothing toward the global base rate."""
+ by_bin = defaultdict(lambda: [0,0])
+ g_hits = g_n = 0
+ for r in train:
+ if r["nws"] is None: continue
+ b = absd_bin(abs(r["nws"] - r["mid"]))
+ by_bin[b][0] += r["hit"]; by_bin[b][1] += 1
+ g_hits += r["hit"]; g_n += 1
+ base = (g_hits / g_n) if g_n else 0.15
+ ALPHA = 4.0
+ table = {}
+ for b,(h,n) in by_bin.items():
+ table[b] = (h + ALPHA*base) / (n + ALPHA)
+ return table, base
+
+def emp_p_hit(table, base, nws, mid):
+ if nws is None: return base
+ return table.get(absd_bin(abs(nws - mid)), base)
+
+def brier(pred, outcome):
+ return (pred - outcome) ** 2
+
+def walk_forward(rows, min_edge):
+ """Chronological. For trade i, train empirical on 0..i-1 only.
+ Simulate: bet NO if p_hit <= mkt_p_hit - min_edge; YES if p_hit >= mkt_p_hit + min_edge."""
+ res = {"gauss": dict(n=0,wins=0,pnl=0.0,brier=[]),
+ "emp": dict(n=0,wins=0,pnl=0.0,brier=[]),
+ "actual_all_brier_g":[], "actual_all_brier_e":[]}
+ usable = [r for r in rows if r["nws"] is not None and r["mkt_p_hit"] is not None
+ and r["pnl"] is not None and r["cnt"]]
+ for i, r in enumerate(usable):
+ train = usable[:i]
+ if len(train) < 15:
+ continue
+ table, base = build_empirical(train)
+ pe = emp_p_hit(table, base, r["nws"], r["mid"])
+ pg = gaussian_p_hit(r["nws"], r["lo"], r["hi"])
+ mp = r["mkt_p_hit"]
+
+ res["actual_all_brier_g"].append(brier(pg, r["hit"]))
+ res["actual_all_brier_e"].append(brier(pe, r["hit"]))
+ per_contract_no_win = (1.0 - r["mkt_price"])
+ for tag, p in (("gauss", pg), ("emp", pe)):
+ d = res[tag]
+ if p <= mp - min_edge:
+ d["n"] += 1
+ won = (r["hit"] == 0)
+ d["pnl"] += (r["cnt"]*per_contract_no_win) if won else (-r["cost"])
+ d["wins"] += 1 if won else 0
+ d["brier"].append(brier(p, r["hit"]))
+ elif p >= mp + min_edge:
+ d["n"] += 1
+ won = (r["hit"] == 1)
+ yes_price = 1.0 - r["mkt_price"]
+ d["pnl"] += (r["cnt"]*(1.0-yes_price)) if won else (-r["cnt"]*yes_price)
+ d["wins"] += 1 if won else 0
+ d["brier"].append(brier(p, r["hit"]))
+ return res, usable
+
+def main():
+ rows = load()
+ print(f"Loaded {len(rows)} bracket rows; "
+ f"{sum(1 for r in rows if r['nws'] is not None)} have NWS forecast.\n")
+
+ print("=== Empirical P(hit) by |NWS - bracket_mid|, vs Gaussian prediction ===")
+ buckets = defaultdict(lambda:[0,0,[]])
+ for r in rows:
+ if r["nws"] is None: continue
+ b = absd_bin(abs(r["nws"]-r["mid"]))
+ buckets[b][0]+=r["hit"]; buckets[b][1]+=1
+ buckets[b][2].append(gaussian_p_hit(r["nws"],r["lo"],r["hi"]))
+ print(f"{'|d|<':>6} {'n':>4} {'hits':>4} {'emp_P(hit)':>10} {'gauss_P(hit)':>12} gap")
+ for b in sorted(buckets):
+ h,n,gs = buckets[b]
+ emp = h/n if n else 0
+ g = statistics.mean(gs) if gs else 0
+ print(f"{b:>6} {n:>4} {h:>4} {emp:>10.3f} {g:>12.3f} {emp-g:+.3f}")
+
+ bg = [brier(gaussian_p_hit(r["nws"],r["lo"],r["hi"]), r["hit"])
+ for r in rows if r["nws"] is not None]
+ print(f"\nGaussian Brier (all {len(bg)} w/ NWS): {statistics.mean(bg):.4f} "
+ f"(0.25=coinflip, lower=better)")
+
+ for thr in (0.45, 0.35, 0.25, 0.15):
+ res, usable = walk_forward(rows, thr)
+ print(f"\n=== WALK-FORWARD @ min_edge={thr} (usable n={len(usable)}, "
+ f"warmup 15) ===")
+ for tag in ("gauss","emp"):
+ d = res[tag]
+ n = d["n"]; wr = (d["wins"]/n*100) if n else 0
+ ev = (d["pnl"]/n) if n else 0
+ bm = statistics.mean(d["brier"]) if d["brier"] else float("nan")
+ print(f" {tag:5s}: trades={n:3d} WR={wr:5.1f}% "
+ f"P&L=${d['pnl']:+7.2f} EV/trade=${ev:+.3f} Brier(traded)={bm:.4f}")
+ if res["actual_all_brier_g"]:
+ print(f" calibration Brier on ALL resolved (walk-fwd): "
+ f"gauss={statistics.mean(res['actual_all_brier_g']):.4f} "
+ f"emp={statistics.mean(res['actual_all_brier_e']):.4f}")
+
+if __name__ == "__main__":
+ main()