| | @@ -0,0 +1,241 @@ |
| + | # Hermes v4.0 - Research Findings & Pending Fixes |
| + | |
| + | **Source:** `kalshi-weather-research.pdf` (compiled March 25, 2026) |
| + | **Date:** 2026-03-26 (updated 2026-04-03) |
| + | **Status:** Fixes 1-2-7 implemented. Bracket fix patch deployed 2026-04-01. Net EV filter deployed 2026-04-03. |
| + | |
| + | --- |
| + | |
| + | ## 2026-04-03 - Net EV Dust Trade Filter (Deployed) |
| + | |
| + | **Root cause found:** Kelly sizing with dampeners can produce bets that buy only 1 contract |
| + | on high-priced markets (e.g. KXHIGHLAX-26APR03-T74: 1 contract @ $0.64 = $0.31 net EV). |
| + | These dust trades waste one of 8 daily trade slots for pennies of expected value. |
| + | |
| + | **Fix deployed (CT-REDACTED + CT-REDACTED):** |
| + | |
| + | | Fix | What | Why | |
| + | |-----|------|-----| |
| + | | `MIN_TRADE_EV_PCT = 0.0015` | New guardrail constant (0.15% of bankroll) | Scales threshold with account size | |
| + | | Net EV gate | `net_ev = contracts × (bet_prob - ask_price) - fees` | Filters on expected dollars, not arbitrary contract counts | |
| + | | Skip + log | Skipped trades logged with EV, threshold, contract count | Full audit trail in scan actions / Discord | |
| + | |
| + | **Why net EV over simpler alternatives:** |
| + | - Min contracts floor (e.g. 3): crude, doesn't account for price differences |
| + | - Raised min bet ($2): ignores whether the trade actually generates value |
| + | - Net EV: directly measures expected dollars per trade slot, captures fee drag, scales with bankroll |
| + | |
| + | **Thresholds at current balances:** |
| + | - Hermes ($348): min EV = $0.52/trade |
| + | - Hermes2 ($49): min EV = $0.07/trade |
| + | |
| + | --- |
| + | |
| + | ## 2026-04-01 - Bracket Fix Patch (Deployed) |
| + | |
| + | **Root cause found:** `_ensemble_gaussian_bracket()` systematically underestimates bracket |
| + | probability when the ensemble has converged (sigma < 2°F). Outliers inflate Gaussian sigma, |
| + | spreading probability outside the bracket. Example: Chicago high ensemble converged to |
| + | 40-41°F range, Gaussian said 31% bracket probability, raw count showed 74%. This inflated |
| + | NO edge above the 0.20 veto ceiling, causing risky bracket trades to bypass Sonnet review. |
| + | |
| + | **4 fixes deployed (CT-REDACTED + CT-REDACTED):** |
| + | |
| + | | Fix | What | Why | |
| + | |-----|------|-----| |
| + | | Hybrid bracket prob | `max(Gaussian, raw_count±0.5°F)` | Gaussian helps far-out; raw catches converged | |
| + | | Bracket veto trigger | Ensemble mean inside bracket → Sonnet review | Safety net for riskiest bracket scenario | |
| + | | Bet sizing hard cap | `min(bet, 8% × bankroll)` after Kelly | Prevents rounding overshoot | |
| + | | Raw prob column | `raw_ensemble_probability` in trades table | Audit: separate raw vs calibrated | |
| + | |
| + | **Rejected after backtesting:** |
| + | - ±2°F NWS guard: Blocked 7 winners, 0 losses = -$16.19 net. NWS distance is NOT a predictor of bracket failure. |
| + | - METAR entry filter: Dead code. Trades placed 12-30h before observations become informative. |
| + | |
| + | **Planned (weekend):** Bracket exit monitor - sells positions when 6 gates confirm edge flip. |
| + | |
| + | **Key data points:** |
| + | - Historical bracket NO: 15W/6L, +$5.56 net, 71% win rate |
| + | - Losses cluster at NWS 3-10°F from bracket (big forecast busts), NOT near-bracket trades |
| + | - The live code already had a 50/50 blend + 5% floor approach - replaced with max() which is more accurate |
| + | |
| + | --- |
| + | |
| + | ## Fixes To Implement (Priority Order) |
| + | |
| + | ### FIX 1: Variable Fee Formula (CRITICAL - blocking profitable trades now) |
| + | **Current:** Flat `TAKER_FEE_PER_CONTRACT = 0.05` applied to all trades. |
| + | **Correct:** `fee = ceil(0.07 * contracts * price * (1 - price))` |
| + | |
| + | | Contract Price | Our Fee (flat) | Actual Fee | We're Wrong By | |
| + | |---------------|---------------|------------|----------------| |
| + | | $0.85 | $0.05 | $0.01 | 5x too high | |
| + | | $0.75 | $0.05 | $0.02 | 2.5x too high | |
| + | | $0.60 | $0.05 | $0.02 | 2.5x too high | |
| + | | $0.50 | $0.05 | $0.02 | 2.5x too high | |
| + | | $0.40 | $0.05 | $0.02 | 2.5x too high | |
| + | |
| + | **Impact:** We're rejecting trades with 7-9% true edge because our inflated fee estimate makes them look below the 10% threshold. This is the single biggest leak in the system right now. |
| + | |
| + | **Implementation:** |
| + | ```python |
| + | def kalshi_taker_fee(price, contracts=1): |
| + | import math |
| + | raw_fee = 0.07 * contracts * price * (1 - price) |
| + | return math.ceil(raw_fee * 100) / 100 |
| + | ``` |
| + | Replace the flat constant everywhere edge is calculated. |
| + | |
| + | --- |
| + | |
| + | ### FIX 2: Reduce Scan Interval from 30 Minutes to 5 Minutes |
| + | **Current:** Scanner runs every 30 minutes. |
| + | **Finding:** The competing bot (suislanchez, $1,325+ profit) scans every 5 minutes. |
| + | **Cost:** Zero. 1,440 API calls/day vs 10,000 limit. |
| + | **Benefit:** Catches mispricing faster, especially after GFS ensemble releases (data available ~3.5h after initialization at 00Z/06Z/12Z/18Z). |
| + | **Risk:** More Sonnet veto calls on Max plan. Mitigated by the filter pipeline - most markets get rejected before reaching Sonnet. |
| + | |
| + | --- |
| + | |
| + | ### FIX 3: Add Maker Orders for Better Fees |
| + | **Current:** All orders are taker (market) orders. |
| + | **Finding:** Maker fee is 25% of taker fee: `ceil(0.0175 * C * P * (1-P))`. At $0.50 contract: taker fee = $0.02, maker fee = $0.01. |
| + | **Implementation:** For trades where market is not about to close (>6h to settlement), place limit orders slightly inside the spread instead of taking the ask. Fall back to taker if not filled within 10 minutes. |
| + | **Complexity:** Medium - requires order monitoring and cancellation logic. |
| + | **Priority:** After fix 1 and 2 are validated. |
| + | |
| + | --- |
| + | |
| + | ### FIX 4: Extremized Aggregation (Replace Simple Calibration Multiply) |
| + | **Current:** `adj_prob = ens_prob * calibration_multiplier` |
| + | **Better:** Combine ensemble + NWS + base rates via log-odds with extremizing factor. |
| + | **Research:** Satopaa et al. (2014), Neyman & Roughgarden (2021) - optimal factor ~1.73 for robust aggregation. |
| + | **Implementation:** |
| + | ```python |
| + | def extremize_aggregate(probabilities, weights=None, factor=1.5): |
| + | import math |
| + | if weights is None: |
| + | weights = [1.0 / len(probabilities)] * len(probabilities) |
| + | clamped = [max(0.001, min(0.999, p)) for p in probabilities] |
| + | log_odds = [math.log(p / (1 - p)) for p in clamped] |
| + | avg_lo = sum(w * lo for w, lo in zip(weights, log_odds)) |
| + | ext_lo = avg_lo * factor |
| + | return 1 / (1 + math.exp(-ext_lo)) |
| + | ``` |
| + | **Notes:** |
| + | - Start with factor 1.5 (conservative - ensemble members share model physics, high info overlap) |
| + | - Weights: 0.5 ensemble, 0.3 NWS, 0.2 historical base rate |
| + | - Factor for weather should be lower than geopolitical (1.73) because ensemble members aren't independent |
| + | **Priority:** After 30 trades validate the current system works. |
| + | |
| + | --- |
| + | |
| + | ### FIX 5: Rain Ensemble Bias Correction |
| + | **Finding:** GFS ensemble over-forecasts light precipitation (false alarm rate too high). Raw member counting overestimates "any rain" probability. |
| + | **Source:** Zhu & Luo (2015), "Precipitation Calibration Based on the Frequency-Matching Method" |
| + | **Implementation:** Maintain rolling 30-day comparison of ensemble rain probability vs observed rain for each city. Apply frequency-matching correction. |
| + | **Priority:** After accumulating 20+ KXRAIN settlements to establish baseline bias. |
| + | |
| + | --- |
| + | |
| + | ### FIX 6: City-Specific Low Temp Adjustments |
| + | **Finding:** Overnight lows are harder to forecast than highs due to radiative cooling, inversions, UHI effects. |
| + | **Risk ranking:** |
| + | | City | Low Temp Risk | Reason | |
| + | |------|-------------|--------| |
| + | | Denver | HIGHEST | Altitude + inversions + dry air + DEN airport 24mi from downtown | |
| + | | Chicago | HIGH | Lake effect + continental + inversion potential | |
| + | | NYC | MEDIUM | UHI 8F+, airport vs Manhattan can differ 5F+ at night | |
| + | | LA | MEDIUM | LAX coastal vs inland can differ 10-15F on summer nights | |
| + | | Miami | LOWEST | Tropical maritime limits radiative cooling | |
| + | |
| + | **Implementation:** Apply higher minimum edge for KXLOWT than KXHIGH. Possible: 12% for low vs 10% for high, with Denver KXLOWT at 15%. |
| + | **Priority:** Can implement now as a constant, tune after data. |
| + | |
| + | --- |
| + | |
| + | ### FIX 7: Reduce Edge Threshold from 10% to 8% |
| + | **Finding:** The suislanchez bot uses 8% edge threshold and is profitable. With the variable fee fix (Fix 1), our true edge calculation becomes more accurate, so a lower threshold is justified. |
| + | **Current:** `MIN_EDGE = 0.10` |
| + | **Proposed:** `MIN_EDGE = 0.08` (matches competitor) |
| + | **Caveat:** Only after Fix 1 (variable fees) is implemented. With the flat $0.05 fee, 8% would let in bad trades. |
| + | **Priority:** Implement together with Fix 1. |
| + | |
| + | --- |
| + | |
| + | ### FIX 8: GFS Ensemble Release-Aware Scanning |
| + | **Finding:** GFS ensemble data becomes available ~3.5h after initialization: |
| + | | Run | Init (UTC) | Data Available | CDT | |
| + | |-----|-----------|----------------|-----| |
| + | | 00Z | 00:00 | ~03:30 UTC | 10:30 PM | |
| + | | 06Z | 06:00 | ~09:30 UTC | 4:30 AM | |
| + | | 12Z | 12:00 | ~15:30 UTC | 10:30 AM | |
| + | | 18Z | 18:00 | ~21:30 UTC | 4:30 PM | |
| + | |
| + | **Implementation:** After switching to 5-minute scans, no special timing needed - the bot naturally picks up new data. But could log which GFS run the ensemble came from for calibration purposes. |
| + | **Priority:** Low - 5-minute scans handle this implicitly. |
| + | |
| + | --- |
| + | |
| + | ## Research Findings (Reference - No Code Changes Needed) |
| + | |
| + | ### FINDING 1: Kalshi Balance Earns 3.75-4% APY |
| + | Kalshi pays yield on total account balance. At $60 this is negligible ($2.25/year), but at $500+ it becomes a consideration - idle cash isn't fully idle. |
| + | |
| + | ### FINDING 2: Maker Fee History - Rounding Exploit Was Real |
| + | Before July 2025, maker fees were flat $0.0025/contract. On $0.02 contracts, this rounded to $0.01 - a 50% effective fee. Kalshi fixed this. Current variable formula eliminates the rounding issue. |
| + | |
| + | ### FINDING 3: Post-2024 Kalshi Regime Change |
| + | Before 2024 Q4, takers made money on average. After Kalshi's legal victory and volume explosion ($30M to $820M/quarter), professional market makers entered. Takers now lose on average. Our edge MUST come from better information (ensemble data), not from market structure. |
| + | |
| + | ### FINDING 4: Weather is in the "Other" Category at ~10% of Volume |
| + | Weather/climate is Kalshi's original niche but only ~10% of total notional volume. Lower volume = potentially wider spreads but also less competition from sophisticated market makers. |
| + | |
| + | ### FINDING 5: Longshot Bias Confirmed with Kalshi Data |
| + | 72.1M trade analysis confirms: contracts below 10% implied probability consistently underperform for buyers. Our $0.40 price floor already exploits this by forcing us to trade in the 40-99 cent range where mispricing exists without the longshot trap. |
| + | |
| + | ### FINDING 6: Becker Dataset Available for Backtesting |
| + | Full Parquet dataset at github.com/Jon-Becker/prediction-market-analysis. Could filter to weather tickers and compute actual historical mispricing, time-of-day effects, and pre/post ensemble release patterns. |
| + | |
| + | ### FINDING 7: NBM May Be Superior to Raw GFS Ensemble |
| + | The National Blend of Models (NBM v4.3) already applies bias correction + quantile mapping to GFS/GEFS/HRRR/ECMWF. Could be used as a third probability source alongside ensemble and NWS for extremized aggregation. |
| + | |
| + | ### FINDING 8: Fan & van den Dool (2011) Key Results |
| + | - GFS ensemble 2m temp errors are dominated by large-scale spatial patterns |
| + | - 30-day mean forecast errors produce more robust bias corrections than 7-day means |
| + | - Cold season shows more removable bias than warm season |
| + | - ~60% of total error variance captured by leading EOF modes |
| + | |
| + | ### FINDING 9: UHI Effect Is Larger at Night |
| + | Urban Heat Island effect is 2-5F warmer at night (more than daytime 1-7F). Since Kalshi settles on airport METAR stations (often outside UHI), the model grid cell may include urban warming the airport doesn't see. This creates a systematic warm bias in low temp ensemble forecasts for urban stations. |
| + | |
| + | ### FINDING 10: GEFS Reforecast Data Available on AWS |
| + | `s3://noaa-gefs-retrospective/GEFSv12/reforecast/` - could build city-specific MAE/bias tables from 2000-present instead of waiting for live trade data. 11 ensemble members (vs 31 operational) but large historical sample. |
| + | |
| + | --- |
| + | |
| + | ## Open-Source Bot Comparison |
| + | |
| + | | Aspect | suislanchez Bot | Hermes v4 | |
| + | |--------|----------------|-----------| |
| + | | Profit | $1,325+ confirmed | $0 (just deployed) | |
| + | | Edge threshold | 8% | 10% (should lower to 8%) | |
| + | | Scan interval | 5 minutes | 30 minutes (should lower to 5) | |
| + | | Kelly fraction | 15% (0.15x) | Variable (0.125-0.375x by confidence) | |
| + | | Markets | KXHIGH only | KXHIGH + KXLOWT + KXRAIN | |
| + | | Fee model | Unknown | Variable formula (pending fix) | |
| + | | NWS cross-check | No | Yes (high, low, rain) | |
| + | | Sonnet veto gate | No | Yes | |
| + | | Correlation guard | No | Yes (2 per city/date) | |
| + | | Calibration | Brier only | Per-type per-city per-season | |
| + | |
| + | --- |
| + | |
| + | ## Implementation Order |
| + | |
| + | | Phase | Fixes | When | |
| + | |-------|-------|------| |
| + | | **Now** | Fix 1 (variable fees) + Fix 7 (lower threshold to 8%) | Immediate | |
| + | | **This week** | Fix 2 (5-min scans) + Fix 6 (city-specific low temp adjustments) | After Fix 1 validated | |
| + | | **After 30 trades** | Fix 4 (extremized aggregation) + Fix 5 (rain bias correction) | Need data first | |
| + | | **After 50 trades** | Fix 3 (maker orders) + Fix 8 (release-aware logging) | Optimization phase | |