The filtered event dict was built with explicit fields and omitted these two columns,
so the frontend always received null despite the DB having valid data.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a one-click upload+parse flow for the TE calendar page HTML.
The TE page (tradingeconomics.com/calendar) contains ~1000 events with
actuals, previous, and analyst consensus (forecast) values.
- Add backend/services/te_html_parser.py:
- parse_html(html): extracts events from <tr data-event> rows
- Maps 14 countries to major currencies (all Eurozone → EUR)
- Impact inferred from data-category (high/medium/low)
- Times converted from Europe/Zurich (CET/CEST) → UTC via zoneinfo
- Actual=td[5], Forecast=td[7] (analyst consensus), Previous=td[6]
- Add POST /api/eco/te-html-upload (saves file to /tmp)
- Add POST /api/eco/te-html-import + GET /status (background parse)
- Add "Import TE HTML" upload button in CalendarPage ImportPanel
Tested locally: 993 events parsed, 854 with forecast, 861 with actual,
date range 2025-03-31 → 2026-06-17, timezone conversion verified.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FXStreet calendar.fxstreet.com/eventdate/ returns ~300-400 events over
6 weeks including consensus forecasts, no authentication required.
FF HTML scraper is blocked by Cloudflare even on residential IPs.
FMP free plan returns 403 on /economic_calendar (requires Starter plan).
- Add backend/services/fxstreet_calendar.py: single GET request returning
all major currencies; maps Volatility 0/1/2 → low/medium/high
- Add POST /api/eco/fxs-sync + GET /api/eco/fxs-sync/status endpoints
- Add FXStreet to daily background sync in main.py (runs every 24h)
- Add "Sync Upcoming (FXStreet)" button in ImportPanel (no key needed)
- Fix FMP 403 error message to say endpoint requires Starter plan
- Keep FMP panel for users who upgrade to FMP Starter ($14.99/month)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TE costs $199/month and its Economic Calendar is not in the free tier.
FMP (Financial Modeling Prep) offers a free API key (250 req/day) with
full economic calendar coverage including consensus estimates (forecasts).
- Add backend/services/fmp_calendar.py: fetches upcoming events from
GET /api/v3/economic_calendar (one request, all countries, date range)
- Replace /api/eco/te-key + te-sync endpoints with fmp-key + fmp-sync
- Update daily background sync in main.py to use fmp_calendar
- Replace TEPanel with FMPPanel in CalendarPage.tsx (link to FMP docs)
- Remove broken Cloudflare-blocked FF HTML scrape button from ImportPanel
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New services/te_calendar.py: fetch_upcoming(weeks_ahead) calls TE API
for 9 countries (USD/EUR/GBP/JPY/AUD/CAD/NZD/CHF/CNY), converts to
ff_calendar format, upserts with source='te_api'
- New endpoints: GET/POST /api/eco/te-key, POST /api/eco/te-sync,
GET /api/eco/te-sync/status
- Daily scheduler in main.py: FF live sync + TE sync (if key configured)
run 60s after startup then every 24h
- CalendarPage: TEPanel with key input (password field, Enter to save,
"Get free key" link to tradingeconomics.com/api/login),
"Sync upcoming (6 weeks)" button with polling
FF HTML scraper kept as fallback but TE API is the primary source
for upcoming forecasts (no Cloudflare blocking on server IPs).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Backend: /api/eco/calendar accepts date_from + date_to query params
when period=custom; get_calendar() uses them directly without override
Limit raised to 5000 for wide date ranges
- Frontend: "Custom" tab at end of period list; shows two date inputs
(from/to) with Apply button; displays day count; fetches on Apply click
(not on every keystroke to avoid hammering the API)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- scrape_upcoming(weeks_ahead=5) in ff_calendar.py:
fetches forexfactory.com/calendar?week=... HTML for N weeks ahead,
parses calendar__table (date/time/currency/impact/event/forecast/previous),
converts ET times to UTC, upserts into ff_calendar
- Daily scheduler in main.py: runs scrape_upcoming at startup (after 30s delay)
then every 24h — no manual action needed
- New endpoints: POST /api/eco/ff-scrape?weeks=5, GET /api/eco/ff-scrape/status
- CalendarPage: "Scrape Upcoming (5w)" button (indigo) with polling + result
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backend:
- ff_calendar: add series_id column (migration) + FF_TO_FRED mapping dict
(NFP→PAYEMS, CPI→CPIAUCSL, Jobless Claims→ICSA, GDP→GDPC1, FEDFUNDS, PCE)
- import_csv + sync_live now populate series_id on each FF event
- New GET /api/eco/series/{id}/history: FRED time series + linked FF events
(surprises, forecast, actual) merged by date — enables context queries
Frontend:
- New MacroSeriesPage.tsx: sidebar with 11 FRED series grouped by category,
recharts ComposedChart with area + z-score surprise reference lines (|z|≥1.5),
KPI cards (latest/prev/min/max), FF events table (actual vs forecast coloring),
z-score bar chart for recent surprises, range selector (1Y/2Y/5Y/10Y/All)
- Route /macro-series + Sidebar entry "Macro Series"
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New ff_calendar table (event_date, time, currency, impact, actual, forecast, previous)
- New service ff_calendar.py: bulk CSV import (83K events 2007-2025) + live sync
from faireconomy.media JSON endpoint (this week / next week)
- New API endpoints: POST /api/eco/ff-import, POST /api/eco/ff-sync,
GET /api/eco/calendar (period filter), GET /api/eco/ff-stats
- CalendarPage.tsx full rewrite: period tabs (Recent/Today/Tomorrow/This Week…),
currency flags filter, impact filter, unified date-grouped table with
Time·Flag·Currency·Impact·Event·Actual·Forecast·Previous columns,
green/red actual vs forecast, TODAY badge, auto-refresh 60s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- fred_bootstrap.py: switch from CSV graph endpoint (CloudFlare-blocked)
to api.stlouisfed.org/fred/series/observations JSON API; reads key
from DB config 'fred_api_key'; returns clear error if key missing
- eco.py: add GET/POST /api/eco/fred-key to check/save the FRED API key
- CalendarPage.tsx: BootstrapPanel shows API key section with status,
input to paste key + save button, disables Launch if key missing
Free key: fred.stlouisfed.org -> My Account -> API Keys
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FRED blocks requests without a valid User-Agent. Added headers matching
a real browser. Also added print() calls so fetch errors appear in
docker logs (logger.warning goes to system_logs table, not stdout).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GDPC1 replaces A191RL1Q225SBEA: compute annualized QoQ growth from
GDP level ((val/prev)^4 - 1)*100 → displays proper ~2-3% not 31 819
- CPIAUCSL/CPILFESL/PCEPILFE: yoy_pct transform (val/val_12m_ago-1)*100
→ displays 3.x% YoY inflation, not raw index level 334
- ICSA: div1000 transform → displays 226 K claims, not 226 000 K
- delta_absolute flag: pp change for rate/% series, % change for levels
- SurprisePct component: shows 'pp' suffix for %, '%' for K/levels
- Column header renamed from 'Δ%' to 'Δ vs préc.' with tooltip
- Deprecated A191RL1Q225SBEA rows cleaned from DB on next bootstrap
- Warm-up periods: 2yr for yoy_pct, 3yr for qoq_annualized, 1yr others
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- backend/services/fred_bootstrap.py: fetch 11 FRED series (PAYEMS, UNRATE, CPI, PCE, FEDFUNDS, ICSA, GDP, HY spread, T10Y2Y, T10Y3M) from public CSV endpoint — no API key needed; computes rolling z-scores and upserts into economic_events table
- backend/routers/eco.py: new /api/eco router with bootstrap (POST + status GET), events list with full filtering (date range, category, series, min z-score, direction, sort/pagination), series catalog, and db status endpoints
- backend/main.py: register eco router
- frontend/src/pages/CalendarPage.tsx: complete rewrite — real data table from /api/eco/events, Bootstrap FRED button with live polling, filter bar (date range, category, series chips, |z| threshold, direction), sort by date/z-score/series, pagination, z-score badges with color coding, sidebar with series inventory + geo alerts + z-score guide
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Database migration:
- Add 'origin' and 'source_refs' columns to market_events ALTER TABLE migration
(sub_type/actual_value/expected_value/surprise_pct were already there)
- All new tables (macro_gauge_snapshots, ai_desks) created via CREATE TABLE IF NOT EXISTS
on next init_db() call (container restart)
Backend:
- GET /api/market-events/db-status — health check returning row counts,
latest dates, and missing columns for all 6 tables needed by the detector
- list_events() now accepts gen_date_from / gen_date_to query params
filtering by date(created_at) — separate from start_date event date filters
Frontend (MarketEvents.tsx):
- MarketEvent interface: add created_at field
- EventRow: show generation date as ⚡MM-DD next to event date
- Extended filters: new '⚡ Date de génération' section with from/to inputs
filtered independently from the event date range
- Clear-all button includes genFrom/genTo reset
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each macro gauge selected in the Sentiment desk can now be independently
configured with three alert criteria:
- Seuil bas (≤): alert when value crosses below
- Seuil haut (≥): alert when value crosses above
- Variation % (Δ%): alert when N-day % change exceeds threshold
Frontend: GaugeThresholdConfig component — one row per selected gauge,
compact grid layout with enable toggle + 3 numeric inputs.
Stored in config.gauge_thresholds[gauge_id].
Backend: _check_sentiment() extended — after CBOE signals, reads
macro_gauge_snapshots history, checks each enabled gauge threshold,
emits sentiment market_events with options_note for each breach.
Gauge → affected_assets mapping covers all 32 gauge keys.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Mark 6 price signals (ma_cross, rsi_extreme, bb_squeeze, new_52w_extreme, price_gap, volume_spike) as desk_type=technical so they no longer appear in the Sentiment desk
- Sentiment desk now shows MacroGaugeSelector (32 gauges grouped by bloc: Liquidité, Crédit, Volatilité, Métaux…) instead of the stock/ETF instrument picker
- Sentiment desk instruments seed updated to macro gauge keys (vix, vvix, skew, hyg, dxy, slope_10y3m, gold_copper_ratio)
- Signal init useEffect extended to cover sentiment desk as well as technical
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
DB:
- New table macro_gauge_snapshots (daily snapshot of all 28+ gauges + dominant + scores)
- save_macro_gauge_snapshot / get_macro_gauge_snapshot_at / get_macro_gauge_history
- Auto-save once per calendar day on every macro-regime fetch (not just force=True)
API:
- GET /api/market/macro-gauges/at?date=YYYY-MM-DD — nearest snapshot ≤ date
- GET /api/market/macro-gauges/history?days=N
Detector (_check_macro_gauges in Eco Desk):
- Regime transition events (goldilocks→stagflation etc.) with severity scoring
- Yield curve inversion / désinversion (slope_10y3m sign change)
- DXY shock (% change over lookback window)
- Credit stress (HYG drop threshold)
- Gold/Copper ratio regime crossings
InstrumentDashboard:
- macroAtDate state: fetches /api/market/macro-gauges/at when crosshair date ≠ last date
- RegimeCard uses historical macro regime when on a past date
- MacroGaugePanel: full breakdown of all gauges by bloc (liquidité, crédit, énergie...)
visible only when on a historical date — shows value + change_pct + regime scores bar
AIDesks: added fundamental + sentiment to AIDesk type
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New ai_desks table with CRUD (get_all/by_type/upsert/delete)
- ai_desks router: REST API + GET /signal-catalog (7 extensible signals)
- News Desk: semantic dedup via AI (±N days window, system_prompt hint)
- Technical Desk: 4 signal detectors driven by desk config
(ma_cross, rsi_extreme, bb_squeeze, new_52w_extreme)
- 3 more signals in catalog ready to enable: price_gap, volume_spike, macd_crossover
- market_event_detector.py loads desk configs at runtime, falls back to legacy params
- AIDesks.tsx: full editor UI with signal toggles, param sliders, instrument multi-select
- Sidebar: Bot icon + /ai-desks route
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backend: merge strategy — category defaults are applied as baseline
(ai_generated=0), AI impacts override/extend them (ai_generated=1).
All N category defaults appear in the impacts list; AI covers them
explicitly with event-specific score/direction adjustments.
Prompt: instruct AI to cover ALL default instruments.
Frontend: remove default impact chips under category selector —
they now appear directly in the instrument impacts list.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
