Adds a one-click upload+parse flow for the TE calendar page HTML.
The TE page (tradingeconomics.com/calendar) contains ~1000 events with
actuals, previous, and analyst consensus (forecast) values.
- Add backend/services/te_html_parser.py:
- parse_html(html): extracts events from <tr data-event> rows
- Maps 14 countries to major currencies (all Eurozone → EUR)
- Impact inferred from data-category (high/medium/low)
- Times converted from Europe/Zurich (CET/CEST) → UTC via zoneinfo
- Actual=td[5], Forecast=td[7] (analyst consensus), Previous=td[6]
- Add POST /api/eco/te-html-upload (saves file to /tmp)
- Add POST /api/eco/te-html-import + GET /status (background parse)
- Add "Import TE HTML" upload button in CalendarPage ImportPanel
Tested locally: 993 events parsed, 854 with forecast, 861 with actual,
date range 2025-03-31 → 2026-06-17, timezone conversion verified.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FXStreet calendar.fxstreet.com/eventdate/ returns ~300-400 events over
6 weeks including consensus forecasts, no authentication required.
FF HTML scraper is blocked by Cloudflare even on residential IPs.
FMP free plan returns 403 on /economic_calendar (requires Starter plan).
- Add backend/services/fxstreet_calendar.py: single GET request returning
all major currencies; maps Volatility 0/1/2 → low/medium/high
- Add POST /api/eco/fxs-sync + GET /api/eco/fxs-sync/status endpoints
- Add FXStreet to daily background sync in main.py (runs every 24h)
- Add "Sync Upcoming (FXStreet)" button in ImportPanel (no key needed)
- Fix FMP 403 error message to say endpoint requires Starter plan
- Keep FMP panel for users who upgrade to FMP Starter ($14.99/month)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TE costs $199/month and its Economic Calendar is not in the free tier.
FMP (Financial Modeling Prep) offers a free API key (250 req/day) with
full economic calendar coverage including consensus estimates (forecasts).
- Add backend/services/fmp_calendar.py: fetches upcoming events from
GET /api/v3/economic_calendar (one request, all countries, date range)
- Replace /api/eco/te-key + te-sync endpoints with fmp-key + fmp-sync
- Update daily background sync in main.py to use fmp_calendar
- Replace TEPanel with FMPPanel in CalendarPage.tsx (link to FMP docs)
- Remove broken Cloudflare-blocked FF HTML scrape button from ImportPanel
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New services/te_calendar.py: fetch_upcoming(weeks_ahead) calls TE API
for 9 countries (USD/EUR/GBP/JPY/AUD/CAD/NZD/CHF/CNY), converts to
ff_calendar format, upserts with source='te_api'
- New endpoints: GET/POST /api/eco/te-key, POST /api/eco/te-sync,
GET /api/eco/te-sync/status
- Daily scheduler in main.py: FF live sync + TE sync (if key configured)
run 60s after startup then every 24h
- CalendarPage: TEPanel with key input (password field, Enter to save,
"Get free key" link to tradingeconomics.com/api/login),
"Sync upcoming (6 weeks)" button with polling
FF HTML scraper kept as fallback but TE API is the primary source
for upcoming forecasts (no Cloudflare blocking on server IPs).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Backend: /api/eco/calendar accepts date_from + date_to query params
when period=custom; get_calendar() uses them directly without override
Limit raised to 5000 for wide date ranges
- Frontend: "Custom" tab at end of period list; shows two date inputs
(from/to) with Apply button; displays day count; fetches on Apply click
(not on every keystroke to avoid hammering the API)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- scrape_upcoming(weeks_ahead=5) in ff_calendar.py:
fetches forexfactory.com/calendar?week=... HTML for N weeks ahead,
parses calendar__table (date/time/currency/impact/event/forecast/previous),
converts ET times to UTC, upserts into ff_calendar
- Daily scheduler in main.py: runs scrape_upcoming at startup (after 30s delay)
then every 24h — no manual action needed
- New endpoints: POST /api/eco/ff-scrape?weeks=5, GET /api/eco/ff-scrape/status
- CalendarPage: "Scrape Upcoming (5w)" button (indigo) with polling + result
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backend:
- ff_calendar: add series_id column (migration) + FF_TO_FRED mapping dict
(NFP→PAYEMS, CPI→CPIAUCSL, Jobless Claims→ICSA, GDP→GDPC1, FEDFUNDS, PCE)
- import_csv + sync_live now populate series_id on each FF event
- New GET /api/eco/series/{id}/history: FRED time series + linked FF events
(surprises, forecast, actual) merged by date — enables context queries
Frontend:
- New MacroSeriesPage.tsx: sidebar with 11 FRED series grouped by category,
recharts ComposedChart with area + z-score surprise reference lines (|z|≥1.5),
KPI cards (latest/prev/min/max), FF events table (actual vs forecast coloring),
z-score bar chart for recent surprises, range selector (1Y/2Y/5Y/10Y/All)
- Route /macro-series + Sidebar entry "Macro Series"
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New ff_calendar table (event_date, time, currency, impact, actual, forecast, previous)
- New service ff_calendar.py: bulk CSV import (83K events 2007-2025) + live sync
from faireconomy.media JSON endpoint (this week / next week)
- New API endpoints: POST /api/eco/ff-import, POST /api/eco/ff-sync,
GET /api/eco/calendar (period filter), GET /api/eco/ff-stats
- CalendarPage.tsx full rewrite: period tabs (Recent/Today/Tomorrow/This Week…),
currency flags filter, impact filter, unified date-grouped table with
Time·Flag·Currency·Impact·Event·Actual·Forecast·Previous columns,
green/red actual vs forecast, TODAY badge, auto-refresh 60s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- fred_bootstrap.py: switch from CSV graph endpoint (CloudFlare-blocked)
to api.stlouisfed.org/fred/series/observations JSON API; reads key
from DB config 'fred_api_key'; returns clear error if key missing
- eco.py: add GET/POST /api/eco/fred-key to check/save the FRED API key
- CalendarPage.tsx: BootstrapPanel shows API key section with status,
input to paste key + save button, disables Launch if key missing
Free key: fred.stlouisfed.org -> My Account -> API Keys
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FRED blocks requests without a valid User-Agent. Added headers matching
a real browser. Also added print() calls so fetch errors appear in
docker logs (logger.warning goes to system_logs table, not stdout).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GDPC1 replaces A191RL1Q225SBEA: compute annualized QoQ growth from
GDP level ((val/prev)^4 - 1)*100 → displays proper ~2-3% not 31 819
- CPIAUCSL/CPILFESL/PCEPILFE: yoy_pct transform (val/val_12m_ago-1)*100
→ displays 3.x% YoY inflation, not raw index level 334
- ICSA: div1000 transform → displays 226 K claims, not 226 000 K
- delta_absolute flag: pp change for rate/% series, % change for levels
- SurprisePct component: shows 'pp' suffix for %, '%' for K/levels
- Column header renamed from 'Δ%' to 'Δ vs préc.' with tooltip
- Deprecated A191RL1Q225SBEA rows cleaned from DB on next bootstrap
- Warm-up periods: 2yr for yoy_pct, 3yr for qoq_annualized, 1yr others
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- backend/services/fred_bootstrap.py: fetch 11 FRED series (PAYEMS, UNRATE, CPI, PCE, FEDFUNDS, ICSA, GDP, HY spread, T10Y2Y, T10Y3M) from public CSV endpoint — no API key needed; computes rolling z-scores and upserts into economic_events table
- backend/routers/eco.py: new /api/eco router with bootstrap (POST + status GET), events list with full filtering (date range, category, series, min z-score, direction, sort/pagination), series catalog, and db status endpoints
- backend/main.py: register eco router
- frontend/src/pages/CalendarPage.tsx: complete rewrite — real data table from /api/eco/events, Bootstrap FRED button with live polling, filter bar (date range, category, series chips, |z| threshold, direction), sort by date/z-score/series, pagination, z-score badges with color coding, sidebar with series inventory + geo alerts + z-score guide
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Database migration:
- Add 'origin' and 'source_refs' columns to market_events ALTER TABLE migration
(sub_type/actual_value/expected_value/surprise_pct were already there)
- All new tables (macro_gauge_snapshots, ai_desks) created via CREATE TABLE IF NOT EXISTS
on next init_db() call (container restart)
Backend:
- GET /api/market-events/db-status — health check returning row counts,
latest dates, and missing columns for all 6 tables needed by the detector
- list_events() now accepts gen_date_from / gen_date_to query params
filtering by date(created_at) — separate from start_date event date filters
Frontend (MarketEvents.tsx):
- MarketEvent interface: add created_at field
- EventRow: show generation date as ⚡MM-DD next to event date
- Extended filters: new '⚡ Date de génération' section with from/to inputs
filtered independently from the event date range
- Clear-all button includes genFrom/genTo reset
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each macro gauge selected in the Sentiment desk can now be independently
configured with three alert criteria:
- Seuil bas (≤): alert when value crosses below
- Seuil haut (≥): alert when value crosses above
- Variation % (Δ%): alert when N-day % change exceeds threshold
Frontend: GaugeThresholdConfig component — one row per selected gauge,
compact grid layout with enable toggle + 3 numeric inputs.
Stored in config.gauge_thresholds[gauge_id].
Backend: _check_sentiment() extended — after CBOE signals, reads
macro_gauge_snapshots history, checks each enabled gauge threshold,
emits sentiment market_events with options_note for each breach.
Gauge → affected_assets mapping covers all 32 gauge keys.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Mark 6 price signals (ma_cross, rsi_extreme, bb_squeeze, new_52w_extreme, price_gap, volume_spike) as desk_type=technical so they no longer appear in the Sentiment desk
- Sentiment desk now shows MacroGaugeSelector (32 gauges grouped by bloc: Liquidité, Crédit, Volatilité, Métaux…) instead of the stock/ETF instrument picker
- Sentiment desk instruments seed updated to macro gauge keys (vix, vvix, skew, hyg, dxy, slope_10y3m, gold_copper_ratio)
- Signal init useEffect extended to cover sentiment desk as well as technical
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
DB:
- New table macro_gauge_snapshots (daily snapshot of all 28+ gauges + dominant + scores)
- save_macro_gauge_snapshot / get_macro_gauge_snapshot_at / get_macro_gauge_history
- Auto-save once per calendar day on every macro-regime fetch (not just force=True)
API:
- GET /api/market/macro-gauges/at?date=YYYY-MM-DD — nearest snapshot ≤ date
- GET /api/market/macro-gauges/history?days=N
Detector (_check_macro_gauges in Eco Desk):
- Regime transition events (goldilocks→stagflation etc.) with severity scoring
- Yield curve inversion / désinversion (slope_10y3m sign change)
- DXY shock (% change over lookback window)
- Credit stress (HYG drop threshold)
- Gold/Copper ratio regime crossings
InstrumentDashboard:
- macroAtDate state: fetches /api/market/macro-gauges/at when crosshair date ≠ last date
- RegimeCard uses historical macro regime when on a past date
- MacroGaugePanel: full breakdown of all gauges by bloc (liquidité, crédit, énergie...)
visible only when on a historical date — shows value + change_pct + regime scores bar
AIDesks: added fundamental + sentiment to AIDesk type
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New ai_desks table with CRUD (get_all/by_type/upsert/delete)
- ai_desks router: REST API + GET /signal-catalog (7 extensible signals)
- News Desk: semantic dedup via AI (±N days window, system_prompt hint)
- Technical Desk: 4 signal detectors driven by desk config
(ma_cross, rsi_extreme, bb_squeeze, new_52w_extreme)
- 3 more signals in catalog ready to enable: price_gap, volume_spike, macd_crossover
- market_event_detector.py loads desk configs at runtime, falls back to legacy params
- AIDesks.tsx: full editor UI with signal toggles, param sliders, instrument multi-select
- Sidebar: Bot icon + /ai-desks route
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backend: merge strategy — category defaults are applied as baseline
(ai_generated=0), AI impacts override/extend them (ai_generated=1).
All N category defaults appear in the impacts list; AI covers them
explicitly with event-specific score/direction adjustments.
Prompt: instruct AI to cover ALL default instruments.
Frontend: remove default impact chips under category selector —
they now appear directly in the instrument impacts list.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The UPDATE statement listed all columns except sub_type, so the
AI-matched category auto-set was written to the object but never
persisted to the DB. The category dropdown stayed blank after evaluate.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The AI was returning 'geopolitical' (the event type/family) instead of
a category name like 'Guerre — Moyen-Orient'. Two fixes:
1. Prompt: rename 'Type' → 'Famille', list categories as CAT_N: "name",
explicit warning not to return a generic type, show valid name examples
2. Code: reject any response that matches a known type string before matching
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The AI often returns 'Guerre - Moyen-Orient' (ASCII dash) instead of
'Guerre — Moyen-Orient' (em-dash), causing exact match to fail silently.
Fix: normalize unicode, collapse all dash variants, then try
exact → normalized → partial substring matching.
Also log the raw AI response for debugging, increase max_tokens 1200→2000.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously: heuristic text matching (sub_type in category.name) — failed
for semantic cases like 'Iran Ceasefire Talks' → 'Guerre — Moyen-Orient'.
Now: all EventCategories are sent in the prompt (name + type + description).
The AI picks matched_category by semantic understanding, returns it in JSON.
- Validates returned name against known categories (case-insensitive)
- Auto-sets event sub_type when AI finds a match and sub_type was empty
- Returns matched_category in evaluate response + shown in UI eval message
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_find_cat() returned only default_impacts, never the category name.
Now returns (name, defaults) tuple; cat_name → matched_cat_name.
Fixes: 'name cat_name is not defined' on AI evaluate.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- DB: colonne origin (migration + UPDATE heuristique sur données legacy)
- save/update_market_event: persist origin
- Tous les points de création taguent leur origine:
bootstrap_macro/eco/ma/legacy | detector_news/eco/technical/report | manual
- UI MarketEvents: badge d'origine avec icône + description dans le panneau détail,
icône tooltip dans la liste gauche, message explicite si pas de source_refs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add macro_regime (goldilocks/stagflation/recession/etc.) to every instrument snapshot via get_macro_gauges() + score_macro_scenarios()
- RegimeCard now shows global macro cycle section (emoji + label + top-3 scenarios) above technical signals
- Fix _detect_regime() confidence: capped at 85% max; add late-bull (dist_MA200 > 20%) and correction-in-bull (MA50 > MA200 but momentum < -3%) detection so regime no longer locks at 100%
- Add macro_events_bootstrap.py with 30 curated historical events (FOMC 2022-2025, CPI surprises, Ukraine/Hamas/Iran geopolitics, BOJ pivots, Bitcoin ETF, Liberation Day tariffs, SVB crisis, etc.)
- POST /api/timeline/bootstrap-macro endpoint (idempotent, deduplicates by name)
- Fix event date filter in _get_relevant_events(): overlap logic instead of start-only filter — events extending into the chart window are now included
- EventTimelineStrip: add "Signaux Techniques" fallback row for events not matched by any driver keyword (MA crossovers are now always visible)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- instruments.json: add keywords array to every driver across 20 instruments
(Fed, BCE, BOJ, OPEC, CPI, AI, EIA, etc.) for event-to-driver matching
- instrument_service.py: add update_instrument_drivers() persisting changes to JSON
and refreshing in-memory cache
- instruments.py: add PUT /api/instruments/{id}/drivers endpoint (DriverUpdate model)
- InstrumentDashboard:
* RegimeCard: replace regime score bars with 6-metric signal grid
(MA50/MA200 position, MA50 slope, MA200 slope, momentum 20j, dist MA200, ATR vol ratio)
with colour-coded values and contextual sub-labels (Golden cross, Surextension, etc.)
* EventTimelineStrip: rows now keyed by top-4 instrument drivers (by weight)
instead of LT/MT/CT; events matched via case-insensitive keyword scan against
title + description + category; fallback dashed line when no events match
* DriversPanel: inline edit panel (toggle via Drivers button in header);
edit label, weight, keywords (comma-separated) per driver; add/remove drivers;
saves via PUT /api/instruments/{id}/drivers; optimistic local state update
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- InstrumentChart: onDateHover callback via subscribeCrosshairMove (useRef pattern)
- EventTimelineStrip: 3 rows LT/MT/CT with CSS-% bars aligned to chart X axis
- Cards date-aware: crosshair drives selectedDate; dateTrend + dateSignals computed
client-side from lookup maps (priceMap/indMap/sortedDates) without extra API calls
- TrendCard: price, RSI, ATR, slopes, momentum, 52W range all at selected date
- RegimeCard: 5 signals recomputed at selected date; regime label from server
- Date badge above cards; blue tint when browsing history, grey on last date
- instrument_service.py: end_date in events; price_data built before events block
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The docker-compose mounts db_data named volume at /app/data which hid the
instruments.json file baked into the image. Moving to /app/config which is
not volume-overlaid resolves the FileNotFoundError on startup.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- TimelineVertical: replace horizontal frise with Y=time vertical layout,
3 columns (Long/Medium/Short), auto-scroll to selected date, sub-columns
for overlapping events, today/selected-date horizontal lines
- ma_analyzer.py: detect MA50/MA200 crossovers + MA100 slope changes +
MA20 direction swings on EUR/USD, Brent, Gold, S&P500, US10Y (5y history)
with 5-bar confirmation, dedup, GPT-4o-mini enrichment, idempotent DB save
- POST /api/timeline/bootstrap-ma endpoint to trigger analysis
- Bootstrap MA button in Timeline page with loading state + result count
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Frise chronologique:
- Sub-lane stacking (assignSubLanes) — overlapping events se décalent verticalement
- Zone d'overlap semi-transparente sur la période commune entre 2 événements
- Hauteur dynamique selon nb de sub-lanes par niveau
- Événements en cours avec flèche ▶ à droite, gradient de fin
- Tri par start_date pour placement greedy
Event Manager (composant EventManager.tsx):
- Tableau filtrable par niveau (Long/Moyen/Court)
- Edit modal complet : tous les champs + absorption_pct éditable
- Bouton "IA — Enrichir" par événement → POST /api/timeline/events/{id}/ai-enrich
→ GPT-4o-mini suggère absorption_pct + indicateurs pertinents par niveau temporel
- Delete avec confirmation double-clic
- Expand row pour voir description + indicateurs
- Intégré Timeline page via bouton "Gérer événements"
Backend:
- Nouvelles colonnes market_events: absorption_pct + relevant_indicators (ALTER idempotent)
- DELETE /api/timeline/events/{id}
- POST /api/timeline/events/{id}/ai-enrich
Snapshot Externe:
- AbsorptionBar par événement dans cellule Géopolitique
- MA indicators : fetch 200j history, compute MA10/MA20/MA100 per level (short/med/long)
- Affichage prix vs MA + % écart dans CellMarkets
- Si relevant_indicators configurés sur l'event → utilise ces symbols au lieu des défauts
- Calendar : horizons exclusifs (short 0-7j, medium 8-30j, long 31-90j) — bug corrigé
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two root causes in the logs:
1. fetch_forward_curves() tried 5 offsets × 8 commodities = 40 individual
yfinance requests for monthly contracts (CLN26, GCQ26, etc.) that Yahoo
Finance does not support — generating ERROR storm and triggering hard
rate limiting that cascades onto front-month CL=F/GC=F calls used by
the main cycle.
2. Ticker format lacked exchange suffix (.NYM/.CMX/.CBT).
Fix: replace the per-ticker loop with two batch yfinance.download() calls
(one for all 8 front-months, one for all deferred candidates). Failed
deferred lookups are logged at DEBUG level and reported as structure='unknown'
rather than ERROR, since Yahoo Finance does not expose monthly commodity
contracts reliably. Added 1s sleep between batches to avoid rate spiking.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace per-endpoint market name matching with unified contract code
lookup on the legacy Socrata endpoint (6dca-aqww.json). COMEX became
"COMMODITY EXCHANGE INC.", CBOT became "CHICAGO BOARD OF TRADE", and
the disaggregated endpoints stopped receiving Natural Gas / financial
instruments after Feb 2022. Contract codes (067651, 023651, etc.) are
stable across rebranding. Fetch now returns 19/19 markets dated
2026-06-16.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
suggest_patterns_from_market_context() was missing the specialist desk
block that score_patterns_with_context() already received. All 7 desks
(forex, metals, agri, energy, indices, crypto, bonds) with their
fundamentals, macro sensitivity, and upcoming reports are now injected
so the AI can generate targeted patterns per desk rather than generic ones.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
On restart the scheduler was counting interval_hours from now, ignoring
when the last cycle actually ran. It now reads last_run_at (in-memory or
DB) and deducts elapsed time so a restart doesn't silently push the next
fire by a full interval.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Detect yfinance quote_type (CURRENCY→forex, FUTURE→energy, INDEX→indices,
ETF→etfs, EQUITY→equities) when adding a custom ticker and persist it in
market_watchlist.asset_class
- get_all_quotes() merges custom tickers into their proper group (e.g. EURUSD=X
appears under Forex) instead of always under a separate "Custom" group
- "Custom" tab only shows tickers whose type couldn't be detected
- Add market_watchlist.asset_class migration; ensure backtest_lab_runs and
market_watchlist are always created at init_db() time
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
