The filtered event dict was built with explicit fields and omitted these two columns,
so the frontend always received null despite the DB having valid data.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a one-click upload+parse flow for the TE calendar page HTML.
The TE page (tradingeconomics.com/calendar) contains ~1000 events with
actuals, previous, and analyst consensus (forecast) values.
- Add backend/services/te_html_parser.py:
- parse_html(html): extracts events from <tr data-event> rows
- Maps 14 countries to major currencies (all Eurozone → EUR)
- Impact inferred from data-category (high/medium/low)
- Times converted from Europe/Zurich (CET/CEST) → UTC via zoneinfo
- Actual=td[5], Forecast=td[7] (analyst consensus), Previous=td[6]
- Add POST /api/eco/te-html-upload (saves file to /tmp)
- Add POST /api/eco/te-html-import + GET /status (background parse)
- Add "Import TE HTML" upload button in CalendarPage ImportPanel
Tested locally: 993 events parsed, 854 with forecast, 861 with actual,
date range 2025-03-31 → 2026-06-17, timezone conversion verified.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FXStreet calendar.fxstreet.com/eventdate/ returns ~300-400 events over
6 weeks including consensus forecasts, no authentication required.
FF HTML scraper is blocked by Cloudflare even on residential IPs.
FMP free plan returns 403 on /economic_calendar (requires Starter plan).
- Add backend/services/fxstreet_calendar.py: single GET request returning
all major currencies; maps Volatility 0/1/2 → low/medium/high
- Add POST /api/eco/fxs-sync + GET /api/eco/fxs-sync/status endpoints
- Add FXStreet to daily background sync in main.py (runs every 24h)
- Add "Sync Upcoming (FXStreet)" button in ImportPanel (no key needed)
- Fix FMP 403 error message to say endpoint requires Starter plan
- Keep FMP panel for users who upgrade to FMP Starter ($14.99/month)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TE costs $199/month and its Economic Calendar is not in the free tier.
FMP (Financial Modeling Prep) offers a free API key (250 req/day) with
full economic calendar coverage including consensus estimates (forecasts).
- Add backend/services/fmp_calendar.py: fetches upcoming events from
GET /api/v3/economic_calendar (one request, all countries, date range)
- Replace /api/eco/te-key + te-sync endpoints with fmp-key + fmp-sync
- Update daily background sync in main.py to use fmp_calendar
- Replace TEPanel with FMPPanel in CalendarPage.tsx (link to FMP docs)
- Remove broken Cloudflare-blocked FF HTML scrape button from ImportPanel
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New services/te_calendar.py: fetch_upcoming(weeks_ahead) calls TE API
for 9 countries (USD/EUR/GBP/JPY/AUD/CAD/NZD/CHF/CNY), converts to
ff_calendar format, upserts with source='te_api'
- New endpoints: GET/POST /api/eco/te-key, POST /api/eco/te-sync,
GET /api/eco/te-sync/status
- Daily scheduler in main.py: FF live sync + TE sync (if key configured)
run 60s after startup then every 24h
- CalendarPage: TEPanel with key input (password field, Enter to save,
"Get free key" link to tradingeconomics.com/api/login),
"Sync upcoming (6 weeks)" button with polling
FF HTML scraper kept as fallback but TE API is the primary source
for upcoming forecasts (no Cloudflare blocking on server IPs).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Backend: /api/eco/calendar accepts date_from + date_to query params
when period=custom; get_calendar() uses them directly without override
Limit raised to 5000 for wide date ranges
- Frontend: "Custom" tab at end of period list; shows two date inputs
(from/to) with Apply button; displays day count; fetches on Apply click
(not on every keystroke to avoid hammering the API)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- scrape_upcoming(weeks_ahead=5) in ff_calendar.py:
fetches forexfactory.com/calendar?week=... HTML for N weeks ahead,
parses calendar__table (date/time/currency/impact/event/forecast/previous),
converts ET times to UTC, upserts into ff_calendar
- Daily scheduler in main.py: runs scrape_upcoming at startup (after 30s delay)
then every 24h — no manual action needed
- New endpoints: POST /api/eco/ff-scrape?weeks=5, GET /api/eco/ff-scrape/status
- CalendarPage: "Scrape Upcoming (5w)" button (indigo) with polling + result
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backend:
- ff_calendar: add series_id column (migration) + FF_TO_FRED mapping dict
(NFP→PAYEMS, CPI→CPIAUCSL, Jobless Claims→ICSA, GDP→GDPC1, FEDFUNDS, PCE)
- import_csv + sync_live now populate series_id on each FF event
- New GET /api/eco/series/{id}/history: FRED time series + linked FF events
(surprises, forecast, actual) merged by date — enables context queries
Frontend:
- New MacroSeriesPage.tsx: sidebar with 11 FRED series grouped by category,
recharts ComposedChart with area + z-score surprise reference lines (|z|≥1.5),
KPI cards (latest/prev/min/max), FF events table (actual vs forecast coloring),
z-score bar chart for recent surprises, range selector (1Y/2Y/5Y/10Y/All)
- Route /macro-series + Sidebar entry "Macro Series"
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New ff_calendar table (event_date, time, currency, impact, actual, forecast, previous)
- New service ff_calendar.py: bulk CSV import (83K events 2007-2025) + live sync
from faireconomy.media JSON endpoint (this week / next week)
- New API endpoints: POST /api/eco/ff-import, POST /api/eco/ff-sync,
GET /api/eco/calendar (period filter), GET /api/eco/ff-stats
- CalendarPage.tsx full rewrite: period tabs (Recent/Today/Tomorrow/This Week…),
currency flags filter, impact filter, unified date-grouped table with
Time·Flag·Currency·Impact·Event·Actual·Forecast·Previous columns,
green/red actual vs forecast, TODAY badge, auto-refresh 60s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- fred_bootstrap.py: switch from CSV graph endpoint (CloudFlare-blocked)
to api.stlouisfed.org/fred/series/observations JSON API; reads key
from DB config 'fred_api_key'; returns clear error if key missing
- eco.py: add GET/POST /api/eco/fred-key to check/save the FRED API key
- CalendarPage.tsx: BootstrapPanel shows API key section with status,
input to paste key + save button, disables Launch if key missing
Free key: fred.stlouisfed.org -> My Account -> API Keys
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FRED blocks requests without a valid User-Agent. Added headers matching
a real browser. Also added print() calls so fetch errors appear in
docker logs (logger.warning goes to system_logs table, not stdout).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
