160-tool MCP server for real-time global intelligence from free public APIs: markets, liquidity, prediction markets, SEC filings, conflict, military, aircraft-cohort early warning, migration and border flows, cyber, climate, news and more. Dated observation history, 29 research prompts, cited briefs, geofences, SSE dashboard. MIT, no paid API keys.
Python ai-toolsai-watchanthropicclaudecybersecuritydashboardgeofencinggeopoliticalgeospatialintelligencemarketsmcpmigrationmilitarymodel-context-protocolnuclearosintreal-timespace-weatherthreat-intelligence
Merge feat/history-backfill: FRED and ECB history into the store, three history extractors (Phase 29)
Phoenix Agent (mac-studio)fef80e5b5e1a
diff --cc CHANGELOG.mdindex b25de08,4ea431c..2b01ab6--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -349,56 -229,60 +349,110 @@@ source here, and no proxy is substitutehistory capture is an error.- Tools 139 -> 143.+### Added: connections with timing offsets (Phase 29)+- `intel_lagged_connections` (CLI: `intel history connections`): Pearson+ correlation between 2 to 8 series in the history store (keys as+ `intel_series_history` lists them), aligned on a UTC day, week (Monday to+ Sunday) or month grid and tested at every lag from `-max_lag` to+ `+max_lag` (default 5, at most 30, reduced so one call runs at most 1,000+ tests). Lag +k compares A at period t with B at t+k, so A leads B by k.+- Default transform: period-over-period changes, never taken across a gap;+ `levels` on request. Responses say which ran and why changes are the+ default (levels of trending series correlate spuriously).+- Per pair: the lag with the smallest p, with r, n (overlapping periods+ after the transform and shift), a two-sided p from Student's t (computed+ from the regularized incomplete beta function, standard library only) and+ the Benjamini-Hochberg q across every test in the call, with the+ Benjamini-Yekutieli `q_by` alongside. Significant at q <= 0.05 by default+ (allowed 0.001 to 0.25); the other significant lags are listed too.+- A pair and lag with fewer than 30 overlapping periods (callers may go down+ to 20), or a constant side, is listed as not tested with its n, never+ dropped. 30 points give about 80% power for a true |r| of 0.5 at alpha+ 0.05 (module docstring).+- One vintage, the latest collected value for each `observed_at`. Rows+ without `observed_at` (Yahoo quotes) are not used, with the reason: rows+ are stored only on change, so a missing day cannot be told from an+ unchanged one. A period takes its last observation; nothing is averaged.+- Serial correlation: each series' lag-1 autocorrelation after the+ transform, and per strongest result an AR(1) effective sample size and+ the p recomputed with it, as a diagnostic outside q. Nothing is+ prewhitened.+- `cannot_prove` in every response: correlation is not causation, a lead is+ not a causal direction, serial correlation inflates significance, a+ shared driver produces correlation, the store holds only what the+ collector recorded, BH under dependence is approximate and covers one+ call, and a small q is not a large effect.+- An unknown grid or transform is an error naming the allowed values;+ numeric inputs are clamped and each clamp is reported.+- Measured 2026-10-17 over a live MCP stdio session and the CLI, on four+ real FRED series (WALCL, WDTGAL, WTREGEN, RRPONTSYD) loaded into an+ isolated store as one capture; illustrative, not findings. Week grid,+ changes: 26 of 66 tests significant; the Wednesday TGA level against its+ week average gives r 0.61 at lag 0 and 0.59 at lag +1, the averaging+ window rather than a cause. Levels: Fed total assets against either TGA+ series gives r 0.74 at lag 0 (n 1,244) with an AR(1) effective sample+ size of 5.3 and 3.4. Day grid: the weekly series have no adjacent-day+ changes, so nothing is tested and the response says why.+- With the default 365-day observation retention+ (`WORLD_INTEL_HISTORY_DAYS`), a store filled only by the collector spans+ about a year, so the month grid cannot reach the 20-point floor without a+ longer retention or backfilled history.+- Tools 152 -> 153.++ ### Added: history backfill and more history extractors (Phase 29)+ - `intel history backfill fred SERIES... [--start YYYY-MM-DD]`,+ `intel history backfill ecb DATASET.KEY... [--start YYYY-MM-DD]` and the+ MCP tool `intel_history_backfill` load the history that FRED's keyless graph CSV+ and the ECB Data Portal (BSI and ILM datasets) already serve into the+ history store. Requests reuse the liquidity tools' fetch and parse code,+ so they go through the Fetcher (FRED rate floor, circuit breaker, cache):+ a copy cached by an earlier call (FRED 1 h, ECB 3 h) is reused without a+ request.+ - Backfilled rows have their own sources, `backfill:fred` and+ `backfill:ecb` (series `backfill:fred:<ID>` and+ `backfill:ecb:<DATASET>.<KEY>`), so they never read as collector+ captures; `intel_collection_health` and `intel history health` list them+ apart (`backfills`). Every run is logged in a new table,+ `history_backfills`, created in place with `CREATE TABLE IF NOT EXISTS`+ (existing rows untouched).+ - Dates: `observed_at` is the publisher's period (FRED's date; ECB months+ as written, ECB weeks as the Sunday that ends the ISO week);+ `published_at` is NULL, because neither response names a release or+ vintage date; `collected_at` is the backfill time, stamped after the+ call's last fetch. Values are the publisher's vintage on the backfill+ day, already revised, so revisions made before the first backfill are+ invisible.+ - Idempotent: a re-run stores only new or changed values, and a changed+ value is a revision in `intel_revisions`. FRED's missing cells (`.` and+ blank) are skipped and counted, never stored as zero. ECB missing values+ cannot be counted (the SDMX-JSON parser drops them) and read `unknown`.+ - Failures are errors, never an empty success: an HTTP error, an empty or+ malformed body, a window without values, and a response available only+ as a stale cache copy each record an error capture and store nothing.+ The CLI exits 1 when any request failed.+ - Caps per call: 5 FRED series or 2 ECB keys (at most 20 series per key),+ the newest 20,000 points per series and 100,000 per call; `--start`+ narrows the window.+ - Retention: observations are pruned by `collected_at`, not `observed_at`,+ so a 1962 value is kept as long as any value stored by the same backfill+ (`WORLD_INTEL_HISTORY_DAYS`, default 365). A backfill prunes before it+ stores, so a pruned value is stored again as new.+ - New history extractors (`history_extractors.py`): `liquidity_board`+ (latest and previous value per indicator; the derived net liquidity row+ is left to its own source), `net_liquidity` (net liquidity and its three+ terms per Wednesday, reverse repo dated by the day it was taken from) and+ `net_migration` (World Bank SM.POP.NETM per country and year). Each+ dates a value only from the payload.+ - Measured 2026-10-17, live, isolated HOME: WALCL 1,244 values+ (2002-12-18 to 2026-10-14), RRPONTSYD 3,348 (2,833 blank cells skipped),+ DGS10 16,182 (721 skipped; units unknown, it is not in this server's+ FRED catalog), euro area M3 560 months (1980-01 to 2026-08, 1+ provisional). A second run from the cached copies and a refetch of WALCL+ and M3 stored 0.+ - FRED's terms of use speak of personal, non-commercial use of FRED data;+ whether a deployment fits them is the operator's call.+ - Tools 152 -> 153; the `intel history` group gains `backfill fred|ecb`.+### Added: news signals (ROADMAP Phase 29)- `intel_hostility_timeline`: per UTC day, GDELT DOC 2.0 coverage of acountry or a pair of countries (`timelinevolraw`): matching articles,diff --cc README.mdindex a3b943a,f6fc7fc..a5570d2--- a/README.md+++ b/README.md@@@ -6,7 -6,7 +6,7 @@@[](https://python.org)[](LICENSE)- Real-time global intelligence across **30+ domains** with **159 MCP tools**, a live ops-center dashboard, a CLI, and a **Qdrant vector store** for enterprise-grade semantic search across accumulated intelligence. All data comes from free, public APIs: no paid subscriptions required.-Real-time global intelligence across **30+ domains** with **153 MCP tools**, a live ops-center dashboard, a CLI, and a **Qdrant vector store** for enterprise-grade semantic search across accumulated intelligence. All data comes from free, public APIs: no paid subscriptions required.++Real-time global intelligence across **30+ domains** with **160 MCP tools**, a live ops-center dashboard, a CLI, and a **Qdrant vector store** for enterprise-grade semantic search across accumulated intelligence. All data comes from free, public APIs: no paid subscriptions required.Built for AI agents that need world awareness: market conditions, geopolitical risk, military posture, aircraft-cohort early warning, refugee and migration flows, supply chain disruptions, cyber threats, and more — all queryable via the Model Context Protocol. The vector store enables natural language queries like *"military activity near Taiwan"* or *"cyber threats targeting healthcare"* across all historical data.@@@ -66,10 -66,9 +66,10 @@@| **Volcanoes** | 1 | Smithsonian GVP weekly activity report || **Tropical Cyclones** | 2 | NHC (Atlantic, E/C Pacific) + JTWC (NW Pacific, N Indian, S Hemisphere), with opt-in forecast tracks: positions, UTC valid times and intensities, plus NHC's cone of uncertainty || **Situation Brief** | 1 | Cited situational awareness brief over MCP: bounded server-side overview synthesized via local Ollama, with a mechanically-cited fallback |- | **Collection History** | 5 | Collector run log (per-source health, overdue and failing sources), saved captures with change detection and replay, dated observation series, revisions, and one UTC day's timeline by collection or observation date |-| **Collection History** | 5 | Collector run log (per-source health, overdue and failing sources), saved captures with change detection and replay, dated observation series, revisions, and on-demand backfill of the history FRED and the ECB Data Portal already serve |++| **Collection History** | 6 | Collector run log (per-source health, overdue and failing sources), saved captures with change detection and replay, dated observation series, revisions, one UTC day's timeline by collection or observation date, and on-demand backfill of the history FRED and the ECB Data Portal already serve |+| **Connections Between Series** | 1 | Correlation between stored series at leads and lags, with a minimum sample, false-discovery control and what it cannot prove |- **Total: 159 tools** across 30+ intelligence domains.-**Total: 153 tools** across 30+ intelligence domains.++**Total: 160 tools** across 30+ intelligence domains.---@@@ -146,9 -142,8 +146,11 @@@ intel prediction-settled # Brier scorintel status # cache + circuit breaker healthintel guide # the research guide (also served as intel://guide)intel history health # per-source collection health from the collector's run log+intel history connections central_bank_rates:federal_funds_rate central_bank_rates:boe_bank_rate_sonia+ # correlation at leads and lags between stored series (day grid by default)+intel history day 2026-10-17 --by observed # values observed on one UTC day (default: captures that day)+ intel history backfill fred WALCL RRPONTSYD --start 2020-01-01 # load FRED history into the history store+ intel history backfill ecb BSI.M.U2.Y.V.M30.X.1.U2.2300.Z01.E # load ECB Data Portal history (euro area M3)```---@@@ -620,8 -598,8 +622,8 @@@ different, openly licensed data source|------|-------------|| `intel_situation_brief` | Cited situational awareness brief, generated on demand over MCP: a bounded server-side overview (earthquakes, military flights, ACLED conflict events, wildfires, cyber threats, disease outbreaks, news, space weather, strategic posture, alert digest), synthesized via local Ollama or a mechanically-cited fallback when Ollama is unreachable |--### Collection History (5)- History accrues only while the collector (`intel-collector`) runs; nothing before its first recorded cycle exists. Every observation keeps three dates: `observed_at` (the period the value describes, from the payload, or null when the payload has none), `published_at` (publisher release, when known) and `collected_at` (when this server saved it).++### Collection History (6)+ History accrues only while the collector (`intel-collector`) runs; nothing before its first recorded cycle exists, except what a backfill loads (`intel_history_backfill`, `intel history backfill`): FRED graph CSV and ECB Data Portal series, stored under `backfill:fred` and `backfill:ecb` with the publisher's current vintage, so revisions made before the first backfill are invisible. Every observation keeps three dates: `observed_at` (the period the value describes, from the payload, or null when the payload has none), `published_at` (publisher release, when known) and `collected_at` (when this server saved it).| Tool | Description ||------|-------------|@@@ -629,15 -607,8 +631,16 @@@| `intel_capture_history` | A source's earlier captures (ok / degraded / error with reason, duration, record count) and whether each changed from the previous one; `capture_id` replays one stored payload || `intel_series_history` | One series' observations with all three dates, or the list of available series || `intel_revisions` | Values for the same `observed_at` that a later capture changed (USGS magnitude revisions, NOAA Kp updates) |+| `intel_day_timeline` | What the history store holds for one UTC day (`YYYY-MM-DD`; a future day is an error). `by=collected` (default): per source, that day's captures (attempts, ok / degraded / error with reasons, changed vs unchanged payloads, first and last capture, observations stored), and for a source with no capture whether the collector did not run that day, ran without it, never ran it, or the day is past run-log retention. `by=observed`: values whose `observed_at` falls on the day; month and year values containing the day are listed apart as period values, and undated observations are counted and excluded. Every item carries `observed_at`, `published_at` and `collected_at`; pruned records are named, never read as a quiet day; per-source and total caps with truncation disclosed. Read-only, never fetches |+ | `intel_history_backfill` | Load the history FRED (graph CSV) or the ECB Data Portal already serve into the history store: one capture per series under `backfill:fred` / `backfill:ecb`, `observed_at` from the publisher's dates, `published_at` null, `collected_at` the backfill time; re-runs store only new or changed values (a change is a revision); missing values skipped and counted; writes and fetches, so capped at 5 FRED series or 2 ECB keys per call |+### Connections Between Series (1)+Reads the history store only, so the sample is what the collector saved, not each series' full history. A connection is a correlation in that sample: it does not show causation, and a lead does not show causal direction.++| Tool | Description |+|------|-------------|+| `intel_lagged_connections` | Correlation between 2 to 8 stored series (keys as `intel_series_history` lists them) on a UTC day, week or month grid, tested at every lag from `-max_lag` to `+max_lag` (default 5, at most 30, at most 1,000 tests a call). Default transform is period-over-period changes, because levels of trending series correlate spuriously; `levels` on request. Per pair: the lag with the smallest p, with Pearson r, overlap n, two-sided p from Student's t and the Benjamini-Hochberg q across every test in the call (significant at q <= 0.05 by default). Lag +k compares A at period t with B at t+k, so A leads B by k. Pair/lag overlaps under 30 points are listed as not tested with their n; revised points use the latest vintage; rows without `observed_at` (Yahoo quotes) are not used. Each strongest result also carries an AR(1) effective sample size as a serial-correlation diagnostic |+---## Watching your own area (geofences/AOIs)diff --cc src/world_intel_mcp/cli.pyindex 9074957,7005d22..45764bf--- a/src/world_intel_mcp/cli.py+++ b/src/world_intel_mcp/cli.py@@@ -3581,300 -3328,106 +3591,400 @@@ def history_revisionsconsole.print(f"[yellow]Cannot prove:[/yellow] {escape(line)}")+from .analysis import lagged_connections as _lagged # noqa: E402+++def _p_text(value: float) -> str:+ return "<1e-300" if value == 0 else f"{value:.2g}"+++@history_group.command(name="connections")[email protected]("series_keys", nargs=-1, required=True)[email protected]("--grid", type=click.Choice(_lagged.GRIDS), default=_lagged.DEFAULT_GRID,+ show_default=True, help="UTC calendar grid (weeks run Monday to Sunday)")[email protected]("--transform", type=click.Choice(_lagged.TRANSFORMS),+ default=_lagged.DEFAULT_TRANSFORM, show_default=True,+ help="changes (period-over-period) or levels")[email protected]("--max-lag", type=int, default=_lagged.DEFAULT_MAX_LAG, show_default=True,+ help=f"Largest lead or lag in grid periods (0 to {_lagged.MAX_LAG_CAP})")[email protected]("--min-overlap", type=int, default=_lagged.DEFAULT_MIN_OVERLAP, show_default=True,+ help=f"Fewest overlapping periods a test needs (at least {_lagged.MIN_OVERLAP_FLOOR})")[email protected]("--q", "q", type=float, default=_lagged.DEFAULT_Q, show_default=True,+ help="False discovery rate threshold (Benjamini-Hochberg)")[email protected]("--start", default=None, help="First date YYYY-MM-DD")[email protected]("--end", default=None, help="Last date YYYY-MM-DD")[email protected]_context+def history_connections(+ ctx: click.Context, series_keys: tuple[str, ...], grid: str, transform: str,+ max_lag: int, min_overlap: int, q: float, start: str | None, end: str | None,+) -> None:+ """Correlation between stored series at leads and lags.++ Lag +k compares the first series of a pair at period t with the second+ at t+k, so the first leads by k. A connection is not a cause.+ """+ data = _with_history_store(+ lambda s: _lagged.lagged_connections(+ s, list(series_keys), grid=grid, transform=transform, max_lag=max_lag,+ min_overlap=min_overlap, q=q, start=start, end=end,+ )+ )+ if _bail_on_error(ctx, data):+ return++ method = data["method"]+ lo, hi = method["lags"]+ table = Table(+ title=f"Connections ({method['grid']} grid, {method['transform']}, lags {lo:+d} to "+ f"{hi:+d}, {data['tests_run']} tests, BH q <= {method['q_threshold']:g})",+ box=box.SIMPLE_HEAVY,+ )+ for name in ("A", "B", "Lag", "Timing", "r", "n", "n_eff", "p", "q", "Significant"):+ justify = "right" if name in ("Lag", "r", "n", "n_eff", "p", "q") else "left"+ table.add_column(name, justify=justify)+ for pair in data["pairs"]:+ if not pair["strongest"]:+ table.add_row(_cell(pair["a"]), _cell(pair["b"]), "-",+ Text("not tested", style="dim"), "-", "-", "-", "-", "-", "-")+ for entry in pair["strongest"]:+ n_eff = entry.get("n_effective_ar1")+ table.add_row(+ _cell(pair["a"]), _cell(pair["b"]), f"{entry['lag']:+d}", entry["timing"],+ f"{entry['r']:+.3f}", str(entry["n"]), "-" if n_eff is None else f"{n_eff:g}",+ _p_text(entry["p"]), _p_text(entry["q"]),+ Text("yes", style="green") if entry["significant"] else Text("no"),+ )+ console.print(table)+ console.print(escape(data["summary"]))+ console.print(+ "n_eff: AR(1) effective sample size from the two series' lag-1 autocorrelations, "+ "a diagnostic only: p and q use n."+ )+ console.print(f"Lag convention: {escape(method['lag_convention'])}")+ console.print(+ f"Transform: {escape(method['transform_rule'])} {escape(method['why_changes_default'])}"+ )+ for pair in data["pairs"]:+ best = {e["lag"] for e in pair["strongest"]}+ others = [e for e in pair["significant"] if e["lag"] not in best]+ if others:+ listed = ", ".join(+ f"lag {e['lag']:+d} (r {e['r']:+.3f}, q {_p_text(e['q'])})" for e in others+ )+ console.print(f"Also significant: {escape(pair['a'])} / {escape(pair['b'])}: {listed}")+ if pair["not_tested"]:+ listed = ", ".join(+ f"lag {nt['lag']:+d} (n {nt['n']}"+ + (", no variation)" if nt["reason"] == "no_variation" else ")")+ for nt in pair["not_tested"]+ )+ console.print(+ f"[yellow]Not tested[/yellow] (minimum {method['min_overlap']} overlapping "+ f"periods): {escape(pair['a'])} / {escape(pair['b'])}: {listed}"+ )+ for item in data["excluded_series"]:+ console.print(+ f"[yellow]Excluded:[/yellow] {escape(item['series_key'])}: {escape(item['reason'])}"+ )+ _print_data_gaps(data)+ for line in data["cannot_prove"]:+ console.print(f"[yellow]Cannot prove:[/yellow] {escape(line)}")+++@history_group.command(name="day")[email protected]("day")+ "--by",+ type=click.Choice(["collected", "observed"]),+ default="collected",+ show_default=True,+ help="collected: what was captured that day; observed: values whose observed_at is that day",+)[email protected]("--source", default=None, help="One collector source name")[email protected]("--limit", default=None, type=int, help="Most items listed in all")[email protected]("--per-source", "per_source", default=None, type=int, help="Most items listed per source")[email protected]_context+def history_day(+ ctx: click.Context,+ day: str,+ by: str,+ source: str | None,+ limit: int | None,+ per_source: int | None,+) -> None:+ """What the history store recorded for one UTC day (YYYY-MM-DD)."""+ import time++ from .collector import SOURCES+ from .history_timeline import (+ DEFAULT_LIMIT,+ DEFAULT_PER_SOURCE_LIMIT,+ day_timeline,+ parse_day,+ )++ now = time.time()+ _, problem = parse_day(day, now)+ if problem:+ raise click.BadParameter(problem, param_hint="DAY")+ data = day_timeline(+ _get_fetcher().cache.db_path,+ day,+ by=by,+ source=source,+ roster=[name for name, _, _, _ in SOURCES],+ limit=DEFAULT_LIMIT if limit is None else limit,+ per_source_limit=DEFAULT_PER_SOURCE_LIMIT if per_source is None else per_source,+ now=now,+ )+ if _bail_on_error(ctx, data):+ return++ window = data["window"]+ span = f"{window['start']} to {window['end']} (end exclusive)"+ if not window["complete"]:+ span += f", in progress through {window['through']}"+ console.print(f"[bold]{data['day']}[/bold] (UTC) by {data['by']} date: {span}")+ if by == "collected":+ _print_day_collected(data)+ else:+ _print_day_observed(data)+ _print_data_gaps(data)+ for line in data["cannot_prove"]:+ console.print(f"[yellow]Cannot prove:[/yellow] {escape(line)}")+++def _print_day_collected(data: dict) -> None:+ c = data["collector"]+ if c["state"] == "ran":+ console.print(+ f"Collector: ran, {c['captures']:,} captures of {c['sources_captured']} "+ f"source(s), {c['first_capture_at']} to {c['last_capture_at']}; longest gap "+ f"{c['longest_gap_s']:,.0f} s ({c['longest_gap_from']} to {c['longest_gap_to']})"+ )+ else:+ console.print(f"Collector: [yellow]{c['state']}[/yellow]. {escape(c['explanation'])}")++ styles = {"changed": "green", "unchanged": "green", "answered": "green",+ "failed": "red", "never_run": "dim"}+ table = Table(title=f"Sources on {data['day']}", box=box.SIMPLE_HEAVY)+ for name in ("Source", "State", "Tries", "OK", "Degr", "Err", "Changed", "Same",+ "First", "Last", "Obs", "Reason / context"):+ table.add_column(name, justify="right" if name in {+ "Tries", "OK", "Degr", "Err", "Changed", "Same", "Obs"} else "left")+ for row in data["sources"]:+ counts = row.get("status_counts") or {}+ changes = row.get("payload_changes") or {}+ if row["attempts"]:+ top = (row.get("reasons") or [None])[0]+ context = f"{top['reason']} (x{top['count']})" if top else ""+ if row.get("more_reasons"):+ context += f" +{row['more_reasons']} more"+ stored = row.get("observations_stored")+ obs = "-" if stored is None else str(stored)+ else:+ parts = []+ if row.get("last_capture_before"):+ parts.append(f"last before {row['last_capture_before']}")+ if row.get("next_capture_after"):+ parts.append(f"next after {row['next_capture_after']}")+ context = ", ".join(parts)+ obs = str(row["observations_retained"]) if row["observations_retained"] else "-"+ table.add_row(+ _cell(row["source"]),+ Text(row["day_state"], style=styles.get(row["day_state"], "yellow")),+ str(row["attempts"]),+ str(counts.get("ok", "-")),+ str(counts.get("degraded", "-")),+ str(counts.get("error", "-")),+ (f"{changes['changed']}" + (f" +{changes['first']} first" if changes["first"] else ""))+ if changes else "-",+ str(changes.get("unchanged", "-")),+ _cell(row.get("first_capture_at") or "-"),+ _cell(row.get("last_capture_at") or "-"),+ obs,+ _cell(context),+ )+ console.print(table)+ states = ", ".join(f"{k} {v}" for k, v in sorted(data["state_counts"].items()))+ console.print(f"States: {states}.")++ items = data["observations"]+ trunc = data["truncation"]+ if items:+ obs_table = Table(+ title=f"Observations saved that day ({trunc['returned']:,} of {trunc['matched']:,})",+ box=box.SIMPLE_HEAVY,+ )+ for name in ("Collected", "Source", "Series", "Observed", "Published", "Value", "Unit"):+ obs_table.add_column(name, justify="right" if name == "Value" else "left")+ for item in items:+ obs_table.add_row(+ item["collected_at"],+ _cell(item["source"]),+ _cell(item["series_key"]),+ _cell(item["observed_at"] or "unknown"),+ _cell(item["published_at"] or "unknown"),+ f"{item['value']:g}",+ _cell(item.get("unit") or ""),+ )+ console.print(obs_table)+ else:+ console.print("[dim]No observations saved that day.[/dim]")+++def _print_day_observed(data: dict) -> None:+ def _points_table(title: str, points: list[dict]) -> Table:+ table = Table(title=title, box=box.SIMPLE_HEAVY)+ for name in ("Observed", "Precision", "Series", "Value", "Unit", "Published",+ "Collected", "Stored"):+ table.add_column(name, justify="right" if name in {"Value", "Stored"} else "left")+ for point in points:+ stored = str(point["stored_values"])+ if point["stored_values"] > 1:+ stored += f" (first {point['first_value']:g} at {point['first_collected_at']})"+ table.add_row(+ _cell(point["observed_at"]),+ point["precision"],+ _cell(point["series_key"]),+ f"{point['value']:g}",+ _cell(point.get("unit") or ""),+ _cell(point["published_at"] or "unknown"),+ point["collected_at"],+ stored,+ )+ return table++ day_trunc = data["truncation"]["day_values"]+ if data["day_values"]:+ console.print(_points_table(+ f"Values observed on {data['day']} ({day_trunc['returned']:,} of "+ f"{day_trunc['matched']:,})",+ data["day_values"],+ ))+ else:+ console.print(f"[yellow]No stored value is observed on {data['day']}.[/yellow]")+ if data["period_values"]:+ period_trunc = data["truncation"]["period_values"]+ console.print(_points_table(+ f"Period values containing the day, not values for it "+ f"({period_trunc['returned']:,} of {period_trunc['matched']:,})",+ data["period_values"],+ ))+ console.print(f"[dim]{escape(data['period_values_note'])}[/dim]")+ if data["sources"]:+ table = Table(title="By source", box=box.SIMPLE_HEAVY)+ for name in ("Source", "Day values", "Revised", "Period values", "No observed_at"):+ table.add_column(name, justify="left" if name == "Source" else "right")+ for row in data["sources"]:+ table.add_row(+ _cell(row["source"]),+ str(row["day_values"]),+ str(row["day_values_revised"]),+ str(row["period_values"]),+ str(row["without_observed_at_collected_that_day"]),+ )+ console.print(table)+ # ---------------------------------------------------------------------------+ # Backfill (history FRED and the ECB already serve, into the history store)+ # ---------------------------------------------------------------------------+++ @history_group.group(name="backfill")+ def backfill_group() -> None:+ """Load history FRED or the ECB Data Portal already serve into the history store.++ Values are the publisher's current vintage, so revisions made before the+ first backfill are invisible. Read them back with intel history series.+ """+++ def _run_backfill(+ ctx: click.Context, provider: str, requests: tuple[str, ...], start: str | None+ ) -> None:+ from . import history_backfill+ from .history import HistoryStore++ fetcher = _get_fetcher()+ run = history_backfill.backfill_fred if provider == "fred" else history_backfill.backfill_ecb+ store = HistoryStore(fetcher.cache.db_path)+ try:+ data = _run(run(fetcher, store, list(requests), start))+ finally:+ store.close()+ if ctx.obj.get("json"):+ _print_json(data)+ else:+ _print_backfill(data)+ if "error" in data or data["totals"]["failed"]:+ ctx.exit(1)+++ def _print_backfill(data: dict) -> None:+ if not data.get("requests"):+ console.print(f"[red]Error:[/red] {escape(str(data['error']))}")+ return+ title = f"Backfill into {data['source']} at {data['collected_at']}"+ if data.get("start"):+ title += f" from {data['start']}"+ table = Table(title=escape(title), box=box.SIMPLE_HEAVY)+ table.add_column("Series", style="bold")+ table.add_column("Status")+ table.add_column("Loaded", justify="right")+ table.add_column("New", justify="right")+ table.add_column("Revised", justify="right")+ table.add_column("Unchanged", justify="right")+ table.add_column("Missing", justify="right")+ table.add_column("Observed from")+ table.add_column("Observed to")+ for req in data["requests"]:+ if req["status"] == "error":+ table.add_row(_cell(req["request"]), Text("error", style="red"), *["-"] * 7)+ continue+ for s in req["series"]:+ table.add_row(+ _cell(s["series_key"]),+ Text("ok", style="green"),+ str(s["loaded"]),+ str(s["stored_new"]),+ str(s["stored_revised"]),+ str(s["unchanged"]),+ "unknown" if s["missing_skipped"] is None else str(s["missing_skipped"]),+ _cell(s["first_observed_at"] or "-"),+ _cell(s["last_observed_at"] or "-"),+ )+ console.print(table)+ totals = data["totals"]+ console.print(+ f"Stored {totals['stored_new']} new and {totals['stored_revised']} revised values, "+ f"{totals['unchanged']} unchanged, in {escape(data['db_path'])}."+ )+ _print_data_gaps(data)+++ @backfill_group.command(name="fred")+ @click.argument("series_ids", nargs=-1, required=True)+ @click.option("--start", default=None, help="Only observations on or after YYYY-MM-DD")+ @click.pass_context+ def backfill_fred_cmd(ctx: click.Context, series_ids: tuple[str, ...], start: str | None) -> None:+ """Load FRED graph CSV series (up to 5).++ For example: intel history backfill fred WALCL RRPONTSYD DGS10 --start 2020-01-01+ """+ _run_backfill(ctx, "fred", series_ids, start)+++ @backfill_group.command(name="ecb")+ @click.argument("keys", nargs=-1, required=True)+ @click.option("--start", default=None, help="Only observations on or after YYYY-MM-DD")+ @click.pass_context+ def backfill_ecb_cmd(ctx: click.Context, keys: tuple[str, ...], start: str | None) -> None:+ """Load ECB Data Portal series (up to 2 keys).++ Each key is DATASET.KEY, for example BSI.M.U2.Y.V.M30.X.1.U2.2300.Z01.E+ (euro area M3); a key with + alternatives loads every series it matches.+ """+ _run_backfill(ctx, "ecb", keys, start)if __name__ == "__main__":diff --cc src/world_intel_mcp/tools/__init__.pyindex 82119e2,fb02ea6..9b2f84d--- a/src/world_intel_mcp/tools/__init__.py+++ b/src/world_intel_mcp/tools/__init__.py@@@ -30,9 -29,9 +30,10 @@@ from . import ( # noqa: E40geospatial,hazards,history,+ history_backfill,infrastructure,intelligence,+ lagged_connections,liquidity,markets,news_market,@@@ -63,8 -60,7 +64,9 @@@ _MODULES =vector,aoi,history,+ lagged_connections,+ day_timeline,+ history_backfill,system,]