160-tool MCP server for real-time global intelligence from free public APIs: markets, liquidity, prediction markets, SEC filings, conflict, military, aircraft-cohort early warning, migration and border flows, cyber, climate, news and more. Dated observation history, 29 research prompts, cited briefs, geofences, SSE dashboard. MIT, no paid API keys.
Python ai-toolsai-watchanthropicclaudecybersecuritydashboardgeofencinggeopoliticalgeospatialintelligencemarketsmcpmigrationmilitarymodel-context-protocolnuclearosintreal-timespace-weatherthreat-intelligence
Merge feat/news-signals a5ced4a: news signals review fixes
Phoenix Agent (mac-studio)7f87ff19cc98
diff --cc CHANGELOG.mdindex 5e653d3,b1d7b9b..6cd1132--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -576,179 -113,39 +595,211 @@@ source here, and no proxy is substitutehave no headline keywords, so the search could never match; the sectionnow says "mentions not searched" instead of reporting zero.`intel dossier` prints `News: not measured` and the gaps in that case.+ - Review fixes for the news signals (independent review, 2026-10-17):+ - The hostility timeline showed an expired cached GDELT copy (days old+ when no collector evicts it) as current counts, with no marker. Such a+ column is now null, named in `data_gaps` with its age and in+ `column_status`, and the response is `degraded` with+ `reason: gdelt_stale`.+ - A day with no buckets in the series that judged completeness read as+ complete (`partial: false`), and a category series covering half a day+ sat unflagged next to full-day counts. Partial days are now judged per+ column; a half-day count gets no `hostile_share`.+ - With no comparable feed, situations reported change 0 and ratio 1.0+ ("no change"), and the CLI printed `+0`; attention read as "nothing+ grew". Both are now null or empty and marked not measured.+ - A 429 inside one GDELT call was retried after 1 s and 2 s, and the+ timeline's retry pass asked again for every failed column; see the+ GDELT spacing entry under Changed.+ - Dossier headline matching was a substring test ("Tirana" counted as+ Iran, "Indiana" as India) and hid its cap: it now matches whole words,+ removes four compound names first, and reports `matching_items`,+ `mention_count_capped`, `items_searched` and `searched_span`.+ - A near-total RSS outage (118 of 119 feeds down) read as normal:+ `fetch_news_feed` now marks a read where under half of the feeds+ answered live `degraded` with `reason: partial_feeds_failed`, and+ `intel_trending_keywords` passes the feed counts and marker through;+ situations, top stories and the dossier rank or count over the feeds+ that answered and say so.+ - Clustering ran on the event loop the collector and the MCP server+ share; unknown codes such as "GBR" were sent to GDELT as text; a limit+ of 0 or -1 returned nothing or dropped a result, and `min_articles: 0`+ divided by zero; feed links of any scheme (`javascript:`, `data:`)+ were passed on and titles had no length cap (now http(s) only, titles+ capped at 500 characters).+### Fixed: vessel snapshot waterway matching+- `intel_vessel_snapshot` counted every NGA warning at all nine waterways,+ so three naval-keyword warnings anywhere on Earth marked Panama and+ Hormuz critical alike. On the live feed of 2026-10-17 the old code marked+ all nine `critical` from 8 such warnings, none within 300 km of any of+ them. A warning now counts only at waterways within 300 km of a position+ it states (degrees-minutes or degrees-minutes-seconds). A warning that+ states no position counts by NAVAREA only when its area code names a+ single NAVAREA (4 is NAVAREA IV, 12 is NAVAREA XII, which of the nine+ contain only the Panama Canal). HYDROLANT, HYDROPAC and HYDROARC each+ span several NAVAREAs, so a positionless naval warning in those series,+ or one whose only stated position is malformed, is listed in+ `unplaced_naval_warnings` and named in `data_gaps`. The response states+ the rule in `matching_rule`.+- Each waterway emits `total_nearby` (warnings of any topic credited to+ it, which the old code computed and dropped), split by how they matched+ in `nearby_by_match` and `naval_by_match`; `warning_details` entries+ carry `match`, `distance_km` and `issue_date`.+- With NGA down every waterway read `clear`. It now reads `unknown`, and+ the response carries `error`, `degraded` and `reason`.++### Fixed: navigational warnings+- An NGA outage returned `warnings: []` with no marker, the same shape as a+ sea with no active warnings. It now carries `error`, `degraded`,+ `reason` (`nga_msi_fetch_failed`) and `data_gaps`. A payload with no+ warnings list is `nga_msi_unexpected_payload`; a stale cache serve is+ degraded and says how old the copy is.+- The NGA MSI API holds no warning issued after 10 May 2024 (measured+ 2026-10-17: `msgYear=2025` with `status=all` and `msgYear=2026` return+ none, `msgYear=2024` with `status=all` ends on 10 May, and+ `/current-warnings` and `/inforce` list the same 2024 state), yet+ responses read as current. `intel_nav_warnings` now reports+ `newest_issue_date` and `newest_issue_age_days`; past `stale_after_days`+ (14, twice the slowest series' longest gap between issues in 2024) the+ response is degraded with the age in `data_gaps`. The vessel snapshot and+ fleet report pass this on.+- `area_codes` maps the codes the API sends in `navArea` (4, 12, A, P, C;+ never the Roman numerals) to the NAVAREAs, with each code's source in+ `sources/maritime.py`.++### Fixed: fleet report+- `intel_fleet_report` turned a failed component into `{}`, so with all+ three down it read `LOW_ACTIVITY` with no marker. A component that raises+ or reports an error is now named in `unavailable_components` and+ `data_gaps` and the report is degraded; with all three down it also+ carries `error` and `reason` (`fleet_components_unavailable`). Component+ `data_gaps` are passed on.++### Tests+- Failing-first regression tests for all three, built on live NGA records+ captured 2026-10-17.++### Added: AOI escalation news and convergence components+- `intel_aoi_escalation` now scores the news and convergence components it+ used to report as null (ROADMAP Phase 23).+- News: the rise in the area's share of all GDELT coverage over the last 7+ days against the 7 days before, for the reverse-geocoded settlement and+ county (the query the AOI brief already uses, sent as `timelinevolraw`).+ Expected articles = prior share x current GDELT volume; change_pct =+ 100 x (current - expected) / max(expected, 20); component =+ min(20, 0.2 x max(0, change_pct)). Shares rather than raw counts because+ GDELT's own intake has multi-day outages (measured 2026-10-17 for+ Pittsburgh: no data for 16 of 85 days), which raw counts would read as a+ surge on recovery; a comparison where one week monitored under half the+ other's volume is refused as a data gap. An area that cannot be+ geocoded is not scored, rather than searched by the AOI's name.+- Convergence: `analysis/convergence.py`, unmodified, over the+ earthquakes, military flights, ACLED events and wildfires inside the+ AOI's exact shape (circle, polygon or corridor, through the same+ `filter_by_aoi` scoping as the brief). With an input missing, a+ convergent cell is reported as a lower bound; no cell at all is null,+ because quiet cannot be told from missing.+- New response keys; existing keys keep their meaning. `computed_from`+ names the components the score was computed from, and+ `zero_filled_components` the ones among them that are a 0 only because+ the military or ACLED fetch failed (`score_hotspot` takes no null for+ those, unchanged); `news_signal` (query, both windows' counts and+ shares, change), `news_scoping` and `convergence_signal` (cells, events+ per domain, inputs used, failed or partial) carry the working. A failed+ news or convergence input makes that component null in `components`,+ listed in `unavailable_components` and explained in `data_gaps`, never+ a 0.++### Changed+- The escalation also fetches earthquakes and wildfires (convergence+ inputs) and one GDELT timeline, in parallel with ACLED and military+ flights; their failures are named in `data_gaps`. The brief, the change+ sweep and the escalation now share one AOI scoping helper.+- `test_aoi.py` stubs every network-bound source by default: its older+ escalation tests stub only ACLED and military flights and would+ otherwise reach Nominatim, GDELT, USGS and FIRMS live.++### Added: border crossings (flows)+- `intel_us_border_encounters`: U.S. CBP encounters per calendar month+ (Border Patrol Title 8 apprehensions, Field Operations Title 8+ inadmissibles, Title 42 expulsions), read from the newest CSV on CBP's+ public dataset page, which spans four fiscal years (October 2022 to+ August 2026 when written). Nationwide or one land border region, all+ components or USBP / OFO, optionally one citizenship as CBP names it;+ with the latest month against the same month a year earlier, fiscal-year+ totals (the year in progress compared with the same months of the prior+ year, only when every one is published), breakdowns by component,+ encounter type and demographic, and the top citizenships with CBP's+ OTHER bucket reported apart and never ranked. CBP's fiscal months are+ mapped to calendar months (FY2026 October is October 2025).+- `intel_uk_small_boat_arrivals`: Home Office daily and weekly counts of+ migrants and boats detected crossing the English Channel in small boats,+ with uncontrolled landings and the weekly migrants and events prevented+ by French authorities; month and year sums of the daily rows, and the+ year to date against the same span a year earlier. Read through the+ GOV.UK Content API and the published ODS time series, parsed with the+ standard library (zipfile + ElementTree, no new dependency). The+ publisher's numbered notes are quoted from the file on every refresh.+- Both are flows (events per period), stated in every response with units,+ date semantics and the publisher's caveats, and kept apart from the+ UNHCR and World Bank year-end stocks. Neither file has a person+ identifier, so neither counts unique people (said in each response as a+ tool note, not as the publisher's words).+- An outage is a degraded error with no figures; a failed refresh serves+ the last good copy marked `stale` with its age. A '-' in the UK file is+ null, never 0; a CBP month with no rows inside the file's coverage is 0,+ because CBP lists only non-zero cells, and months outside it are never+ invented. Weekly UK totals are checked against their seven days and any+ mismatch is named in `data_gaps`.+- Requests use an identifying User-Agent, two per refresh (the listing,+ then the file), cached 12 h (CBP) and 3 h (UK). Paths that robots.txt+ disallows are never requested; a newer-looking CBP file at such a path+ is named in `data_gaps` instead.+- Frontex is not built: its only data file is an XLSX under `/assets/`,+ which its robots.txt disallows (measured 2026-10-17, see ROADMAP Phase 28).+- Tools 139 -> 141.++### Added: Kalshi and cross-venue prediction markets+- `intel_kalshi_markets`: Kalshi open markets from its public market-data+ API (no account, no key), ranked by 24h then lifetime volume. Prices are+ YES implied probabilities (bid/ask midpoint when the book is two-sided,+ else the last trade, with the basis named); Kalshi's raw dollar quotes+ are kept. Filter by category (the category's highest-volume series) or+ series ticker. The response names the events it scanned and whether the+ scan was complete: Kalshi lists more than 8,000 open events, furthest+ close date first.+- `intel_prediction_compare`: the same event on Kalshi and Polymarket side+ by side. Matching is deterministic: quantities (with their direction,+ so "more than 25 bps" is not "25 bps"), years, end dates, negation and+ the outcome side must all agree, and each pair carries a+ `match_confidence` and its reasons. Below the threshold (default 0.6) a+ pair is not a match. Returns the price gap per pair, the closest+ rejected pairs with the reason, and unmatched counts per venue.+- `intel_prediction_settled`: recently settled markets on both venues with+ the outcome and the YES price at fixed offsets before close (default 24h+ and 168h) from each venue's price-history endpoint, and a Brier score per+ venue and offset with sample size, distinct events, selection and+ exclusions stated. A venue whose price history cannot be read is not+ scored. The two samples are different markets, and the response says so.+- Every number these three tools take has a stated range in its input+ schema: at most 25 Kalshi series or pages per scan, 2,000 Polymarket+ markets, 500 results, 100 settled markets per venue, and 10 offsets of+ at most 8,760 hours, so one call cannot fan out into an unbounded number+ of upstream requests. The MCP server refuses a value outside its range;+ the source functions clamp counts again for the CLI and other callers,+ and refuse a value that is not a number before any request.+- CLI: `intel kalshi`, `intel prediction-compare`, `intel prediction-settled`.+- Collector: `kalshi_markets` and `prediction_compare` (Economics+ category) join the `prediction` domain. Sources 55 -> 57. Tools 139 -> 142.++### Fixed: prediction markets outage read as "no markets"+- `intel_prediction_markets` returned `{"markets": [], "count": 0}` with no+ marker when Polymarket was down, the same shape as a quiet market. It now+ adds `error` and `degraded: true` on an outage or an unexpected response,+ and `stale: true` when the list came from an expired cache entry. Keys,+ parameters and the tool name are unchanged.+## 0.11.0 - 2026-10-12Cross-border migration (MCP tools, CLI, collector and a dashboarddiff --cc ROADMAP.mdindex 5a3fed0,2955cb0..7a1604c--- a/ROADMAP.md+++ b/ROADMAP.md@@@ -587,18 -571,18 +587,18 @@@ dashboard| Wave | Feature | What it means here | Status ||------|---------|--------------------|--------|-| 1 | Observation history store | Every collector capture saved with `observed_at` (period the data describe), `published_at` (publisher release, when known) and `collected_at` (when we saved it); unchanged payloads stored once; bounded retention; backfill for sources that serve history (FRED CSV) | :red_circle: |-| 1 | Capture history and revisions | Earlier saved copies of a source, and values a later copy revised | :red_circle: |-| 1 | Collection health | Per-source expected cadence; overdue / failing / never-run per source, measured from the run log, not inferred from cache freshness. Says what it cannot prove | :red_circle: |-| 1 | Research guide over MCP | MCP resources (guide generated from the live registry and source health, measures glossary, "what this server will not do") and MCP prompts (worked research questions, each naming the tools it needs and the limit to respect). Server exposed no prompts or resources before | :red_circle: |+| 1 | Observation history store | Every collector capture saved with `observed_at` (period the data describe), `published_at` (publisher release, when known) and `collected_at` (when we saved it); unchanged payloads stored once; bounded retention; backfill for sources that serve history (FRED CSV) | :white_check_mark: Unreleased: store, three dates, dedupe, retention and `register_extractor()` done (`history.py`; extractors for Kp, USGS magnitudes, FRED central bank rates, Yahoo quotes, and in `history_extractors.py` the liquidity board, net liquidity and World Bank net migration). Backfill built 2026-10-17 (`history_backfill.py`, `intel_history_backfill`, `intel history backfill fred|ecb`) for FRED graph CSV and ECB Data Portal (BSI, ILM) series, under its own sources `backfill:fred` and `backfill:ecb`; `published_at` is NULL (neither response names a release date) and values are the vintage of the backfill day. Measured live 2026-10-17: WALCL 1,244 values (2002-12-18 to 2026-10-14), RRPONTSYD 3,348 (2,833 blank cells skipped), DGS10 16,182 (721 skipped), euro area M3 560 months (1980-01 to 2026-08); a second run and a refetch stored 0 |+| 1 | Capture history and revisions | Earlier saved copies of a source, and values a later copy revised | :white_check_mark: Unreleased (`intel_capture_history` with payload replay, `intel_revisions`) |+| 1 | Collection health | Per-source expected cadence; overdue / failing / never-run per source, measured from the run log, not inferred from cache freshness. Says what it cannot prove | :white_check_mark: Unreleased (`intel_collection_health`, `intel history health`) |+| 1 | Research guide over MCP | MCP resources (guide generated from the live registry and source health, measures glossary, "what this server will not do") and MCP prompts (worked research questions, each naming the tools it needs and the limit to respect). Server exposed no prompts or resources before | :white_check_mark: Built 2026-10-17, unreleased: 3 resources (`intel://guide`, `intel://guide/source-health`, `intel://guide/measures`), 29 prompts, `intel guide`; verified over a live stdio session |- | 1 | Hostility timeline | Per country pair or country, daily counts of threat / sanction / force / criticism coverage plus tone; sortable by most covered, most hostile, biggest change | :white_check_mark: Built 2026-10-17, unreleased: `intel_hostility_timeline` (GDELT DOC 2.0 `timelinevolraw` + `timelinetone`, one call per category, failed calls and missing days null) and `intel_situations` (RSS headlines, ranked by covered / hostile / change, change defined over feeds whose history spans both windows). Measured 2026-10-17: timeline modes answer (HTTP 200, `date_resolution` `hour` for `timespan=7d` and `15m` for `3d`, explicit zero buckets, `norm` = all monitored articles), but a `timespan=7d` request returned 54 hours (2026-10-15T10Z onward), and about half of calls drew 429 even 30 s apart (later probes, 16:51 to 16:53Z, 4 of 4 drew 429, so the resolution for 14 days and coverage before 2026-10-15T10Z are unmeasured; 30 days was not probed; a live MCP run at 17:00Z got 4 of 6 columns after the retry pass, one at 17:20Z got 1 of 6, consistent with the server's breaker opening after 3 failed calls, which also skips the retry pass). The fetcher now spaces every live GDELT call at least 5 s apart, per process. Artlist was not used for daily counts: 250 records sorted by date covered 15 minutes. Situations see only the 22-country entity table |- | 1 | Story breadth | News clusters carry a distinct-outlet count; a top-stories view ranks by how many outlets carry a story | :white_check_mark: Built 2026-10-17, unreleased: clusters gain `outlets` / `outlet_count` (sister feeds of one publisher count once); `intel_top_stories` ranks by outlet count with an attention list (countries and organizations by mention growth). Found on the way: cluster `sources` was empty in every live cluster (key mismatch, `feed_name` vs `source_name`) and RSS times were shifted by the host UTC offset; both fixed. Concurrent RSS reads now share one request per feed (a live collector news-group cycle: 119 requests for 119 feeds) |+ | 1 | Hostility timeline | Per country pair or country, daily counts of threat / sanction / force / criticism coverage plus tone (`intel_hostility_timeline`); situations between and inside countries, sortable by most covered, most hostile, biggest change (`intel_situations`) | :white_check_mark: Built 2026-10-17, unreleased: `intel_hostility_timeline` (GDELT DOC 2.0 `timelinevolraw` + `timelinetone`, one call per category; failed calls, expired cached copies and missing days null; `column_status` per call; partial days judged per column) and `intel_situations` (RSS headlines, ranked by covered / hostile / change, change defined over feeds whose history spans both windows, null with `change_measured: false` when no feed does). Review fixes 2026-10-17: expired GDELT copies were shown as current counts; a day with no buckets in the judging series read as complete; "no change" was reported where nothing was compared; a GDELT 429 was retried after 1 s and then 2 s; unknown codes such as "GBR" were searched as text; a partial RSS outage was unmarked; limits below 1 misbehaved. All fixed with failing-first tests. Measured 2026-10-17: timeline modes answer (HTTP 200, `date_resolution` `hour` for `timespan=7d` and `15m` for `3d`, explicit zero buckets, `norm` = all monitored articles), but a `timespan=7d` request returned 54 hours (2026-10-15T10Z onward), and about half of calls drew 429 even 30 s apart (later probes, 16:51 to 16:53Z, 4 of 4 drew 429, so the resolution for 14 days and coverage before 2026-10-15T10Z are unmeasured; 30 days was not probed; a live MCP run at 17:00Z got 4 of 6 columns after the retry pass, one at 17:20Z got 1 of 6, consistent with the server's breaker opening after 3 failed calls, which also skips the retry pass). The fetcher now spaces every live GDELT call at least 5 s apart across the processes that share one cache DB (`shared_floor.py`), and retries a 429 no sooner than that; the timeline's retry pass stops at the first retry that fails again. The AOI sweep makes one GDELT call per AOI, so the floor bounds it: against a mocked GDELT, 45 AOIs took 220 s of the collector's 240 s `aoi_digest` budget and 50 timed out after 48 calls (every other AOI domain stubbed; `analysis/aoi.py` not changed here). Artlist was not used for daily counts: 250 records sorted by date covered 15 minutes. Situations see only the 22-country entity table, whose matcher still counts "South Sudan" as Sudan and "Indian Ocean" as India (`analysis/entities.py`, not changed here) |+ | 1 | Story breadth | News clusters carry a distinct-outlet count; a top-stories view ranks by how many outlets carry a story | :white_check_mark: Built 2026-10-17, unreleased: clusters gain `outlets` / `outlet_count` (sister feeds of one publisher count once); `intel_top_stories` ranks by outlet count with an attention list (countries and organizations by mention growth). Found on the way: cluster `sources` was empty in every live cluster (key mismatch, `feed_name` vs `source_name`) and RSS times were shifted by the host UTC offset; both fixed. Concurrent RSS reads now share one request per feed (a live collector news-group cycle: 119 requests for 119 feeds). Review fixes 2026-10-17: clustering runs off the event loop; attention is empty with `attention_change_measured: false` when no feed spans both windows, never "nothing grew" |-| 1 | Liquidity and flows | Fed balance sheet, Treasury General Account, reverse repo and net liquidity (FRED keyless CSV), money supply (US M2, euro area M1/M2/M3, never summed), ECB, ICI fund flows and ETF net issuance, FINRA margin statistics, BLS series; amounts kept separate from flows | :red_circle: |-| 1 | Kalshi + cross-venue markets | Kalshi public market data next to Polymarket; same-event comparison with an explicit match confidence; settled markets scored against pre-settlement prices (Brier) | :red_circle: |-| 1 | News vs market reaction | Companies and countries named in story clusters mapped to tickers / country indices; same-day move against a benchmark | :red_circle: |-| 2 | Connections with timing offsets | Correlation between retained series over a shared sample, tested at leads and lags, with minimum sample size and false-discovery control; states that it does not prove causation | :red_circle: |-| 2 | Day timeline | What every retained source recorded for a given day, by observation date or collection date | :red_circle: |-| 3 | Dashboard views | Liquidity board, situations, top stories, collection health panels | :red_circle: |+| 1 | Liquidity and flows | Fed balance sheet, Treasury General Account, reverse repo and net liquidity (FRED keyless CSV), money supply (US M2, euro area M1/M2/M3, never summed), ECB, ICI fund flows and ETF net issuance, FINRA margin statistics, BLS series; amounts kept separate from flows | :yellow_circle: Built, unreleased: 5 tools (`intel_liquidity_board`, `intel_net_liquidity`, `intel_money_supply`, `intel_fed_liquidity_series`, `intel_ecb_series`), 5 CLI commands, 2 collector sources. FRED graph CSV and ECB Data Portal live-verified 2026-10-17. The TGA term is WDTGAL (Wednesday level, USD millions): WTREGEN is a week average; the net liquidity `method` block states each term's units, conversion and how RRPONTSYD (daily, USD billions) is aligned to the Wednesday. Blocked, measured 2026-10-17, and listed in the board's `blocked` field; no proxy is substituted, so fund flows and ETF net issuance have no source: ICI (money market fund assets at /research/stats/mmf, ETF net issuance at /research/stats/etf) answers 403 Akamai "Access Denied" on every path, robots.txt included; FINRA margin statistics XLSX answers 200, but the finra.org Terms of Use (Restrictions d and e) forbid building a database from the site and data mining, scraping or harvesting tools, robots included, and the page says data feeds are not available; BLS public API v1 answers 200 without a key (CPI-U, payrolls), but api.bls.gov robots.txt is `Disallow: /` and the terms page on www.bls.gov answers 403 to this server's User-Agent, so it is not used |+| 1 | Kalshi + cross-venue markets | Kalshi public market data next to Polymarket; same-event comparison with an explicit match confidence; settled markets scored against pre-settlement prices (Brier) | :white_check_mark: Unreleased: `intel_kalshi_markets`, `intel_prediction_compare`, `intel_prediction_settled` (`sources/kalshi.py`, `sources/prediction_cross.py`, `analysis/prediction_matching.py`), CLI and 2 collector sources. Kalshi endpoints measured 2026-10-17, all HTTP 200 without an account: `/events` (nested markets), `/series` (`include_volume`), `/markets?status=settled`, `/markets/candlesticks` (batch, capped at 10,000 candles per call), `/historical/cutoff`. The `/events` `category` parameter is ignored by the API, so categories resolve through `/series?category=`, whose filter is loose too (7 of the 869 series it returned for Economics were other categories), so series are matched exactly; an unknown category answers `{"series": null}` and is reported as a gap, not an outage. The exchange-wide settled listing is intraday sports (1,000 rows covered about 40 minutes; none open 24h), so settled samples default to the Economics category's top series. Polymarket Gamma caps a page at 100 rows. Matching is deliberately conservative: live 2026-10-17 it matched the Fed 25 bps hike/cut pairs for October and December and rejected 25 vs 50+ bps, 2026 vs 2027, and the Fed-hold pair ("no change" vs "Hike 0bps": a known false negative). Team and person aliases are not resolved, so most sports pairs stay unmatched. Kalshi requests are spaced 0.2 s apart in-process: the two collector sources scanning the same series at once drew HTTP 429, and at 0.1 s a 26-request category scan still drew one (none at 0.2 s in one live run). Settled samples are the most recently settled markets and cluster in time (one football match supplied 11 of 25 Polymarket markets in a live run), so a Brier score describes its sample, not a venue's skill. Every number a caller passes has a range stated in the input schema (at most 25 Kalshi series or pages per scan), so one call cannot fan out into unbounded upstream requests |+| 1 | News vs market reaction | Companies and countries named in story clusters mapped to tickers / country indices; same-day move against a benchmark | :yellow_circle: Built 2026-10-17, unreleased; 7 test expectations await correction at merge: `intel_news_market_reaction` and `intel news-reaction`. Company rows: stock, benchmark (SPY, or the exchange country's index) and excess return on the story's session; country rows: index return with a trailing z-score. Session rule in the exchange's timezone (after close = next session; weekends and holidays read from the listing's own bars); a last bar Yahoo has not finalized is pending. Open: `^TASI` and `^TA125` (Saudi and Israel in the country-index table) return 404 on Yahoo, so those benchmarks are data gaps; story times inherit `sources/news.py`'s RSS timestamp offset until that is fixed |+| 2 | Connections with timing offsets | Correlation between retained series over a shared sample, tested at leads and lags, with minimum sample size and false-discovery control; states that it does not prove causation | :white_check_mark: Built 2026-10-17, unreleased: `intel_lagged_connections` and `intel history connections`. Pearson r on a UTC day, week or month grid at every lag from -max_lag to +max_lag (default 5, cap 30, at most 1,000 tests a call), period-over-period changes by default, p from Student's t (regularized incomplete beta, standard library), Benjamini-Hochberg q across the call (default 0.05), a 30-point minimum with every untested pair and lag listed with its n, latest vintage, rows without `observed_at` refused with the reason, an AR(1) effective sample size as a diagnostic, and a `cannot_prove` list. p-values match scipy to 3.3e-13 relative error. Verified 2026-10-17 over a live MCP stdio session and the CLI on four real FRED series loaded into an isolated store. Not done: rank correlation, prewhitening |+| 2 | Day timeline | What every retained source recorded for a given day, by observation date or collection date | :white_check_mark: Built 2026-10-17, unreleased: `intel_day_timeline` and `intel history day` (`history_timeline.py`; read-only `mode=ro` connection, never fetches). by=collected: per-source run-log state (changed, unchanged, answered, degraded, failed, not_attempted, collector_did_not_run, never_run, unknown_past_retention) with attempts, status counts, reasons and payload changes; by=observed: date and UTC-instant values on the day, month and year values listed apart as period values, undated observations counted and excluded. Retention is stated per record kind for the day. Measured 2026-10-17: two live collector cycles of 6 sources four minutes apart (12 captures: ok 8, degraded 2, error 2; every payload in the second cycle matched the first) read back over a live MCP stdio session and the CLI; 2026-10-16 in the same store reads `collector_did_not_run` (context `before_first_capture`). On a synthetic store (475,200 captures, 1.75 million observations, 10,982 series) one call took 24 to 46 ms, and no statement full-scans a history table under SQLite 3.45.3, 3.53.4 and 3.54.0. Not measured: a live store with weeks of history |+| 3 | Dashboard views | Liquidity board, situations, top stories, collection health panels | :white_check_mark: Built 2026-10-17, unreleased: four tabs behind RESEARCH (key `p`), each loaded from its own route that calls the tool's function (`/api/liquidity-board`, `/api/situations`, `/api/top-stories`, and `/api/collection-health` on the dashboard cache's history store); a failed call renders as UNAVAILABLE or REFRESH FAILED with the function's reason, never as an empty list. Checked 2026-10-17 in headless Chrome with real upstreams (9 liquidity rows; 20 situations and 9 stories over 95 of 119 feeds; collection health after one collector cycle: 38 ok, 5 degraded, 16 failing), with FRED and ECB, every RSS feed and the history store made unreachable (each tab showed the outage and its reason), and with markup injected into a headline (shown as text). Situations and top stories take 5 to 15 s per call |Considered and not planned:@@@ -650,11 -634,11 +650,11 @@@ fail-quietly bugs below were fixed in t| UNHCR outage returned all-zero global totals with no error/degraded key ("zero refugees worldwide") | `sources/displacement.py` | fail-reads-as-success | :white_check_mark: Fixed (#22) || A climate zone whose fetch failed was silently omitted; full outage yielded `{"zones": {}}` with no marker | `sources/climate.py` | silent degradation | :white_check_mark: Fixed (#23) || "Resolved: Major outage" classified as an active critical incident; a dead provider feed still listed as checked | `sources/service_status.py` | misclassification | :white_check_mark: Fixed (#24) |-| Naval warnings apply to all 9 waterways identically (no proximity/NAVAREA filter); `total_nearby` computed but never emitted | `sources/intelligence.py` vessel snapshot | precision | :red_circle: |+| Naval warnings apply to all 9 waterways identically (no proximity/NAVAREA filter); `total_nearby` computed but never emitted | `sources/intelligence.py` vessel snapshot | precision | :white_check_mark: Fixed (Unreleased): a warning counts only at waterways within 300 km of a position it states, or, when it states none, by single-NAVAREA area code (4, 12); the rule is in `matching_rule`, naval warnings no rule can place are in `unplaced_naval_warnings`; `total_nearby` emitted; an NGA outage reads `unknown`, not `clear` || Country/company/leader entity matching was substring-based ("usa" inside "thousand", "hamas" inside "Bahamas") | `analysis/entities.py` | false positives | :white_check_mark: Fixed in 0.5.0 (word-boundary alternations, measured faster) || Category keywords matched substrings ("airstrike" bumped severity via "strike"; "denied" fired "ied") | `analysis/classifier.py` | false positives | :white_check_mark: Fixed in 0.5.0 (boundary-anchored stems, per-keyword overrides) |- | Silent-empty degradation without a marker on RSS/API failure | `sources/prediction.py` (documented as intended), `sources/maritime.py`, `sources/news.py` RSS path, `sources/fleet.py` `_safe` wrapper | silent degradation | :yellow_circle: `sources/news.py` RSS path fixed 2026-10-17 (every call reports `feeds_attempted` / `feeds_ok` / `feeds_failed` / `feeds_stale`; no feed answering live is `error` + `degraded` + `reason: all_feeds_failed`, including the warm-cache case where failed feeds came back as expired copies and counted as healthy; failing-first regression tests). `maritime.py` and `fleet.py` fixed (Unreleased): outages carry `error`/`degraded`/`reason` and `data_gaps`, failed fleet components are named in `unavailable_components`, and nav warnings flag a feed whose newest warning is over 14 days old (NGA's API holds nothing issued after 10 May 2024, measured 2026-10-17). `sources/prediction.py` fixed, unreleased: an outage or unexpected response adds `error` + `degraded`, an expired-cache serve adds `stale`; keys unchanged; regression tests written failing-first (3 failed on the old module). The fetcher's stale fallback marks only dict payloads, so other text and XML sources can serve expired copies unmarked |- | Country dossier news was always empty with no marker, for every country: it asked the feed for `category="all"` (not a category, so `fetch_news_feed` answered with an error dict), read `articles` / `source` while the feed emits `items` / `feed_name`, and checked only its own `_error`, never the feed's `error`. Also, 28 of the 50 accepted codes have no headline keywords, so their search could never match | `analysis/dossier.py` news section | fail-reads-as-success | :white_check_mark: Fixed 2026-10-17, unreleased (reads every feed; a feed error or outage passes through as `error` / `degraded` / `reason` with `mention_count: null` and a data gap; failed and stale feeds named; keywordless countries report "mentions not searched"; `intel dossier` prints `News: not measured`; failing-first tests drive the real `fetch_news_feed` output) |-| Silent-empty degradation without a marker on RSS/API failure | `sources/prediction.py` (documented as intended), `sources/maritime.py`, `sources/news.py` RSS path, `sources/fleet.py` `_safe` wrapper | silent degradation | :yellow_circle: `sources/news.py` RSS path fixed 2026-10-17 (every call reports `feeds_attempted` / `feeds_ok` / `feeds_failed` / `feeds_stale`; no feed answering live is `error` + `degraded` + `reason: all_feeds_failed`, including the warm-cache case where failed feeds came back as expired copies and counted as healthy; failing-first regression tests). Since the 2026-10-17 review: fewer than half of the feeds answering live is `degraded` + `reason: partial_feeds_failed` with no error (measured that day: 95 of 119 answered, the rest permanent 404 / 403 / 410 plus one timeout), passed through by trending keywords, situations, top stories and the dossier; links are kept only when http(s) and titles are capped at 500 characters. prediction, maritime and fleet still open. The fetcher's stale fallback marks only dict payloads, so other text and XML sources can serve expired copies unmarked |++| Silent-empty degradation without a marker on RSS/API failure | `sources/prediction.py` (documented as intended), `sources/maritime.py`, `sources/news.py` RSS path, `sources/fleet.py` `_safe` wrapper | silent degradation | :yellow_circle: `sources/news.py` RSS path fixed 2026-10-17 (every call reports `feeds_attempted` / `feeds_ok` / `feeds_failed` / `feeds_stale`; no feed answering live is `error` + `degraded` + `reason: all_feeds_failed`, including the warm-cache case where failed feeds came back as expired copies and counted as healthy; failing-first regression tests). Since the 2026-10-17 review: fewer than half of the feeds answering live is `degraded` + `reason: partial_feeds_failed` with no error (measured that day: 95 of 119 answered, the rest permanent 404 / 403 / 410 plus one timeout), passed through by trending keywords, situations, top stories and the dossier; links are kept only when http(s) and titles are capped at 500 characters. `maritime.py` and `fleet.py` fixed (Unreleased): outages carry `error`/`degraded`/`reason` and `data_gaps`, failed fleet components are named in `unavailable_components`, and nav warnings flag a feed whose newest warning is over 14 days old (NGA's API holds nothing issued after 10 May 2024, measured 2026-10-17). `sources/prediction.py` fixed, unreleased: an outage or unexpected response adds `error` + `degraded`, an expired-cache serve adds `stale`; keys unchanged; regression tests written failing-first (3 failed on the old module). The fetcher's stale fallback marks only dict payloads, so other text and XML sources can serve expired copies unmarked |+ | Country dossier news was always empty with no marker, for every country: it asked the feed for `category="all"` (not a category, so `fetch_news_feed` answered with an error dict), read `articles` / `source` while the feed emits `items` / `feed_name`, and checked only its own `_error`, never the feed's `error`. Also, 28 of the 50 accepted codes have no headline keywords, so their search could never match | `analysis/dossier.py` news section | fail-reads-as-success | :white_check_mark: Fixed 2026-10-17, unreleased (reads every feed; a feed error or outage passes through as `error` / `degraded` / `reason` with `mention_count: null` and a data gap; failed and stale feeds named; keywordless countries report "mentions not searched"; `intel dossier` prints `News: not measured`; failing-first tests drive the real `fetch_news_feed` output). Since the 2026-10-17 review: headlines match on whole words (the substring test counted "Tirana" as Iran and "Indiana" as India), four compound names are removed first, and the section reports `matching_items`, `mention_count_capped`, `items_searched` and `searched_span` || 33 CLI commands rendered upstream errors as healthy empty states | `cli.py` | fail-reads-as-success | :white_check_mark: Fixed in 0.6.0 (shared bail-on-error path; 43 failing-first tests) || Rich markup swallowed lowercase bracketed values and let remote titles inject markup | `cli.py` | output corruption | :white_check_mark: Fixed in 0.6.0 (per-site escaping; injection test) || Remote data inside Rich TABLE cells was not markup-escaped | `cli.py` | output corruption | :white_check_mark: Fixed in 0.6.0 (`_cell` Text-wrapping convention, 11 injection tests; error styles unified across all 53 commands) |diff --cc src/world_intel_mcp/fetcher.pyindex e4cea13,d0fb705..7590972--- a/src/world_intel_mcp/fetcher.py+++ b/src/world_intel_mcp/fetcher.py@@@ -7,9 -7,11 +7,11 @@@ response is returned (marked with _stalimport asyncioimport logging+ import os+ import sqlite3import timefrom dataclasses import dataclass-from typing import Any+from typing import Any, Callableimport httpx