160-tool MCP server for real-time global intelligence from free public APIs: markets, liquidity, prediction markets, SEC filings, conflict, military, aircraft-cohort early warning, migration and border flows, cyber, climate, news and more. Dated observation history, 29 research prompts, cited briefs, geofences, SSE dashboard. MIT, no paid API keys.
Python ai-toolsai-watchanthropicclaudecybersecuritydashboardgeofencinggeopoliticalgeospatialintelligencemarketsmcpmigrationmilitarymodel-context-protocolnuclearosintreal-timespace-weatherthreat-intelligence
News signals: fix the 2026-10-17 review findings (H1, M1-M5, L1-L6)
Phoenix Agent (mac-studio)a5ced4a10f55
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex c8eac17..b1d7b9b 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,30 +8,41 @@articles that also use threat / sanction / force / criticism terms (oneGDELT query per category, every term listed in `queries`), GDELT's totalmonitored articles, and average tone (`timelinetone`, weighted by the- per-bucket article counts). A failed call leaves its column null for- every day; a day GDELT did not return is null in every column; both are- listed in `data_gaps`. Partial days (GDELT answered 2026-10-17 in- 15-minute buckets for 3 days and hourly buckets for 7) are flagged- `partial`, judged from the first category series that answered when the- base call failed, and today's unfinished UTC day is flagged- `in_progress`. A- failed call is retried once.+ per-bucket article counts). A failed call, or one only an expired cached+ copy answered, leaves its column null for every day (`column_status`+ per call, `stale_age_seconds` per expired column); a day GDELT did not+ return is null in every column, and a day one series lacks is null in+ that column; all are listed in `data_gaps`. Partial days (GDELT answered+ 2026-10-17 in 15-minute buckets for 3 days and hourly buckets for 7) are+ flagged `partial` per volume column (`partial_columns`,+ `coverage_by_column`), and today's unfinished UTC day is flagged+ `in_progress`. Failed and expired columns get one retry pass, which+ stops at the first retry that fails again. Countries resolve from the+ 22-country table by name, ISO3 or ISO2 code; other names are searched+ verbatim, and other 2- and 3-letter codes are refused.- `intel_situations`: country pairs ("between": a headline naming two ormore countries) and single countries ("inside": a headline naming one)ranked over a recent RSS window by most covered, most hostile (headlinekeyword heuristic, `classifier.classify_hostility`) or biggest change.Change is defined in the response: current minus prior window, countedonly over feeds whose retained history reaches back past the prior- window's start; the feeds that do not are named.+ window's start; the feeds that do not are named. With no such feed,+ change fields are null and `change_measured` is false, and a change+ sort ranks by coverage (`sort_applied`); a change figure resting on+ under half of the window's items is named a thin sample.- `intel_top_stories`: story clusters in the window carried by at least`min_outlets` distinct outlets, plus attention: the countries and- organizations whose headline mentions grew most against the prior window.+ organizations whose headline mentions grew most against the prior window+ (empty with `attention_change_measured: false` when no feed spans both+ windows).- `intel_news_clusters` clusters gain `outlets` and `outlet_count`(distinct publishers among all members; sister feeds such as BBC Worldand BBC Mundo count once). Existing keys are unchanged.- CLI: `intel hostility COUNTRY [OTHER]`, `intel situations`,`intel top-stories`. Nulls render as `n/a`, never 0; partial days are- marked `*` and today `~`.+ marked `*` (and each count from a partial column), today `~`, and+ expired columns are named with their age; unmeasured change and+ attention print as not measured.- Collector: `situations` and `top_stories` (57 sources).### Changed@@ -45,9 +56,17 @@- Live GDELT calls are spaced at least 5 s apart by the fetcher(`_SOURCE_RATE_LIMITS["gdelt"]`), for every GDELT caller: GDELT's 429body asks for "one every 5 seconds" (measured 2026-10-17). Cache hits are- not delayed. The limiter is per process, so the MCP server and the- collector do not share it, and the fetcher's own retries inside one call- are not spaced by it.+ not delayed. The floor holds across the processes that share one cache+ DB (collector, MCP servers, dashboard, CLI): every request that passes+ the fetcher's GDELT floor also reserves a slot in a `source_floor` table+ under `BEGIN IMMEDIATE` (`shared_floor.py`); a reservation that cannot+ be made is logged and the request keeps the in-process floor. A 429 from+ any source with a floor is retried no sooner than that floor+ (`get_json`; it was 1 s, then 2 s). Processes on different cache DB+ paths still do not coordinate. The AOI sweep makes one GDELT call per+ AOI, so the floor bounds it: against a mocked GDELT, 45 AOIs took 220 s+ of the collector's 240 s `aoi_digest` budget and 50 AOIs timed out+ after 48 calls (measured 2026-10-17, every other AOI domain stubbed).### Fixed- RSS outage read as a slow news day (ROADMAP Phase 24.5):@@ -94,6 +113,38 @@have no headline keywords, so the search could never match; the sectionnow says "mentions not searched" instead of reporting zero.`intel dossier` prints `News: not measured` and the gaps in that case.+- Review fixes for the news signals (independent review, 2026-10-17):+ - The hostility timeline showed an expired cached GDELT copy (days old+ when no collector evicts it) as current counts, with no marker. Such a+ column is now null, named in `data_gaps` with its age and in+ `column_status`, and the response is `degraded` with+ `reason: gdelt_stale`.+ - A day with no buckets in the series that judged completeness read as+ complete (`partial: false`), and a category series covering half a day+ sat unflagged next to full-day counts. Partial days are now judged per+ column; a half-day count gets no `hostile_share`.+ - With no comparable feed, situations reported change 0 and ratio 1.0+ ("no change"), and the CLI printed `+0`; attention read as "nothing+ grew". Both are now null or empty and marked not measured.+ - A 429 inside one GDELT call was retried after 1 s and 2 s, and the+ timeline's retry pass asked again for every failed column; see the+ GDELT spacing entry under Changed.+ - Dossier headline matching was a substring test ("Tirana" counted as+ Iran, "Indiana" as India) and hid its cap: it now matches whole words,+ removes four compound names first, and reports `matching_items`,+ `mention_count_capped`, `items_searched` and `searched_span`.+ - A near-total RSS outage (118 of 119 feeds down) read as normal:+ `fetch_news_feed` now marks a read where under half of the feeds+ answered live `degraded` with `reason: partial_feeds_failed`, and+ `intel_trending_keywords` passes the feed counts and marker through;+ situations, top stories and the dossier rank or count over the feeds+ that answered and say so.+ - Clustering ran on the event loop the collector and the MCP server+ share; unknown codes such as "GBR" were sent to GDELT as text; a limit+ of 0 or -1 returned nothing or dropped a result, and `min_articles: 0`+ divided by zero; feed links of any scheme (`javascript:`, `data:`)+ were passed on and titles had no length cap (now http(s) only, titles+ capped at 500 characters).## 0.11.0 - 2026-10-12diff --git a/README.md b/README.mdindex 2624367..331a904 100644--- a/README.md+++ b/README.md@@ -296,9 +296,9 @@ collector.py (daemon) ─┘### News Signals (3)| Tool | Description ||------|-------------|-| `intel_hostility_timeline` | Per-day GDELT coverage of a country or a pair: article count, articles using threat / sanction / force / criticism terms (GDELT keyword match, terms listed in the response), average tone. A failed call or a day GDELT did not return is null and listed in `data_gaps`, never zero |-| `intel_situations` | Situations between countries (headlines naming two or more) and inside a country (headlines naming one), ranked by most covered, most hostile (headline keyword heuristic) or biggest change against the prior window; the response defines each measure and names the feeds excluded from change figures |-| `intel_top_stories` | Stories carried by at least `min_outlets` distinct outlets (sister feeds of one publisher count once), plus the countries and organizations whose headline mentions grew most |+| `intel_hostility_timeline` | Per-day GDELT coverage of a country or a pair: article count, articles using threat / sanction / force / criticism terms (GDELT keyword match, terms listed in the response), average tone. A failed call, an expired cached copy, or a day GDELT did not return is null and listed in `data_gaps` (`column_status` per call), never zero; partial days are flagged per column |+| `intel_situations` | Situations between countries (headlines naming two or more) and inside a country (headlines naming one), ranked by most covered, most hostile (headline keyword heuristic) or biggest change against the prior window; the response defines each measure and names the feeds excluded from change figures. With no comparable feed, change is null and `change_measured` false |+| `intel_top_stories` | Stories carried by at least `min_outlets` distinct outlets (sister feeds of one publisher count once), plus the countries and organizations whose headline mentions grew most (`attention_change_measured` false when nothing could be compared) |### Intelligence Analysis (8)| Tool | Description |diff --git a/ROADMAP.md b/ROADMAP.mdindex 1b53c90..2955cb0 100644--- a/ROADMAP.md+++ b/ROADMAP.md@@ -575,8 +575,8 @@ dashboard.| 1 | Capture history and revisions | Earlier saved copies of a source, and values a later copy revised | :red_circle: || 1 | Collection health | Per-source expected cadence; overdue / failing / never-run per source, measured from the run log, not inferred from cache freshness. Says what it cannot prove | :red_circle: || 1 | Research guide over MCP | MCP resources (guide generated from the live registry and source health, measures glossary, "what this server will not do") and MCP prompts (worked research questions, each naming the tools it needs and the limit to respect). Server exposed no prompts or resources before | :red_circle: |-| 1 | Hostility timeline | Per country pair or country, daily counts of threat / sanction / force / criticism coverage plus tone; sortable by most covered, most hostile, biggest change | :white_check_mark: Built 2026-10-17, unreleased: `intel_hostility_timeline` (GDELT DOC 2.0 `timelinevolraw` + `timelinetone`, one call per category, failed calls and missing days null) and `intel_situations` (RSS headlines, ranked by covered / hostile / change, change defined over feeds whose history spans both windows). Measured 2026-10-17: timeline modes answer (HTTP 200, `date_resolution` `hour` for `timespan=7d` and `15m` for `3d`, explicit zero buckets, `norm` = all monitored articles), but a `timespan=7d` request returned 54 hours (2026-10-15T10Z onward), and about half of calls drew 429 even 30 s apart (later probes, 16:51 to 16:53Z, 4 of 4 drew 429, so the resolution for 14 days and coverage before 2026-10-15T10Z are unmeasured; 30 days was not probed; a live MCP run at 17:00Z got 4 of 6 columns after the retry pass, one at 17:20Z got 1 of 6, consistent with the server's breaker opening after 3 failed calls, which also skips the retry pass). The fetcher now spaces every live GDELT call at least 5 s apart, per process. Artlist was not used for daily counts: 250 records sorted by date covered 15 minutes. Situations see only the 22-country entity table |-| 1 | Story breadth | News clusters carry a distinct-outlet count; a top-stories view ranks by how many outlets carry a story | :white_check_mark: Built 2026-10-17, unreleased: clusters gain `outlets` / `outlet_count` (sister feeds of one publisher count once); `intel_top_stories` ranks by outlet count with an attention list (countries and organizations by mention growth). Found on the way: cluster `sources` was empty in every live cluster (key mismatch, `feed_name` vs `source_name`) and RSS times were shifted by the host UTC offset; both fixed. Concurrent RSS reads now share one request per feed (a live collector news-group cycle: 119 requests for 119 feeds) |+| 1 | Hostility timeline | Per country pair or country, daily counts of threat / sanction / force / criticism coverage plus tone (`intel_hostility_timeline`); situations between and inside countries, sortable by most covered, most hostile, biggest change (`intel_situations`) | :white_check_mark: Built 2026-10-17, unreleased: `intel_hostility_timeline` (GDELT DOC 2.0 `timelinevolraw` + `timelinetone`, one call per category; failed calls, expired cached copies and missing days null; `column_status` per call; partial days judged per column) and `intel_situations` (RSS headlines, ranked by covered / hostile / change, change defined over feeds whose history spans both windows, null with `change_measured: false` when no feed does). Review fixes 2026-10-17: expired GDELT copies were shown as current counts; a day with no buckets in the judging series read as complete; "no change" was reported where nothing was compared; a GDELT 429 was retried after 1 s and then 2 s; unknown codes such as "GBR" were searched as text; a partial RSS outage was unmarked; limits below 1 misbehaved. All fixed with failing-first tests. Measured 2026-10-17: timeline modes answer (HTTP 200, `date_resolution` `hour` for `timespan=7d` and `15m` for `3d`, explicit zero buckets, `norm` = all monitored articles), but a `timespan=7d` request returned 54 hours (2026-10-15T10Z onward), and about half of calls drew 429 even 30 s apart (later probes, 16:51 to 16:53Z, 4 of 4 drew 429, so the resolution for 14 days and coverage before 2026-10-15T10Z are unmeasured; 30 days was not probed; a live MCP run at 17:00Z got 4 of 6 columns after the retry pass, one at 17:20Z got 1 of 6, consistent with the server's breaker opening after 3 failed calls, which also skips the retry pass). The fetcher now spaces every live GDELT call at least 5 s apart across the processes that share one cache DB (`shared_floor.py`), and retries a 429 no sooner than that; the timeline's retry pass stops at the first retry that fails again. The AOI sweep makes one GDELT call per AOI, so the floor bounds it: against a mocked GDELT, 45 AOIs took 220 s of the collector's 240 s `aoi_digest` budget and 50 timed out after 48 calls (every other AOI domain stubbed; `analysis/aoi.py` not changed here). Artlist was not used for daily counts: 250 records sorted by date covered 15 minutes. Situations see only the 22-country entity table, whose matcher still counts "South Sudan" as Sudan and "Indian Ocean" as India (`analysis/entities.py`, not changed here) |+| 1 | Story breadth | News clusters carry a distinct-outlet count; a top-stories view ranks by how many outlets carry a story | :white_check_mark: Built 2026-10-17, unreleased: clusters gain `outlets` / `outlet_count` (sister feeds of one publisher count once); `intel_top_stories` ranks by outlet count with an attention list (countries and organizations by mention growth). Found on the way: cluster `sources` was empty in every live cluster (key mismatch, `feed_name` vs `source_name`) and RSS times were shifted by the host UTC offset; both fixed. Concurrent RSS reads now share one request per feed (a live collector news-group cycle: 119 requests for 119 feeds). Review fixes 2026-10-17: clustering runs off the event loop; attention is empty with `attention_change_measured: false` when no feed spans both windows, never "nothing grew" || 1 | Liquidity and flows | Fed balance sheet, Treasury General Account, reverse repo and net liquidity (FRED keyless CSV), money supply (US M2, euro area M1/M2/M3, never summed), ECB, ICI fund flows and ETF net issuance, FINRA margin statistics, BLS series; amounts kept separate from flows | :red_circle: || 1 | Kalshi + cross-venue markets | Kalshi public market data next to Polymarket; same-event comparison with an explicit match confidence; settled markets scored against pre-settlement prices (Brier) | :red_circle: || 1 | News vs market reaction | Companies and countries named in story clusters mapped to tickers / country indices; same-day move against a benchmark | :red_circle: |@@ -637,8 +637,8 @@ fail-quietly bugs below were fixed in the same-day follow-up| Naval warnings apply to all 9 waterways identically (no proximity/NAVAREA filter); `total_nearby` computed but never emitted | `sources/intelligence.py` vessel snapshot | precision | :red_circle: || Country/company/leader entity matching was substring-based ("usa" inside "thousand", "hamas" inside "Bahamas") | `analysis/entities.py` | false positives | :white_check_mark: Fixed in 0.5.0 (word-boundary alternations, measured faster) || Category keywords matched substrings ("airstrike" bumped severity via "strike"; "denied" fired "ied") | `analysis/classifier.py` | false positives | :white_check_mark: Fixed in 0.5.0 (boundary-anchored stems, per-keyword overrides) |-| Silent-empty degradation without a marker on RSS/API failure | `sources/prediction.py` (documented as intended), `sources/maritime.py`, `sources/news.py` RSS path, `sources/fleet.py` `_safe` wrapper | silent degradation | :yellow_circle: `sources/news.py` RSS path fixed 2026-10-17 (every call reports `feeds_attempted` / `feeds_ok` / `feeds_failed` / `feeds_stale`; no feed answering live is `error` + `degraded` + `reason: all_feeds_failed`, including the warm-cache case where failed feeds came back as expired copies and counted as healthy; failing-first regression tests). prediction, maritime and fleet still open. The fetcher's stale fallback marks only dict payloads, so other text and XML sources can serve expired copies unmarked |-| Country dossier news was always empty with no marker, for every country: it asked the feed for `category="all"` (not a category, so `fetch_news_feed` answered with an error dict), read `articles` / `source` while the feed emits `items` / `feed_name`, and checked only its own `_error`, never the feed's `error`. Also, 28 of the 50 accepted codes have no headline keywords, so their search could never match | `analysis/dossier.py` news section | fail-reads-as-success | :white_check_mark: Fixed 2026-10-17, unreleased (reads every feed; a feed error or outage passes through as `error` / `degraded` / `reason` with `mention_count: null` and a data gap; failed and stale feeds named; keywordless countries report "mentions not searched"; `intel dossier` prints `News: not measured`; failing-first tests drive the real `fetch_news_feed` output) |+| Silent-empty degradation without a marker on RSS/API failure | `sources/prediction.py` (documented as intended), `sources/maritime.py`, `sources/news.py` RSS path, `sources/fleet.py` `_safe` wrapper | silent degradation | :yellow_circle: `sources/news.py` RSS path fixed 2026-10-17 (every call reports `feeds_attempted` / `feeds_ok` / `feeds_failed` / `feeds_stale`; no feed answering live is `error` + `degraded` + `reason: all_feeds_failed`, including the warm-cache case where failed feeds came back as expired copies and counted as healthy; failing-first regression tests). Since the 2026-10-17 review: fewer than half of the feeds answering live is `degraded` + `reason: partial_feeds_failed` with no error (measured that day: 95 of 119 answered, the rest permanent 404 / 403 / 410 plus one timeout), passed through by trending keywords, situations, top stories and the dossier; links are kept only when http(s) and titles are capped at 500 characters. prediction, maritime and fleet still open. The fetcher's stale fallback marks only dict payloads, so other text and XML sources can serve expired copies unmarked |+| Country dossier news was always empty with no marker, for every country: it asked the feed for `category="all"` (not a category, so `fetch_news_feed` answered with an error dict), read `articles` / `source` while the feed emits `items` / `feed_name`, and checked only its own `_error`, never the feed's `error`. Also, 28 of the 50 accepted codes have no headline keywords, so their search could never match | `analysis/dossier.py` news section | fail-reads-as-success | :white_check_mark: Fixed 2026-10-17, unreleased (reads every feed; a feed error or outage passes through as `error` / `degraded` / `reason` with `mention_count: null` and a data gap; failed and stale feeds named; keywordless countries report "mentions not searched"; `intel dossier` prints `News: not measured`; failing-first tests drive the real `fetch_news_feed` output). Since the 2026-10-17 review: headlines match on whole words (the substring test counted "Tirana" as Iran and "Indiana" as India), four compound names are removed first, and the section reports `matching_items`, `mention_count_capped`, `items_searched` and `searched_span` || 33 CLI commands rendered upstream errors as healthy empty states | `cli.py` | fail-reads-as-success | :white_check_mark: Fixed in 0.6.0 (shared bail-on-error path; 43 failing-first tests) || Rich markup swallowed lowercase bracketed values and let remote titles inject markup | `cli.py` | output corruption | :white_check_mark: Fixed in 0.6.0 (per-site escaping; injection test) || Remote data inside Rich TABLE cells was not markup-escaped | `cli.py` | output corruption | :white_check_mark: Fixed in 0.6.0 (`_cell` Text-wrapping convention, 11 injection tests; error styles unified across all 53 commands) |diff --git a/src/world_intel_mcp/analysis/clustering.py b/src/world_intel_mcp/analysis/clustering.pyindex 0a4a0b0..a221f5c 100644--- a/src/world_intel_mcp/analysis/clustering.py+++ b/src/world_intel_mcp/analysis/clustering.py@@ -123,13 +123,18 @@ async def fetch_news_clusters(see ``news.FEED_OUTLETS``). An RSS outage is passed through as``degraded``/``reason``/``error``, never a quiet cluster-free day."""+ import asynciofrom datetime import datetime, timezonefrom ..sources import newsfeed_data = await news.fetch_news_feed(fetcher, category=category, limit=limit)items = feed_data.get("items", [])- raw_clusters = cluster_articles(items, similarity_threshold=threshold)+ # Pairwise Jaccard is quadratic in titles: off the event loop the+ # collector and the MCP server share.+ raw_clusters = await asyncio.to_thread(+ cluster_articles, items, similarity_threshold=threshold+ )out_clusters = []for c in raw_clusters:@@ -186,7 +191,10 @@ async def fetch_news_clusters("source": "jaccard-clustering","timestamp": datetime.now(timezone.utc).isoformat(),}- for key in ("feeds_ok", "feeds_failed", "feeds_stale", "error", "degraded", "reason"):+ for key in (+ "feeds_attempted", "feeds_ok", "feeds_failed", "feeds_stale",+ "error", "degraded", "reason",+ ):if key in feed_data:result[key] = feed_data[key]return resultdiff --git a/src/world_intel_mcp/analysis/dossier.py b/src/world_intel_mcp/analysis/dossier.pyindex 4bf2937..ef2b584 100644--- a/src/world_intel_mcp/analysis/dossier.py+++ b/src/world_intel_mcp/analysis/dossier.py@@ -10,11 +10,63 @@ All sub-queries are run in parallel for speed.from __future__ import annotationsimport asyncio+import functoolsimport logging+import refrom datetime import datetime, timezonelogger = logging.getLogger("world-intel-mcp.analysis.dossier")+# Mentions listed in the news section; the match count is not capped.+_MAX_MENTIONS = 15++# Compound names that contain a tier-1 keyword but name somewhere else.+# They are removed before the whole-word keyword test (checked 2026-10-17+# against headlines the plain keyword test counted for the wrong country).+_COMPOUND_EXCLUSIONS: dict[str, re.Pattern[str]] = {+ "SDN": re.compile(r"\bsouth sudan\w*"),+ "IND": re.compile(r"\bindian oceans?\b"),+ "USA": re.compile(r"\b(?:latin|south|central|north) americans?\b"),+ "MEX": re.compile(r"\bnew mexic\w*"),+}++_MATCH_METHOD = (+ "whole-word match of each headline against the country's tier-1 keywords "+ "(name, demonym, capital; plurals of words of five letters or more), so "+ "'Tirana' is not Iran and 'Indiana' is not India; South Sudan, the Indian "+ "Ocean, Latin/South/Central/North American and New Mexico are removed "+ "first. Other compound names still count (Lagos in Portugal, the Republic "+ "of the Congo). Searched: the newest items across every RSS feed."+)++[email protected]_cache(maxsize=None)+def _keyword_pattern(iso3: str) -> re.Pattern[str] | None:+ from ..config.countries import TIER1_COUNTRIES++ keywords = (TIER1_COUNTRIES.get(iso3) or {}).get("keywords") or []+ if not keywords:+ return None+ parts = []+ for kw in sorted(keywords, key=len, reverse=True):+ body = re.escape(kw.lower())+ if kw.isalpha() and len(kw) >= 5:+ body += "s?"+ parts.append(body)+ return re.compile(r"\b(?:" + "|".join(parts) + r")\b")+++def _names_country(title: str, iso3: str) -> bool:+ """Whether a headline names the tier-1 country ``iso3`` (whole words)."""+ pattern = _keyword_pattern(iso3)+ if pattern is None:+ return False+ text = (title or "").lower()+ exclusion = _COMPOUND_EXCLUSIONS.get(iso3)+ if exclusion is not None:+ text = exclusion.sub(" ", text)+ return pattern.search(text) is not None+# Minimal ISO-2 <-> ISO-3 mapping for the 22 tier-1 countries + major extras._ISO2_TO_ISO3: dict[str, str] = {"US": "USA", "CN": "CHN", "RU": "RUS", "UA": "UKR", "SY": "SYR",@@ -181,9 +233,13 @@ async def fetch_country_dossier(# and "source" (the producer emits "items" and "feed_name") and checked# only "_error": the section was always empty, with no marker. A failed# feed is now a gap with mention_count None, never zero mentions.+ # Headlines are matched on whole words (a substring test counted+ # "Tirana" as Iran); mention_count stays the listed (capped) count and+ # matching_items counts every match among the items searched.news_mentions = []news_gaps: list[str] = []news_marker: dict = {}+ news_reach: dict = {}country_keywords = country_config.get("keywords")if "_error" in news_data:news_gaps.append(f"news feed failed: {news_data['_error']}; mentions not measured")@@ -201,9 +257,13 @@ async def fetch_country_dossier("countries have them); mentions not searched")else:- for article in news_data.get("items", []):- title = (article.get("title") or "").lower()- if any(kw in title for kw in country_keywords):+ items = news_data.get("items", [])+ matching = 0+ for article in items:+ if not _names_country(article.get("title") or "", iso3):+ continue+ matching += 1+ if len(news_mentions) < _MAX_MENTIONS:news_mentions.append({"title": article.get("title"),"source": article.get("feed_name"),@@ -211,8 +271,24 @@ async def fetch_country_dossier("link": article.get("link"),"category": article.get("category"),})- if len(news_mentions) >= 15:- break+ stamps = sorted(+ a["published"] for a in items if isinstance(a.get("published"), str) and a["published"]+ )+ news_reach = {+ "matching_items": matching,+ "mention_count_capped": matching > len(news_mentions),+ "items_searched": len(items),+ "searched_span": {"oldest": stamps[0], "newest": stamps[-1]} if stamps else None,+ "match_method": _MATCH_METHOD,+ }+ if news_data.get("degraded"):+ # Fewer than half of the feeds answered live (fetch_news_feed's+ # partial_feeds_failed): counted, but over a minority.+ news_marker = {k: news_data[k] for k in ("degraded", "reason") if k in news_data}+ news_gaps.append(+ f"only {news_data.get('feeds_ok')} of {news_data.get('feeds_attempted')} RSS "+ "feeds answered live; mention counts rest on a minority of the feeds"+ )failed_feeds = news_data.get("feeds_failed") or []if failed_feeds:news_gaps.append(@@ -225,7 +301,7 @@ async def fetch_country_dossier(f"{len(stale_feeds)} RSS feeds were served from an expired cache copy: "f"{', '.join(stale_feeds)}")- news_measured = not news_marker and bool(country_keywords)+ news_measured = "error" not in news_marker and bool(country_keywords)# --- Section 7: Security (hotspots + conflict zones) ---associated_hotspots = []@@ -260,6 +336,7 @@ async def fetch_country_dossier("news": {"mentions": news_mentions,"mention_count": len(news_mentions) if news_measured else None,+ **news_reach,**news_marker,"data_gaps": news_gaps,},diff --git a/src/world_intel_mcp/analysis/news_signals.py b/src/world_intel_mcp/analysis/news_signals.pyindex 6772644..bc9f7da 100644--- a/src/world_intel_mcp/analysis/news_signals.py+++ b/src/world_intel_mcp/analysis/news_signals.py@@ -14,9 +14,12 @@ Three views over coverage the server already reads:outlets, plus the countries and organizations whose coverage grew most.Honesty contract: a failed fetch is a data gap, never a zero. A day GDELT-did not return is null, not a quiet day; a feed that cannot reach back-across the comparison window, or that came back as an expired cached-copy, is excluded from change figures and named.+did not return is null, not a quiet day; a GDELT column that came back+only as an expired cached copy is null and named with its age, never shown+as current counts; a feed that cannot reach back across the comparison+window, or that came back as an expired cached copy, is excluded from+change figures and named. When no feed can be compared, change is null+and ``change_measured`` is false, never "no change".Gaps / not covered:- Hostility categories are keyword heuristics (GDELT's own keyword match@@ -25,13 +28,20 @@ Gaps / not covered:- Country detection uses ``entities.extract_entities`` (the 22-countryTIER1 table plus leader names); countries outside it (France, Egypt,Saudi Arabia, ...) are invisible to ``fetch_situations`` and to the- attention ranking. The hostility timeline searches any name verbatim.+ attention ranking. Its whole-word match does not know compound names:+ it counts "South Sudan" as Sudan, "Indian Ocean" as India, "Latin+ American" as the United States and "New Mexico" as Mexico (checked+ 2026-10-17). The hostility timeline searches any name verbatim and+ refuses unresolved 2- and 3-letter codes.- RSS feeds keep different amounts of history; change figures use onlyfeeds whose oldest retained item predates the prior window, so a feedthat keeps ten items never fakes a surge, but it also never counts.-- When the base GDELT call fails, partial days are judged from a category- series' buckets. That every series shares the base series' bucket grid- was measured on one query (``Iran Israel``, 2026-10-17), not in general.+- Partial days are judged per GDELT column from the hours its buckets+ cover. GDELT sent explicit zero buckets on the one query measured+ (``Iran Israel``, 2026-10-17); if it omits them for a sparse query, quiet+ hours read as missing and the day is over-flagged as partial, never+ under-flagged. Tone is not judged: a missing hour narrows a day's+ average tone but does not shrink it the way it shrinks a count."""from __future__ import annotations@@ -48,6 +58,7 @@ from ..config.countries import TIER1_COUNTRIESfrom ..sources import newsfrom .classifier import HOSTILITY_CATEGORIES, HOSTILITY_METHOD, classify_hostilityfrom .clustering import _tokenize, cluster_articles+from .dossier import _ISO2_TO_ISO3from .entities import extract_entitieslogger = logging.getLogger("world-intel-mcp.analysis.news_signals")@@ -193,8 +204,37 @@ class _Windows:"feeds_stale": self.stale_feeds,}+ @property+ def change_measured(self) -> bool:+ """True when at least one feed covers both windows."""+ return bool(self.comparable_feeds)++ def change_gaps(self, what: str) -> list[str]:+ """Why ``what`` (change or attention) is unmeasured or thin."""+ if not self.change_measured:+ return [+ f"{what} not measured: no feed answered live with history back to "+ f"{_iso(self.prior_start)} (the prior window's start), so no feed "+ "covers both windows; change, change_ratio and prior are null, not zero"+ ]+ comparable = sum(1 for r in self.current if r.get("feed_name") in self.comparable_feeds)+ if comparable * 2 < len(self.current):+ return [+ f"{what} figures rest on {comparable} of {len(self.current)} "+ f"current-window items ({len(self.comparable_feeds)} comparable feeds); "+ "a thin sample"+ ]+ return []+def gaps(self, feed: dict) -> list[str]:out: list[str] = []+ if feed.get("degraded") and not feed.get("error"):+ # fetch_news_feed marks a read where fewer than half of the+ # feeds answered live (reason partial_feeds_failed).+ out.append(+ f"only {feed.get('feeds_ok')} of {feed.get('feeds_attempted')} RSS feeds "+ "answered live; rankings rest on a minority of the feeds"+ )failed = feed.get("feeds_failed") or []if failed:out.append(@@ -238,12 +278,29 @@ def _change_fields(current_comparable: int, prior: int) -> dict:}+# With no comparable feed, a 0 - 0 change would read as "no change".+_UNMEASURED_CHANGE = {+ "current_comparable": None,+ "prior": None,+ "change": None,+ "change_ratio": None,+}+++def _feed_marker(feed: dict) -> dict:+ """A partial RSS outage's degraded/reason, passed on with the ranking."""+ return {k: feed[k] for k in ("degraded", "reason") if k in feed}++_CHANGE_DEFINITION = ("change = articles in the current window minus articles in the prior ""window of equal length, both counted only over comparable feeds (feeds ""whose oldest retained item is at or before the prior window's start). ""change_ratio = (current_comparable + 1) / (prior + 1). 'change' sort "- "orders by change, then by current articles."+ "orders by change, then by current articles. With no comparable feed "+ "nothing is compared: change_measured (attention_change_measured for "+ "attention) is false, the change fields are null and attention is empty, "+ "and a 'change' sort ranks by coverage instead (sort_applied).")@@ -291,9 +348,11 @@ GDELT_HOSTILITY_TERMS: dict[str, list[str]] = {}# GDELT asks for at most one request per 5 s (its 429 body, measured-# 2026-10-17). The fetcher enforces that floor for every source="gdelt"-# call (fetcher._SOURCE_RATE_LIMITS; cache hits are not delayed). This is-# the extra pause before the one retry pass over failed calls.+# 2026-10-17). The fetcher enforces that floor for every live source="gdelt"+# request, across the processes that share one cache DB+# (fetcher._SOURCE_RATE_LIMITS, shared_floor; cache hits are not delayed),+# and backs off a 429 by at least the floor. This is the extra pause before+# each call of the one retry pass over failed calls._GDELT_RETRY_WAIT_S = 10.0# The DOC API serves roughly the last three months.@@ -306,26 +365,51 @@ _MAX_DAYS = 90# daily total (732) as hourly mode. Other resolutions are unmeasured._SUBDAILY_RESOLUTIONS = {"15m", "hour"}+# The five volume columns, in response order; "tone" is the sixth call.+_VOLUME_COLUMNS = ("articles", *GDELT_HOSTILITY_TERMS)++# How a column's GDELT call ended, worst first: "failed" (nothing usable),+# "stale" (the live call failed and the fetcher served an expired cached+# copy, which is excluded), "ok" (a live answer or a cache entry within+# its TTL). A failed retry never replaces a better earlier outcome.+_STATUS_RANK = {"failed": 0, "stale": 1, "ok": 2}+def _gdelt_term(name: str) -> str:return f'"{name}"' if (" " in name or "-" in name) else namedef _resolve_country(value: str) -> tuple[str, str | None]:- """(search name, iso3 or None) for a country given as ISO3 or name."""+ """(search name, iso3 or None) for a country given as a name or code.++ Resolves the 22-country table's ISO3 codes, the ISO2 codes ``dossier``+ maps onto it (IR, UA, ...), its names and its keywords.+ """raw = (value or "").strip()- if raw.upper() in TIER1_COUNTRIES:- iso3 = raw.upper()+ upper = raw.upper()+ iso3 = upper if upper in TIER1_COUNTRIES else _ISO2_TO_ISO3.get(upper)+ if iso3 in TIER1_COUNTRIES:return TIER1_COUNTRIES[iso3]["name"], iso3lower = raw.lower()- for iso3, info in TIER1_COUNTRIES.items():+ for code, info in TIER1_COUNTRIES.items():if lower == info["name"].lower() or lower in (k.lower() for k in info["keywords"]):- return info["name"], iso3+ return info["name"], codereturn raw, None+def _looks_like_code(value: str) -> bool:+ """A 2- or 3-letter input: searched verbatim, GDELT would count the+ letters ("GBR", "fra"), not the country, and reject 2-letter terms."""+ raw = (value or "").strip()+ return raw.isascii() and raw.isalpha() and 2 <= len(raw) <= 3++async def _gdelt_timeline(fetcher, query: str, mode: str, days: int) -> dict | None:- """One GDELT timeline call (the fetcher spaces live GDELT calls)."""+ """One GDELT timeline call (the fetcher spaces live GDELT calls).++ When the live call fails the fetcher may return an expired cached copy,+ marked ``_stale`` with ``_stale_age_seconds``; ``_call_outcome`` reads it.+ """digest = hashlib.sha1(f"{mode}|{days}|{query}".encode()).hexdigest()[:16]cache_key = f"news_signals:gdelt:{mode}:{days}:{digest}"data = await fetcher.get_json(@@ -341,6 +425,20 @@ async def _gdelt_timeline(fetcher, query: str, mode: str, days: int) -> dict | Nreturn data+def _call_outcome(data: dict | None) -> tuple[str, float | None]:+ """(status, age in seconds of the expired copy when status is "stale")."""+ if data is None:+ return "failed", None+ if data.get("_stale"):+ age = data.get("_stale_age_seconds")+ return "stale", float(age) if isinstance(age, (int, float)) else None+ return "ok", None+++def _age_text(age: float | None) -> str:+ return f"{age / 3600:.1f} h old" if age is not None else "age unknown"++def _series_by_day(data: dict) -> tuple[dict[str, list[dict]], str | None]:"""Group a timeline's first series into UTC days (YYYY-MM-DD)."""resolution = (data.get("query_details") or {}).get("date_resolution")@@ -354,6 +452,11 @@ def _series_by_day(data: dict) -> tuple[dict[str, list[dict]], str | None]:return days, resolution+def _hours_of(points: list[dict]) -> set[str]:+ """The UTC hours ("00".."23") a day's buckets fall in."""+ return {str(p["date"])[9:11] for p in points if len(str(p["date"])) >= 11}++async def fetch_hostility_timeline(fetcher,country: str,@@ -365,20 +468,37 @@ async def fetch_hostility_timeline(Six GDELT calls: matching-article volume, one volume per hostilitycategory (the base query AND that category's terms), and average- tone. A call that fails leaves its column null for every day and is- listed in ``data_gaps``; a day GDELT did not return is null in every- column. Neither is ever reported as zero.+ tone. A call that fails, or that only an expired cached copy answers,+ leaves its column null for every day; ``column_status`` and+ ``data_gaps`` name it (with the copy's age). A day GDELT did not return+ is null in every column, and a day one series lacks is null in that+ column. None of these is ever reported as zero. An unresolved 2- or+ 3-letter input is refused before any call: it reads as a country code."""now = now or datetime.now(timezone.utc)days = max(1, min(int(days), _MAX_DAYS))- name_a, iso_a = _resolve_country(country)- parties = [{"input": country, "name": name_a, "iso3": iso_a}]- base_query = _gdelt_term(name_a)- if other:- name_b, iso_b = _resolve_country(other)- parties.append({"input": other, "name": name_b, "iso3": iso_b})- base_query = f"{base_query} {_gdelt_term(name_b)}"+ parties = []+ for value in [country] + ([other] if other else []):+ name, iso3 = _resolve_country(value)+ parties.append({"input": value, "name": name, "iso3": iso3})+ codes = [p["input"] for p in parties if p["iso3"] is None and _looks_like_code(p["input"])]+ if codes:+ return {+ "parties": parties,+ "days_requested": days,+ "days": [],+ "error": "; ".join(+ f"'{c}' looks like a country code but is not in the "+ f"{len(TIER1_COUNTRIES)}-country table"+ for c in codes+ )+ + "; pass the country's name instead (names are searched verbatim)",+ "source": "gdelt-doc-2.0",+ "timestamp": _iso(now),+ }++ base_query = " ".join(_gdelt_term(p["name"]) for p in parties)queries = {"articles": base_query}for cat, terms in GDELT_HOSTILITY_TERMS.items():queries[cat] = f"{base_query} ({' OR '.join(terms)})"@@ -393,47 +513,93 @@ async def fetch_hostility_timeline(jobs = [(col, q, "timelinevolraw") for col, q in queries.items()]jobs.append(("tone", base_query, "timelinetone"))fetched: dict[str, dict] = {}- for col, query, mode in jobs:- data = await _gdelt_timeline(fetcher, query, mode, days)- if data is not None:+ status: dict[str, str] = {}+ stale_age: dict[str, float | None] = {}++ def _record(col: str, data: dict | None) -> None:+ outcome, age = _call_outcome(data)+ if col in status and _STATUS_RANK[outcome] < _STATUS_RANK[status[col]]:+ return+ status[col] = outcome+ if outcome == "ok":fetched[col] = data- # One spaced retry for failed calls: measured 2026-10-17, about half of- # GDELT calls drew a 429 even 30 s apart, and most succeeded on retry.+ stale_age.pop(col, None)+ elif outcome == "stale":+ stale_age[col] = age+for col, query, mode in jobs:- if col in fetched or not fetcher.breaker.is_available("gdelt"):- continue+ _record(col, await _gdelt_timeline(fetcher, query, mode, days))+ # One spaced retry pass over failed and expired columns: measured+ # 2026-10-17, about half of GDELT calls drew a 429 even 30 s apart, and+ # most succeeded on retry. A retry that fails again means GDELT is still+ # refusing, so the pass stops there rather than queue more requests.+ pending = [job for job in jobs if status[job[0]] != "ok"]+ for i, (col, query, mode) in enumerate(pending):+ rest = [c for c, _q, _m in pending[i:]]+ if not fetcher.breaker.is_available("gdelt"):+ data_gaps.append(+ "retry pass skipped: the GDELT circuit breaker is open after repeated "+ "failures; not retried: " + ", ".join(rest)+ )+ breakawait asyncio.sleep(_GDELT_RETRY_WAIT_S)- data = await _gdelt_timeline(fetcher, query, mode, days)- if data is not None:- fetched[col] = data+ _record(col, await _gdelt_timeline(fetcher, query, mode, days))+ if status[col] != "ok":+ if rest[1:]:+ data_gaps.append(+ f"retry pass stopped after {col} failed again; not retried: "+ + ", ".join(rest[1:])+ )+ break- by_series: dict[str, dict[str, list[dict]]] = {}- resolution: str | None = Nonefor col, _query, mode in jobs:- if col not in fetched:- kind = "tone" if mode == "timelinetone" else "volume"+ kind = "tone" if mode == "timelinetone" else "volume"+ if status[col] == "stale":+ data_gaps.append(+ f"{col}: GDELT {kind} timeline call failed and only an expired cached copy "+ f"({_age_text(stale_age[col])}) was available; column is null, not shown "+ "as current counts"+ )+ elif status[col] == "failed":data_gaps.append(f"{col}: GDELT {kind} timeline fetch failed; column is null")- continue- series, res = _series_by_day(fetched[col])- by_series[col] = series- resolution = resolution or res++ by_series: dict[str, dict[str, list[dict]]] = {}+ resolution: str | None = None+ for col, _query, _mode in jobs:+ if col in fetched:+ series, res = _series_by_day(fetched[col])+ by_series[col] = series+ resolution = resolution or restone_days = by_series.pop("tone", None)+ any_stale = "stale" in status.values()base_out = {"parties": parties,"days_requested": days,"queries": dict(queries, tone=base_query),+ "column_status": {col: status[col] for col, _q, _m in jobs},+ "stale_age_seconds": {+ col: stale_age[col] for col, _q, _m in jobs if status[col] == "stale"+ },"source": "gdelt-doc-2.0","timestamp": _iso(now),}if not by_series and tone_days is None:out = dict(base_out)+ if any_stale:+ n_stale = sum(1 for s in status.values() if s == "stale")+ error = (+ f"Every GDELT timeline call failed; {n_stale} of {len(jobs)} columns came "+ "back only as an expired cached copy, which is never shown as current counts"+ )+ else:+ error = "Every GDELT timeline call failed (rate limited, API error, or malformed response)"out.update({"days": [],- "error": "Every GDELT timeline call failed (rate limited, API error, or malformed response)",+ "error": error,"degraded": True,- "reason": "gdelt_fetch_failed",+ "reason": "gdelt_stale" if any_stale else "gdelt_fetch_failed","data_gaps": data_gaps,})@@ -445,78 +611,96 @@ async def fetch_hostility_timeline(f"GDELT date_resolution {resolution!r} is not one this server has measured; ""partial-day detection is off")- # The series whose buckets judge partial days: the base series, or when- # that call failed, the first category series that answered (measured- # 2026-10-17: category series carry the base series' bucket grid, zeros- # included). Without one, a surviving column could show a partial day's- # count unflagged.- coverage_col = next((c for c in ("articles", *GDELT_HOSTILITY_TERMS) if c in by_series), None)- if subdaily and coverage_col is None:+ volume_cols = [c for c in _VOLUME_COLUMNS if c in by_series]+ # The series that gives a row's buckets_returned and hours_covered: the+ # base series, or the first category series that answered when the base+ # call failed. partial itself is judged per column.+ judge_col = volume_cols[0] if volume_cols else None+ if subdaily and judge_col is None:data_gaps.append("no volume timeline answered, so partial days are not flagged")- elif subdaily and coverage_col != "articles":+ elif subdaily and judge_col != "articles":data_gaps.append(- f"partial days are judged from the {coverage_col} series' buckets "- "(the base articles call failed)"+ f"buckets_returned and hours_covered are judged from the {judge_col} series "+ "(the base articles call failed); partial is judged per column "+ "(coverage_by_column)")+ all_series = dict(by_series)+ if tone_days is not None:+ all_series["tone"] = tone_days+today = now.date()rows = []missing_days: list[str] = []+ col_missing: dict[str, list[str]] = defaultdict(list)for offset in range(days - 1, -1, -1):day = (today - timedelta(days=offset)).isoformat()- base_points = by_series["articles"].get(day) if "articles" in by_series else None- coverage_points = by_series[coverage_col].get(day) if coverage_col else None+ points = {c: series.get(day) or [] for c, series in all_series.items()}+ returned = [c for c, pts in points.items() if pts]+ if not returned:+ missing_days.append(day)+ else:+ for c in all_series:+ if not points[c]:+ col_missing[c].append(day)# Today's UTC day has not ended: its totals are not comparable with# a full day's, whatever GDELT returned for the hours so far.row: dict[str, Any] = {"date": day, "in_progress": day == today.isoformat()}- returned = None- if coverage_col is not None:- returned = len(coverage_points or [])- row["buckets_returned"] = returned- if subdaily and returned is not None:- # partial = some elapsed hour of the day has no bucket at all- # (missing data). Today counts only wholly elapsed hours, so the- # current one may be absent: GDELT's newest bucket lags the clock.- hours = {- str(p["date"])[9:11] for p in coverage_points or [] if len(str(p["date"])) >= 11- }- elapsed = now.hour if row["in_progress"] else 24- missing_hours = sorted({f"{h:02d}" for h in range(elapsed)} - hours)- row["hours_covered"] = len(hours)- row["partial"] = bool(hours) and bool(missing_hours)- if row["partial"]:- row["hours_missing"] = missing_hours- if "articles" in by_series:- if not base_points:- row["articles"] = None- missing_days.append(day)- else:- row["articles"] = int(sum(p["value"] for p in base_points))- norms = [p.get("norm") for p in base_points]- row["all_monitored_articles"] = (- int(sum(norms)) if all(isinstance(n, (int, float)) for n in norms) else None- )+ row["buckets_returned"] = len(points[judge_col]) if judge_col else None+ hours = {c: _hours_of(points[c]) for c in volume_cols}+ if subdaily and judge_col is not None:+ # A column is partial on a day when some elapsed hour has no+ # bucket at all, or the day has none (missing data, never a+ # quiet hour: GDELT sends zero buckets). Today counts only wholly+ # elapsed hours: GDELT's newest bucket lags the clock.+ elapsed = {f"{h:02d}" for h in range(now.hour if row["in_progress"] else 24)}+ coverage: dict[str, dict] = {}+ partial_columns: list[str] = []+ hours_missing: set[str] = set()+ for c in volume_cols:+ gone = elapsed - hours[c]+ col_partial = not points[c] or bool(gone)+ coverage[c] = {+ "buckets_returned": len(points[c]),+ "hours_covered": len(hours[c]),+ "partial": col_partial,+ }+ if col_partial:+ partial_columns.append(c)+ hours_missing |= gone+ row["hours_covered"] = len(hours[judge_col])+ row["partial"] = bool(partial_columns)+ row["partial_columns"] = partial_columns+ if partial_columns:+ row["hours_missing"] = sorted(hours_missing)+ row["coverage_by_column"] = coverage+ base_points = points.get("articles")+ if base_points:+ row["articles"] = int(sum(p["value"] for p in base_points))+ norms = [p.get("norm") for p in base_points]+ row["all_monitored_articles"] = (+ int(sum(norms)) if all(isinstance(n, (int, float)) for n in norms) else None+ )else:row["articles"] = Nonehostile: dict[str, int | None] = {}for cat in GDELT_HOSTILITY_TERMS:- if cat not in by_series:- hostile[cat] = None- continue# GDELT returns explicit zero buckets (measured 2026-10-17: a# category series carried the base series' exact buckets,# zeros included), so a day with no buckets is missing data.- pts = by_series[cat].get(day)+ pts = points.get(cat)hostile[cat] = int(sum(p["value"] for p in pts)) if pts else Nonerow["hostile"] = hostileif row["articles"]:+ # A count over hours the base count does not cover is not a share.row["hostile_share"] = {- cat: (round(v / row["articles"], 4) if v is not None else None)+ cat: (+ round(v / row["articles"], 4)+ if v is not None and not (subdaily and hours[cat] != hours["articles"])+ else None+ )for cat, v in hostile.items()}- row["tone"] = _day_tone(- (tone_days or {}).get(day) if tone_days is not None else None,- base_points,- )+ row["tone"] = _day_tone(points.get("tone"), base_points)rows.append(row)if missing_days:@@ -524,6 +708,14 @@ async def fetch_hostility_timeline(f"GDELT returned no buckets for {len(missing_days)} of {days} requested days "f"({missing_days[0]} .. {missing_days[-1]}); those days are null, not zero")+ for c in all_series:+ if col_missing.get(c):+ gone_days = col_missing[c]+ data_gaps.append(+ f"{c}: GDELT returned no buckets on {len(gone_days)} of {days} requested "+ f"days that other series covered ({gone_days[0]} .. {gone_days[-1]}); "+ "null there, not zero"+ )out = dict(base_out)out.update(@@ -550,24 +742,39 @@ async def fetch_hostility_timeline(),"definitions": {"partial": (- "GDELT returned no bucket for at least one elapsed hour of that "- "UTC day (for today, only wholly elapsed hours count); "- "hours_covered says how many hours had one. Judged from the "- "base series, or the first category series that answered when "- "the base call failed; absent when it cannot be judged"+ "a volume column (articles or a hostility category) has no bucket "+ "for at least one elapsed hour of that UTC day, or none at all (for "+ "today, only wholly elapsed hours count). Judged per column: "+ "partial_columns names them, coverage_by_column gives each column's "+ "buckets and hours, and hours_missing is the union of their missing "+ "hours. buckets_returned and hours_covered come from the base "+ "series, or the first category series that answered when the base "+ "call failed. Tone is not judged. Absent when it cannot be judged"),"in_progress": ("the row is today's UTC day, which has not ended; its totals ""are not comparable with a full day's"),- "null": "not measured (call failed or day not returned), never zero",+ "hostile_share": (+ "a category's count over the day's articles; null when either is "+ "null or, with hourly or finer buckets, when the two series cover "+ "different hours of the day"+ ),+ "column_status": (+ "per GDELT call: ok (a live answer or a cache entry within its "+ "30-minute TTL), stale (the live call failed and only an expired "+ "cached copy was available: the column is null, and "+ "stale_age_seconds gives the copy's age), failed (nothing usable: "+ "the column is null)"+ ),+ "null": "not measured (call failed, expired copy, or day not returned), never zero",},"data_gaps": data_gaps,})- if len(by_series) < len(queries) or tone_days is None:+ if any(s != "ok" for s in status.values()):out["degraded"] = True- out["reason"] = "gdelt_partial"+ out["reason"] = "gdelt_stale" if any_stale else "gdelt_partial"return out@@ -602,25 +809,33 @@ async def fetch_situations(A headline naming two or more countries counts toward each pair itnames ("between"); a headline naming exactly one counts toward thatcountry ("inside"). Hostile = the headline matches any hostility- category.+ category. When no feed covers both windows, change fields are null+ and ``change_measured`` is false; a "change" sort then falls back to+ coverage and ``sort_applied`` says so."""now = now or datetime.now(timezone.utc)hours = _clamp_hours(hours)sort = sort if sort in _SORTS else "covered"kind = kind if kind in ("all", "between", "inside") else "all"+ limit = max(1, int(limit))+ min_articles = max(1, int(min_articles))base = {"situations": [],"count": 0,"sort": sort,"kind": kind,+ "limit": limit,"source": "rss-aggregator+news-signals","timestamp": _iso(now),}feed = await news.fetch_news_feed(fetcher, limit=_ALL_ITEMS)- if feed.get("degraded") or feed.get("error"):+ # Every feed failed: nothing to rank. A partial outage (degraded, no+ # error) is ranked over the feeds that answered, and says so.+ if feed.get("error"):return _degraded_feed_response(feed, base)win = _Windows(feed, hours, now)+ measured = win.change_measureddef _blank() -> dict:return {@@ -693,24 +908,37 @@ async def fetch_situations("hostile_share": round(s["hostile"] / s["articles"], 3),"by_category": s["by_category"],"outlet_count": len(s["outlets"]),- **_change_fields(s["current_comparable"], s["prior"]),+ **(+ _change_fields(s["current_comparable"], s["prior"])+ if measured+ else _UNMEASURED_CHANGE+ ),+ "change_measured": measured,"sample_headlines": s["headlines"],})- if sort == "hostile":+ sort_applied = sort if (sort != "change" or measured) else "covered"+ if sort_applied == "hostile":situations.sort(key=lambda x: (x["hostile_articles"], x["hostile_share"], x["articles"]), reverse=True)- elif sort == "change":+ elif sort_applied == "change":situations.sort(key=lambda x: (x["change"], x["articles"]), reverse=True)else:situations.sort(key=lambda x: (x["articles"], x["hostile_articles"]), reverse=True)situations = situations[:limit]+ data_gaps = win.gaps(feed) + win.change_gaps("change")+ if sort_applied != sort:+ data_gaps.append(+ "sort=change was asked for but change was not measured; ranked by coverage instead"+ )out = dict(base)out.update({"situations": situations,"count": len(situations),+ "sort_applied": sort_applied,+ "change_measured": measured,"window": win.describe(),"min_articles": min_articles,"definitions": {@@ -734,7 +962,8 @@ async def fetch_situations("once. Hostility: " + HOSTILITY_METHOD),"coverage": win.coverage(feed),- "data_gaps": win.gaps(feed),+ **_feed_marker(feed),+ "data_gaps": data_gaps,})return out@@ -773,26 +1002,34 @@ async def fetch_top_stories(``intel_news_clusters``), counts distinct publishers per cluster(sister feeds count once), and ranks by outlet count, then articlecount. ``attention`` ranks named countries and organizations by- growth in mentions against the prior window.+ growth in mentions against the prior window; when no feed covers both+ windows it is empty and ``attention_change_measured`` is false."""now = now or datetime.now(timezone.utc)hours = _clamp_hours(hours)min_outlets = max(1, int(min_outlets))+ limit = max(1, int(limit))+ attention_limit = max(1, int(attention_limit))base = {"stories": [],"count": 0,"attention": [],"min_outlets": min_outlets,+ "limit": limit,+ "attention_limit": attention_limit,"source": "rss-aggregator+news-signals","timestamp": _iso(now),}feed = await news.fetch_news_feed(fetcher, limit=_ALL_ITEMS)- if feed.get("degraded") or feed.get("error"):+ # As in fetch_situations: only a total outage stops the ranking.+ if feed.get("error"):return _degraded_feed_response(feed, base)win = _Windows(feed, hours, now)rows = win.current- raw = cluster_articles(rows, similarity_threshold=threshold)+ # Pairwise Jaccard is quadratic in titles: off the event loop the+ # collector and the MCP server share.+ raw = await asyncio.to_thread(cluster_articles, rows, similarity_threshold=threshold)stories = []for c in raw:@@ -830,19 +1067,22 @@ async def fetch_top_stories(stories.sort(key=lambda s: (s["outlet_count"], s["articles"], s["last_seen"]), reverse=True)stories = stories[:limit]- cur = _attention_counts(win.current, win.comparable_feeds)- prior = _attention_counts(win.prior, win.comparable_feeds)+ # With no comparable feed every growth would be 0 - 0: the list stays+ # empty and attention_change_measured says why, never "nothing grew".attention = []- for key in set(cur) | set(prior):- etype, ident = key- entry: dict[str, Any] = {"type": etype, "name": ident}- if etype == "country":- entry = {"type": etype, "name": _country_name(ident), "iso3": ident}- entry.update(_change_fields(cur.get(key, 0), prior.get(key, 0)))- attention.append(entry)- attention = [a for a in attention if a["change"] > 0]- attention.sort(key=lambda a: (a["change"], a["current_comparable"]), reverse=True)- attention = attention[:attention_limit]+ if win.change_measured:+ cur = _attention_counts(win.current, win.comparable_feeds)+ prior = _attention_counts(win.prior, win.comparable_feeds)+ for key in set(cur) | set(prior):+ etype, ident = key+ entry: dict[str, Any] = {"type": etype, "name": ident}+ if etype == "country":+ entry = {"type": etype, "name": _country_name(ident), "iso3": ident}+ entry.update(_change_fields(cur.get(key, 0), prior.get(key, 0)))+ attention.append(entry)+ attention = [a for a in attention if a["change"] > 0]+ attention.sort(key=lambda a: (a["change"], a["current_comparable"]), reverse=True)+ attention = attention[:attention_limit]out = dict(base)out.update(@@ -850,6 +1090,7 @@ async def fetch_top_stories("stories": stories,"count": len(stories),"attention": attention,+ "attention_change_measured": win.change_measured,"window": win.describe(),"definitions": {"outlet": (@@ -873,7 +1114,8 @@ async def fetch_top_stories("several stories, and headlines in different languages never merge."),"coverage": win.coverage(feed),- "data_gaps": win.gaps(feed),+ **_feed_marker(feed),+ "data_gaps": win.gaps(feed) + win.change_gaps("attention"),})return outdiff --git a/src/world_intel_mcp/cli.py b/src/world_intel_mcp/cli.pyindex 8e1fb20..ea99039 100644--- a/src/world_intel_mcp/cli.py+++ b/src/world_intel_mcp/cli.py@@ -775,6 +775,7 @@ def news_cmd(ctx: click.Context, category: str | None, limit: int) -> None:if _bail_on_error(ctx, data):return+ _print_partial_feed_outage(data)items = data.get("items", [])if not items:console.print("[yellow]No news items available[/yellow]")@@ -812,6 +813,7 @@ def trending(ctx: click.Context, min_count: int) -> None:console.print(f"[bold]Trending keywords[/bold] (from {data.get('total_items_analyzed', 0)} items)\n")+ _print_partial_feed_outage(data)table = Table(box=box.SIMPLE_HEAVY)table.add_column("#", justify="right")@@ -855,6 +857,20 @@ def _na(value: Any) -> str:return "n/a" if value is None else str(value)+def _print_partial_feed_outage(data: dict) -> None:+ """Say when a result rests on a minority of the RSS feeds.++ fetch_news_feed marks that read degraded with no error (reason+ partial_feeds_failed); a total outage carries an error and never+ reaches here (``_bail_on_error``).+ """+ if data.get("degraded") and "error" not in data and data.get("feeds_attempted"):+ console.print(+ f"[yellow]Partial outage:[/yellow] only {data.get('feeds_ok')} of "+ f"{data.get('feeds_attempted')} RSS feeds answered live"+ )++@main.command(name="hostility")@click.argument("country")@click.argument("other", required=False)@@ -880,21 +896,34 @@ def hostility_cmd(ctx: click.Context, country: str, other: str | None, days: inttable.add_column(col, justify="left" if col == "Date" else "right")for d in data.get("days", []):h = d.get("hostile", {})+ partial_cols = set(d.get("partial_columns") or [])++ def _count(col: str, value: Any) -> str:+ # A count from a column whose buckets miss an elapsed hour.+ return _na(value) + ("*" if value is not None and col in partial_cols else "")+date = d["date"] + (" *" if d.get("partial") else "") + (" ~" if d.get("in_progress") else "")table.add_row(date,- _na(d.get("articles")),- _na(h.get("threat")),- _na(h.get("sanction")),- _na(h.get("force")),- _na(h.get("criticism")),+ _count("articles", d.get("articles")),+ _count("threat", h.get("threat")),+ _count("sanction", h.get("sanction")),+ _count("force", h.get("force")),+ _count("criticism", h.get("criticism")),_na(d.get("tone")),)console.print(table)console.print(- "[dim]* partial day (GDELT returned fewer hours than elapsed); ~ day in progress; "- "n/a = not measured (fetch failed or day not returned)[/dim]"+ "[dim]* after a date: partial day; after a count: that column's buckets miss an "+ "elapsed hour; ~ day in progress; n/a = not measured (fetch failed, expired "+ "copy, or day not returned)[/dim]")+ for col, age in (data.get("stale_age_seconds") or {}).items():+ age_text = f"{age / 3600:.1f} h old" if isinstance(age, (int, float)) else "age unknown"+ console.print(+ f"[yellow]Stale:[/yellow] {escape(str(col))} is an expired cached copy "+ f"({age_text}); shown as n/a"+ )_print_data_gaps(data)@@ -920,9 +949,16 @@ def situations_cmd(ctx: click.Context, hours: float, sort: str, kind: str, limitwin = data.get("window", {})console.print(- f"[bold]Situations[/bold] by {sort}, window {win.get('current', {}).get('start', '?')} "- f".. {win.get('current', {}).get('end', '?')}\n"+ f"[bold]Situations[/bold] by {data.get('sort_applied', sort)}, window "+ f"{win.get('current', {}).get('start', '?')} .. {win.get('current', {}).get('end', '?')}\n")+ _print_partial_feed_outage(data)+ if data.get("change_measured") is False:+ # n/a in the Change column, never +0.+ console.print(+ "[yellow]Change not measured[/yellow] (no feed covers both windows); "+ "the Change column shows n/a"+ )table = Table(box=box.SIMPLE_HEAVY)table.add_column("#", justify="right")table.add_column("Kind")@@ -966,6 +1002,7 @@ def top_stories_cmd(ctx: click.Context, hours: float, min_outlets: int, limit: ireturnconsole.print(f"[bold]Top stories[/bold] (>= {min_outlets} outlets)\n")+ _print_partial_feed_outage(data)table = Table(box=box.SIMPLE_HEAVY)table.add_column("#", justify="right")table.add_column("Outlets", justify="right")@@ -992,6 +1029,11 @@ def top_stories_cmd(ctx: click.Context, hours: float, min_outlets: int, limit: i_signed(a["change"]),)console.print(att)+ elif data.get("attention_change_measured") is False:+ # An empty list here is not "nothing grew".+ console.print(+ "[yellow]Attention:[/yellow] not measured (no feed covers both windows)"+ )_print_data_gaps(data)@@ -1521,6 +1563,12 @@ def dossier(ctx: click.Context, country: str) -> None:if "mention_count" in news and news["mention_count"] is None:# A failed feed (or a country without keywords) is not zero mentions.console.print("[blue]News:[/blue] not measured")+ elif news.get("mention_count_capped"):+ # The listed mentions are capped; the match count is not.+ console.print(+ f"[blue]News:[/blue] {news.get('matching_items')} matching headlines in the "+ f"{news.get('items_searched')} newest items ({news.get('mention_count')} listed)"+ )else:console.print(f"[blue]News:[/blue] {news.get('mention_count', 0)} recent mentions")for art in news.get("mentions", [])[:3]:diff --git a/src/world_intel_mcp/fetcher.py b/src/world_intel_mcp/fetcher.pyindex fbbcc11..d0fb705 100644--- a/src/world_intel_mcp/fetcher.py+++ b/src/world_intel_mcp/fetcher.py@@ -7,12 +7,15 @@ response is returned (marked with _stale=True) so dashboards never go blank.import asyncioimport logging+import os+import sqlite3import timefrom dataclasses import dataclassfrom typing import Anyimport httpx+from . import shared_floorfrom .cache import Cachefrom .circuit_breaker import CircuitBreaker@@ -201,6 +204,13 @@ class Fetcher:last_error = excif attempt < self.max_retries:wait = min(2**attempt, 30)+ if (+ isinstance(exc, httpx.HTTPStatusError)+ and exc.response.status_code == 429+ ):+ # A 429 asks for slower requests: never retry sooner+ # than the source's own floor (GDELT: 5 s).+ wait = max(wait, _SOURCE_RATE_LIMITS.get(source, 0.0))logger.debug("Retry %d/%d for %s (%s), waiting %.1fs",attempt + 1,@@ -367,8 +377,30 @@ class Fetcher:elapsed = now - lastif elapsed < min_interval:await asyncio.sleep(min_interval - elapsed)+ if source in shared_floor.SHARED_SOURCES:+ await self._shared_floor_wait(source, min_interval)_source_last_call[source] = time.time()+ async def _shared_floor_wait(self, source: str, min_interval: float) -> None:+ """Wait out another process's reserved slot (see ``shared_floor``)."""+ db_path = getattr(self.cache, "db_path", None)+ if not isinstance(db_path, (str, os.PathLike)):+ return+ try:+ slot = await asyncio.to_thread(+ shared_floor.reserve, db_path, source, min_interval+ )+ except (sqlite3.Error, OSError) as exc:+ logger.warning(+ "Cross-process %s floor unavailable (%s); spacing this process only",+ source,+ exc,+ )+ return+ wait = slot - time.time()+ if wait > 0:+ await asyncio.sleep(wait)+async def _yahoo_throttle(self) -> None:"""Enforce Yahoo Finance rate limit (600ms between calls)."""global _yahoo_last_calldiff --git a/src/world_intel_mcp/shared_floor.py b/src/world_intel_mcp/shared_floor.pynew file mode 100644index 0000000..12b85c3--- /dev/null+++ b/src/world_intel_mcp/shared_floor.py@@ -0,0 +1,80 @@+"""Cross-process spacing for rate-floored upstreams.++The fetcher's per-source floor (``fetcher._SOURCE_RATE_LIMITS``) is an+asyncio lock plus a timestamp, so on its own it spaces calls made inside+one process only. GDELT limits per IP address ("Please limit requests to+one every 5 seconds", its 429 body, measured 2026-10-17), and this server+routinely runs as several processes on one host against one cache DB: the+collector daemon, one MCP stdio server per client, the dashboard, and+every ``intel`` CLI run. For the sources in ``SHARED_SOURCES`` each live+request therefore also reserves a slot in a one-row-per-source table of+that cache DB. The reservation runs under ``BEGIN IMMEDIATE``, so two+processes can never read the same last slot.++A process's own earlier reservation is left to its in-process floor:+only another process's slot pushes this one back. A slot more than+``MAX_AHEAD_S`` ahead of the clock is treated as a clock jump or a corrupt+row, not a queue, and ignored.++Gaps / not covered: processes that resolve different cache DB paths+(``WORLD_INTEL_CACHE_DB`` / ``XDG_CACHE_HOME`` decide it) do not see each+other's slots; other hosts are separate IP addresses with their own DBs;+a reservation that fails (database locked past the timeout, read-only+file) is logged by the fetcher and the call goes ahead on the in-process+floor alone; sources outside ``SHARED_SOURCES`` keep a per-process floor.+"""++from __future__ import annotations++import os+import sqlite3+import time+from pathlib import Path++# Sources whose floor holds across processes. Others keep the in-process+# floor only.+SHARED_SOURCES = frozenset({"gdelt"})++# Ten minutes of queued 5 s slots is far beyond any real queue (one waiter+# per process at a time); a slot further ahead than this is not one.+MAX_AHEAD_S = 600.0++_LOCK_TIMEOUT_S = 5.0+++def reserve(+ db_path: str | Path, source: str, interval: float, now: float | None = None+) -> float:+ """Reserve the next request slot for ``source``; return its Unix time.++ The slot is ``now`` unless another process reserved one less than+ ``interval`` before ``now`` (or one still ahead of it), in which case it+ is that slot plus ``interval``. Raises ``sqlite3.Error`` when the DB+ cannot be locked or written.+ """+ now = time.time() if now is None else now+ pid = os.getpid()+ conn = sqlite3.connect(str(db_path), timeout=_LOCK_TIMEOUT_S, isolation_level=None)+ try:+ conn.execute("BEGIN IMMEDIATE")+ conn.execute(+ "CREATE TABLE IF NOT EXISTS source_floor ("+ "source TEXT PRIMARY KEY, slot REAL NOT NULL, pid INTEGER NOT NULL)"+ )+ row = conn.execute(+ "SELECT slot, pid FROM source_floor WHERE source = ?", (source,)+ ).fetchone()+ slot = now+ if row is not None:+ last, last_pid = float(row[0]), int(row[1])+ if last_pid != pid and last - now <= MAX_AHEAD_S:+ slot = max(now, last + interval)+ conn.execute(+ "INSERT OR REPLACE INTO source_floor (source, slot, pid) VALUES (?, ?, ?)",+ (source, slot, pid),+ )+ conn.execute("COMMIT")+ return slot+ finally:+ # Closing with the transaction still open (an error above) rolls it back.+ conn.close()diff --git a/src/world_intel_mcp/sources/news.py b/src/world_intel_mcp/sources/news.pyindex b1a4ff7..0f1615e 100644--- a/src/world_intel_mcp/sources/news.py+++ b/src/world_intel_mcp/sources/news.py@@ -549,6 +549,26 @@ def _truncate(text: str | None, max_len: int = 200) -> str:return text[:max_len] + "..."+# Feed titles are upstream text: capped like summaries, at a length no real+# headline reaches.+_MAX_TITLE_CHARS = 500++# A partial RSS outage is marked degraded when fewer than this share of the+# attempted feeds answered live. Measured 2026-10-17: 95 of 119 feeds (80 %)+# answered, and the 24 that did not were permanent (16 x 404, 6 x 403, one+# 410) plus one timeout, so the normal state stays unflagged with margin;+# below half, a result rests on a minority of its feeds.+_MIN_LIVE_FEED_SHARE = 0.5+++def _web_link(value: object) -> str:+ """The link if it is an http(s) URL, else "" (javascript:, data:, relative)."""+ if not isinstance(value, str):+ return ""+ link = value.strip()+ return link if link.lower().startswith(("http://", "https://")) else ""++# ---------------------------------------------------------------------------# Function 1: RSS News Feed Aggregation# ---------------------------------------------------------------------------@@ -595,7 +615,11 @@ async def fetch_news_feed(limit: Maximum number of items to return.Returns:- Dict with items list, count, categories fetched, source, and timestamp.+ Dict with items list, count, categories fetched, source, and timestamp,+ plus feeds_attempted/feeds_ok/feeds_failed/feeds_stale. ``degraded``+ and ``reason`` mark an outage: ``all_feeds_failed`` (with ``error``)+ when no feed answered live, ``partial_feeds_failed`` (no error) when+ fewer than half did. Links are kept only when they are http(s)."""if feedparser is None:return {@@ -663,8 +687,8 @@ async def fetch_news_feed(summary_raw = entry.get("summary") or entry.get("description") or ""items.append({- "title": entry.get("title", ""),- "link": entry.get("link", ""),+ "title": _truncate(entry.get("title", ""), _MAX_TITLE_CHARS),+ "link": _web_link(entry.get("link", "")),"published": published,"summary": _truncate(summary_raw, 200),"feed_name": feed_name,@@ -755,6 +779,11 @@ async def fetch_news_feed(result["error"] = errorresult["degraded"] = Trueresult["reason"] = "all_feeds_failed"+ elif feeds_attempted and result["feeds_ok"] / feeds_attempted < _MIN_LIVE_FEED_SHARE:+ # Most feeds failed: items still flow (no error), but the result+ # rests on a minority of the feeds and says so.+ result["degraded"] = True+ result["reason"] = "partial_feeds_failed"return result@@ -782,7 +811,8 @@ async def fetch_trending_keywords(Returns:Dict with keywords list (word + count), total items analyzed,- source, and timestamp.+ source, and timestamp, plus the feed counts and any outage marker+ (error/degraded/reason) of the underlying ``fetch_news_feed`` read."""now = datetime.now(timezone.utc)@@ -822,8 +852,12 @@ async def fetch_trending_keywords("source": "keyword-analysis","timestamp": now.strftime("%Y-%m-%dT%H:%M:%SZ"),}- # An RSS outage is not "nothing trending".- for key in ("error", "degraded", "reason"):+ # An RSS outage is not "nothing trending", and a partial one says how+ # many feeds the counts rest on.+ for key in (+ "feeds_attempted", "feeds_ok", "feeds_failed", "feeds_stale",+ "error", "degraded", "reason",+ ):if key in feed_data:result[key] = feed_data[key]return resultdiff --git a/src/world_intel_mcp/tests/test_dossier_news.py b/src/world_intel_mcp/tests/test_dossier_news.pyindex b7787a5..0304aed 100644--- a/src/world_intel_mcp/tests/test_dossier_news.py+++ b/src/world_intel_mcp/tests/test_dossier_news.py@@ -11,9 +11,11 @@ dict), read "articles" (the producer emits "items") and "source" (itemscarry "feed_name"), and checked only its own "_error", never theproducer's "error".-Gaps / not covered: headline matching is a substring test on the tier-1-keyword lists ("american" also matches "Latin American", "iran" matches-"Tirana"); only the 200 newest items are searched; real feed depth.+Gaps / not covered: headline matching is a whole-word test on the tier-1+keyword lists with a short list of compound names excluded (the trap+headlines below); compounds outside that list ("Lagos, Portugal", the+Republic of the Congo) still count; only the 200 newest items are+searched; real feed depth."""import asyncio as asyncio_mod@@ -179,3 +181,145 @@ def test_cli_dossier_prints_a_news_outage_as_not_measured(assert "News: not measured" in result.outputassert f"Gap: {gap}" in result.outputassert "recent mentions" not in result.output+++# ---------------------------------------------------------------------------+# Review fixes (cloud review of 3c0ebff, 2026-10-17)+# ---------------------------------------------------------------------------++_TIRANA = "Tirana mayor resigns after corruption probe"+_IRAN = "Iran's foreign minister visits Beijing"+++def _rss_doc(items: list[tuple[str, str, str]]) -> str:+ body = "".join(+ f"<item><title>{title}</title><link>{link}</link><pubDate>{pub}</pubDate></item>"+ for title, link, pub in items+ )+ return (+ '<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"><channel>'+ f"<title>Feed</title>{body}</channel></rss>"+ )+++ "iso3,title,expected",+ [+ # The review's false mentions, then true ones beside them.+ ("IRN", "Tirana mayor resigns", False),+ ("IRN", _IRAN, True),+ ("IND", "Indiana governor signs bill", False),+ ("IND", "Storm builds over the Indian Ocean", False),+ ("IND", "Indian navy deploys to the Gulf", True),+ ("IND", "Indians vote in state polls", True),+ ("SDN", "South Sudan holds vote", False),+ ("SDN", "South Sudanese refugees cross the border", False),+ ("SDN", "Sudan army retakes Khartoum", True),+ ("USA", "Latin American leaders meet", False),+ ("USA", "Americans head to the polls", True),+ ("USA", "American and Latin American envoys meet", True),+ ("MEX", "New Mexico wildfire spreads", False),+ ("MEX", "Mexican peso slides", True),+ ],+)+def test_dossier_headline_country_match_is_whole_word(iso3: str, title: str, expected: bool) -> None:+ assert dossier._names_country(title, iso3) is expected+++async def test_dossier_news_matches_whole_words_and_reports_its_reach(+ fetcher, _other_sections+) -> None:+ # Review M4: the match was a substring test, so "Tirana" counted as+ # Iran; the response also hid the 15-mention cap and how far back the+ # 200 searched items reached.+ respx.route().mock(return_value=httpx.Response(200, text=_rss_doc([+ (_TIRANA, "https://news.example/tirana", "Sat, 17 Oct 2026 17:00:00 GMT"),+ (_IRAN, "https://news.example/iran", "Sat, 17 Oct 2026 16:00:00 GMT"),+ (_POPE, "https://news.example/pope", "Sat, 17 Oct 2026 15:00:00 GMT"),+ ])))+ r = await dossier.fetch_country_dossier(fetcher, country="IR")+ section = r["news"]+ assert {m["title"] for m in section["mentions"]} == {_IRAN}+ assert section["mention_count"] == 15+ # The 200 newest items: every Tirana copy, then the newest Iran copies.+ assert section["items_searched"] == 200+ assert section["matching_items"] == 200 - _N_FEEDS+ assert section["mention_count_capped"] is True+ assert section["searched_span"] == {+ "oldest": "2026-10-17T16:00:00Z",+ "newest": "2026-10-17T17:00:00Z",+ }+ assert "whole-word" in section["match_method"]+ assert section["data_gaps"] == []+++async def test_dossier_news_names_feeds_served_from_an_expired_copy(+ fetcher, _other_sections, _no_sleep+) -> None:+ # Review mutant M27: nothing pinned the dossier's stale-feed gap.+ stale_name, stale_url = _RSS_FEEDS["geopolitics"][0]+ fetcher.cache.set(+ f"news:rss:{stale_name.lower().replace(' ', '_')}", _RSS, ttl_seconds=-1+ )+ respx.get(stale_url).mock(return_value=httpx.Response(503))+ respx.route().mock(return_value=httpx.Response(200, text=_RSS))++ r = await dossier.fetch_country_dossier(fetcher, country="RU")+ section = r["news"]+ assert section["mention_count"] == 15+ assert section["data_gaps"] == [+ f"1 RSS feeds were served from an expired cache copy: {stale_name}"+ ]+++async def test_dossier_news_minority_of_feeds_is_degraded_but_measured(+ fetcher, _other_sections, _no_sleep+) -> None:+ # Review M5: only the geopolitics feeds answer; mentions are still+ # counted over them, and the section says the count rests on a+ # minority of the feeds.+ live = {url for _, url in _RSS_FEEDS["geopolitics"]}+ for feeds in _RSS_FEEDS.values():+ for _, url in feeds:+ if url not in live:+ respx.get(url).mock(return_value=httpx.Response(503))+ respx.route().mock(return_value=httpx.Response(200, text=_RSS))++ r = await dossier.fetch_country_dossier(fetcher, country="RU")+ section = r["news"]+ n_live = len(_RSS_FEEDS["geopolitics"])+ assert section["mention_count"] == n_live+ assert section["degraded"] is True+ assert section["reason"] == "partial_feeds_failed"+ assert "error" not in section+ assert any(+ g.startswith(f"only {n_live} of {_N_FEEDS} RSS feeds answered live")+ for g in section["data_gaps"]+ )+++def test_cli_dossier_says_when_mentions_are_capped(+ tmp_path, monkeypatch: pytest.MonkeyPatch+) -> None:+ from click.testing import CliRunner++ from world_intel_mcp import cli++ monkeypatch.setenv("WORLD_INTEL_CACHE_DB", str(tmp_path / "cli-cache.db"))+ monkeypatch.setattr(cli, "_fetcher", None)++ async def _dossier(*args, **kwargs) -> dict:+ return {+ "overview": {"country": "Iran", "iso2": "IR", "iso3": "IRN"},+ "news": {"mentions": [{"title": _IRAN}], "mention_count": 15,+ "mention_count_capped": True, "matching_items": 81,+ "items_searched": 200, "data_gaps": []},+ }++ monkeypatch.setattr(dossier, "fetch_country_dossier", _dossier)+ result = CliRunner().invoke(cli.main, ["dossier", "-c", "IR"], env={"COLUMNS": "200"})+ assert result.exit_code == 0+ assert "News: 81 matching headlines in the 200 newest items (15 listed)" in result.outputdiff --git a/src/world_intel_mcp/tests/test_gdelt_floor.py b/src/world_intel_mcp/tests/test_gdelt_floor.pynew file mode 100644index 0000000..56dc2ca--- /dev/null+++ b/src/world_intel_mcp/tests/test_gdelt_floor.py@@ -0,0 +1,183 @@+"""GDELT request spacing: the 429 back-off and the cross-process floor.++GDELT's DOC API answers 429 "Please limit requests to one every 5 seconds"+(measured 2026-10-17) and limits per IP address. The fetcher's floor+(``_SOURCE_RATE_LIMITS["gdelt"]``) used to be an asyncio lock plus a+timestamp, so it spaced calls inside one process only, while this server+runs as several processes on one host (collector, one MCP server per+client, dashboard, CLI). ``shared_floor`` keeps the last reserved slot in+the shared cache DB. The cloud review of 2026-10-17 (finding M3) also+measured retries of one call going 1 s and 2 s apart on a 429.++Gaps / not covered: processes that resolve different cache DB paths (they+do not coordinate, by design of the shared file); a lock held longer than+the reservation's busy timeout under real write load (simulated only by a+raised error); real GDELT behaviour under the new spacing (no live probe).+"""++import asyncio+import logging+import os+import sqlite3+import subprocess+import sys+import textwrap+import time++import httpx+import pytest+import respx++from world_intel_mcp import fetcher as fetcher_mod+from world_intel_mcp import shared_floor+from world_intel_mcp.sources import news++_GDELT_URL = r".*api\.gdeltproject\.org.*"+_OTHER_PID = os.getpid() + 1+_T = 1_000_000.0+++def _seed(db, source: str, slot: float, pid: int) -> None:+ conn = sqlite3.connect(str(db))+ try:+ conn.execute(+ "INSERT OR REPLACE INTO source_floor (source, slot, pid) VALUES (?, ?, ?)",+ (source, slot, pid),+ )+ conn.commit()+ finally:+ conn.close()+++def test_reserve_defers_only_to_another_process(tmp_path) -> None:+ db = tmp_path / "floor.db"+ assert shared_floor.reserve(db, "gdelt", 5.0, now=_T) == _T+ # This process again: its own in-process floor spaces it, not the row.+ assert shared_floor.reserve(db, "gdelt", 5.0, now=_T + 1) == _T + 1+ # Another process went 1 s ago: this call goes 4 s from now.+ _seed(db, "gdelt", slot=_T + 1, pid=_OTHER_PID)+ assert shared_floor.reserve(db, "gdelt", 5.0, now=_T + 2) == _T + 6+ # Another process holds a slot still ahead: this one queues behind it.+ _seed(db, "gdelt", slot=_T + 20, pid=_OTHER_PID)+ assert shared_floor.reserve(db, "gdelt", 5.0, now=_T + 10) == _T + 25+ # Sources keep separate rows.+ assert shared_floor.reserve(db, "other", 5.0, now=_T + 10) == _T + 10+++def test_reserve_ignores_a_slot_far_in_the_future(tmp_path) -> None:+ # A slot beyond MAX_AHEAD_S is a clock jump or a corrupt row, not a+ # queue: honouring it would stall every caller.+ db = tmp_path / "floor.db"+ shared_floor.reserve(db, "gdelt", 5.0, now=_T)+ _seed(db, "gdelt", slot=_T + shared_floor.MAX_AHEAD_S + 50, pid=_OTHER_PID)+ assert shared_floor.reserve(db, "gdelt", 5.0, now=_T + 1) == _T + 1+++async def test_source_throttle_waits_out_another_process_slot(+ fetcher, monkeypatch: pytest.MonkeyPatch+) -> None:+ waits: list[float] = []++ async def _record(delay, *args, **kwargs) -> None:+ waits.append(delay)++ monkeypatch.setattr(asyncio, "sleep", _record)+ shared_floor.reserve(fetcher.cache.db_path, "gdelt", 5.0)+ _seed(fetcher.cache.db_path, "gdelt", slot=time.time() + 2.0, pid=_OTHER_PID)++ await fetcher._source_throttle("gdelt")+ assert len(waits) == 1+ assert 6.5 < waits[0] <= 7.0+++async def test_source_throttle_keeps_the_local_floor_when_the_db_fails(+ fetcher, monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture+) -> None:+ def _locked(*args, **kwargs):+ raise sqlite3.OperationalError("database is locked")++ monkeypatch.setattr(shared_floor, "reserve", _locked)+ with caplog.at_level(logging.WARNING, logger="world-intel-mcp.fetcher"):+ await fetcher._source_throttle("gdelt")+ assert any("Cross-process gdelt floor unavailable" in r.message for r in caplog.records)+ assert fetcher_mod._source_last_call["gdelt"] > 0+++_CHILD = textwrap.dedent(+ """+ import asyncio, sys, time+ from pathlib import Path+ from world_intel_mcp import fetcher as fetcher_mod+ from world_intel_mcp.cache import Cache+ from world_intel_mcp.circuit_breaker import CircuitBreaker++ db, start, interval = sys.argv[1], float(sys.argv[2]), float(sys.argv[3])+ fetcher_mod._SOURCE_RATE_LIMITS["gdelt"] = interval+ f = fetcher_mod.Fetcher(cache=Cache(db_path=Path(db)), breaker=CircuitBreaker())++ async def main():+ while time.time() < start:+ await asyncio.sleep(0.005)+ await f._source_throttle("gdelt")+ print(repr(time.time()))++ asyncio.run(main())+ """+)+++def test_gdelt_floor_spaces_calls_across_processes(tmp_path) -> None:+ # Three processes on one cache DB all ask for a GDELT slot at the same+ # instant; each must go at least one interval after the one before.+ db = tmp_path / "shared.db"+ interval = 0.5+ start = time.time() + 3.0+ procs = [+ subprocess.Popen(+ [sys.executable, "-c", _CHILD, str(db), repr(start), str(interval)],+ stdout=subprocess.PIPE,+ stderr=subprocess.PIPE,+ text=True,+ env=dict(os.environ),+ )+ for _ in range(3)+ ]+ outs = [p.communicate(timeout=60) for p in procs]+ for p, (_, err) in zip(procs, outs):+ assert p.returncode == 0, err+ passed = sorted(float(out.strip()) for out, _ in outs)+ gaps = [b - a for a, b in zip(passed, passed[1:])]+ assert all(g >= interval - 0.05 for g in gaps), gaps+++async def test_a_429_is_retried_no_sooner_than_the_gdelt_floor(+ fetcher, monkeypatch: pytest.MonkeyPatch+) -> None:+ # Review M3: one GDELT call against an always-429 server went REQUEST,+ # sleep 1, REQUEST, sleep 2, REQUEST.+ events: list[tuple[str, float]] = []++ async def _record(delay, *args, **kwargs) -> None:+ events.append(("sleep", delay))++ def _answer(request: httpx.Request) -> httpx.Response:+ events.append(("request", 0.0))+ return httpx.Response(429, text="Please limit requests to one every 5 seconds")++ monkeypatch.setattr(asyncio, "sleep", _record)+ respx.get(url__regex=_GDELT_URL).mock(side_effect=_answer)++ result = await news.fetch_gdelt_search(fetcher, "Iran")+ assert result["reason"] == "gdelt_fetch_failed"+ gaps: list[float] = []+ waited: float | None = None+ for kind, delay in events:+ if kind == "request":+ if waited is not None:+ gaps.append(waited)+ waited = 0.0+ elif waited is not None:+ waited += delay+ assert len(gaps) == 2+ assert all(g >= fetcher_mod._SOURCE_RATE_LIMITS["gdelt"] for g in gaps), gapsdiff --git a/src/world_intel_mcp/tests/test_news_feed_outages.py b/src/world_intel_mcp/tests/test_news_feed_outages.pynew file mode 100644index 0000000..5042a78--- /dev/null+++ b/src/world_intel_mcp/tests/test_news_feed_outages.py@@ -0,0 +1,188 @@+"""RSS outage markers and feed-content checks in sources/news.py.++Covers the cloud review of 2026-10-17 (findings M5, L4, L6): a near-total+RSS outage used to read as a normal result (fetch_news_feed marked only a+total outage, and fetch_trending_keywords passed on only error/degraded/+reason); feed links were returned whatever their scheme and titles had no+length cap. Feeds are served by respx in the live RSS item shape.++Gaps / not covered: the 50 % live-share threshold is pinned at its edges+(2 of 8 and 4 of 8 feeds), not tuned against a history of real outages;+feedparser's own handling of malformed links beyond the schemes below.+"""++import asyncio as asyncio_mod++import httpx+import pytest+import respx+from click.testing import CliRunner++from world_intel_mcp import cli+from world_intel_mcp.fetcher import Fetcher+from world_intel_mcp.sources import news+from world_intel_mcp.sources.news import _RSS_FEEDS, fetch_news_feed, fetch_trending_keywords++_RSS = """<?xml version="1.0" encoding="UTF-8"?>+<rss version="2.0"><channel><title>Feed</title>+<item>+ <title>Sanctions talks resume in Geneva</title>+ <link>https://news.example/1</link>+ <pubDate>Sat, 17 Oct 2026 14:00:00 GMT</pubDate>+</item>+<item>+ <title>Sanctions relief on the table</title>+ <link>https://news.example/2</link>+ <pubDate>Sat, 17 Oct 2026 13:00:00 GMT</pubDate>+</item>+</channel></rss>"""++_SECURITY = _RSS_FEEDS["security"]+_ALL_FEEDS = [entry for feeds in _RSS_FEEDS.values() for entry in feeds]+++def _no_sleep(monkeypatch: pytest.MonkeyPatch) -> None:+ async def _skip(*args, **kwargs) -> None:+ return None++ monkeypatch.setattr(asyncio_mod, "sleep", _skip)+++def _serve(dead_urls: set[str], body: str = _RSS) -> None:+ for url in dead_urls:+ respx.get(url).mock(return_value=httpx.Response(503))+ respx.route().mock(return_value=httpx.Response(200, text=body))+++async def test_fetch_news_feed_minority_of_feeds_answering_is_degraded(+ fetcher: Fetcher, _no_sleep+) -> None:+ dead = [name for name, _ in _SECURITY[:6]]+ _serve({url for _, url in _SECURITY[:6]})++ result = await fetch_news_feed(fetcher, category="security")+ assert result["feeds_ok"] == 2+ assert result["feeds_attempted"] == len(_SECURITY)+ assert result["feeds_failed"] == sorted(dead)+ assert result["degraded"] is True+ assert result["reason"] == "partial_feeds_failed"+ # Not a total outage: the items that did arrive are kept, and no error.+ assert "error" not in result+ assert result["count"] == 4+++async def test_fetch_news_feed_half_of_the_feeds_answering_is_not_degraded(+ fetcher: Fetcher, _no_sleep+) -> None:+ _serve({url for _, url in _SECURITY[:4]})++ result = await fetch_news_feed(fetcher, category="security")+ assert result["feeds_ok"] == 4+ assert "degraded" not in result+ assert "reason" not in result+++async def test_fetch_news_feed_all_feeds_down_names_the_count_and_no_stale_copy(+ fetcher: Fetcher, _no_sleep+) -> None:+ # Review L6: the all-feeds-down test checked neither the error text nor+ # that no feed was counted as an expired copy.+ respx.route().mock(return_value=httpx.Response(500))++ result = await fetch_news_feed(fetcher, category="security")+ assert result["error"] == f"All {len(_SECURITY)} RSS feeds failed or timed out"+ assert result["feeds_stale"] == []+ assert result["reason"] == "all_feeds_failed"+++async def test_trending_passes_feed_counts_and_a_near_total_outage(+ fetcher: Fetcher, _no_sleep+) -> None:+ # Review M5: with 118 of 119 feeds down, trending returned a normal+ # table with no marker and no feed counts.+ live_name, live_url = _ALL_FEEDS[0]+ _serve({url for _, url in _ALL_FEEDS[1:]})++ result = await fetch_trending_keywords(fetcher, min_count=1)+ assert result["feeds_attempted"] == len(_ALL_FEEDS)+ assert result["feeds_ok"] == 1+ assert len(result["feeds_failed"]) == len(_ALL_FEEDS) - 1+ assert live_name not in result["feeds_failed"]+ assert result["feeds_stale"] == []+ assert result["degraded"] is True+ assert result["reason"] == "partial_feeds_failed"+ assert {k["word"] for k in result["keywords"]} >= {"sanctions"}+++async def test_fetch_news_feed_keeps_only_web_links_and_caps_titles(fetcher: Fetcher) -> None:+ # Review L4: links were returned whatever their scheme (javascript:,+ # data:) and titles had no length cap; feedparser 6.0.14 passes both+ # through unchanged (checked 2026-10-17).+ long_title = "Sanctions " + "x" * 2000+ body = f"""<?xml version="1.0" encoding="UTF-8"?>+<rss version="2.0"><channel><title>Feed</title>+<item><title>{long_title}</title><link>javascript:alert(1)</link>+ <pubDate>Sat, 17 Oct 2026 14:00:00 GMT</pubDate></item>+<item><title>Data link</title><link>data:text/html;base64,PHNjcmlwdD4=</link>+ <pubDate>Sat, 17 Oct 2026 13:00:00 GMT</pubDate></item>+<item><title>Upper-case scheme</title><link>HTTPS://news.example/ok</link>+ <pubDate>Sat, 17 Oct 2026 12:00:00 GMT</pubDate></item>+<item><title>Plain http</title><link>http://news.example/plain</link>+ <pubDate>Sat, 17 Oct 2026 11:00:00 GMT</pubDate></item>+</channel></rss>"""+ respx.route().mock(return_value=httpx.Response(200, text=body))++ result = await fetch_news_feed(fetcher, category="security")+ by_pub = {}+ for item in result["items"]:+ by_pub.setdefault(item["published"], item)+ assert by_pub["2026-10-17T14:00:00Z"]["link"] == ""+ assert by_pub["2026-10-17T14:00:00Z"]["title"] == long_title[:500] + "..."+ assert by_pub["2026-10-17T13:00:00Z"]["link"] == ""+ assert by_pub["2026-10-17T12:00:00Z"]["link"] == "HTTPS://news.example/ok"+ assert by_pub["2026-10-17T11:00:00Z"]["link"] == "http://news.example/plain"+++def _cli(monkeypatch: pytest.MonkeyPatch, tmp_path, target: str, payload: dict, *args: str):+ monkeypatch.setenv("WORLD_INTEL_CACHE_DB", str(tmp_path / "cli-cache.db"))+ monkeypatch.setattr(cli, "_fetcher", None)++ async def _fake(*a, **kw) -> dict:+ return payload++ monkeypatch.setattr(news, target, _fake)+ return CliRunner().invoke(cli.main, list(args), env={"COLUMNS": "200"})+++_PARTIAL = {+ "feeds_attempted": 119,+ "feeds_ok": 1,+ "feeds_failed": ["BBC World"],+ "feeds_stale": [],+ "degraded": True,+ "reason": "partial_feeds_failed",+}+++def test_cli_news_and_trending_say_when_most_feeds_failed(+ tmp_path, monkeypatch: pytest.MonkeyPatch+) -> None:+ feed = dict(_PARTIAL, items=[{"title": "Sanctions talks resume", "feed_name": "DW News",+ "category": "europe", "published": "2026-10-17T14:00:00Z"}],+ count=1, categories_fetched=["europe"])+ result = _cli(monkeypatch, tmp_path, "fetch_news_feed", feed, "news")+ assert result.exit_code == 0+ assert "Partial outage: only 1 of 119 RSS feeds answered live" in result.output+ assert "Sanctions talks resume" in result.output++ trend = dict(_PARTIAL, keywords=[{"word": "sanctions", "count": 2}], total_items_analyzed=2)+ result = _cli(monkeypatch, tmp_path, "fetch_trending_keywords", trend, "trending")+ assert result.exit_code == 0+ assert "Partial outage: only 1 of 119 RSS feeds answered live" in result.output+ assert "sanctions" in result.outputdiff --git a/src/world_intel_mcp/tests/test_news_signals.py b/src/world_intel_mcp/tests/test_news_signals.pyindex f787ad8..1251997 100644--- a/src/world_intel_mcp/tests/test_news_signals.py+++ b/src/world_intel_mcp/tests/test_news_signals.py@@ -458,9 +458,10 @@ async def test_hostility_timeline_spaces_live_calls_and_not_cache_hits(async def test_gdelt_search_and_timeline_share_one_rate_limit(fetcher, monkeypatch: pytest.MonkeyPatch) -> None:- # Every GDELT caller goes through the fetcher's per-source floor, so an- # intel_gdelt_search call and the timeline's calls cannot burst- # together (GDELT's 429 text: "one every 5 seconds").+ # Every GDELT caller goes through the fetcher's per-source floor, so in+ # one process an intel_gdelt_search call and the timeline's calls cannot+ # burst together (GDELT's 429 text: "one every 5 seconds"). Across+ # processes the floor holds through shared_floor (test_gdelt_floor.py).waits: list[float] = []async def _record(delay, *args, **kwargs) -> None:@@ -989,3 +990,589 @@ def test_cli_top_stories_json_output_is_raw(_cli_env, monkeypatch: pytest.Monkeyresult = _invoke("--json-output", "top-stories")assert result.exit_code == 0assert '"degraded": true' in result.output+++# ---------------------------------------------------------------------------+# Review fixes (cloud review of 3c0ebff, 2026-10-17): expired GDELT copies,+# per-column partial days, unmeasured change, retry spacing, odd inputs.+# ---------------------------------------------------------------------------++import sqlite3 # noqa: E402+import time # noqa: E402+from collections import Counter # noqa: E402++from world_intel_mcp.analysis import clustering # noqa: E402++_COLUMNS = ("articles", "threat", "sanction", "force", "criticism", "tone")+_GDELT_URL = r".*api\.gdeltproject\.org.*"+++def _expire_gdelt_copies(fetcher, age_s: float) -> int:+ """Turn every cached GDELT timeline into an expired copy fetched+ ``age_s`` ago: what the cache holds when no collector evicts it."""+ now = time.time()+ conn = sqlite3.connect(str(fetcher.cache.db_path))+ try:+ cur = conn.execute(+ "UPDATE cache SET expires_at = ?, created_at = ? WHERE key LIKE ?",+ (now - 1, now - age_s, "news_signals:gdelt:%"),+ )+ conn.commit()+ return cur.rowcount+ finally:+ conn.close()+++def _column(request: httpx.Request) -> str:+ if request.url.params["mode"] == "timelinetone":+ return "tone"+ return _category_of(request.url.params["query"])+++def _volraw_dropping(title: str, values: list[int], drop) -> dict:+ """The live volume shape without the buckets whose stamp ``drop`` picks."""+ payload = _volraw(title, values)+ series = payload["timeline"][0]+ series["data"] = [p for p in series["data"] if not drop(p["date"])]+ return payload+++async def test_hostility_timeline_expired_copies_are_never_shown_as_current(+ fetcher, _fast+) -> None:+ # Review H1: every live call fails, so the fetcher serves each column's+ # expired cached copy (marked _stale). The timeline used to return those+ # copies as current counts with no gap, marker or degraded flag.+ route, _ = _gdelt_router()+ down = {"on": False}++ def _answer(request: httpx.Request) -> httpx.Response:+ return httpx.Response(503) if down["on"] else route(request)++ respx.get(url__regex=_GDELT_URL).mock(side_effect=_answer)+ first = await ns.fetch_hostility_timeline(fetcher, "Iran", "Israel", days=3, now=_NOW_GDELT)+ assert first["column_status"] == {c: "ok" for c in _COLUMNS}+ assert first["stale_age_seconds"] == {}+ assert _expire_gdelt_copies(fetcher, age_s=72 * 3600) == 6+ down["on"] = True++ r = await ns.fetch_hostility_timeline(fetcher, "Iran", "Israel", days=3, now=_NOW_GDELT)+ assert r["degraded"] is True+ assert r["reason"] == "gdelt_stale"+ assert "expired cached copy" in r["error"]+ assert r["days"] == []+ assert r["column_status"] == {c: "stale" for c in _COLUMNS}+ assert sorted(r["stale_age_seconds"]) == sorted(_COLUMNS)+ assert all(abs(a - 72 * 3600) < 120 for a in r["stale_age_seconds"].values())+ for col in _COLUMNS:+ assert any(+ g.startswith(f"{col}:") and "expired cached copy (72.0 h old)" in g+ for g in r["data_gaps"]+ ), col+++async def test_hostility_timeline_one_expired_column_is_null_and_named(fetcher, _fast) -> None:+ route, _ = _gdelt_router()+ force_down = {"on": False}++ def _answer(request: httpx.Request) -> httpx.Response:+ if force_down["on"] and _column(request) == "force":+ return httpx.Response(503)+ return route(request)++ respx.get(url__regex=_GDELT_URL).mock(side_effect=_answer)+ await ns.fetch_hostility_timeline(fetcher, "Iran", "Israel", days=3, now=_NOW_GDELT)+ _expire_gdelt_copies(fetcher, age_s=26 * 3600)+ force_down["on"] = True++ r = await ns.fetch_hostility_timeline(fetcher, "Iran", "Israel", days=3, now=_NOW_GDELT)+ assert r["degraded"] is True+ assert r["reason"] == "gdelt_stale"+ assert "error" not in r+ assert r["column_status"] == dict({c: "ok" for c in _COLUMNS}, force="stale")+ assert list(r["stale_age_seconds"]) == ["force"]+ for d in r["days"]:+ assert d["hostile"]["force"] is None+ assert d["hostile_share"]["force"] is None+ assert d["articles"] == _DAY_ARTICLES[d["date"]]+ assert d["hostile"]["sanction"] == _DAY_SANCTION[d["date"]]+ assert any(+ g.startswith("force:") and "expired cached copy (26.0 h old)" in g for g in r["data_gaps"]+ )+++async def test_hostility_timeline_retry_pass_replaces_an_expired_copy(fetcher, _fast) -> None:+ # The initial force call falls back to its expired copy; the retry pass+ # asks again and the fresh answer (different values) replaces it.+ route, _ = _gdelt_router()+ second = {"on": False, "force_429s": 3}+ fresh_force = [v * 3 for v in _DERIVED["force"]]++ def _answer(request: httpx.Request) -> httpx.Response:+ if second["on"] and _column(request) == "force":+ if second["force_429s"] > 0:+ second["force_429s"] -= 1+ return httpx.Response(429, text=_RATE_LIMIT_BODY)+ return httpx.Response(200, json=_volraw(request.url.params["query"], fresh_force))+ return route(request)++ respx.get(url__regex=_GDELT_URL).mock(side_effect=_answer)+ await ns.fetch_hostility_timeline(fetcher, "Iran", "Israel", days=3, now=_NOW_GDELT)+ _expire_gdelt_copies(fetcher, age_s=26 * 3600)+ second["on"] = True++ r = await ns.fetch_hostility_timeline(fetcher, "Iran", "Israel", days=3, now=_NOW_GDELT)+ assert "degraded" not in r+ assert r["column_status"]["force"] == "ok"+ assert r["stale_age_seconds"] == {}+ d16 = next(d for d in r["days"] if d["date"] == "2026-10-16")+ assert d16["hostile"]["force"] == sum(fresh_force[14:38])+++async def test_hostility_timeline_partial_category_column_is_flagged_per_column(+ fetcher, _fast+) -> None:+ # Review M1 (C): the base series covers all of 10-16 but the force+ # series returned only 00-11Z; the half-day force count sat next to+ # full-day counts with partial False.+ route, _ = _gdelt_router()++ def _answer(request: httpx.Request) -> httpx.Response:+ if _column(request) == "force":+ return httpx.Response(200, json=_volraw_dropping(+ request.url.params["query"], _DERIVED["force"],+ lambda d: d.startswith("20261016") and int(d[9:11]) >= 12,+ ))+ return route(request)++ respx.get(url__regex=_GDELT_URL).mock(side_effect=_answer)+ r = await ns.fetch_hostility_timeline(fetcher, "Iran", "Israel", days=3, now=_NOW_GDELT)+ days = {d["date"]: d for d in r["days"]}+ d16 = days["2026-10-16"]+ assert d16["hostile"]["force"] == sum(_DERIVED["force"][14:26])+ assert d16["partial"] is True+ assert d16["partial_columns"] == ["force"]+ assert d16["hours_missing"] == [f"{h:02d}" for h in range(12, 24)]+ assert d16["coverage_by_column"]["force"] == {+ "buckets_returned": 12, "hours_covered": 12, "partial": True,+ }+ assert d16["coverage_by_column"]["articles"] == {+ "buckets_returned": 24, "hours_covered": 24, "partial": False,+ }+ # A half-day count over a full-day count is not a share.+ assert d16["hostile_share"]["force"] is None+ assert d16["hostile_share"]["sanction"] == round(79 / 732, 4)+ assert days["2026-10-17"]["partial"] is False+ assert days["2026-10-17"]["partial_columns"] == []+ # 10-15 starts at 10:00 in every series: partial everywhere, and the+ # shares compare like with like.+ assert days["2026-10-15"]["partial_columns"] == ["articles", "threat", "sanction", "force", "criticism"]+ assert days["2026-10-15"]["hostile_share"]["force"] is not None+++async def test_hostility_timeline_day_missing_from_the_judging_series_is_not_complete(+ fetcher, _fast+) -> None:+ # Review M1 (A): with the base call failed, the threat series judges+ # partial days, and it has no buckets for 10-16. That day read partial+ # False with hours_covered 0 (surviving mutant M22), and the other+ # columns' counts carried no gap.+ route, _ = _gdelt_router(fail={"articles"})++ def _answer(request: httpx.Request) -> httpx.Response:+ if _column(request) == "threat":+ return httpx.Response(200, json=_volraw_dropping(+ request.url.params["query"], _DERIVED["threat"],+ lambda d: d.startswith("20261016"),+ ))+ return route(request)++ respx.get(url__regex=_GDELT_URL).mock(side_effect=_answer)+ r = await ns.fetch_hostility_timeline(fetcher, "Iran", "Israel", days=3, now=_NOW_GDELT)+ d16 = next(d for d in r["days"] if d["date"] == "2026-10-16")+ assert d16["hostile"]["threat"] is None+ assert d16["hostile"]["sanction"] == _DAY_SANCTION["2026-10-16"]+ assert d16["partial"] is True+ assert d16["partial_columns"] == ["threat"]+ assert d16["coverage_by_column"]["threat"] == {+ "buckets_returned": 0, "hours_covered": 0, "partial": True,+ }+ assert d16["coverage_by_column"]["sanction"]["partial"] is False+ assert any(+ g.startswith("threat:") and "2026-10-16" in g and "not zero" in g for g in r["data_gaps"]+ )+++async def test_hostility_timeline_failed_base_still_names_days_no_series_returned(+ fetcher, _fast+) -> None:+ # The "no buckets" gap used to be computed from the base series only,+ # so with the base call failed, unreturned days went unnamed.+ route, _ = _gdelt_router(fail={"articles"})+ respx.get(url__regex=_GDELT_URL).mock(side_effect=route)+ r = await ns.fetch_hostility_timeline(fetcher, "Iran", "Israel", days=5, now=_NOW_GDELT)+ days = {d["date"]: d for d in r["days"]}+ assert days["2026-10-13"]["hostile"] == {c: None for c in ns.GDELT_HOSTILITY_TERMS}+ assert days["2026-10-13"]["partial"] is True+ assert any("no buckets for 2 of 5" in g for g in r["data_gaps"])+++async def test_hostility_timeline_retry_requests_are_spaced_by_the_floor(+ fetcher, monkeypatch: pytest.MonkeyPatch+) -> None:+ # Review M3: attempts inside one call went 1 s and 2 s apart, and the+ # timeline's retry pass added three more such requests per failed+ # column. Every GDELT request now follows the previous one by at least+ # GDELT's 5 s floor (sleeps recorded, not slept).+ events: list[tuple[str, float]] = []++ async def _record(delay, *args, **kwargs) -> None:+ events.append(("sleep", delay))++ monkeypatch.setattr(asyncio_mod, "sleep", _record)+ route, calls = _gdelt_router(fail={"sanction"})++ def _answer(request: httpx.Request) -> httpx.Response:+ events.append(("request", 0.0))+ return route(request)++ respx.get(url__regex=_GDELT_URL).mock(side_effect=_answer)+ r = await ns.fetch_hostility_timeline(fetcher, "Iran", "Israel", days=3, now=_NOW_GDELT)+ assert Counter(c for c, _ in calls)["sanction"] == 6+ assert r["column_status"]["sanction"] == "failed"+ gaps: list[float] = []+ waited: float | None = None+ for kind, delay in events:+ if kind == "request":+ if waited is not None:+ gaps.append(waited)+ waited = 0.0+ elif waited is not None:+ waited += delay+ assert len(gaps) == 10+ assert min(gaps) > 4.5, gaps+++async def test_hostility_timeline_retry_pass_stops_when_a_retry_fails_again(+ fetcher, _fast+) -> None:+ route, calls = _gdelt_router(fail={"sanction", "force"})+ respx.get(url__regex=_GDELT_URL).mock(side_effect=route)+ r = await ns.fetch_hostility_timeline(fetcher, "Iran", "Israel", days=3, now=_NOW_GDELT)+ counts = Counter(c for c, _ in calls)+ # sanction: three attempts, then three more on the retry pass, which+ # then stops: GDELT is still refusing, so force is not asked again.+ assert counts["sanction"] == 6+ assert counts["force"] == 3+ assert r["column_status"]["force"] == "failed"+ assert any("retry pass stopped" in g and "force" in g for g in r["data_gaps"])+++def test_country_resolution_maps_two_letter_codes() -> None:+ assert ns._resolve_country("IR") == ("Iran", "IRN")+ assert ns._resolve_country("ua") == ("Ukraine", "UKR")+++async def test_hostility_timeline_refuses_codes_it_cannot_name(fetcher, _fast) -> None:+ # Review L2: "GBR" or "FRA" used to be sent to GDELT as literal text,+ # which reads as low coverage; "UK" is too short for GDELT.+ route, calls = _gdelt_router()+ respx.get(url__regex=_GDELT_URL).mock(side_effect=route)+ for code in ("GBR", "UK", "fra"):+ r = await ns.fetch_hostility_timeline(fetcher, code, now=_NOW_GDELT)+ assert f"'{code}' looks like a country code" in r["error"]+ assert r["days"] == []+ r = await ns.fetch_hostility_timeline(fetcher, "Iran", "DE", now=_NOW_GDELT)+ assert "'DE' looks like a country code" in r["error"]+ assert calls == []+++_SHALLOW_ITEMS = [+ # Every feed keeps about 10 h of history, so none reaches back to a+ # 24 h prior window's start: nothing can be compared.+ _item("Russia launches missile strikes on Kyiv as Ukraine war grinds on", "BBC World", 2),+ _item("Russia and Ukraine trade prisoners", "BBC World", 10),+ _item("Russian drone attack hits Ukrainian grid", "Al Jazeera", 1),+ _item("Mexico unveils new budget plan", "Al Jazeera", 9),+ _item("Mexico budget vote delayed", "NPR World", 3),+]+++async def test_situations_change_is_not_measured_when_no_feed_is_comparable(+ fetcher, monkeypatch: pytest.MonkeyPatch+) -> None:+ # Review M2: with no comparable feed, change read 0 and change_ratio 1.0+ # ("no change"), and the CLI printed +0.+ _patch_feed(monkeypatch, _feed(_SHALLOW_ITEMS))+ r = await ns.fetch_situations(fetcher, min_articles=1, now=_NOW)+ assert r["coverage"]["feeds_comparable"] == 0+ assert r["change_measured"] is False+ rus_ukr = next(s for s in r["situations"] if s["key"] == "RUS-UKR")+ assert rus_ukr["articles"] == 3+ assert rus_ukr["change_measured"] is False+ for key in ("current_comparable", "prior", "change", "change_ratio"):+ assert rus_ukr[key] is None, key+ assert any(g.startswith("change not measured") for g in r["data_gaps"])++ by_change = await ns.fetch_situations(fetcher, sort="change", min_articles=1, now=_NOW)+ assert by_change["sort"] == "change"+ assert by_change["sort_applied"] == "covered"+ assert [s["key"] for s in by_change["situations"]] == ["RUS-UKR", "MEX"]+ assert any("ranked by coverage instead" in g for g in by_change["data_gaps"])+++async def test_situations_measured_change_says_so_and_flags_a_thin_sample(+ fetcher, monkeypatch: pytest.MonkeyPatch+) -> None:+ _patch_feed(monkeypatch, _feed(_SITUATION_ITEMS))+ r = await ns.fetch_situations(fetcher, sort="change", min_articles=1, now=_NOW)+ assert r["change_measured"] is True+ assert r["sort_applied"] == "change"+ assert all(s["change_measured"] is True for s in r["situations"])+ # 4 of the 6 current-window items come from comparable feeds.+ assert not any("thin sample" in g for g in r["data_gaps"])++ thin = _SITUATION_ITEMS + [+ _item(f"Russia and Ukraine story {i}", "NPR World", 1 + i / 10) for i in range(6)+ ]+ _patch_feed(monkeypatch, _feed(thin))+ r = await ns.fetch_situations(fetcher, min_articles=1, now=_NOW)+ assert any("4 of 12 current-window items" in g and "thin sample" in g for g in r["data_gaps"])+++async def test_situations_item_exactly_at_the_prior_start_is_in_neither_window(+ fetcher, monkeypatch: pytest.MonkeyPatch+) -> None:+ # Review mutant M16: windows are (start, end], so an item stamped+ # exactly at the prior window's start belongs to neither window; it+ # still shows that its feed's history reaches back that far.+ items = [+ _item("Mexico unveils new budget plan", "Al Jazeera", 5),+ _item("Mexico budget vote delayed", "Al Jazeera", 48),+ ]+ _patch_feed(monkeypatch, _feed(items))+ r = await ns.fetch_situations(fetcher, min_articles=1, now=_NOW)+ mex = next(s for s in r["situations"] if s["key"] == "MEX")+ assert (mex["current_comparable"], mex["prior"], mex["change"]) == (1, 0, 1)+ assert r["coverage"]["feeds_comparable"] == 1+ assert r["coverage"]["items_prior_window"] == 0+++async def test_top_stories_attention_is_not_measured_when_no_feed_is_comparable(+ fetcher, monkeypatch: pytest.MonkeyPatch+) -> None:+ # Review M2: an empty attention list read as "nothing grew".+ _patch_feed(monkeypatch, _feed(_SHALLOW_ITEMS))+ r = await ns.fetch_top_stories(fetcher, now=_NOW)+ assert r["attention"] == []+ assert r["attention_change_measured"] is False+ assert any(g.startswith("attention not measured") for g in r["data_gaps"])++ _patch_feed(monkeypatch, _feed(_top_items()))+ measured = await ns.fetch_top_stories(fetcher, now=_NOW)+ assert measured["attention_change_measured"] is True+ assert measured["attention"][0]["name"] == "Yemen"+++async def test_situations_and_top_stories_proceed_on_a_partial_feed_outage(+ fetcher, monkeypatch: pytest.MonkeyPatch+) -> None:+ # Review M5: a partial outage now carries degraded/reason; it is not a+ # total outage, so rankings are built and the marker passed on.+ marker = {"degraded": True, "reason": "partial_feeds_failed", "feeds_ok": 1,+ "feeds_attempted": 3, "feeds_failed": ["CFR", "DW News"]}+ _patch_feed(monkeypatch, _feed(_SITUATION_ITEMS, **marker))+ r = await ns.fetch_situations(fetcher, min_articles=1, now=_NOW)+ assert r["situations"]+ assert "error" not in r+ assert (r["degraded"], r["reason"]) == (True, "partial_feeds_failed")+ assert any(g.startswith("only 1 of 3 RSS feeds answered live") for g in r["data_gaps"])++ _patch_feed(monkeypatch, _feed(_top_items(), **marker))+ t = await ns.fetch_top_stories(fetcher, now=_NOW)+ assert t["count"] == 1+ assert (t["degraded"], t["reason"]) == (True, "partial_feeds_failed")+ assert any(g.startswith("only 1 of 3 RSS feeds answered live") for g in t["data_gaps"])+++async def test_situations_and_top_stories_clamp_limits_to_one(+ fetcher, monkeypatch: pytest.MonkeyPatch+) -> None:+ # Review L3: limit -1 dropped the last result and 0 returned nothing;+ # min_articles 0 divided by zero on a situation seen only in the prior+ # window.+ prior_only = _item("Iraq and Syria close their border", "BBC World", 30)+ _patch_feed(monkeypatch, _feed(_SITUATION_ITEMS + [prior_only]))+ for bad in (0, -1):+ r = await ns.fetch_situations(fetcher, limit=bad, min_articles=1, now=_NOW)+ assert [s["key"] for s in r["situations"]] == ["RUS-UKR"]+ assert r["limit"] == 1+ r = await ns.fetch_situations(fetcher, min_articles=0, now=_NOW)+ assert r["min_articles"] == 1+ assert "IRQ-SYR" not in {s["key"] for s in r["situations"]}++ _patch_feed(monkeypatch, _feed(_top_items()))+ t = await ns.fetch_top_stories(fetcher, limit=0, attention_limit=-1, min_outlets=1, now=_NOW)+ assert t["count"] == 1+ assert len(t["attention"]) == 1+ assert (t["limit"], t["attention_limit"]) == (1, 1)+++def _blocking_clusterer(monkeypatch: pytest.MonkeyPatch, module) -> None:+ real = module.cluster_articles++ def _slow(*args, **kwargs):+ time.sleep(0.3)+ return real(*args, **kwargs)++ monkeypatch.setattr(module, "cluster_articles", _slow)+++async def _ticks_during(coro) -> tuple[int, object]:+ """How often a 10 ms ticker ran while ``coro`` was awaited."""+ ticks = 0+ done = asyncio_mod.Event()++ async def _ticker() -> None:+ nonlocal ticks+ while not done.is_set():+ await asyncio_mod.sleep(0.01)+ ticks += 1++ task = asyncio_mod.create_task(_ticker())+ try:+ result = await coro+ finally:+ done.set()+ await task+ return ticks, result+++async def test_top_stories_and_clusters_cluster_off_the_event_loop(+ fetcher, monkeypatch: pytest.MonkeyPatch+) -> None:+ # Review L1: clustering 3,500 titles took 6.4 s on the event loop the+ # collector and the MCP server share.+ _patch_feed(monkeypatch, _feed(_top_items()))+ _blocking_clusterer(monkeypatch, ns)+ ticks, r = await _ticks_during(ns.fetch_top_stories(fetcher, now=_NOW))+ assert r["count"] == 1+ assert ticks >= 10, ticks++ _blocking_clusterer(monkeypatch, clustering)+ ticks, c = await _ticks_during(clustering.fetch_news_clusters(fetcher))+ assert c["cluster_count"] >= 1+ assert ticks >= 10, ticks+++def test_cli_hostility_marks_partial_and_stale_columns(+ _cli_env, monkeypatch: pytest.MonkeyPatch+) -> None:+ payload = {+ "parties": [{"name": "Iran"}, {"name": "Israel"}],+ "days_requested": 1,+ "date_resolution": "hour",+ "column_status": dict({c: "ok" for c in _COLUMNS}, criticism="stale"),+ "stale_age_seconds": {"criticism": 93600.0},+ "days": [+ {"date": "2026-10-16", "articles": 732, "partial": True, "partial_columns": ["force"],+ "tone": -2.6654,+ "hostile": {"threat": 140, "sanction": 79, "force": 120, "criticism": None}},+ ],+ "data_gaps": [],+ }+ monkeypatch.setattr(ns, "fetch_hostility_timeline", _fake(payload))+ result = _invoke("hostility", "Iran", "Israel", "-d", "1")+ assert result.exit_code == 0+ row = next(line for line in result.output.splitlines() if "2026-10-16" in line)+ assert row.split() == ["2026-10-16", "*", "732", "140", "79", "120*", "n/a", "-2.6654"]+ assert "criticism is an expired cached copy (26.0 h old)" in result.output+++def test_cli_situations_prints_unmeasured_change_as_not_measured(+ _cli_env, monkeypatch: pytest.MonkeyPatch+) -> None:+ payload = {+ "window": {"current": {"start": "2026-10-16T16:00:00Z", "end": "2026-10-17T16:00:00Z"}},+ "change_measured": False,+ "situations": [+ {"kind": "between", "countries": [{"name": "Russia"}, {"name": "Ukraine"}],+ "articles": 3, "hostile_articles": 2, "change": None, "change_measured": False,+ "outlet_count": 2, "sample_headlines": [{"title": "Russia launches missile strikes"}]},+ ],+ "data_gaps": [],+ }+ monkeypatch.setattr(ns, "fetch_situations", _fake(payload))+ result = _invoke("situations")+ assert result.exit_code == 0+ assert "Change not measured" in result.output+ assert "+0" not in result.output+++def test_cli_top_stories_prints_unmeasured_attention(+ _cli_env, monkeypatch: pytest.MonkeyPatch+) -> None:+ payload = {"stories": [], "attention": [], "attention_change_measured": False,+ "data_gaps": []}+ monkeypatch.setattr(ns, "fetch_top_stories", _fake(payload))+ result = _invoke("top-stories")+ assert result.exit_code == 0+ assert "Attention: not measured" in result.output+++async def test_hostility_timeline_failed_retry_keeps_the_expired_copy_status(+ fetcher, monkeypatch: pytest.MonkeyPatch, _fast+) -> None:+ # The force call finds only an expired copy and its retry finds nothing+ # (the copy evicted in between): the column stays "stale" with the+ # copy's age instead of turning into a plain failure.+ calls: Counter = Counter()++ async def _timeline(fetcher, query, mode, days):+ col = "tone" if mode == "timelinetone" else _category_of(query)+ calls[col] += 1+ if col == "force":+ if calls[col] == 1:+ return dict(_volraw(query, _DERIVED["force"]), _stale=True, _stale_age_seconds=7200.0)+ return None+ if col == "tone":+ return _tone(query)+ if col == "articles":+ return _volraw(query, [v for v, _ in _BASE])+ if col == "sanction":+ return _volraw(query, _SANCTION)+ return _volraw(query, _DERIVED[col])++ monkeypatch.setattr(ns, "_gdelt_timeline", _timeline)+ r = await ns.fetch_hostility_timeline(fetcher, "Iran", "Israel", days=3, now=_NOW_GDELT)+ assert calls["force"] == 2+ assert r["column_status"]["force"] == "stale"+ assert r["stale_age_seconds"] == {"force": 7200.0}+ assert all(d["hostile"]["force"] is None for d in r["days"])+ assert any(+ g.startswith("force:") and "expired cached copy (2.0 h old)" in g for g in r["data_gaps"]+ )+++async def test_situations_half_comparable_items_is_not_a_thin_sample(+ fetcher, monkeypatch: pytest.MonkeyPatch+) -> None:+ # The thin-sample note's edge: 4 comparable of 8 current-window items is+ # half, not under half.+ items = _SITUATION_ITEMS + [+ _item(f"Russia and Ukraine story {i}", "NPR World", 1 + i / 10) for i in range(2)+ ]+ _patch_feed(monkeypatch, _feed(items))+ r = await ns.fetch_situations(fetcher, min_articles=1, now=_NOW)+ assert r["coverage"]["items_current_window"] == 8+ assert r["coverage"]["items_current_window_comparable"] == 4+ assert not any("thin sample" in g for g in r["data_gaps"])diff --git a/src/world_intel_mcp/tools/news_signals.py b/src/world_intel_mcp/tools/news_signals.pyindex 583fbe8..91102f0 100644--- a/src/world_intel_mcp/tools/news_signals.py+++ b/src/world_intel_mcp/tools/news_signals.py@@ -16,19 +16,26 @@ TOOLS: list[Tool] = ["Per-day news coverage of a country, or of a pair of countries, from GDELT: ""article count, articles using threat / sanction / force / criticism terms ""(GDELT keyword match, listed in the response), and GDELT average tone. "- "A failed call or a day GDELT did not return is null and listed in "- "data_gaps, never zero."+ "A failed call, an expired cached copy, or a day GDELT did not return is "+ "null and listed in data_gaps (column_status per call), never zero; "+ "partial days are flagged per column."),inputSchema={"type": "object","properties": {"country": {"type": "string",- "description": "Country name or ISO3 code (e.g. 'Iran' or 'IRN').",+ "description": (+ "Country name, or an ISO3 or ISO2 code from the 22-country table "+ "(e.g. 'Iran', 'IRN' or 'IR'). Other names are searched verbatim; "+ "other 2- and 3-letter codes are refused."+ ),},"other": {"type": "string",- "description": "Optional second country: coverage naming both.",+ "description": (+ "Optional second country, same forms as country: coverage naming both."+ ),},"days": {"type": "integer",@@ -46,7 +53,9 @@ TOOLS: list[Tool] = ["inside a country (headlines naming one) over a recent window of the RSS ""feeds. Sort by most covered, most hostile (headline keyword heuristic), ""or biggest change against the preceding window; the response defines "- "each measure and names excluded feeds."+ "each measure and names excluded feeds. When no feed covers both windows, "+ "change is null with change_measured false and a change sort ranks by "+ "coverage (sort_applied)."),inputSchema={"type": "object",@@ -87,7 +96,8 @@ TOOLS: list[Tool] = ["Stories carried by at least min_outlets distinct outlets in a recent RSS ""window (sister feeds of one publisher count once), ranked by outlet count, ""plus attention: the countries and organizations whose headline mentions "- "grew most against the preceding window."+ "grew most against the preceding window (empty, with "+ "attention_change_measured false, when no feed covers both windows)."),inputSchema={"type": "object",