Skip to content

160-tool MCP server for real-time global intelligence from free public APIs: markets, liquidity, prediction markets, SEC filings, conflict, military, aircraft-cohort early warning, migration and border flows, cyber, climate, news and more. Dated observation history, 29 research prompts, cited briefs, geofences, SSE dashboard. MIT, no paid API keys.

Geo-scoped AOI news; fix every live GDELT fetch silently failing

Phoenix Agent (Marcs-Mac-Studio)7ee1866b777d
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 0542ead..e1f1ffd 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,49 @@
# Changelog
+## Unreleased
+
+### Changed
+- AOI news is now scoped by where the area IS, not what it is named.
+ Every other AOI domain was filtered geometrically; news alone was a
+ GDELT search for the AOI's name, so an area called "Home" returned a
+ Chinese A-share IPO article and "PGH Square" an Indian op-ed on Gaza
+ (both measured live 2026-09-02, both for a fence around Pittsburgh).
+ New `sources/geocode.py` reverse-geocodes the AOI centre via OSM
+ Nominatim (no key; 1 req/s floor and an identifying User-Agent per
+ its usage policy; 30-day cache since places do not move) and the
+ brief/changes/digest search GDELT for the settlement and county
+ (`("Pittsburgh" OR "Allegheny County")`). The brief reports how news
+ was scoped in `news_scoping`; when geocoding is unavailable (geocoder
+ down, open ocean) it falls back to the AOI name and says so in
+ `data_gaps`, because that is exactly when articles may be unrelated.
+
+### Fixed
+- Every live GDELT fetch had been failing and hiding behind stale
+ cache. Root cause, measured: api.gdeltproject.org TCP-connects in
+ 0.03 s but its TLS handshake took 19.7 s, 17.4 s, then >45 s across
+ three samples (Nominatim and USGS: 0.05 s on the same network), and
+ the Fetcher's 15 s default counts TLS inside the connect phase, so
+ every request was a `ConnectTimeout`. Both GDELT call sites (news
+ search, company profile) now use a shared 45 s `GDELT_TIMEOUT`;
+ live-verified afterwards with 10 real articles for the Pittsburgh
+ AOI. Handshakes slower than that still degrade to the existing honest
+ `error`/`degraded` shape.
+- GDELT DOC API requires OR clauses in parentheses: a bare `"a" OR "b"`
+ returns nothing at all (measured live). The news query builder wraps
+ multi-term queries; a regression test pins the exact form.
+- A failed Nominatim request (e.g. HTTP 429) was reported as "returned
+ an unexpected response shape", pointing a reader at parsing when the
+ cause was rate limiting. A failed fetch and a genuinely odd payload
+ now carry distinct messages.
+
+### Testing
+- The AOI test suite had briefly become non-hermetic: the shared
+ domain stub did not cover the new geocoder, so ~150 tests were each
+ making a live Nominatim request (suite time 0.9 s -> 15.8 s, and a
+ real 429 from OSM). The stub now covers it (back to 0.7 s), proven by
+ temporarily making any live geocode call raise: only the six tests
+ that mock Nominatim themselves touched it.
+
## 0.9.0 - 2026-09-02
AOI geofences become a continuous watch: the collector daemon sweeps
diff --git a/CLAUDE.md b/CLAUDE.md
index 212beab..3a1ed11 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -73,7 +73,9 @@ collector.py (daemon) ──┘
## Key Patterns
-- **Source name string**: The `source` parameter in `fetcher.get_json()` identifies the API for circuit breaking and rate limiting. Must match entries in `_SOURCE_RATE_LIMITS` if rate-limited (e.g., `"yahoo-finance"`, `"coingecko"`, `"adsblol"`).
+- **Source name string**: The `source` parameter in `fetcher.get_json()` identifies the API for circuit breaking and rate limiting. Must match entries in `_SOURCE_RATE_LIMITS` if rate-limited (e.g., `"yahoo-finance"`, `"coingecko"`, `"adsblol"`, `"nominatim"`).
+- **GDELT needs its own timeout**: `api.gdeltproject.org` TCP-connects instantly but its TLS handshake runs 17-20 s and sometimes longer (measured 2026-09-02), and the Fetcher's 15 s default counts TLS inside connect. Every GDELT `get_json` call must pass `timeout=GDELT_TIMEOUT` (from `sources/news.py`, 45 s) or it silently fails and serves stale cache.
+- **AOI news is geographic** (`sources/geocode.py`): AOI briefs reverse-geocode the area centre with OSM Nominatim and search GDELT for the settlement/county, never the AOI's name. Nominatim policy: 1 req/s (`_SOURCE_RATE_LIMITS`), identifying User-Agent with contact URL, results cached 30 days. Tests must stub `geocode.fetch_place_context` (the shared `_patch_all_domains` helper in `test_aoi.py` does) — a suite that hits Nominatim live gets rate-limited within seconds.
- **Tool dispatch**: `server.py` uses Python `match/case` to route tool names to source functions. Tool names follow `intel_*` convention.
- **All source functions take `fetcher` as first arg** — never construct your own httpx client.
- **Tests strip proxy env vars** automatically via `conftest.py` fixture (prevents SOCKS proxy interference). The conftest also resets global fetcher rate-limit locks between tests to avoid cross-event-loop binding.
diff --git a/README.md b/README.md
index cca29d0..3bedea3 100644
--- a/README.md
+++ b/README.md
@@ -57,7 +57,7 @@ Built for AI agents that need world awareness: market conditions, geopolitical r
| **Cross-Domain Analytics** | 3 | Correlation, domain summary, trend detection |
| **Reports** | 1 | PDF/HTML multi-domain intelligence reports |
| **Daily Digest** | 1 | Cited markdown morning brief: top events, headlines, trends, and timeline |
-| **AOI Geofences** | 10 | User-defined areas of interest in three shapes (circle, polygon, corridor): define/list/update/delete, a cited multi-domain brief, hotspot escalation scoring, and enter/leave change detection for a user's own area |
+| **AOI Geofences** | 10 | User-defined areas of interest in three shapes (circle, polygon, corridor): define/list/update/delete, a cited multi-domain brief, hotspot escalation scoring, and enter/leave change detection for a user's own area. News is scoped to the place the area sits in (OSM Nominatim reverse geocoding, no key), not to the area's name |
| **Severe Weather** | 2 | NWS active CAP alerts (US), Meteoalarm (39 EU countries) |
| **Space Launches** | 1 | Launch Library 2 upcoming launches |
| **Volcanoes** | 1 | Smithsonian GVP weekly activity report |
diff --git a/ROADMAP.md b/ROADMAP.md
index 4d90996..a58ebe9 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -531,7 +531,7 @@ Ordered by value; numbers are proposals, not commitments.
| Corridor AOIs | Waypoint route + width; distances measured to the route | :white_check_mark: Shipped in 0.5.0 (`intel_aoi_define_corridor`) |
| AOI groups / watchlists | `intel_aoi_digest` sweeps all (or named) AOIs in one call, advancing snapshots | :white_check_mark: Shipped in 0.8.0 |
| Escalation news + convergence components | `intel_aoi_escalation` reports null for news/convergence; wire GDELT name-mention counts and the existing convergence grid | :yellow_circle: |
-| Geo-scoped news | AOI news is name-mention only; a generic AOI name ("Home") yields junk. Investigate GDELT geo filters / GKG location fields | :yellow_circle: |
+| Geo-scoped news | AOI centre reverse-geocoded via OSM Nominatim (`sources/geocode.py`); GDELT searched for the settlement + county, e.g. `("Pittsburgh" OR "Allegheny County")`. Brief reports `news_scoping`; name-mention fallback is disclosed as a data gap. GDELT GEO 2.0 API was tried first and is 404 (both path variants, measured), so per-article coordinates are not available; the curated 105-city set could not do this either (nearest to Pittsburgh: Toronto, ~360 km) | :white_check_mark: Shipped (unreleased) |
| CLI parity | `intel aoi` group with all 9 subcommands, shared store semantics, live-smoked | :white_check_mark: Shipped in 0.8.0 |
| Dashboard AOI layer | Draw defined AOIs on the Leaflet map; show per-AOI counts | :red_circle: |
| Scheduled AOI sweeps | `aoi_digest` is a collector source (`fetch_aoi_sweep`, 240 s budget): every daemon cycle advances all AOI snapshots, so `--daemon` + the launchd wrapper IS the schedule. Change digests land in the collector log and vector store | :white_check_mark: Shipped in 0.9.0 |
diff --git a/src/world_intel_mcp/analysis/aoi.py b/src/world_intel_mcp/analysis/aoi.py
index ba2fade..5255f76 100644
--- a/src/world_intel_mcp/analysis/aoi.py
+++ b/src/world_intel_mcp/analysis/aoi.py
@@ -1191,6 +1191,67 @@ async def _fetch_military_merged(fetcher, bboxes: list[str]) -> dict:
return out
+async def _aoi_news_query(fetcher, aoi_row: dict) -> tuple[str, dict]:
+ """Build the AOI's news query from where the area *is*.
+
+ Every other domain in an AOI brief is scoped geometrically; news
+ was the exception, searched by mention of whatever the user named
+ the area. Measured live 2026-09-02: an AOI named "Home" returned a
+ Chinese A-share IPO article and one named "PGH Square" an Indian
+ op-ed, both for a fence around Pittsburgh.
+
+ Reverse-geocoding the AOI centre gives the settlement and county
+ containing it, which is what a reader means by "news near here".
+ When that is unavailable (geocoder down, or genuinely unnamed
+ ground such as open ocean) the AOI name is the only thing left, so
+ it is used - and the returned record says so, because that is
+ precisely when the articles may be unrelated.
+
+ Returns ``(query, scoping)``.
+ """
+ from ..sources import geocode
+
+ aoi_name = str(aoi_row.get("name") or "").strip()
+ place_ctx = await geocode.fetch_place_context(
+ fetcher, aoi_row.get("lat"), aoi_row.get("lon")
+ )
+
+ if isinstance(place_ctx, dict) and place_ctx.get("error"):
+ return aoi_name, {
+ "basis": "aoi_name",
+ "query": aoi_name,
+ "reason": f"Reverse geocoding failed: {place_ctx['error']}",
+ }
+
+ terms = place_ctx.get("terms") or [] if isinstance(place_ctx, dict) else []
+ if not terms:
+ return aoi_name, {
+ "basis": "aoi_name",
+ "query": aoi_name,
+ "reason": (
+ place_ctx.get("note")
+ if isinstance(place_ctx, dict) and place_ctx.get("note")
+ else "No named place at these coordinates."
+ ),
+ }
+
+ # Quote each term so multi-word places stay phrases, and OR them so
+ # a county-level story about the area still lands. The parentheses
+ # are load-bearing: GDELT's DOC API returns nothing at all for a
+ # bare `"a" OR "b"` and works for `("a" OR "b")` (measured live
+ # 2026-09-02), so omitting them would silently empty AOI news.
+ quoted = [f'"{t}"' for t in terms]
+ query = quoted[0] if len(quoted) == 1 else "(" + " OR ".join(quoted) + ")"
+ return query, {
+ "basis": "geocoded",
+ "query": query,
+ "place": place_ctx.get("place"),
+ "county": place_ctx.get("county"),
+ "display_name": place_ctx.get("display_name"),
+ "source": "nominatim-reverse",
+ }
+
+
async def _gather_scoped_domains(fetcher, aoi_row: dict) -> dict[str, dict]:
"""Fetch all six dynamic domains in parallel and scope each to the
AOI. The single shared scoping path for ``fetch_aoi_brief`` and
@@ -1203,11 +1264,13 @@ async def _gather_scoped_domains(fetcher, aoi_row: dict) -> dict[str, dict]:
regions) use the AOI's bounding circle; membership uses the exact
shape. Military may additionally carry ``partial_errors``
(antimeridian AOIs query two boxes; one can fail alone). News items
- are the raw articles — scoped by AOI-name mention, not geography."""
+ are the raw articles, scoped by the place the AOI sits in (see
+ ``_aoi_news_query``); ``scoped["news"]["scoping"]`` records whether
+ that came from geocoding or fell back to the AOI's own name."""
from ..sources import aviation, conflict, news, seismology
lat, lon, radius_km = aoi_row["lat"], aoi_row["lon"], aoi_row["radius_km"]
- aoi_name = aoi_row["name"]
+ news_query, news_scoping = await _aoi_news_query(fetcher, aoi_row)
bboxes = bboxes_from_radius_km(lat, lon, radius_km)
wildfire_regions = _overlapping_wildfire_regions(lat, lon, radius_km)
@@ -1225,7 +1288,9 @@ async def _gather_scoped_domains(fetcher, aoi_row: dict) -> dict[str, dict]:
_safe_fetch(_fetch_wildfires_for_aoi(fetcher, wildfire_regions), "wildfires"),
_safe_fetch(aviation.fetch_domestic_flights(fetcher), "domestic_flights"),
_safe_fetch(
- news.fetch_gdelt_search(fetcher, query=aoi_name, mode="artlist", limit=10),
+ news.fetch_gdelt_search(
+ fetcher, query=news_query, mode="artlist", limit=10
+ ),
"news",
),
)
@@ -1253,12 +1318,13 @@ async def _gather_scoped_domains(fetcher, aoi_row: dict) -> dict[str, dict]:
_scope(aviation_result, "sampled", "aviation", "lat", "lon")
if isinstance(news_result, dict) and news_result.get("error"):
- scoped["news"] = {"error": str(news_result["error"])}
+ scoped["news"] = {"error": str(news_result["error"]), "scoping": news_scoping}
else:
scoped["news"] = {
"items": (
news_result.get("articles", []) if isinstance(news_result, dict) else []
- )
+ ),
+ "scoping": news_scoping,
}
return scoped
@@ -1411,7 +1477,14 @@ async def fetch_aoi_brief(fetcher, store: AOIStore, name: Any) -> dict:
f"- [{n}] {len(aviation_in_range)} sampled aircraft observed",
)
- # News headline mentions -----------------------------------------------
+ # News (scoped to the place the AOI sits in) ---------------------------
+ news_scoping = scoped["news"].get("scoping") or {}
+ if news_scoping.get("basis") == "aoi_name":
+ data_gaps.append(
+ "News is scoped by the AOI's name, not its location "
+ f"({news_scoping.get('reason', 'reverse geocoding unavailable')}); "
+ "articles may be unrelated to the area."
+ )
if scoped["news"].get("error"):
data_gaps.append(f"News: {scoped['news']['error']}")
counts["news"] = 0
@@ -1518,6 +1591,7 @@ async def fetch_aoi_brief(fetcher, store: AOIStore, name: Any) -> dict:
"sources": sources,
"cited": cited,
"data_gaps": data_gaps,
+ "news_scoping": news_scoping,
"source": "aoi-brief",
"timestamp": _utc_now_iso(),
}
diff --git a/src/world_intel_mcp/analysis/company.py b/src/world_intel_mcp/analysis/company.py
index 94a956e..143cc45 100644
--- a/src/world_intel_mcp/analysis/company.py
+++ b/src/world_intel_mcp/analysis/company.py
@@ -117,11 +117,14 @@ def _raw_val(field) -> float | int | None:
async def _fetch_company_news(fetcher: Fetcher, query: str) -> list[dict]:
"""Fetch recent news about the company from GDELT."""
+ from ..sources.news import GDELT_TIMEOUT
+
data = await fetcher.get_json(
_GDELT_DOC_URL,
source="gdelt",
cache_key=f"company:news:{query}",
cache_ttl=1800,
+ timeout=GDELT_TIMEOUT,
params={
"query": f'"{query}"',
"mode": "artlist",
diff --git a/src/world_intel_mcp/fetcher.py b/src/world_intel_mcp/fetcher.py
index b4c992f..c31ed1e 100644
--- a/src/world_intel_mcp/fetcher.py
+++ b/src/world_intel_mcp/fetcher.py
@@ -25,6 +25,7 @@ _YAHOO_MIN_INTERVAL = 0.6 # seconds
# Per-source rate limits (min seconds between calls).
# Sources not listed here have no enforced limit.
_SOURCE_RATE_LIMITS: dict[str, float] = {
+ "nominatim": 1.1, # OSM usage policy: absolute max 1 req/sec
"ripestat": 1.0,
"yahoo-finance": 0.6, # unofficial — ~100 req/min safe
"opensky": 6.0, # free tier: 10 req/min
diff --git a/src/world_intel_mcp/sources/geocode.py b/src/world_intel_mcp/sources/geocode.py
new file mode 100644
index 0000000..77945a2
--- /dev/null
+++ b/src/world_intel_mcp/sources/geocode.py
@@ -0,0 +1,145 @@
+"""Reverse geocoding via OpenStreetMap Nominatim.
+
+Turns an AOI's coordinates into the names of the place it sits in, so
+area news can be searched by where the area *is* rather than by what
+the user happened to call it. An AOI named "Home" or "PGH Square"
+previously matched those strings in GDELT and returned unrelated
+articles from anywhere in the world.
+
+No API key. Nominatim's usage policy caps this at one request per
+second and requires an identifying User-Agent with a contact address,
+both of which are honored here; place names do not move, so results
+are cached for 30 days.
+"""
+
+from datetime import datetime, timezone
+from typing import Any
+
+from ..fetcher import Fetcher
+
+_REVERSE_URL = "https://nominatim.openstreetmap.org/reverse"
+_SOURCE = "nominatim"
+
+# A place name does not change; the only reason to refetch is a new AOI.
+_CACHE_TTL = 60 * 60 * 24 * 30
+
+# Nominatim requires an identifying UA with a way to make contact.
+_USER_AGENT = "world-intel-mcp/0.9 (+https://github.com/marc-shade/world-intel-mcp)"
+
+# Settlement keys in Nominatim's address object, most specific first.
+_SETTLEMENT_KEYS = ("city", "town", "village", "hamlet", "municipality", "suburb")
+
+
+def _utc_now_iso() -> str:
+ return datetime.now(timezone.utc).isoformat()
+
+
+async def fetch_place_context(fetcher: Fetcher, lat: Any, lon: Any) -> dict:
+ """Resolve coordinates to the names of the place containing them.
+
+ Returns a dict with ``place`` (the settlement, or None where
+ nothing is named — mid-ocean, empty desert), ``county``, ``state``,
+ ``country``/``country_code``, ``display_name``, and ``terms``: the
+ place names most specific first, suitable for a news query.
+
+ A point with no named place is a successful lookup reporting
+ ``place: None``, deliberately distinct from ``{"error": ...}`` for
+ an upstream failure — a caller must be able to tell "nowhere is
+ named here" from "the geocoder is down", because only the second
+ is worth retrying or reporting as an outage.
+ """
+ try:
+ lat_f, lon_f = float(lat), float(lon)
+ except (TypeError, ValueError):
+ return {
+ "error": f"Invalid coordinates: lat={lat!r}, lon={lon!r}",
+ "place": None,
+ }
+ if not (-90.0 <= lat_f <= 90.0) or not (-180.0 <= lon_f <= 180.0):
+ return {
+ "error": (
+ f"Coordinates out of range: lat={lat_f} (-90..90), "
+ f"lon={lon_f} (-180..180)"
+ ),
+ "place": None,
+ }
+
+ data = await fetcher.get_json(
+ _REVERSE_URL,
+ params={
+ "lat": f"{lat_f:.5f}",
+ "lon": f"{lon_f:.5f}",
+ "format": "jsonv2",
+ # City/county granularity: a street address would make a
+ # useless news query.
+ "zoom": 10,
+ "addressdetails": 1,
+ },
+ source=_SOURCE,
+ cache_key=f"geocode:reverse:{lat_f:.4f}:{lon_f:.4f}",
+ cache_ttl=_CACHE_TTL,
+ headers={"User-Agent": _USER_AGENT},
+ )
+
+ if data is None:
+ # get_json exhausted its retries and found no stale entry. The
+ # specific cause (rate limit, 5xx, timeout, DNS) is in the
+ # fetcher's log; naming a parse problem here would send a
+ # reader to the wrong place, which a live 429 did on
+ # 2026-09-02.
+ return {
+ "error": (
+ "Reverse geocoding unavailable: the Nominatim request failed "
+ "(rate limit, upstream error, or network). See fetcher logs "
+ "for the specific failure."
+ ),
+ "place": None,
+ }
+ if not isinstance(data, dict):
+ return {
+ "error": (
+ f"Nominatim returned an unexpected response shape "
+ f"({type(data).__name__}, expected object)."
+ ),
+ "place": None,
+ }
+ if data.get("error"):
+ # Nominatim's own "unable to geocode" — a real answer about the
+ # point, not a transport failure.
+ return {
+ "place": None,
+ "county": None,
+ "state": None,
+ "country": None,
+ "country_code": None,
+ "display_name": None,
+ "terms": [],
+ "note": f"No named place at {lat_f}, {lon_f} (open water or unnamed area).",
+ "source": "nominatim-reverse",
+ "timestamp": _utc_now_iso(),
+ }
+
+ address = data.get("address") or {}
+ place = next(
+ (address[k] for k in _SETTLEMENT_KEYS if address.get(k)),
+ None,
+ )
+ county = address.get("county")
+ state = address.get("state")
+
+ # Most specific first: a news query leads with the settlement and
+ # widens only as far as the county, since a state-wide term buys
+ # noise rather than locality.
+ terms = [t for t in (place, county) if t]
+
+ return {
+ "place": place,
+ "county": county,
+ "state": state,
+ "country": address.get("country"),
+ "country_code": address.get("country_code"),
+ "display_name": data.get("display_name"),
+ "terms": terms,
+ "source": "nominatim-reverse",
+ "timestamp": _utc_now_iso(),
+ }
diff --git a/src/world_intel_mcp/sources/news.py b/src/world_intel_mcp/sources/news.py
index 6ff549b..7c11078 100644
--- a/src/world_intel_mcp/sources/news.py
+++ b/src/world_intel_mcp/sources/news.py
@@ -465,6 +465,14 @@ _STOPWORDS: set[str] = {
_GDELT_DOC_URL = "https://api.gdeltproject.org/api/v2/doc/doc"
+# GDELT's TLS handshake is slow and erratic: measured 2026-09-02 at
+# 19.7 s, 17.4 s, then >45 s across three samples (TCP connect 0.03 s;
+# Nominatim and USGS handshake in 0.05 s on the same network). The
+# Fetcher's 15 s default counts TLS inside the connect phase, so every
+# live GDELT fetch was a ConnectTimeout that stale cache then hid.
+# Shared by every GDELT call site.
+GDELT_TIMEOUT = 45.0
+
# Regex to strip punctuation from words
_PUNCT_RE = re.compile(f"[{re.escape(string.punctuation)}]")
@@ -758,6 +766,7 @@ async def fetch_gdelt_search(
cache_key=f"news:gdelt:{safe_query}:{mode}",
cache_ttl=600,
params=params,
+ timeout=GDELT_TIMEOUT,
)
if data is None:
diff --git a/src/world_intel_mcp/tests/test_aoi.py b/src/world_intel_mcp/tests/test_aoi.py
index 5b31989..eaf9b4f 100644
--- a/src/world_intel_mcp/tests/test_aoi.py
+++ b/src/world_intel_mcp/tests/test_aoi.py
@@ -22,6 +22,7 @@ from world_intel_mcp.analysis.aoi import AOIAlreadyExists, AOIStore
from world_intel_mcp.sources import (
aviation,
conflict,
+ geocode,
military,
news,
seismology,
@@ -426,7 +427,23 @@ def aoi_store_with_pittsburgh(tmp_path: Path) -> AOIStore:
s.close()
+async def _fake_place_context(fetcher, lat, lon):
+ """Default reverse-geocode stub. Without this every AOI test would
+ make a live Nominatim request: slow, flaky, and rude to a free
+ service that rate-limits to one request per second."""
+ return {
+ "place": "Pittsburgh",
+ "county": "Allegheny County",
+ "state": "Pennsylvania",
+ "country_code": "us",
+ "display_name": "Pittsburgh, Allegheny County, Pennsylvania, United States",
+ "terms": ["Pittsburgh", "Allegheny County"],
+ "source": "nominatim-reverse",
+ }
+
+
def _patch_all_domains(monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setattr(geocode, "fetch_place_context", _fake_place_context)
monkeypatch.setattr(seismology, "fetch_earthquakes", _fake_earthquakes)
monkeypatch.setattr(military, "fetch_military_flights", _fake_military)
monkeypatch.setattr(conflict, "fetch_acled_events", _fake_acled)
@@ -1973,3 +1990,135 @@ async def test_sweep_notification_unknown_format_rejected(
assert note["sent"] is False
assert "xml" in note["error"] and "json" in note["error"]
assert not route.called
+
+
+# ---------------------------------------------------------------------------
+# Geo-scoped AOI news (place names, not AOI-name mention)
+# ---------------------------------------------------------------------------
+
+
+def _patch_geocode(monkeypatch: pytest.MonkeyPatch, result: dict) -> None:
+ async def _fake(fetcher_, lat, lon):
+ return dict(result)
+
+ monkeypatch.setattr(geocode, "fetch_place_context", _fake)
+
+
+async def test_news_query_uses_geocoded_place_not_aoi_name(
+ fetcher, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ """The defect this fixes, measured live 2026-09-02: an AOI named
+ "Home" queried GDELT for "Home" and got a Chinese IPO article. The
+ query must come from where the AOI IS."""
+ _patch_geocode(
+ monkeypatch,
+ {
+ "place": "Pittsburgh",
+ "county": "Allegheny County",
+ "terms": ["Pittsburgh", "Allegheny County"],
+ },
+ )
+ row = {
+ "name": "Home",
+ "lat": _PGH_LAT,
+ "lon": _PGH_LON,
+ "radius_km": _PGH_RADIUS_KM,
+ }
+
+ query, basis = await aoi._aoi_news_query(fetcher, row)
+
+ assert "Pittsburgh" in query
+ assert "Home" not in query
+ assert basis["basis"] == "geocoded"
+ assert basis["place"] == "Pittsburgh"
+
+
+async def test_news_query_falls_back_to_name_and_says_so(
+ fetcher, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ """When geocoding fails the old name-mention behavior is the only
+ option left, but the caller must be told, because that is exactly
+ when results are unreliable."""
+ _patch_geocode(monkeypatch, {"error": "Nominatim HTTP 503", "place": None})
+ row = {
+ "name": "Pittsburgh",
+ "lat": _PGH_LAT,
+ "lon": _PGH_LON,
+ "radius_km": _PGH_RADIUS_KM,
+ }
+
+ query, basis = await aoi._aoi_news_query(fetcher, row)
+
+ assert query == "Pittsburgh"
+ assert basis["basis"] == "aoi_name"
+ assert "503" in basis["reason"]
+
+
+async def test_news_query_unnamed_place_falls_back_to_name(
+ fetcher, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ """Mid-ocean AOI: the lookup succeeded but named nothing."""
+ _patch_geocode(monkeypatch, {"place": None, "terms": [], "note": "open water"})
+ row = {"name": "Bering Watch", "lat": 60.0, "lon": -179.0, "radius_km": 200.0}
+
+ query, basis = await aoi._aoi_news_query(fetcher, row)
+
+ assert query == "Bering Watch"
+ assert basis["basis"] == "aoi_name"
+
+
+async def test_brief_reports_news_scoping_basis(
+ fetcher, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ """The brief must disclose how its news was scoped; a name-mention
+ fallback is a data gap, since those results may be unrelated."""
+ _patch_all_domains(monkeypatch)
+ _patch_geocode(monkeypatch, {"error": "Nominatim HTTP 503", "place": None})
+ s = AOIStore(tmp_path / "newsbasis.db")
+ aoi.define_aoi(s, "Home", _PGH_LAT, _PGH_LON, _PGH_RADIUS_KM)
+
+ result = await aoi.fetch_aoi_brief(fetcher, s, "Home")
+
+ assert result["news_scoping"]["basis"] == "aoi_name"
+ assert any("name" in g.lower() for g in result["data_gaps"])
+ s.close()
+
+
+async def test_news_query_parenthesizes_multi_term_or(
+ fetcher, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ """Measured live 2026-09-02: GDELT's DOC API returns NOTHING for a
+ bare `"a" OR "b"` and works for `("a" OR "b")`. Unparenthesized
+ would have been worse than the defect being fixed - silent empty
+ news instead of wrong news."""
+ _patch_geocode(
+ monkeypatch,
+ {
+ "place": "Pittsburgh",
+ "county": "Allegheny County",
+ "terms": ["Pittsburgh", "Allegheny County"],
+ },
+ )
+ row = {"name": "Home", "lat": _PGH_LAT, "lon": _PGH_LON, "radius_km": _PGH_RADIUS_KM}
+
+ query, _ = await aoi._aoi_news_query(fetcher, row)
+
+ assert query.startswith("(") and query.endswith(")")
+ assert query == '("Pittsburgh" OR "Allegheny County")'
+
+
+async def test_news_query_single_term_needs_no_parens(
+ fetcher, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ _patch_geocode(monkeypatch, {"place": "Reykjavik", "county": None, "terms": ["Reykjavik"]})
+ row = {"name": "North Watch", "lat": 64.15, "lon": -21.94, "radius_km": 30.0}
+
+ query, _ = await aoi._aoi_news_query(fetcher, row)
+
+ assert query == '"Reykjavik"'
diff --git a/src/world_intel_mcp/tests/test_company.py b/src/world_intel_mcp/tests/test_company.py
index 47888bc..6995d09 100644
--- a/src/world_intel_mcp/tests/test_company.py
+++ b/src/world_intel_mcp/tests/test_company.py
@@ -4,6 +4,7 @@ import httpx
import pytest
import respx
+from world_intel_mcp.analysis import company
from world_intel_mcp.fetcher import Fetcher
@@ -233,3 +234,21 @@ async def test_fetch_company_profile_total_failure(fetcher: Fetcher) -> None:
assert result["stock"] == {}
assert result["recent_news"] == []
assert "fetched_at" in result
+
+
+async def test_company_news_uses_extended_gdelt_timeout(fetcher, monkeypatch) -> None:
+ """Same GDELT slow-TLS finding as test_sources_news; the company
+ profile's news fetch is the second GDELT call site and must not
+ keep the 15 s default that fails every live request."""
+ seen: dict = {}
+
+ async def _record(url, source, **kwargs):
+ seen.update(kwargs)
+ return {"articles": []}
+
+ monkeypatch.setattr(fetcher, "get_json", _record)
+ await company._fetch_company_news(fetcher, "Nvidia")
+
+ assert seen.get("timeout") is not None, "no timeout passed to get_json"
+ assert seen["timeout"] >= 30.0
diff --git a/src/world_intel_mcp/tests/test_sources_geocode.py b/src/world_intel_mcp/tests/test_sources_geocode.py
new file mode 100644
index 0000000..7b87dd6
--- /dev/null
+++ b/src/world_intel_mcp/tests/test_sources_geocode.py
@@ -0,0 +1,157 @@
+"""Tests for sources/geocode.py — respx-mocked Nominatim reverse lookups.
+
+Reverse geocoding exists here for one reason: AOI news was scoped by
+mention of the AOI's *name*, so an area a user called "Home" or "PGH
+Square" pulled whatever GDELT matched that string. Measured live
+2026-09-02: query "Home" returned a Chinese A-share IPO article and
+"PGH Square" an Indian op-ed on Gaza, both for a Pittsburgh geofence.
+Turning coordinates into real place names is what makes AOI news
+geographic.
+
+Gaps / not covered: Nominatim's rate-limit (429) and usage-policy
+blocking responses are mocked, not provoked live; the zoom level (10,
+city/county granularity) is asserted as sent but its result quality
+across countries is verified only for the US case measured above.
+"""
+
+import httpx
+import pytest
+import respx
+
+from world_intel_mcp.sources import geocode
+
+_URL = r".*nominatim\.openstreetmap\.org/reverse.*"
+
+_PITTSBURGH = {
+ "name": "Pittsburgh",
+ "display_name": "Pittsburgh, Allegheny County, Pennsylvania, United States",
+ "address": {
+ "city": "Pittsburgh",
+ "county": "Allegheny County",
+ "state": "Pennsylvania",
+ "country": "United States",
+ "country_code": "us",
+ },
+}
+
+
+async def test_reverse_returns_place_terms(fetcher) -> None:
+ route = respx.get(url__regex=_URL).mock(
+ return_value=httpx.Response(200, json=_PITTSBURGH)
+ )
+ result = await geocode.fetch_place_context(fetcher, 40.44, -79.99)
+
+ assert "error" not in result
+ assert result["place"] == "Pittsburgh"
+ assert result["county"] == "Allegheny County"
+ assert result["state"] == "Pennsylvania"
+ assert result["country_code"] == "us"
+ # Most specific first: a news query should lead with the city.
+ assert result["terms"][0] == "Pittsburgh"
+ assert "Allegheny County" in result["terms"]
+ assert route.called
+
+
+async def test_reverse_sends_contact_user_agent(fetcher) -> None:
+ """Nominatim's usage policy requires identifying the application
+ with a contact URL; the default fetcher UA does not carry one."""
+ route = respx.get(url__regex=_URL).mock(
+ return_value=httpx.Response(200, json=_PITTSBURGH)
+ )
+ await geocode.fetch_place_context(fetcher, 40.44, -79.99)
+
+ ua = route.calls.last.request.headers.get("User-Agent", "")
+ assert "world-intel-mcp" in ua
+ assert "github.com" in ua
+
+
+async def test_reverse_rejects_out_of_range_coordinates(fetcher) -> None:
+ route = respx.get(url__regex=_URL).mock(
+ return_value=httpx.Response(200, json=_PITTSBURGH)
+ )
+ result = await geocode.fetch_place_context(fetcher, 91.0, 0.0)
+ assert "error" in result
+ assert not route.called # never spend a request on invalid input
+
+
+async def test_reverse_open_ocean_has_no_place(fetcher) -> None:
+ """Nominatim returns an error object for points with no address.
+ That is a real answer ("nowhere named here"), not a failure to
+ hide: the caller must be able to tell it apart from an outage."""
+ respx.get(url__regex=_URL).mock(
+ return_value=httpx.Response(200, json={"error": "Unable to geocode"})
+ )
+ result = await geocode.fetch_place_context(fetcher, 0.0, -140.0)
+ assert result["place"] is None
+ assert result["terms"] == []
+ assert "error" not in result # the lookup succeeded; the place does not exist
+
+
+async def test_reverse_upstream_failure_is_an_error(fetcher) -> None:
+ respx.get(url__regex=_URL).mock(return_value=httpx.Response(503))
+ result = await geocode.fetch_place_context(fetcher, 40.44, -79.99)
+ assert "error" in result
+ assert result.get("place") is None
+
+
+async def test_town_and_village_count_as_place(fetcher) -> None:
+ """Smaller settlements come back under town/village/hamlet rather
+ than city; an AOI over a village must still get its own name."""
+ respx.get(url__regex=_URL).mock(
+ return_value=httpx.Response(
+ 200,
+ json={
+ "display_name": "Grafton, Taylor County, West Virginia",
+ "address": {
+ "town": "Grafton",
+ "county": "Taylor County",
+ "state": "West Virginia",
+ "country_code": "us",
+ },
+ },
+ )
+ )
+ result = await geocode.fetch_place_context(fetcher, 39.34, -80.01)
+ assert result["place"] == "Grafton"
+ assert result["terms"][0] == "Grafton"
+
+
+async def test_failed_request_is_not_reported_as_a_shape_problem(fetcher) -> None:
+ """Caught live 2026-09-02: a real Nominatim 429 surfaced to the AOI
+ brief as "returned an unexpected response shape", which points a
+ reader at parsing when the cause was rate limiting. A failed fetch
+ (get_json returns None) and a genuinely odd payload are different
+ failures and must not share a message."""
+ respx.get(url__regex=_URL).mock(return_value=httpx.Response(429))
+ result = await geocode.fetch_place_context(fetcher, 40.44, -79.99)
+
+ assert "error" in result
+ err = result["error"].lower()
+ assert "shape" not in err
+ assert "request" in err or "unavailable" in err
+
+
+async def test_genuinely_odd_payload_still_reports_shape(fetcher) -> None:
+ """The shape message keeps its own real case: a 200 that parses to
+ something that is not an object."""
+ respx.get(url__regex=_URL).mock(return_value=httpx.Response(200, json=[1, 2, 3]))
+ result = await geocode.fetch_place_context(fetcher, 40.44, -79.99)
+
+ assert "error" in result
+ assert "shape" in result["error"].lower()
diff --git a/src/world_intel_mcp/tests/test_sources_news.py b/src/world_intel_mcp/tests/test_sources_news.py
index 49086d4..b07abad 100644
--- a/src/world_intel_mcp/tests/test_sources_news.py
+++ b/src/world_intel_mcp/tests/test_sources_news.py
@@ -255,3 +255,24 @@ def test_parse_published_fallbacks() -> None:
assert _parse_published({"published": "raw"}) == "raw"
assert _parse_published({"updated": "raw2"}) == "raw2"
assert _parse_published({}) is None
+
+
+async def test_gdelt_search_uses_extended_timeout(fetcher, monkeypatch) -> None:
+ """Measured live 2026-09-02 from mac-studio: api.gdeltproject.org
+ TCP-connects in 0.03 s but its TLS handshake took 19.7 s, 17.4 s,
+ then >45 s across three samples, while Nominatim and USGS handshake
+ in 0.05 s. The Fetcher's 15 s default counts TLS inside connect, so
+ every live GDELT fetch was a ConnectTimeout hidden behind stale
+ cache. GDELT must be given its own budget."""
+ seen: dict = {}
+
+ async def _record(url, source, **kwargs):
+ seen.update(kwargs)
+ return {"articles": []}
+
+ monkeypatch.setattr(fetcher, "get_json", _record)
+ await news.fetch_gdelt_search(fetcher, query="test", mode="artlist", limit=1)
+
+ assert seen.get("timeout") is not None, "no timeout passed to get_json"
+ assert seen["timeout"] >= 30.0