160-tool MCP server for real-time global intelligence from free public APIs: markets, liquidity, prediction markets, SEC filings, conflict, military, aircraft-cohort early warning, migration and border flows, cyber, climate, news and more. Dated observation history, 29 research prompts, cited briefs, geofences, SSE dashboard. MIT, no paid API keys.
Python ai-toolsai-watchanthropicclaudecybersecuritydashboardgeofencinggeopoliticalgeospatialintelligencemarketsmcpmigrationmilitarymodel-context-protocolnuclearosintreal-timespace-weatherthreat-intelligence
Add cross-border migration tools; fix displacement summary returning one global row
Phoenix Agent (mac-studio)9508d102d93c
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 4fa5fcb..597e007 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -2,6 +2,48 @@## Unreleased+Cross-border migration and a fix to the displacement summary.++### Added: migration+- `intel_migration_flows`: every UNHCR origin -> destination corridor for a+ year (about 6,000 with people in them for 2025: refugees, asylum seekers+ and others in need of international protection), filterable by country+ (ISO-3, ISO-2 or name) or region (UNHCR major areas and UN sub-regions),+ with each corridor's change against the prior year and top origins and+ destinations of the selection.+- `intel_migration_surges`: largest year-over-year increases and decreases+ in people abroad, by origin and by corridor, and in internal displacement,+ with each origin's change as a share of its population (World Bank+ SP.POP.TOTL). Levels mass / major / notable are stated heuristics.+ UNHCR's non-country buckets ("Unknown" origin) are reported separately+ and never ranked.+- `intel_migration_history`: one country's yearly UNHCR series since 1951,+ as an origin or as a host, with its peak year and largest annual waves.+- `intel_net_migration`: World Bank net migration (all reasons, economic+ included) and international migrant stock: latest-year ranking in+ absolute and per-1,000 terms with world regions, a World Bank region+ filter, or one country's series since 1960.++### Fixed: displacement summary+- `intel_displacement_summary` never sent UNHCR `coo_all=true`, so UNHCR+ answered with one global row and `by_origin` held a single entry named+ "-" (found by a live call 2026-10-11). It now returns the top 30 origins+ with ISO-3 codes. Names prefer the short TIER1 name ("Syria") so+ `intel_country_signals` can join displacement with the conflict sources;+ that join had no displacement input before.++### Data honesty: migration+- UNHCR figures are year-end stocks, not border crossings; every response+ carries a `measure` string saying so. World Bank net migration is+ modelled by the UN World Population Prospects, and its latest years can+ be projections.+- UNHCR's own country codes are not ISO ("AUS" is Austria), so requests+ send `cf_type=ISO` and rows are read through `coo_iso` / `coa_iso`.+- Not included: near-real-time displacement (IOM DTM and HDX HAPI need+ keys), regional border-crossing series, and bilateral economic migration+ flows (only national net totals exist in a keyless API).++Aircraft-cohort early warning, approach inspired by[kylemcdonald/ews](https://github.com/kylemcdonald/ews) (independentimplementation; no ews code used).diff --git a/CLAUDE.md b/CLAUDE.mdindex d9ea0e4..9fbbe3a 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co## What This Is-World Intelligence MCP Server: 135 tools across 30+ domains providing real-time global intelligence from free public APIs. Serves four interfaces: MCP stdio (for Claude Code/Cursor), a live Starlette dashboard with SSE, a Click CLI with Rich output, and a collector daemon for 24/7 vector store population. Python 3.11+, built with hatchling.+World Intelligence MCP Server: 139 tools across 30+ domains providing real-time global intelligence from free public APIs. Serves four interfaces: MCP stdio (for Claude Code/Cursor), a live Starlette dashboard with SSE, a Click CLI with Rich output, and a collector daemon for 24/7 vector store population. Python 3.11+, built with hatchling.## Commands@@ -47,7 +47,7 @@ collector.py (daemon) ──┘- `CircuitBreaker`: Per-source tracking. 3 consecutive failures trips the breaker for 5 minutes. Each RSS feed gets its own breaker (`rss:bbc_world`).- `Cache`: SQLite WAL-mode TTL cache. `get()` returns live data, `get_stale()` returns expired data for fallback.-**Tool registry** (`tools/*.py` + `runtime.py`, Phase 26): server.py is a thin shell; the 135 tools live in 12 domain modules under `tools/`, each exporting `TOOLS` (list of Tool definitions) and `HANDLERS` (tool name -> `async def h(arguments)`). Shared infrastructure (cache, circuit breaker, AOI store, vector store, fetcher) is constructed once in `runtime.py`; handlers reach it as `runtime.fetcher` etc. `tools/aggregate()` enforces TOOLS/HANDLERS parity and cross-module name uniqueness at import time — a drifted registry refuses to start.+**Tool registry** (`tools/*.py` + `runtime.py`, Phase 26): server.py is a thin shell; the 139 tools live in 12 domain modules under `tools/`, each exporting `TOOLS` (list of Tool definitions) and `HANDLERS` (tool name -> `async def h(arguments)`). Shared infrastructure (cache, circuit breaker, AOI store, vector store, fetcher) is constructed once in `runtime.py`; handlers reach it as `runtime.fetcher` etc. `tools/aggregate()` enforces TOOLS/HANDLERS parity and cross-module name uniqueness at import time — a drifted registry refuses to start.**Source modules** (`sources/*.py`): Each module exports `async def fetch_*(fetcher: Fetcher, **kwargs) -> dict`. Pure data fetching — no MCP awareness. 30+ modules covering markets, seismology, military, cyber, health, tech, environmental, etc.diff --git a/README.md b/README.mdindex eed48a4..882fd6b 100644--- a/README.md+++ b/README.md@@ -6,7 +6,7 @@[](https://python.org)[](LICENSE)-Real-time global intelligence across **30+ domains** with **135 MCP tools**, a live ops-center dashboard, a CLI, and a **Qdrant vector store** for enterprise-grade semantic search across accumulated intelligence. All data comes from free, public APIs: no paid subscriptions required.+Real-time global intelligence across **30+ domains** with **139 MCP tools**, a live ops-center dashboard, a CLI, and a **Qdrant vector store** for enterprise-grade semantic search across accumulated intelligence. All data comes from free, public APIs: no paid subscriptions required.Built for AI agents that need world awareness: market conditions, geopolitical risk, military posture, supply chain disruptions, cyber threats, and more — all queryable via the Model Context Protocol. The vector store enables natural language queries like *"military activity near Taiwan"* or *"cyber threats targeting healthcare"* across all historical data.@@ -47,7 +47,7 @@ Built for AI agents that need world awareness: market conditions, geopolitical r| **Country Intelligence** | 3 | Country brief, country stocks, financial centers || **Prediction Markets** | 1 | Polymarket event contracts || **Elections** | 1 | Global election calendar with risk scoring |-| **Displacement** | 1 | UNHCR refugee/IDP data |+| **Displacement and Migration** | 5 | UNHCR refugee/IDP data, origin -> destination corridors since 1951, year-over-year surge detection; World Bank net migration and migrant stock || **Shipping** | 1 | Dry bulk shipping stress index || **Government** | 1 | USAspending.gov federal contracts || **Traffic** | 2 | Road traffic flow, real-time incidents |@@ -64,7 +64,7 @@ Built for AI agents that need world awareness: market conditions, geopolitical r| **Tropical Cyclones** | 2 | NHC (Atlantic, E/C Pacific) + JTWC (NW Pacific, N Indian, S Hemisphere) || **Situation Brief** | 1 | Cited situational awareness brief over MCP: bounded server-side overview synthesized via local Ollama, with a mechanically-cited fallback |-**Total: 135 tools** across 30+ intelligence domains.+**Total: 139 tools** across 30+ intelligence domains.---@@ -329,10 +329,14 @@ collector.py (daemon) ─┘|------|-------------|| `intel_election_calendar` | Global election calendar with risk scoring |-### Displacement (1)+### Displacement and migration (5)| Tool | Description ||------|-------------|-| `intel_displacement_summary` | UNHCR refugee/IDP statistics |+| `intel_displacement_summary` | UNHCR refugee/IDP statistics, top 30 origins |+| `intel_migration_flows` | UNHCR origin -> destination corridors (country or region filters), with change vs the prior year |+| `intel_migration_surges` | Largest year-over-year increases and decreases by origin and corridor, plus IDPs, sized against population |+| `intel_migration_history` | One country's yearly series since 1951 as an origin or a host, with its peak and largest waves |+| `intel_net_migration` | World Bank net migration and migrant stock: latest ranking, regions, or one country since 1960 |### Aviation (5)| Tool | Description |diff --git a/ROADMAP.md b/ROADMAP.mdindex 2994345..d4fac0f 100644--- a/ROADMAP.md+++ b/ROADMAP.md@@ -2,7 +2,7 @@**Benchmark**: [koala73/worldmonitor](https://github.com/koala73/worldmonitor)**Updated**: 2026-09-02 (v0.10.0)-**Current tools**: 135 (134 intel + 1 status)+**Current tools**: 139 (138 intel + 1 status)---diff --git a/src/world_intel_mcp/sources/displacement.py b/src/world_intel_mcp/sources/displacement.pyindex c74e48f..8f43424 100644--- a/src/world_intel_mcp/sources/displacement.py+++ b/src/world_intel_mcp/sources/displacement.py@@ -7,6 +7,7 @@ Population Statistics API. No API key required.import loggingfrom datetime import datetime, timezone+from ..config.countries import TIER1_COUNTRIESfrom ..fetcher import Fetcherlogger = logging.getLogger("world-intel-mcp.sources.displacement")@@ -42,17 +43,21 @@ async def fetch_displacement_summary(# Ensure year is an int even if passed as stringyear = int(year)+ # coo_all=true asks for one row per origin country. Without it UNHCR+ # returns a single global row (coo_name "-"), which is what this tool+ # served until 2026-10-11: "by_origin" held one entry named "-".params = {- "limit": 100,+ "limit": 1000,"yearFrom": year,"yearTo": year,+ "coo_all": "true","cf_type": "ISO",}data = await fetcher.get_json(url=_UNHCR_POPULATION_URL,source="unhcr",- cache_key=f"displacement:unhcr:{year}",+ cache_key=f"displacement:unhcr:by_origin:{year}",cache_ttl=43200,params=params,)@@ -86,11 +91,20 @@ async def fetch_displacement_summary(items = data.get("items", [])- # Aggregate by country of origin (coo_name)+ # Aggregate by country of origin. Rows carry coo_iso (ISO-3) under+ # cf_type=ISO; the display name prefers the TIER1 short name ("Syria",+ # not UNHCR's "Syrian Arab Rep.") so signals.py can join it with the+ # conflict sources, which use short names.by_country: dict[str, dict[str, int]] = {}+ names: dict[str, tuple[str, str | None]] = {}for item in items:- coo_name = item.get("coo_name") or "Unknown"+ iso3 = (item.get("coo_iso") or "").upper()+ iso3 = iso3 if len(iso3) == 3 else None+ tier1 = TIER1_COUNTRIES.get(iso3) if iso3 else None+ coo_name = tier1["name"] if tier1 else (item.get("coo_name") or "Unknown").strip()+ key = iso3 or coo_name+ names[key] = (coo_name, iso3)refugees = _safe_int(item.get("refugees"))asylum_seekers = _safe_int(item.get("asylum_seekers"))@@ -98,8 +112,8 @@ async def fetch_displacement_summary(stateless = _safe_int(item.get("stateless"))ooc = _safe_int(item.get("ooc"))- if coo_name not in by_country:- by_country[coo_name] = {+ if key not in by_country:+ by_country[key] = {"refugees": 0,"asylum_seekers": 0,"idps": 0,@@ -107,7 +121,7 @@ async def fetch_displacement_summary("ooc": 0,}- entry = by_country[coo_name]+ entry = by_country[key]entry["refugees"] += refugeesentry["asylum_seekers"] += asylum_seekersentry["idps"] += idps@@ -116,7 +130,8 @@ async def fetch_displacement_summary(# Build sorted list with total_displacedby_origin = []- for country, totals in by_country.items():+ for key, totals in by_country.items():+ country, iso3 = names[key]total_displaced = (totals["refugees"]+ totals["asylum_seekers"]@@ -127,6 +142,7 @@ async def fetch_displacement_summary(by_origin.append({"country": country,+ "iso3": iso3,"refugees": totals["refugees"],"asylum_seekers": totals["asylum_seekers"],"internally_displaced": totals["idps"],diff --git a/src/world_intel_mcp/sources/migration.py b/src/world_intel_mcp/sources/migration.pynew file mode 100644index 0000000..9bd9a1d--- /dev/null+++ b/src/world_intel_mcp/sources/migration.py@@ -0,0 +1,661 @@+"""Cross-border migration: UNHCR origin->destination corridors and World Bank net migration.++Two public, keyless APIs:++* UNHCR Population Statistics (https://api.unhcr.org/population/v1/): refugees,+ asylum seekers and other people in need of international protection for every+ origin/asylum country pair, 1951 to the latest reporting year. Figures are+ YEAR-END STOCKS (people present at the end of the year), not border crossings,+ so a year-over-year change is net arrivals minus returns, resettlement onward,+ naturalisation and deaths.+* World Bank WDI (https://api.worldbank.org/v2/): net migration (SM.POP.NETM,+ immigrants minus emigrants of all kinds, not only forced displacement),+ international migrant stock (SM.POP.TOTL) and population (SP.POP.TOTL).++UNHCR's own country codes differ from ISO-3 (its "AUS" is Austria), so every+UNHCR request sends ``cf_type=ISO`` and every row is read through its+``coo_iso`` / ``coa_iso`` fields.++Gaps / not covered: no near-real-time source (IOM DTM and HDX HAPI need keys);+UNHCR publishes once a year, so the latest year lags by months; economic+migration appears only as World Bank net totals, never as country-to-country+flows; the surge levels are heuristics (stated in each response), not a UN+classification.+"""++from __future__ import annotations++import logging+from datetime import datetime, timezone+from typing import Any++from ..config.countries import TIER1_COUNTRIES+from ..fetcher import Fetcher++logger = logging.getLogger("world-intel-mcp.sources.migration")++_UNHCR = "https://api.unhcr.org/population/v1/"+_WB = "https://api.worldbank.org/v2/"++_UNHCR_TTL = 43200 # 12 h; the data changes once a year+_META_TTL = 604800 # 7 d for country lists+_MAX_PAGES = 10++UNHCR_MEASURE = (+ "UNHCR year-end stock: people from the origin present in the destination at "+ "the end of the year (refugees + asylum seekers + others in need of "+ "international protection). Year-over-year change = net arrivals minus "+ "returns, onward movement, naturalisation and deaths; not a crossing count."+)++# UNHCR's country list carries two buckets that are not countries.+NON_COUNTRY_ISO = {"UNK": "Unknown origin", "XXA": "Stateless"}++# Surge levels (heuristic, reported with every result).+MASS_SHARE = 0.01 # >= 1% of the origin population in one year+MASS_ABS = 500_000+MAJOR_ABS = 100_000+++def _now() -> str:+ return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")+++def _int(value: Any) -> int:+ """UNHCR mixes ints, numeric strings and "-"; anything unparseable is 0."""+ if value is None:+ return 0+ try:+ return int(value)+ except (ValueError, TypeError):+ return 0+++def _degraded(error: str, **extra: Any) -> dict:+ return {"error": error, "degraded": True, "source": extra.pop("source", "unhcr"),+ "timestamp": _now(), **extra}+++# ---------------------------------------------------------------------------+# Raw fetches+# ---------------------------------------------------------------------------++async def _unhcr_items(fetcher: Fetcher, endpoint: str, params: dict, cache_key: str,+ ttl: int = _UNHCR_TTL) -> list[dict] | None:+ """All pages of a UNHCR list endpoint, or None if the first page failed."""+ items: list[dict] = []+ page = 1+ while page <= _MAX_PAGES:+ data = await fetcher.get_json(+ url=_UNHCR + endpoint, source="unhcr",+ cache_key=f"{cache_key}:p{page}", cache_ttl=ttl,+ params={**params, "page": page},+ )+ if not isinstance(data, dict):+ return None if page == 1 else items+ items.extend(data.get("items") or [])+ if page >= _int(data.get("maxPages")):+ return items+ page += 1+ logger.warning("UNHCR %s exceeded %d pages; result truncated", endpoint, _MAX_PAGES)+ return items+++async def unhcr_countries(fetcher: Fetcher) -> dict[str, dict] | None:+ """ISO-3 -> {name, region, major_area, names} from UNHCR's country list."""+ items = await _unhcr_items(fetcher, "countries/", {"limit": 1000},+ "migration:unhcr:countries", ttl=_META_TTL)+ if items is None:+ return None+ out: dict[str, dict] = {}+ for c in items:+ iso = (c.get("iso") or "").upper()+ if len(iso) != 3:+ continue+ names = {c.get(k) for k in ("name", "nameShort", "nameLong", "nameOrigin", "nameFormal")}+ names.discard(None)+ tier1 = TIER1_COUNTRIES.get(iso)+ if tier1:+ names.add(tier1["name"])+ names.update(tier1.get("keywords", []))+ out[iso] = {+ "name": NON_COUNTRY_ISO.get(iso) or (tier1["name"] if tier1 else (c.get("name") or iso).strip()),+ "is_country": iso not in NON_COUNTRY_ISO,+ "iso2": c.get("iso2"),+ "region": c.get("region"),+ "major_area": c.get("majorArea"),+ "names": sorted(n for n in names if n),+ }+ return out+++async def unhcr_pairs(fetcher: Fetcher, year: int) -> list[dict] | None:+ """Every origin/asylum pair row for one year (about 6,300 rows for 2025)."""+ return await _unhcr_items(+ fetcher, "population/",+ {"yearFrom": year, "yearTo": year, "coo_all": "true", "coa_all": "true",+ "cf_type": "ISO", "limit": 10000},+ f"migration:unhcr:pairs:{year}",+ )+++async def _latest_pairs(fetcher: Fetcher, year: int | None) -> tuple[int, list[dict]] | None:+ """Rows for ``year``, or for the newest year that has data when None."""+ if year is not None:+ rows = await unhcr_pairs(fetcher, int(year))+ return None if rows is None else (int(year), rows)+ this_year = datetime.now(timezone.utc).year+ for candidate in (this_year - 1, this_year - 2):+ rows = await unhcr_pairs(fetcher, candidate)+ if rows is None:+ return None+ if rows:+ return candidate, rows+ return this_year - 2, []+++async def _wb_indicator(fetcher: Fetcher, indicator: str, country: str = "all",+ date: str | None = None) -> list[dict] | None:+ """World Bank indicator rows (all pages). Latest non-empty value when no date."""+ params: dict[str, Any] = {"format": "json", "per_page": 1000}+ if date:+ params["date"] = date+ else:+ params["mrnev"] = 1+ rows: list[dict] = []+ page = 1+ while page <= _MAX_PAGES:+ data = await fetcher.get_json(+ url=f"{_WB}country/{country}/indicator/{indicator}", source="worldbank",+ cache_key=f"migration:wb:{indicator}:{country}:{date or 'latest'}:p{page}",+ cache_ttl=_UNHCR_TTL, params={**params, "page": page},+ )+ if not isinstance(data, list) or len(data) < 2 or not isinstance(data[0], dict):+ return None if page == 1 else rows+ rows.extend(data[1] or [])+ if page >= _int(data[0].get("pages")):+ return rows+ page += 1+ return rows+++async def wb_countries(fetcher: Fetcher) -> dict[str, dict] | None:+ """ISO-3 -> {name, region, income, aggregate} from the World Bank country list."""+ data = await fetcher.get_json(+ url=_WB + "country", source="worldbank", cache_key="migration:wb:countries",+ cache_ttl=_META_TTL, params={"format": "json", "per_page": 1000},+ )+ if not isinstance(data, list) or len(data) < 2:+ return None+ out = {}+ for c in data[1] or []:+ region = (c.get("region") or {})+ out[c["id"]] = {+ "name": c.get("name", "").strip(),+ "iso2": c.get("iso2Code"),+ "region": (region.get("value") or "").strip(),+ "region_id": region.get("id"),+ "income": ((c.get("incomeLevel") or {}).get("value") or "").strip(),+ "aggregate": region.get("id") == "NA",+ }+ return out+++# ---------------------------------------------------------------------------+# Place resolution+# ---------------------------------------------------------------------------++def resolve_place(query: str, countries: dict[str, dict]) -> tuple[str, str, set[str]]:+ """Resolve a country (ISO-3, ISO-2 or name) or a region to ISO-3 codes.++ Returns (kind, label, codes) with kind "country" or "region". Regions are+ UNHCR's major areas ("Europe") and UN sub-regions ("Eastern Africa").+ Raises ValueError when nothing, or more than one country, matches.+ """+ q = query.strip()+ if not q:+ raise ValueError("empty place")+ up, low = q.upper(), q.lower()+ if len(up) == 3 and up in countries:+ return "country", countries[up]["name"], {up}+ if len(up) == 2:+ hits = [iso for iso, c in countries.items() if (c.get("iso2") or "").upper() == up]+ if len(hits) == 1:+ return "country", countries[hits[0]]["name"], set(hits)+ for field in ("major_area", "region"):+ members = {iso for iso, c in countries.items() if (c.get(field) or "").lower() == low}+ if members:+ return "region", next(c[field] for c in countries.values() if (c.get(field) or "").lower() == low), members+ exact = [iso for iso, c in countries.items()+ if low == c["name"].lower() or any(low == n.lower() for n in c.get("names", []))]+ if len(exact) == 1:+ return "country", countries[exact[0]]["name"], set(exact)+ partial = exact or [iso for iso, c in countries.items()+ if len(low) >= 4 and any(low in n.lower() for n in [c["name"], *c.get("names", [])])]+ if len(partial) == 1:+ return "country", countries[partial[0]]["name"], set(partial)+ if partial:+ names = sorted(f"{countries[i]['name']} ({i})" for i in partial)[:8]+ raise ValueError(f"'{query}' is ambiguous: {', '.join(names)}")+ raise ValueError(f"unknown country or region: '{query}'")+++def _name(iso: str, row_name: str | None, countries: dict[str, dict]) -> str:+ c = countries.get(iso)+ return c["name"] if c else (row_name or iso or "Unknown").strip()+++# ---------------------------------------------------------------------------+# Corridors+# ---------------------------------------------------------------------------++def _corridor_table(rows: list[dict]) -> dict[tuple[str, str], dict]:+ """(origin_iso3, destination_iso3) -> summed counts; same-country rows dropped."""+ table: dict[tuple[str, str], dict] = {}+ for r in rows:+ o = (r.get("coo_iso") or "").upper()+ d = (r.get("coa_iso") or "").upper()+ if not o or not d or o == "-" or d == "-" or o == d:+ continue+ e = table.setdefault((o, d), {"refugees": 0, "asylum_seekers": 0, "oip": 0,+ "origin_name": r.get("coo_name"), "destination_name": r.get("coa_name")})+ e["refugees"] += _int(r.get("refugees"))+ e["asylum_seekers"] += _int(r.get("asylum_seekers"))+ e["oip"] += _int(r.get("oip"))+ for e in table.values():+ e["people"] = e["refugees"] + e["asylum_seekers"] + e["oip"]+ return table+++def _idps_by_origin(rows: list[dict]) -> dict[str, int]:+ out: dict[str, int] = {}+ for r in rows:+ o = (r.get("coo_iso") or "").upper()+ if o and o == (r.get("coa_iso") or "").upper():+ out[o] = out.get(o, 0) + _int(r.get("idps"))+ return out+++async def fetch_migration_flows(+ fetcher: Fetcher,+ year: int | None = None,+ origin: str | None = None,+ destination: str | None = None,+ limit: int = 25,+ min_people: int = 0,+) -> dict:+ """Largest origin->destination corridors, optionally filtered by country or region."""+ countries = await unhcr_countries(fetcher)+ if countries is None:+ return _degraded("UNHCR country list unavailable", reason="unhcr_fetch_failed")+ try:+ o_sel = resolve_place(origin, countries) if origin else None+ d_sel = resolve_place(destination, countries) if destination else None+ except ValueError as exc:+ return {"error": str(exc), "source": "unhcr", "timestamp": _now()}++ got = await _latest_pairs(fetcher, year)+ if got is None:+ return _degraded("UNHCR population API unavailable", reason="unhcr_fetch_failed")+ year, rows = got+ prior_rows = await unhcr_pairs(fetcher, year - 1)+ cur = _corridor_table(rows)+ prev = _corridor_table(prior_rows) if prior_rows else None++ selected = []+ for (o, d), e in cur.items():+ if o_sel and o not in o_sel[2]:+ continue+ if d_sel and d not in d_sel[2]:+ continue+ if e["people"] <= 0 or e["people"] < min_people:+ continue+ p = prev.get((o, d), {}).get("people", 0) if prev is not None else None+ selected.append({+ "origin": _name(o, e["origin_name"], countries), "origin_iso3": o,+ "destination": _name(d, e["destination_name"], countries), "destination_iso3": d,+ "people": e["people"], "refugees": e["refugees"],+ "asylum_seekers": e["asylum_seekers"], "other_in_need_of_protection": e["oip"],+ "prior_year_people": p, "change": None if p is None else e["people"] - p,+ })+ selected.sort(key=lambda x: x["people"], reverse=True)++ def top(key: str, iso_key: str) -> list[dict]:+ agg: dict[str, dict] = {}+ for c in selected:+ a = agg.setdefault(c[iso_key], {"name": c[key], "iso3": c[iso_key], "people": 0, "corridors": 0})+ a["people"] += c["people"]+ a["corridors"] += 1+ return sorted(agg.values(), key=lambda x: x["people"], reverse=True)[:10]++ return {+ "year": year,+ "origin_filter": None if not o_sel else {"kind": o_sel[0], "name": o_sel[1]},+ "destination_filter": None if not d_sel else {"kind": d_sel[0], "name": d_sel[1]},+ "total_people": sum(c["people"] for c in selected),+ "corridor_count": len(selected),+ "corridors": selected[: max(1, int(limit))],+ "top_origins": top("origin", "origin_iso3"),+ "top_destinations": top("destination", "destination_iso3"),+ "prior_year_available": prev is not None,+ "measure": UNHCR_MEASURE,+ "source": "unhcr",+ "timestamp": _now(),+ }+++def _surge_level(increase: int, population: int | None) -> str:+ if increase >= MASS_ABS or (population and increase >= MASS_SHARE * population):+ return "mass"+ if increase >= MAJOR_ABS:+ return "major"+ return "notable"+++async def fetch_migration_surges(+ fetcher: Fetcher,+ year: int | None = None,+ min_change: int = 50_000,+ limit: int = 20,+) -> dict:+ """Largest year-over-year increases and decreases, by origin and by corridor."""+ countries = await unhcr_countries(fetcher)+ if countries is None:+ return _degraded("UNHCR country list unavailable", reason="unhcr_fetch_failed")+ got = await _latest_pairs(fetcher, year)+ if got is None:+ return _degraded("UNHCR population API unavailable", reason="unhcr_fetch_failed")+ year, rows = got+ prior_rows = await unhcr_pairs(fetcher, year - 1)+ if not rows or not prior_rows:+ return _degraded(f"UNHCR has no data for {year} and {year - 1} to compare",+ reason="no_comparison_year", year=year)++ cur, prev = _corridor_table(rows), _corridor_table(prior_rows)+ idp_cur, idp_prev = _idps_by_origin(rows), _idps_by_origin(prior_rows)++ pop_rows = await _wb_indicator(fetcher, "SP.POP.TOTL")+ population = {r["countryiso3code"]: _int(r.get("value")) for r in pop_rows or []+ if r.get("countryiso3code") and r.get("value") is not None}++ corridors = []+ for key in set(cur) | set(prev):+ now_p = cur.get(key, {}).get("people", 0)+ was_p = prev.get(key, {}).get("people", 0)+ delta = now_p - was_p+ if abs(delta) < min_change:+ continue+ e = cur.get(key) or prev[key]+ corridors.append({+ "origin": _name(key[0], e["origin_name"], countries), "origin_iso3": key[0],+ "destination": _name(key[1], e["destination_name"], countries), "destination_iso3": key[1],+ "people": now_p, "prior_year_people": was_p, "change": delta,+ "pct_change": round(100.0 * delta / was_p, 1) if was_p else None,+ })++ origins: dict[str, dict] = {}+ for table, sign in ((cur, 1), (prev, -1)):+ for (o, _d), e in table.items():+ a = origins.setdefault(o, {"origin_name": e["origin_name"], "now": 0, "was": 0})+ a["now" if sign == 1 else "was"] += e["people"]+ origin_changes, unattributed = [], []+ for o, a in origins.items():+ delta = a["now"] - a["was"]+ idp_delta = idp_cur.get(o, 0) - idp_prev.get(o, 0)+ if abs(delta) < min_change and abs(idp_delta) < min_change:+ continue+ pop = population.get(o)+ target = origin_changes if countries.get(o, {}).get("is_country") else unattributed+ target.append({+ "origin": _name(o, a["origin_name"], countries), "origin_iso3": o,+ "people_abroad": a["now"], "prior_year_people_abroad": a["was"], "change_abroad": delta,+ "idps": idp_cur.get(o, 0), "change_idps": idp_delta,+ "population": pop,+ "change_abroad_pct_of_population": round(100.0 * delta / pop, 2) if pop else None,+ "level": _surge_level(delta, pop) if delta >= min_change else None,+ })++ attributed = [c for c in corridors if countries.get(c["origin_iso3"], {}).get("is_country")]++ def ranked(items: list[dict], key: str, rising: bool) -> list[dict]:+ picked = [i for i in items if (i[key] > 0) == rising and i[key] != 0]+ return sorted(picked, key=lambda i: i[key], reverse=rising)[: max(1, int(limit))]++ return {+ "year": year,+ "compared_with": year - 1,+ "min_change": min_change,+ "origin_outflow_increases": ranked(origin_changes, "change_abroad", True),+ "origin_outflow_decreases": ranked(origin_changes, "change_abroad", False),+ "internal_displacement_increases": sorted(+ [o for o in origin_changes if o["change_idps"] >= min_change],+ key=lambda o: o["change_idps"], reverse=True)[: max(1, int(limit))],+ "corridor_increases": ranked(attributed, "change", True),+ "corridor_decreases": ranked(attributed, "change", False),+ # UNHCR's "Unknown" origin and "Stateless" are not countries; a jump in+ # them is usually a host's reporting change, so they stay out of the rankings.+ "unattributed_origin_changes": sorted(unattributed, key=lambda o: abs(o["change_abroad"]), reverse=True),+ "levels": {+ "mass": f"increase >= {MASS_ABS:,} people or >= {MASS_SHARE:.0%} of the origin population",+ "major": f"increase >= {MAJOR_ABS:,} people",+ "notable": f"increase >= min_change ({min_change:,})",+ "basis": "heuristic thresholds of this tool, not a UNHCR classification",+ },+ "population_source": "worldbank SP.POP.TOTL" if population else None,+ "measure": UNHCR_MEASURE,+ "source": "unhcr",+ "timestamp": _now(),+ }+++async def fetch_migration_history(+ fetcher: Fetcher,+ country: str,+ role: str = "origin",+ year_from: int = 1951,+ year_to: int | None = None,+) -> dict:+ """Yearly series for one country as an origin or a host, with its largest waves."""+ if role not in ("origin", "host"):+ return {"error": "role must be 'origin' or 'host'", "source": "unhcr", "timestamp": _now()}+ countries = await unhcr_countries(fetcher)+ if countries is None:+ return _degraded("UNHCR country list unavailable", reason="unhcr_fetch_failed")+ try:+ kind, label, codes = resolve_place(country, countries)+ except ValueError as exc:+ return {"error": str(exc), "source": "unhcr", "timestamp": _now()}+ if kind != "country":+ return {"error": f"'{country}' is a region; history needs one country", "source": "unhcr",+ "timestamp": _now()}+ iso = next(iter(codes))+ year_to = int(year_to or datetime.now(timezone.utc).year)+ field = "coo" if role == "origin" else "coa"+ items = await _unhcr_items(+ fetcher, "population/",+ {field: iso, "cf_type": "ISO", "yearFrom": int(year_from), "yearTo": year_to, "limit": 1000},+ f"migration:unhcr:history:{role}:{iso}:{year_from}:{year_to}",+ )+ if items is None:+ return _degraded("UNHCR population API unavailable", reason="unhcr_fetch_failed")++ by_year: dict[int, dict] = {}+ for r in items:+ y = _int(r.get("year"))+ if not y:+ continue+ e = by_year.setdefault(y, {"year": y, "refugees": 0, "asylum_seekers": 0,+ "other_in_need_of_protection": 0, "idps": 0, "returned_refugees": 0})+ e["refugees"] += _int(r.get("refugees"))+ e["asylum_seekers"] += _int(r.get("asylum_seekers"))+ e["other_in_need_of_protection"] += _int(r.get("oip"))+ e["idps"] += _int(r.get("idps"))+ e["returned_refugees"] += _int(r.get("returned_refugees"))+ series = [by_year[y] for y in sorted(by_year)]+ for e in series:+ e["people"] = e["refugees"] + e["asylum_seekers"] + e["other_in_need_of_protection"]++ waves = []+ for prev, cur in zip(series, series[1:]):+ if cur["year"] != prev["year"] + 1:+ continue+ delta = cur["people"] - prev["people"]+ if delta > 0:+ waves.append({"year": cur["year"], "increase": delta, "people": cur["people"],+ "pct_change": round(100.0 * delta / prev["people"], 1) if prev["people"] else None})+ waves.sort(key=lambda w: w["increase"], reverse=True)+ peak = max(series, key=lambda e: e["people"]) if series else None++ return {+ "country": label, "iso3": iso, "role": role,+ "meaning": ("people FROM this country living abroad (and IDPs inside it)" if role == "origin"+ else "people from other countries hosted IN this country"),+ "years": len(series),+ "first_year": series[0]["year"] if series else None,+ "latest": series[-1] if series else None,+ "peak": None if not peak else {"year": peak["year"], "people": peak["people"]},+ "largest_waves": waves[:5],+ "series": series,+ "measure": UNHCR_MEASURE,+ "source": "unhcr",+ "timestamp": _now(),+ }+++# ---------------------------------------------------------------------------+# World Bank net migration (all migration, not only forced displacement)+# ---------------------------------------------------------------------------++WB_MEASURE = (+ "World Bank SM.POP.NETM: immigrants minus emigrants over the year, citizens and "+ "non-citizens, all reasons (work, family, study, forced). Positive = net "+ "inflow. Source is the UN World Population Prospects: modelled values, and the "+ "most recent years can be WPP projections rather than observed counts."+)+++def _wb_place(query: str, wb: dict[str, dict]) -> tuple[str, str, set[str]]:+ """Resolve a World Bank country (code or name) or region (code or name)."""+ countries = {iso: {"name": c["name"], "iso2": c["iso2"], "region": c["region"],+ "names": [c["name"], *(TIER1_COUNTRIES.get(iso, {}).get("keywords", []))]}+ for iso, c in wb.items() if not c["aggregate"]}+ for iso, c in TIER1_COUNTRIES.items():+ if iso in countries:+ countries[iso]["names"].append(c["name"])+ up = query.strip().upper()+ region_codes = {c["region_id"]: c["region"] for c in wb.values() if not c["aggregate"]}+ if up in region_codes:+ label = region_codes[up]+ return "region", label, {i for i, c in countries.items() if c["region"] == label}+ return resolve_place(query, countries)+++async def fetch_net_migration(+ fetcher: Fetcher,+ country: str | None = None,+ region: str | None = None,+ limit: int = 20,+) -> dict:+ """Net migration ranking (latest year) or one country's series since 1960."""+ wb = await wb_countries(fetcher)+ if wb is None:+ return _degraded("World Bank country list unavailable", reason="worldbank_fetch_failed",+ source="worldbank")++ if country:+ try:+ kind, label, codes = _wb_place(country, wb)+ except ValueError as exc:+ return {"error": str(exc), "source": "worldbank", "timestamp": _now()}+ if kind != "country":+ return {"error": f"'{country}' is a region; pass it as region=", "source": "worldbank",+ "timestamp": _now()}+ iso = next(iter(codes))+ span = f"1960:{datetime.now(timezone.utc).year}"+ net = await _wb_indicator(fetcher, "SM.POP.NETM", iso, span)+ if net is None:+ return _degraded("World Bank API unavailable", reason="worldbank_fetch_failed", source="worldbank")+ stock = await _wb_indicator(fetcher, "SM.POP.TOTL", iso, span) or []+ pop = await _wb_indicator(fetcher, "SP.POP.TOTL", iso, span) or []+ pop_by = {_int(r["date"]): _int(r["value"]) for r in pop if r.get("value") is not None}+ series: list[dict[str, Any]] = []+ for r in net:+ if r.get("value") is None:+ continue+ y, v = _int(r["date"]), _int(r["value"])+ series.append({"year": y, "net_migration": v,+ "per_1000": round(1000.0 * v / pop_by[y], 2) if pop_by.get(y) else None})+ series.sort(key=lambda e: e["year"])+ stock_series = sorted(({"year": _int(r["date"]), "migrant_stock": _int(r["value"])}+ for r in stock if r.get("value") is not None), key=lambda e: e["year"])+ return {+ "country": label, "iso3": iso, "region": wb[iso]["region"],+ "latest": series[-1] if series else None,+ "largest_net_inflow_years": sorted((e for e in series if e["net_migration"] > 0),+ key=lambda e: e["net_migration"], reverse=True)[:5],+ "largest_net_outflow_years": sorted((e for e in series if e["net_migration"] < 0),+ key=lambda e: e["net_migration"])[:5],+ "migrant_stock": stock_series,+ "series": series,+ "measure": WB_MEASURE,+ "source": "worldbank",+ "timestamp": _now(),+ }++ selected: set[str] | None = None+ region_label = None+ if region:+ try:+ kind, region_label, selected = _wb_place(region, wb)+ except ValueError as exc:+ return {"error": str(exc), "source": "worldbank", "timestamp": _now()}+ if kind != "region":+ return {"error": f"'{region}' is a country; pass it as country=", "source": "worldbank",+ "timestamp": _now()}++ net = await _wb_indicator(fetcher, "SM.POP.NETM")+ if net is None:+ return _degraded("World Bank API unavailable", reason="worldbank_fetch_failed", source="worldbank")+ pop = {r["countryiso3code"]: _int(r["value"]) for r in await _wb_indicator(fetcher, "SP.POP.TOTL") or []+ if r.get("value") is not None}+ stock = {r["countryiso3code"]: _int(r["value"]) for r in await _wb_indicator(fetcher, "SM.POP.TOTL") or []+ if r.get("value") is not None}++ rows, aggregates = [], []+ for r in net:+ iso = r.get("countryiso3code") or ""+ meta = wb.get(iso)+ if meta is None or r.get("value") is None:+ continue+ value = _int(r["value"])+ entry = {"name": meta["name"], "iso3": iso, "year": _int(r.get("date")), "net_migration": value,+ "per_1000": round(1000.0 * value / pop[iso], 2) if pop.get(iso) else None,+ "migrant_stock": stock.get(iso),+ "migrant_stock_pct": round(100.0 * stock[iso] / pop[iso], 1)+ if stock.get(iso) and pop.get(iso) else None}+ if meta["aggregate"]:+ aggregates.append(entry)+ elif selected is None or iso in selected:+ entry["region"] = meta["region"]+ rows.append(entry)++ n = max(1, int(limit))+ geo = {"WLD", "EAS", "ECS", "LCN", "MEA", "NAC", "SAS", "SSF"}+ return {+ "region_filter": region_label,+ "countries": len(rows),+ "net_total": sum(r["net_migration"] for r in rows),+ "top_net_immigration": sorted((r for r in rows if r["net_migration"] > 0),+ key=lambda r: r["net_migration"], reverse=True)[:n],+ "top_net_emigration": sorted((r for r in rows if r["net_migration"] < 0),+ key=lambda r: r["net_migration"])[:n],+ "highest_rate_immigration": sorted((r for r in rows if r["per_1000"] is not None and r["per_1000"] > 0),+ key=lambda r: r["per_1000"], reverse=True)[:n],+ "highest_rate_emigration": sorted((r for r in rows if r["per_1000"] is not None and r["per_1000"] < 0),+ key=lambda r: r["per_1000"])[:n],+ "world_regions": sorted((a for a in aggregates if a["iso3"] in geo),+ key=lambda a: a["net_migration"], reverse=True) if region is None else [],+ "measure": WB_MEASURE,+ "source": "worldbank",+ "timestamp": _now(),+ }diff --git a/src/world_intel_mcp/tests/test_migration.py b/src/world_intel_mcp/tests/test_migration.pynew file mode 100644index 0000000..e31e8ed--- /dev/null+++ b/src/world_intel_mcp/tests/test_migration.py@@ -0,0 +1,348 @@+"""Tests for sources/migration.py (UNHCR corridors, surges, history; World Bank net migration).++respx serves fixtures shaped like the live APIs as probed on 2026-10-11: UNHCR rows+carry both its own codes (coo/coa, where "AUS" is AUSTRIA) and ISO-3 (coo_iso/coa_iso);+World Bank responses are [meta, rows] with aggregates flagged by region id "NA".++Gaps / not covered: live API calls (run through the MCP stdio path by hand, see the+CHANGELOG entry); UNHCR responses spanning more than one page; WB pagination.+"""++from __future__ import annotations++import asyncio as asyncio_mod++import httpx+import pytest+import respx++from world_intel_mcp.fetcher import Fetcher+from world_intel_mcp.sources import migration+from world_intel_mcp.sources.displacement import fetch_displacement_summary++_COUNTRIES = {"maxPages": 1, "items": [+ {"iso": "SYR", "iso2": "SY", "name": "Syrian Arab Rep.", "nameShort": "Syrian Arab Rep.",+ "region": "Western Asia", "majorArea": "Asia"},+ {"iso": "TUR", "iso2": "TR", "name": "Türkiye", "region": "Western Asia", "majorArea": "Asia"},+ {"iso": "DEU", "iso2": "DE", "name": "Germany", "region": "Western Europe", "majorArea": "Europe"},+ {"iso": "AUT", "iso2": "AT", "name": "Austria", "region": "Western Europe", "majorArea": "Europe"},+ {"iso": "SDN", "iso2": "SD", "name": "Sudan", "region": "Northern Africa", "majorArea": "Africa"},+ {"iso": "TCD", "iso2": "TD", "name": "Chad", "region": "Middle Africa", "majorArea": "Africa"},+ {"iso": "COG", "iso2": "CG", "name": "Congo", "region": "Middle Africa", "majorArea": "Africa"},+ {"iso": "COD", "iso2": "CD", "name": "Dem. Rep. of the Congo", "region": "Middle Africa",+ "majorArea": "Africa"},+ # Live UNHCR lists these two non-country buckets as if they were countries.+ {"code": "UKN", "iso": "UNK", "name": "Unknown ", "region": None, "majorArea": None},+ {"code": "STA", "iso": "XXA", "name": "Stateless", "region": "Stateless", "majorArea": "Stateless"},+]}+++def _row(o, d, refugees=0, asylum=0, oip=0, idps=0, coo=None):+ return {"coo": coo or o, "coo_iso": o, "coo_name": o, "coa_iso": d, "coa_name": d,+ "refugees": refugees, "asylum_seekers": asylum, "oip": oip if oip else "-", "idps": idps}+++_PAIRS = {+ 2025: [+ _row("SYR", "TUR", refugees=2_000_000),+ _row("SYR", "DEU", refugees=600_000, asylum=60_000),+ _row("SYR", "SYR", idps=5_000_000),+ _row("SDN", "TCD", refugees=1_200_000),+ _row("SDN", "SDN", idps=9_000_000),+ # UNHCR's own code AUS means Austria; only coa_iso/coo_iso are trustworthy.+ _row("AUT", "DEU", asylum=10, coo="AUS"),+ _row("UNK", "DEU", refugees=900_000),+ ],+ 2024: [+ _row("SYR", "TUR", refugees=2_900_000),+ _row("SYR", "DEU", refugees=650_000, asylum=60_000),+ _row("SYR", "SYR", idps=7_000_000),+ _row("SDN", "TCD", refugees=400_000),+ _row("SDN", "SDN", idps=11_000_000),+ _row("UNK", "DEU", refugees=100_000),+ ],+}++_HISTORY_SYR = [ # deliberately unordered, as the live API returns it+ {"year": 2013, "refugees": 2_400_000, "asylum_seekers": 0, "idps": 0},+ {"year": 2011, "refugees": 20_000, "asylum_seekers": 0, "idps": 0},+ {"year": 2012, "refugees": 700_000, "asylum_seekers": 29_000, "idps": 0},+ {"year": 2014, "refugees": 3_800_000, "asylum_seekers": 87_000, "idps": 0},+]+++def _unhcr(request: httpx.Request) -> httpx.Response:+ path, q = request.url.path, request.url.params+ if path.endswith("/countries/"):+ return httpx.Response(200, json=_COUNTRIES)+ assert q.get("cf_type") == "ISO", "every UNHCR population call must ask for ISO codes"+ if q.get("coo") == "SYR":+ return httpx.Response(200, json={"maxPages": 1, "items": _HISTORY_SYR})+ year = int(q["yearFrom"])+ return httpx.Response(200, json={"maxPages": 1, "items": _PAIRS.get(year, [])})+++_WB_COUNTRIES = [{"page": 1, "pages": 1}, [+ {"id": "SYR", "iso2Code": "SY", "name": "Syrian Arab Republic", "region": {"id": "MEA", "value": "Middle East"},+ "incomeLevel": {"value": "Low income"}},+ {"id": "DEU", "iso2Code": "DE", "name": "Germany", "region": {"id": "ECS", "value": "Europe & Central Asia"},+ "incomeLevel": {"value": "High income"}},+ {"id": "SDN", "iso2Code": "SD", "name": "Sudan", "region": {"id": "SSF", "value": "Sub-Saharan Africa"},+ "incomeLevel": {"value": "Low income"}},+ {"id": "TCD", "iso2Code": "TD", "name": "Chad", "region": {"id": "SSF", "value": "Sub-Saharan Africa"},+ "incomeLevel": {"value": "Low income"}},+ {"id": "WLD", "iso2Code": "1W", "name": "World", "region": {"id": "NA", "value": "Aggregates"}},+ {"id": "SSF", "iso2Code": "ZG", "name": "Sub-Saharan Africa", "region": {"id": "NA", "value": "Aggregates"}},+]]++_WB_LATEST = {+ "SM.POP.NETM": {"SYR": 400_000, "DEU": -300_000, "SDN": 290_000, "TCD": -120_000, "WLD": 0, "SSF": -200_000},+ "SP.POP.TOTL": {"SYR": 25_000_000, "DEU": 83_000_000, "SDN": 50_000_000, "TCD": 20_000_000,+ "WLD": 8_000_000_000, "SSF": 1_200_000_000},+ "SM.POP.TOTL": {"SYR": 900_000, "DEU": 16_000_000, "SDN": 2_400_000, "TCD": 1_200_000},+}+++def _wb(request: httpx.Request) -> httpx.Response:+ path, q = request.url.path, request.url.params+ if path.rstrip("/").endswith("/country"):+ return httpx.Response(200, json=_WB_COUNTRIES)+ _, _, country, _, indicator = path.rstrip("/").split("/")[-5:]+ if country == "all":+ assert q.get("mrnev") == "1"+ rows = [{"countryiso3code": iso, "date": "2025", "value": v}+ for iso, v in _WB_LATEST[indicator].items()]+ rows.append({"countryiso3code": "XKX", "date": "2025", "value": None})+ return httpx.Response(200, json=[{"page": 1, "pages": 1}, rows])+ series = {"SM.POP.NETM": {"2014": 500_000, "2015": 1_175_000, "2016": 600_000, "1998": -750_000},+ "SP.POP.TOTL": {"2014": 81_000_000, "2015": 82_000_000, "2016": 82_500_000, "1998": 82_000_000},+ "SM.POP.TOTL": {"2015": 12_000_000, "2020": 15_000_000}}[indicator]+ rows = [{"countryiso3code": country, "date": y, "value": v} for y, v in series.items()]+ rows.append({"countryiso3code": country, "date": "2026", "value": None})+ return httpx.Response(200, json=[{"page": 1, "pages": 1}, rows])+++def apis():+ with respx.mock(assert_all_called=False) as mock:+ mock.get(url__regex=r".*api\.unhcr\.org.*").mock(side_effect=_unhcr)+ mock.get(url__regex=r".*api\.worldbank\.org.*").mock(side_effect=_wb)+ yield mock+++# ---------------------------------------------------------------------------+# Place resolution+# ---------------------------------------------------------------------------++def _countries():+ return {+ "SYR": {"name": "Syria", "iso2": "SY", "region": "Western Asia", "major_area": "Asia",+ "names": ["Syrian Arab Rep.", "syria", "syrian"]},+ "COG": {"name": "Congo", "iso2": "CG", "region": "Middle Africa", "major_area": "Africa",+ "names": ["Congo"]},+ "COD": {"name": "Dem. Rep. of the Congo", "iso2": "CD", "region": "Middle Africa",+ "major_area": "Africa", "names": ["Dem. Rep. of the Congo"]},+ "DEU": {"name": "Germany", "iso2": "DE", "region": "Western Europe", "major_area": "Europe",+ "names": ["Germany"]},+ }+++def test_resolve_place_by_code_name_and_region() -> None:+ c = _countries()+ assert migration.resolve_place("syr", c) == ("country", "Syria", {"SYR"})+ assert migration.resolve_place("DE", c) == ("country", "Germany", {"DEU"})+ assert migration.resolve_place("Syrian Arab Rep.", c)[2] == {"SYR"}+ assert migration.resolve_place("syrian", c)[2] == {"SYR"}+ assert migration.resolve_place("Africa", c) == ("region", "Africa", {"COG", "COD"})+ assert migration.resolve_place("western europe", c) == ("region", "Western Europe", {"DEU"})+ # An exact name wins over the substring that would make it ambiguous.+ assert migration.resolve_place("Congo", c)[2] == {"COG"}+++def test_resolve_place_rejects_ambiguous_and_unknown() -> None:+ c = _countries()+ with pytest.raises(ValueError, match="ambiguous"):+ migration.resolve_place("Cong", c)+ with pytest.raises(ValueError, match="unknown"):+ migration.resolve_place("Atlantis", c)+++# ---------------------------------------------------------------------------+# Corridors+# ---------------------------------------------------------------------------++def test_corridor_table_uses_iso_fields_and_drops_internal_rows() -> None:+ table = migration._corridor_table(_PAIRS[2025])+ assert ("SYR", "SYR") not in table+ assert table[("AUT", "DEU")]["people"] == 10 # coo "AUS" ignored, coo_iso AUT used+ assert table[("SYR", "DEU")]["people"] == 660_000+ assert migration._idps_by_origin(_PAIRS[2025]) == {"SYR": 5_000_000, "SDN": 9_000_000}+++async def test_flows_filtered_by_origin_with_prior_year_change(fetcher: Fetcher, apis) -> None:+ result = await migration.fetch_migration_flows(fetcher, year=2025, origin="Syria")+ assert result["year"] == 2025+ assert result["origin_filter"] == {"kind": "country", "name": "Syria"}+ assert [c["destination_iso3"] for c in result["corridors"]] == ["TUR", "DEU"]+ tur = result["corridors"][0]+ assert tur["people"] == 2_000_000+ assert tur["prior_year_people"] == 2_900_000+ assert tur["change"] == -900_000+ assert result["total_people"] == 2_660_000+ assert result["top_destinations"][0]["iso3"] == "TUR"+ assert "year-end stock" in result["measure"].lower()+++async def test_flows_destination_region_and_min_people(fetcher: Fetcher, apis) -> None:+ result = await migration.fetch_migration_flows(fetcher, year=2025, destination="Europe", min_people=100)+ assert result["destination_filter"] == {"kind": "region", "name": "Europe"}+ assert {(c["origin_iso3"], c["destination_iso3"]) for c in result["corridors"]} == {+ ("UNK", "DEU"), ("SYR", "DEU")}+++async def test_flows_default_year_falls_back_when_latest_is_empty(fetcher: Fetcher, apis,+ monkeypatch: pytest.MonkeyPatch) -> None:+ class _Clock:+ @staticmethod+ def now(tz=None):+ from datetime import datetime, timezone+ return datetime(2027, 3, 1, tzinfo=timezone.utc)++ monkeypatch.setattr(migration, "datetime", _Clock)+ result = await migration.fetch_migration_flows(fetcher)+ assert result["year"] == 2025 # 2026 has no rows yet, so the newest year with data is used+++async def test_flows_unknown_place_is_an_error_not_an_empty_result(fetcher: Fetcher, apis) -> None:+ result = await migration.fetch_migration_flows(fetcher, origin="Atlantis")+ assert "unknown" in result["error"]+ assert "corridors" not in result+++async def test_flows_api_down_is_degraded(fetcher: Fetcher, monkeypatch: pytest.MonkeyPatch) -> None:+ async def _no_sleep(*args, **kwargs) -> None:+ return None++ monkeypatch.setattr(asyncio_mod, "sleep", _no_sleep)+ with respx.mock:+ respx.get(url__regex=r".*api\.unhcr\.org.*").mock(return_value=httpx.Response(500))+ result = await migration.fetch_migration_flows(fetcher, year=2025)+ assert result["degraded"] is True+ assert result["reason"] == "unhcr_fetch_failed"+++# ---------------------------------------------------------------------------+# Surges+# ---------------------------------------------------------------------------++async def test_surges_rank_origins_levels_and_population_share(fetcher: Fetcher, apis) -> None:+ result = await migration.fetch_migration_surges(fetcher, year=2025, min_change=50_000)+ assert result["compared_with"] == 2024++ up = result["origin_outflow_increases"]+ assert [o["origin_iso3"] for o in up] == ["SDN"]+ sdn = up[0]+ assert sdn["change_abroad"] == 800_000+ assert sdn["population"] == 50_000_000+ assert sdn["change_abroad_pct_of_population"] == 1.6+ assert sdn["level"] == "mass"+ assert sdn["change_idps"] == -2_000_000++ down = result["origin_outflow_decreases"]+ assert down[0]["origin_iso3"] == "SYR"+ assert down[0]["change_abroad"] == -950_000+ assert down[0]["level"] is None++ assert result["corridor_increases"][0]["origin_iso3"] == "SDN"+ assert result["corridor_decreases"][0]["destination_iso3"] == "TUR"+ # "Unknown" origin is not a country: reported separately, never ranked.+ assert [o["origin_iso3"] for o in result["unattributed_origin_changes"]] == ["UNK"]+ assert all(c["origin_iso3"] != "UNK" for c in result["corridor_increases"])+ assert "heuristic" in result["levels"]["basis"]+++def test_surge_levels() -> None:+ assert migration._surge_level(600_000, None) == "mass"+ assert migration._surge_level(60_000, 5_000_000) == "mass" # 1.2% of population+ assert migration._surge_level(150_000, 100_000_000) == "major"+ assert migration._surge_level(60_000, 100_000_000) == "notable"+++async def test_surges_without_a_comparison_year_is_degraded(fetcher: Fetcher, apis) -> None:+ result = await migration.fetch_migration_surges(fetcher, year=2024) # no 2023 rows+ assert result["degraded"] is True+ assert result["reason"] == "no_comparison_year"+++# ---------------------------------------------------------------------------+# History+# ---------------------------------------------------------------------------++async def test_history_sorts_years_and_finds_waves(fetcher: Fetcher, apis) -> None:+ result = await migration.fetch_migration_history(fetcher, "SYR", year_from=2011, year_to=2014)+ assert result["iso3"] == "SYR" and result["role"] == "origin"+ assert [e["year"] for e in result["series"]] == [2011, 2012, 2013, 2014]+ assert result["peak"] == {"year": 2014, "people": 3_887_000}+ assert result["largest_waves"][0] == {"year": 2013, "increase": 1_671_000, "people": 2_400_000,+ "pct_change": 229.2}+ request = apis.calls.last.request+ assert request.url.params["coo"] == "SYR" and request.url.params["cf_type"] == "ISO"+++async def test_history_rejects_regions_and_bad_role(fetcher: Fetcher, apis) -> None:+ assert "region" in (await migration.fetch_migration_history(fetcher, "Europe"))["error"]+ assert "role" in (await migration.fetch_migration_history(fetcher, "SYR", role="both"))["error"]+++# ---------------------------------------------------------------------------+# World Bank net migration+# ---------------------------------------------------------------------------++async def test_net_migration_ranking_excludes_aggregates(fetcher: Fetcher, apis) -> None:+ result = await migration.fetch_net_migration(fetcher, limit=5)+ assert result["countries"] == 4+ assert result["net_total"] == 270_000+ assert [r["iso3"] for r in result["top_net_immigration"]] == ["SYR", "SDN"]+ assert [r["iso3"] for r in result["top_net_emigration"]] == ["DEU", "TCD"]+ syr = result["top_net_immigration"][0]+ assert syr["per_1000"] == 16.0+ assert syr["migrant_stock_pct"] == 3.6+ assert result["highest_rate_emigration"][0]["iso3"] == "TCD" # -6.0 per 1,000 beats DEU -3.61+ assert [a["iso3"] for a in result["world_regions"]] == ["WLD", "SSF"]+ assert "projection" in result["measure"]+++async def test_net_migration_region_filter(fetcher: Fetcher, apis) -> None:+ result = await migration.fetch_net_migration(fetcher, region="SSF")+ assert result["region_filter"] == "Sub-Saharan Africa"+ assert {r["iso3"] for r in result["top_net_immigration"] + result["top_net_emigration"]} == {"SDN", "TCD"}+ assert result["world_regions"] == []+ assert "country" in (await migration.fetch_net_migration(fetcher, region="Germany"))["error"]+++async def test_net_migration_country_series(fetcher: Fetcher, apis) -> None:+ result = await migration.fetch_net_migration(fetcher, country="Germany")+ assert result["iso3"] == "DEU"+ assert [e["year"] for e in result["series"]] == [1998, 2014, 2015, 2016] # null 2026 dropped+ assert result["latest"]["year"] == 2016+ assert result["largest_net_inflow_years"][0]["year"] == 2015+ assert result["largest_net_inflow_years"][0]["per_1000"] == 14.33+ assert result["largest_net_outflow_years"][0]["year"] == 1998+ assert [s["year"] for s in result["migrant_stock"]] == [2015, 2020]+++# ---------------------------------------------------------------------------+# intel_displacement_summary regression (2026-10-11)+# ---------------------------------------------------------------------------++async def test_displacement_summary_requests_per_origin_rows(fetcher: Fetcher) -> None:+ """Without coo_all=true UNHCR answers with one global row named "-", and the+ tool reported that single row as its whole "by_origin" list."""+ items = [{"coo_iso": "SYR", "coo_name": "Syrian Arab Rep.", "refugees": 10},+ {"coo_iso": "SDN", "coo_name": "Sudan", "refugees": 20}]+ with respx.mock:+ route = respx.get(url__regex=r".*api\.unhcr\.org.*").mock(+ return_value=httpx.Response(200, json={"items": items}))+ result = await fetch_displacement_summary(fetcher, year=2025)+ assert route.calls.last.request.url.params["coo_all"] == "true"+ assert [(e["country"], e["iso3"]) for e in result["by_origin"]] == [("Sudan", "SDN"), ("Syria", "SYR")]diff --git a/src/world_intel_mcp/tests/test_server_registry.py b/src/world_intel_mcp/tests/test_server_registry.pyindex 22c2c70..31a7062 100644--- a/src/world_intel_mcp/tests/test_server_registry.py+++ b/src/world_intel_mcp/tests/test_server_registry.py@@ -123,11 +123,11 @@ def test_aggregate_refuses_drift_and_collisions() -> None:def test_tool_count_matches_documented_surface(server) -> None:- """135 = 134 intelligence tools + intel_status. If this fails after+ """139 = 138 intelligence tools + intel_status. If this fails afteradding a tool, update README.md, ROADMAP.md, and this number in thesame change: the repo's history shows doc counts drifting stalewithin weeks when nothing enforced them."""- assert len(server.TOOLS) == 135+ assert len(server.TOOLS) == 139@pytest.mark.asynciodiff --git a/src/world_intel_mcp/tools/society.py b/src/world_intel_mcp/tools/society.pyindex d019ee6..41568e0 100644--- a/src/world_intel_mcp/tools/society.py+++ b/src/world_intel_mcp/tools/society.py@@ -1,5 +1,5 @@"""News and public-signal tools: RSS news, trending keywords,-GDELT, prediction markets, displacement, cyber threats, AI watch,+GDELT, prediction markets, displacement and migration, cyber threats, AI watch,disease outbreaks, sanctions, elections, social signals, tech andscience feeds, government spending.@@ -17,6 +17,7 @@ from ..sources import (news,prediction,displacement,+ migration,cyber,ai_watch,health,@@ -45,7 +46,7 @@ TOOLS: list[Tool] = [},},),- # --- Displacement (1 tool) ---+ # --- Displacement and migration (5 tools) ---Tool(name="intel_displacement_summary",description="Get UNHCR refugee/displacement statistics by country of origin. No API key needed.",@@ -59,6 +60,77 @@ TOOLS: list[Tool] = [},},),+ Tool(+ name="intel_migration_flows",+ description=(+ "Cross-border displacement corridors from UNHCR: people from an origin country living in a "+ "destination country (refugees + asylum seekers + others in need of protection), year-end "+ "stock, with change vs the prior year. Filter origin/destination by country (ISO-3, ISO-2 or "+ "name) or region ('Europe', 'Eastern Africa'). No API key needed."+ ),+ inputSchema={+ "type": "object",+ "properties": {+ "origin": {"type": "string", "description": "Origin country or region"},+ "destination": {"type": "string", "description": "Destination country or region"},+ "year": {"type": "integer", "description": "Reporting year (default: latest with data)"},+ "limit": {"type": "integer", "description": "Corridors to return (default 25)"},+ "min_people": {"type": "integer", "description": "Drop corridors smaller than this (default 0)"},+ },+ },+ ),+ Tool(+ name="intel_migration_surges",+ description=(+ "Mass-movement detector: largest year-over-year increases and decreases in people displaced "+ "abroad (by origin and by corridor) and in internal displacement, from UNHCR, with each "+ "origin's change as a share of its population (World Bank). Levels mass/major/notable are "+ "stated heuristics. No API key needed."+ ),+ inputSchema={+ "type": "object",+ "properties": {+ "year": {"type": "integer", "description": "Reporting year (default: latest with data)"},+ "min_change": {"type": "integer", "description": "Minimum absolute change to report (default 50000)"},+ "limit": {"type": "integer", "description": "Entries per list (default 20)"},+ },+ },+ ),+ Tool(+ name="intel_migration_history",+ description=(+ "Yearly UNHCR series since 1951 for one country, as an origin (its people abroad, plus IDPs) "+ "or as a host (people it hosts), with the peak year and the largest annual waves. "+ "No API key needed."+ ),+ inputSchema={+ "type": "object",+ "properties": {+ "country": {"type": "string", "description": "Country (ISO-3, ISO-2 or name)"},+ "role": {"type": "string", "enum": ["origin", "host"], "description": "Default origin"},+ "year_from": {"type": "integer", "description": "First year (default 1951)"},+ "year_to": {"type": "integer", "description": "Last year (default current)"},+ },+ "required": ["country"],+ },+ ),+ Tool(+ name="intel_net_migration",+ description=(+ "World Bank net migration (immigrants minus emigrants, all reasons incl. economic) and "+ "international migrant stock. Without a country: latest-year ranking of net immigration and "+ "emigration (absolute and per 1,000) plus world regions; with a country: its series since "+ "1960. No API key needed."+ ),+ inputSchema={+ "type": "object",+ "properties": {+ "country": {"type": "string", "description": "Country (ISO-3, ISO-2 or name) for its series"},+ "region": {"type": "string", "description": "World Bank region code or name to filter the ranking (e.g. SSF, ECS)"},+ "limit": {"type": "integer", "description": "Entries per ranking (default 20)"},+ },+ },+ ),# --- Cyber (1 tool) ---Tool(name="intel_cyber_threats",@@ -328,6 +400,45 @@ async def _displacement_summary(arguments: dict[str, Any]) -> Any:)+async def _migration_flows(arguments: dict[str, Any]) -> Any:+ return await migration.fetch_migration_flows(+ runtime.fetcher,+ year=arguments.get("year"),+ origin=arguments.get("origin"),+ destination=arguments.get("destination"),+ limit=arguments.get("limit", 25),+ min_people=arguments.get("min_people", 0),+ )+++async def _migration_surges(arguments: dict[str, Any]) -> Any:+ return await migration.fetch_migration_surges(+ runtime.fetcher,+ year=arguments.get("year"),+ min_change=arguments.get("min_change", 50_000),+ limit=arguments.get("limit", 20),+ )+++async def _migration_history(arguments: dict[str, Any]) -> Any:+ return await migration.fetch_migration_history(+ runtime.fetcher,+ country=arguments["country"],+ role=arguments.get("role", "origin"),+ year_from=arguments.get("year_from", 1951),+ year_to=arguments.get("year_to"),+ )+++async def _net_migration(arguments: dict[str, Any]) -> Any:+ return await migration.fetch_net_migration(+ runtime.fetcher,+ country=arguments.get("country"),+ region=arguments.get("region"),+ limit=arguments.get("limit", 20),+ )++async def _cyber_threats(arguments: dict[str, Any]) -> Any:return await cyber.fetch_cyber_threats(runtime.fetcher, limit=arguments.get("limit", 50)@@ -428,6 +539,10 @@ async def _usa_spending(arguments: dict[str, Any]) -> Any:HANDLERS = {"intel_prediction_markets": _prediction_markets,"intel_displacement_summary": _displacement_summary,+ "intel_migration_flows": _migration_flows,+ "intel_migration_surges": _migration_surges,+ "intel_migration_history": _migration_history,+ "intel_net_migration": _net_migration,"intel_cyber_threats": _cyber_threats,"intel_news_feed": _news_feed,"intel_trending_keywords": _trending_keywords,