160-tool MCP server for real-time global intelligence from free public APIs: markets, liquidity, prediction markets, SEC filings, conflict, military, aircraft-cohort early warning, migration and border flows, cyber, climate, news and more. Dated observation history, 29 research prompts, cited briefs, geofences, SSE dashboard. MIT, no paid API keys.
Python ai-toolsai-watchanthropicclaudecybersecuritydashboardgeofencinggeopoliticalgeospatialintelligencemarketsmcpmigrationmilitarymodel-context-protocolnuclearosintreal-timespace-weatherthreat-intelligence
Prediction markets: re-review fixes (event gates, per-pair staleness, bounded last-trade lookups)
Phoenix Agent (mac-studio)ca9708047d08
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 2d699ca..2b1b784 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -76,6 +76,29 @@characters. `intel_prediction_markets` names how many markets it leftout for an unusable price.+### Fixed: prediction-market re-review findings+- Matching rejects a pair when both titles name different days of the+ same month, no shared month, or different clock times; when a quantity+ differs in unit ($, %, bps) or kind (to a level, by a change) or the+ same numbers come in a different order; on opposite wording (rise or+ fall, positive or negative, before or after); and when a listed name+ (chamber, party, country, a qualifier such as core or cash) or another+ capitalized name appears on one venue only. More negation words+ (won't, fail to), rate verbs (tighten, ease, boost, reduce and their+ forms), number words, dash minus signs, "between" ranges and bps read+ as hundredths of a percent. "Hold" counts as a rate action only in a+ rate question.+- `intel_prediction_compare`: a side priced from an expired copy carries+ `stale` and its age and its pair is flagged and not comparable. The+ Kalshi last-trade lookup reads the trade's price with its time, flags a+ pair whose trade price differs from the listing, skips sides priced from+ an expired listing, stops after 10 lookups or 10 s, and says per side+ what a null time means (`last_trade_lookup`). Matching runs in a worker+ thread. A pair with no price is flagged; Kalshi quotes outside 0 to 1+ are left null and counted.+- `intel_prediction_markets` sets `degraded` with `stale`; `intel+ predictions` prints its data gaps and titles an expired list as such.+## 0.11.0 - 2026-10-12Cross-border migration (MCP tools, CLI, collector and a dashboarddiff --git a/ROADMAP.md b/ROADMAP.mdindex 16c5c50..7c41126 100644--- a/ROADMAP.md+++ b/ROADMAP.md@@ -578,7 +578,7 @@ dashboard.| 1 | Hostility timeline | Per country pair or country, daily counts of threat / sanction / force / criticism coverage plus tone; sortable by most covered, most hostile, biggest change | :red_circle: || 1 | Story breadth | News clusters carry a distinct-outlet count; a top-stories view ranks by how many outlets carry a story | :red_circle: || 1 | Liquidity and flows | Fed balance sheet, Treasury General Account, reverse repo and net liquidity (FRED keyless CSV), money supply (US M2, euro area M1/M2/M3, never summed), ECB, ICI fund flows and ETF net issuance, FINRA margin statistics, BLS series; amounts kept separate from flows | :red_circle: |-| 1 | Kalshi + cross-venue markets | Kalshi public market data next to Polymarket; same-event comparison with an explicit match confidence; settled markets scored against pre-settlement prices (Brier) | :white_check_mark: Unreleased: `intel_kalshi_markets`, `intel_prediction_compare`, `intel_prediction_settled` (`sources/kalshi.py`, `sources/prediction_cross.py`, `analysis/prediction_matching.py`), CLI and 2 collector sources. Kalshi endpoints measured 2026-10-17, all HTTP 200 without an account: `/events` (nested markets), `/series` (`include_volume`), `/markets?status=settled`, `/markets/candlesticks` (batch, capped at 10,000 candles per call), `/historical/cutoff`. The `/events` `category` parameter is ignored by the API, so categories resolve through `/series?category=`, whose filter is loose too (7 of the 869 series it returned for Economics were other categories), so series are matched exactly; an unknown category answers `{"series": null}` and is reported as a gap, not an outage. The exchange-wide settled listing is intraday sports (1,000 rows covered about 40 minutes; none open 24h), so settled samples default to the Economics category's top series. Polymarket Gamma caps a page at 100 rows. Matching is deliberately conservative: live 2026-10-17 it matched the Fed 25 bps hike/cut pairs for October and December and rejected 25 vs 50+ bps, 2026 vs 2027, and the Fed-hold pair ("no change" vs "Hike 0bps": a known false negative). Team and person aliases are not resolved, so most sports pairs stay unmatched. Kalshi requests are spaced 0.2 s apart in-process: the two collector sources scanning the same series at once drew HTTP 429, and at 0.1 s a 26-request category scan still drew one (none at 0.2 s in one live run). Settled samples are the most recently settled markets and cluster in time (one football match supplied 11 of 25 Polymarket markets in a live run), so a Brier score describes its sample, not a venue's skill. Every number a caller passes has a range stated in the input schema (at most 25 Kalshi series or pages per scan), so one call cannot fan out into unbounded upstream requests. Review round (2026-10-17): expired-cache serves are marked on every Kalshi read and on Polymarket's settled listing and histories, and a stale history is not scored; matching adds rate-action and direction gates, keeps signs and ordinals, reads a $ amount as an amount, and its candidate blocking is complete against a brute-force scan in tests; compare carries each side's price basis, spread and Kalshi last-trade time (public `/markets/trades`, HTTP 200 without an account, measured 2026-10-17) and ranks flagged pairs after comparable ones; Polymarket is scored against the earlier of endDate and closedTime (on 20 closed markets measured 2026-10-17, closedTime came 2.7 to 5.5 h after endDate for football and 164 h before it for one early resolution, and umaEndDate equalled closedTime in all 20, so no outcome-known time is published); Kalshi pacing applies only on a cache miss. The shared `polymarket` circuit breaker (three failed history calls pause all Polymarket reads for 300 s) is documented, not changed |+| 1 | Kalshi + cross-venue markets | Kalshi public market data next to Polymarket; same-event comparison with an explicit match confidence; settled markets scored against pre-settlement prices (Brier) | :white_check_mark: Unreleased: `intel_kalshi_markets`, `intel_prediction_compare`, `intel_prediction_settled` (`sources/kalshi.py`, `sources/prediction_cross.py`, `analysis/prediction_matching.py`), CLI and 2 collector sources. Kalshi endpoints measured 2026-10-17, all HTTP 200 without an account: `/events` (nested markets), `/series` (`include_volume`), `/markets?status=settled`, `/markets/candlesticks` (batch, capped at 10,000 candles per call), `/historical/cutoff`. The `/events` `category` parameter is ignored by the API, so categories resolve through `/series?category=`, whose filter is loose too (7 of the 869 series it returned for Economics were other categories), so series are matched exactly; an unknown category answers `{"series": null}` and is reported as a gap, not an outage. The exchange-wide settled listing is intraday sports (1,000 rows covered about 40 minutes; none open 24h), so settled samples default to the Economics category's top series. Polymarket Gamma caps a page at 100 rows. Matching is deliberately conservative: live 2026-10-17 it matched the Fed 25 bps hike/cut pairs for October and December and rejected 25 vs 50+ bps, 2026 vs 2027, and the Fed-hold pair ("no change" vs "Hike 0bps": a known false negative). Team and person aliases are not resolved, so most sports pairs stay unmatched. Kalshi requests are spaced 0.2 s apart in-process: the two collector sources scanning the same series at once drew HTTP 429, and at 0.1 s a 26-request category scan still drew one (none at 0.2 s in one live run). Settled samples are the most recently settled markets and cluster in time (one football match supplied 11 of 25 Polymarket markets in a live run), so a Brier score describes its sample, not a venue's skill. Every number a caller passes has a range stated in the input schema (at most 25 Kalshi series or pages per scan), so one call cannot fan out into unbounded upstream requests. Review round (2026-10-17): expired-cache serves are marked on every Kalshi read and on Polymarket's settled listing and histories, and a stale history is not scored; matching adds rate-action and direction gates, keeps signs and ordinals, reads a $ amount as an amount, and its candidate blocking is complete against a brute-force scan in tests; compare carries each side's price basis, spread and Kalshi last-trade time (public `/markets/trades`, HTTP 200 without an account, measured 2026-10-17) and ranks flagged pairs after comparable ones; Polymarket is scored against the earlier of endDate and closedTime (on 20 closed markets measured 2026-10-17, closedTime came 2.7 to 5.5 h after endDate for football and 164 h before it for one early resolution, and umaEndDate equalled closedTime in all 20, so no outcome-known time is published); Kalshi pacing applies only on a cache miss. The shared `polymarket` circuit breaker (three failed history calls pause all Polymarket reads for 300 s) is documented, not changed. Re-review round (2026-10-17): matching also rejects different days of the same month, disjoint months, different clock times, quantity unit or kind mismatches, reordered quantities, opposite wording and a name on one venue only (adversarial set: 42 of 46 cases right, the 4 misses are same-event pairs rejected); a side priced from an expired copy is flagged and not comparable; Kalshi last-trade lookups read the trade price, skip expired listings, stop after 10 lookups or 10 s and name each null time; matching runs in a worker thread (2,127 Kalshi x 2,000 Polymarket in 3.4 s on mac-studio, scratch benchmark) || 1 | News vs market reaction | Companies and countries named in story clusters mapped to tickers / country indices; same-day move against a benchmark | :red_circle: || 2 | Connections with timing offsets | Correlation between retained series over a shared sample, tested at leads and lags, with minimum sample size and false-discovery control; states that it does not prove causation | :red_circle: || 2 | Day timeline | What every retained source recorded for a given day, by observation date or collection date | :red_circle: |diff --git a/src/world_intel_mcp/analysis/prediction_matching.py b/src/world_intel_mcp/analysis/prediction_matching.pyindex 6c02b8f..e91333c 100644--- a/src/world_intel_mcp/analysis/prediction_matching.py+++ b/src/world_intel_mcp/analysis/prediction_matching.py@@ -4,12 +4,16 @@ Pure functions, no I/O. Used by ``sources/prediction_cross.py`` to putthe same event on Kalshi and Polymarket side by side and to scoresettled markets against their pre-settlement prices.-Matching is deterministic: normalized title tokens, quantities, years,-negation, rate actions (cut / raise / hold), directions (above /-below), end dates and the outcome side must all agree before a pair is-scored, and a pair below ``DEFAULT_MIN_CONFIDENCE`` is not a match. A-wrong match is worse than no match, so every gate is a hard reject and-every reject carries its reason.+Matching is deterministic: normalized title tokens, quantities (value,+unit, "to" vs "by" and order), years, months, days of month, clock+times, negation, rate actions (cut / raise / hold), directions (above /+below), opposite words (rise / fall, positive / negative, before /+after, win / lose, approve / reject), distinguishing names, end dates+and the outcome side must all agree before a pair is scored, and a+pair below ``DEFAULT_MIN_CONFIDENCE`` is not a match. A wrong match is+worse than no match, so every gate is a hard reject and every reject+carries its reason. A gate decides from the two markets alone, never+from what else either venue lists.``match_confidence`` (``confidence`` in ``score_pair``) is an ordinalsimilarity score for ranking candidate pairs: title-word Dice overlap@@ -27,10 +31,11 @@ Normalized market shape (both venues) consumed here::"price_basis": str, "spread": float | None} # quote fields (optional)Gaps / not covered:-- No synonym coverage beyond the small table below (month names, rate- verbs, above/below). Pairs that word the same event differently- ("Fed cuts" vs "FOMC lowers the target range") fall below the- threshold and are reported unmatched, not matched.+- No synonym coverage beyond the small tables below (month names, rate+ verbs and their inflections, above/below, a few central-bank and+ country names). Pairs that word the same event differently ("Fed+ cuts" vs "FOMC lowers the target range") fall below the threshold+ and are reported unmatched, not matched.- Team, ticker and person aliases ("KC" vs "Kansas City Chiefs") arenot resolved, so most sports pairs stay unmatched.- Resolution-rule equivalence is not checked: two markets can share a@@ -38,10 +43,26 @@ Gaps / not covered:list says what was compared; it does not claim more.- Ordinals written as words ("first", "second") are plain words, notquantities, and a number glued to a unit ("100mm") loses the unit.+ Number words are read from "two" to "twelve"; "one" stays a word.- A fractional threshold worded strict on one venue and inclusive onthe other ("above 4.5%" vs "4.5% or higher") is matched with aboundary flag; an integer one is not, because "more than 50" means"51 or more".+- Distinguishing names are read from capitalization (plus a short+ list of chambers, parties, compass points, qualifiers and+ countries), not from a name database: an all-lowercase title names+ nothing, and a title-case one names every capitalized word, which+ only rejects more pairs. A name on one venue only (an insertion) is+ a Dice penalty, except for the listed qualifiers ("core CPI" vs+ "CPI", "Bitcoin Cash" vs "Bitcoin"), which reject.+- The opposite-word and month/day/clock gates reject only when both+ titles state one; a title that leaves it out ("CPI above 0.3%" vs+ "CPI rise more than 0.3%", "October 2026" vs "Oct 28, 2026") is+ left to the other gates and the end-date window. A day written+ before its month ("20 October") is not read as a day.+- A deadline worded as a date on one venue and as a calendar year on+ the other ("before Jan 1, 2027" vs "in 2026") is rejected on the+ year gate (a false negative, not a wrong match)."""from __future__ import annotations@@ -92,32 +113,131 @@ _SYNONYMS = {"less": "below", "wins": "win", "won": "win", "winner": "win",# "U.S." tokenizes to "u.s" (edge dots are stripped)."usa": "us", "u.s": "us", "u.s.a": "us", "bp": "bps", "basis": "bps",+ # Inflections of the rate verbs, and verbs that name the same move.+ "cutting": "cut", "lowered": "cut", "lowering": "cut", "decreased": "cut", "decreasing": "cut",+ "reduce": "cut", "reduces": "cut", "reduced": "cut", "reducing": "cut", "trim": "cut",+ "trims": "cut", "trimmed": "cut", "trimming": "cut", "ease": "cut",+ "eases": "cut", "eased": "cut", "easing": "cut", "raised": "raise", "raising": "raise",+ "hiked": "raise", "hiking": "raise", "increased": "raise", "increasing": "raise",+ "boost": "raise", "boosts": "raise", "boosted": "raise", "boosting": "raise",+ "tighten": "raise", "tightens": "raise", "tightened": "raise", "tightening": "raise",+ "held": "hold", "holding": "hold", "paused": "hold", "maintained": "hold",+ # Inflections of the opposite-word pairs (_POLES).+ "rises": "rise", "rising": "rise", "rose": "rise", "risen": "rise", "falls": "fall",+ "falling": "fall", "fell": "fall", "fallen": "fall", "drops": "drop", "dropped": "drop",+ "dropping": "drop", "loses": "lose", "losing": "lose", "lost": "lose", "loser": "lose",+ "approves": "approve", "approved": "approve", "approval": "approve", "rejects": "reject",+ "rejected": "reject", "rejection": "reject",}_MONTHS = frozenset("january february march april may june july august september october ""november december".split())-_NEGATIONS = frozenset({"not", "no", "never", "without"})+# "May" is a month only next to a date or after one of these words; "who+# may win" is a verb.+_MAY_BEFORE = frozenset("in by before after of until since end early mid late from through".split())+_NEGATIONS = frozenset({+ "not", "no", "never", "without", "wont", "cannot", "cant", "dont", "doesnt", "didnt",+ "isnt", "arent", "wasnt", "fail", "fails", "failed", "neither", "nor", "none", "nobody",+})_YES_NO = frozenset({"yes", "no"})# Hard gates: two titles must name the same rate action and the same# direction (after synonym folding), or the pair is rejected._ACTIONS = frozenset({"cut", "raise", "hold"})_DIRECTIONS = {"above": "above", "below": "below", "atleast": "above", "atmost": "below", "ormore": "above"}+# "hold" is the rate action only next to one of these (after synonym and+# name folding); "hold an election" is not a rate decision.+_RATE_CONTEXT = frozenset(+ "rate rates interest bps fed ecb boe boj boc rba snb pboc monetary funds".split()+)+# Opposite words: when both titles use a word on the same axis, the poles+# must agree. A title that uses none is left to the other gates ("after+# the October meeting" vs "at their October meeting" is one event).+_POLES = {+ "rise": ("move", "up"), "climb": ("move", "up"), "climbs": ("move", "up"), "reach": ("move", "up"),+ "reaches": ("move", "up"), "hit": ("move", "up"), "hits": ("move", "up"), "surpass": ("move", "up"),+ "surpasses": ("move", "up"), "fall": ("move", "down"), "drop": ("move", "down"), "dip": ("move", "down"),+ "dips": ("move", "down"), "decline": ("move", "down"), "declines": ("move", "down"),+ "sink": ("move", "down"), "sinks": ("move", "down"), "plunge": ("move", "down"),+ "positive": ("sign", "positive"), "negative": ("sign", "negative"),+ "before": ("time", "before"), "after": ("time", "after"),+ "win": ("result", "win"), "lose": ("result", "lose"),+ "approve": ("approval", "approve"), "reject": ("approval", "reject"),+}_SCALE_WORDS = {"thousand": 1e3, "million": 1e6, "billion": 1e9, "trillion": 1e12}_MULTIPLIERS = {"": 1.0, "k": 1e3, "m": 1e6, "mn": 1e6, "b": 1e9, "bn": 1e9, "t": 1e12, "tn": 1e12}+_NUMBER_WORDS = {+ "once": 1, "twice": 2, "thrice": 3, "two": 2, "three": 3, "four": 4, "five": 5, "six": 6, "seven": 7, "eight": 8, "nine": 9,+ "ten": 10, "eleven": 11, "twelve": 12,+}_NUM_RE = re.compile(r"^(-?)\$?(\d+(?:\.\d+)?)(k|mn|m|bn|b|tn|t)?$")_ORDINAL_RE = re.compile(r"^(\d+)(st|nd|rd|th)$")# Ordinal | number (a leading minus only when it is not a hyphen inside# a word or range; never starting inside a digit run, so "100mm" cannot-# leave a partial "10") | word.+# leave a partial "10") | word | percent sign (marks the number before it)._TOKEN_RE = re.compile(r"(?<![\d.,])\d+(?:st|nd|rd|th)(?![a-z0-9])"r"|(?:(?<![a-z0-9.,])-)?\$?(?<![\d.,])\d(?:[\d,]*\d)?(?:\.\d+)?(?:(?:k|mn|m|bn|b|tn|t)(?![a-z]))?"r"|[a-z][a-z.&']*"+ r"|%")_QUANTITY_RE = re.compile(r"^(>=|<=|>|<)?(.+)$")+# Dash characters that stand for a minus sign before a digit (U+2212 is+# mapped in _prepare too).+_DASH_RE = re.compile("[‐-―﹣-](?=\\s*\\d)")+_TIME_ZONES = r"(?:\s*\b(?:et|edt|est|ct|cdt|cst|mt|mdt|mst|pt|pdt|pst|utc|gmt)\b)?"+# Clock times: "11:59 pm et", "5pm", "17:00". Read into their own gate+# (both titles state one: they must agree) and kept out of quantities.+_CLOCK_12_RE = re.compile(r"(?<![\d.:])(\d{1,2})(?::([0-5]\d))?\s*([ap])\.?m\.?(?![a-z])" + _TIME_ZONES)+_CLOCK_24_RE = re.compile(r"(?<![\d.:])([01]?\d|2[0-3]):([0-5]\d)(?![\d:])" + _TIME_ZONES)+# "between 2.5% and 2.7%" is the two thresholds it names.+_BETWEEN_RE = re.compile(r"\bbetween\s+(\$?-?\d[\d.,]*\s*%?)\s*(?:and|to|-)\s*(?=\$?-?\d)")++# Distinguishing names. A capitalized word (or one in _LISTED_NAMES) that+# one title names and the other's text lacks tells two events apart when+# both titles have one ("House" vs "Senate", "Navarro" vs "Bannon"); a+# listed qualifier rejects even when only one title has it ("core CPI"+# vs "CPI"). Phrases fold to one name first so "Federal Reserve" is+# "Fed"; these folds feed only the name gate, not the overlap score.+_NAME_PHRASES = [+ (re.compile(r"\breserve bank of australia\b", re.I), "RBA"),+ (re.compile(r"\bpeople'?s bank of china\b", re.I), "PBoC"),+ (re.compile(r"\bfederal reserve\b", re.I), "Fed"),+ (re.compile(r"\bbank of england\b", re.I), "BoE"),+ (re.compile(r"\bbank of japan\b", re.I), "BoJ"),+ (re.compile(r"\bbank of canada\b", re.I), "BoC"),+ (re.compile(r"\beuropean central bank\b", re.I), "ECB"),+ (re.compile(r"\bunited states\b", re.I), "US"),+ (re.compile(r"\b(?:united kingdom|great britain)\b", re.I), "UK"),+]+_NAME_ALIASES = {+ "fomc": "fed", "democratic": "democrat", "democrats": "democrat", "dem": "democrat",+ "dems": "democrat", "republicans": "republican", "gop": "republican", "btc": "bitcoin",+ "britain": "uk", "british": "uk", "american": "us", "americans": "us", "chinese": "china",+ "japanese": "japan", "german": "germany", "french": "france", "russian": "russia",+ "ukrainian": "ukraine", "israeli": "israel", "iranian": "iran", "canadian": "canada",+ "mexican": "mexico", "brazilian": "brazil", "indian": "india", "korean": "korea",+ "taiwanese": "taiwan", "european": "europe", "eurozone": "europe", "turkish": "turkey",+}+_COUNTRIES = frozenset(+ """+ uk china japan germany france italy spain canada mexico brazil india russia ukraine israel+ iran iraq syria gaza lebanon yemen taiwan korea australia england scotland ireland europe eu+ argentina turkey saudi venezuela poland netherlands switzerland sweden norway egypt nigeria+ pakistan indonesia vietnam philippines colombia chile peru greece portugal belgium austria+ denmark finland hungary romania qatar uae cuba belarus+ """.split()+)+_LISTED_NAMES = _COUNTRIES | frozenset(+ "house senate democrat republican labour conservative tory north south east west core "+ "headline cash classic primary runoff".split()+)+_NOT_NAMES = frozenset(+ "who what which when where how why can could should would may might must yes no vs versus "+ "monday tuesday wednesday thursday friday saturday sunday today tomorrow tonight".split()+)def clip_text(value: object, limit: int = MAX_TEXT_CHARS) -> str:@@ -140,9 +260,44 @@ def _canonical_number(raw: str, scale: float = 1.0) -> str | None:return f"{value:.12g}" if value != int(value) else str(int(value))-def _prepare(text: str) -> str:+def _ascii(text: str) -> str:+ """Typographic signs folded, dashes before a digit made minus signs,+ then ASCII; case kept."""text = (text or "").replace("’", "'").replace("≥", ">=").replace("≤", "<=").replace("−", "-")- text = unicodedata.normalize("NFKD", text).encode("ascii", "ignore").decode().lower()+ text = _DASH_RE.sub("-", text)+ return unicodedata.normalize("NFKD", text).encode("ascii", "ignore").decode()+++def _clock_minutes(hour: str, minute: str | None, meridiem: str | None) -> str | None:+ h, m = int(hour), int(minute or 0)+ if meridiem:+ if not 1 <= h <= 12:+ return None+ h = h % 12 + (12 if meridiem == "p" else 0)+ return f"{h:02d}:{m:02d}"+++def clock_times(text: str) -> set[str]:+ """Clock times a title states, as 24-hour "HH:MM" ("11:59 PM ET" ->+ "23:59", "5pm" -> "17:00")."""+ low = _ascii(text).lower()+ found = {_clock_minutes(*m.groups()) for m in _CLOCK_12_RE.finditer(low)}+ found |= {_clock_minutes(m.group(1), m.group(2), None) for m in _CLOCK_24_RE.finditer(low)}+ return {t for t in found if t is not None}+++def _prepare(text: str) -> str:+ text = _ascii(text).lower()+ # Clock times are read by clock_times(); here they would leave their+ # hour and minute behind as quantities ("11:59 PM" as 11 and 59).+ text = _CLOCK_12_RE.sub(" ", text)+ text = _CLOCK_24_RE.sub(" ", text)+ text = _BETWEEN_RE.sub(r" atleast \1 and atmost ", text)+ text = re.sub(+ r"\b(" + "|".join(_NUMBER_WORDS) + r")\b", lambda m: str(_NUMBER_WORDS[m.group(1)]), text+ )+ text = re.sub(r"(\d)\s*%", r"\1 %", text)+ text = re.sub(r"\b(?:negative|minus)\s+(?=\d)", " -", text)# Comparators become words so a quantity keeps its direction:# ">25bps" and "25 bps" are different contracts.text = text.replace(">=", " atleast ").replace("<=", " atmost ")@@ -159,11 +314,24 @@ def _prepare(text: str) -> str:return re.sub(r"(\d)(bps|bp)\b", r"\1 \2", text)+def _fold_word(raw: str) -> str:+ """A lowercase word token as compared: edge punctuation, possessive+ and apostrophes dropped, synonyms folded ("" when nothing is left)."""+ word = raw.strip(".'&")+ if word.endswith("'s"):+ word = word[:-2]+ word = word.replace("'", "")+ return _SYNONYMS.get(word, word) if word else ""++def _scan(text: str) -> list[tuple[str, str]]:- """``(token, kind)`` pairs; kind is word, number, money, year or ordinal.+ """``(token, kind)`` pairs; kind is word, number, money, percent, year+ or ordinal.``year`` is only the shape (four plain digits, 1900-2100); whether itis a year or an amount depends on what precedes it (text_features).+ A "%" or "percent" after a number makes it ``percent`` and is not a+ token itself."""raws = _TOKEN_RE.findall(_prepare(text))out: list[tuple[str, str]] = []@@ -171,6 +339,10 @@ def _scan(text: str) -> list[tuple[str, str]]:while i < len(raws):raw = raws[i]i += 1+ if raw == "%" or (raw in ("percent", "pct") and out and out[-1][1] in ("number", "year")):+ if out and out[-1][1] in ("number", "year"):+ out[-1] = (out[-1][0], "percent")+ continueordinal = _ORDINAL_RE.match(raw)if ordinal:out.append((f"{int(ordinal.group(1))}{ordinal.group(2)}", "ordinal"))@@ -191,12 +363,9 @@ def _scan(text: str) -> list[tuple[str, str]]:kind = "number"out.append((num, kind))continue- word = raw.strip(".'&")- if word.endswith("'s"):- word = word[:-2]- word = word.replace("'", "")+ word = _fold_word(raw)if word:- out.append((_SYNONYMS.get(word, word), "word"))+ out.append((word, "word"))return out@@ -237,26 +406,73 @@ def _day_of_month(tok: str, kind: str) -> str | None:return None+def _is_month(toks: list[str], kinds: list[str], i: int) -> bool:+ if toks[i] != "may":+ return toks[i] in _MONTHS+ after = kinds[i + 1] if i + 1 < len(kinds) else ""+ return after in ("number", "year", "ordinal") or (i > 0 and toks[i - 1] in _MAY_BEFORE)+++def _mode(toks: list[str], i: int) -> str | None:+ """"to" (a level) or "by" (a change) before the number at ``toks[i]``."""+ j = i - 1+ while j >= 0 and toks[j] in ("than", "the", "$", "above", "below", "atleast", "atmost"):+ j -= 1+ return toks[j] if j >= 0 and toks[j] in ("to", "by") else None+++def _typed_quantity(toks: list[str], kinds: list[str], i: int, comparator: str) -> dict:+ """One quantity with its unit ("$", "%" with bps read as hundredths+ of a percent, or None), its "to"/"by" mode and the key the gates+ compare (comparator plus the value in that unit)."""+ tok, kind = toks[i], kinds[i]+ unit = {"money": "$", "percent": "%"}.get(kind)+ unit_raw = unit+ value = tok+ if i + 1 < len(toks) and toks[i + 1] == "bps" and kind == "number":+ unit, unit_raw = "%", "bps"+ value = _canonical_number(str(float(tok) / 100)) or tok+ return {+ "raw": comparator + tok,+ "key": comparator + value,+ "unit": unit,+ "unit_raw": unit_raw,+ "mode": _mode(toks, i),+ # Strictness matters for whole numbers ("more than 50" is 51 or+ # more), not for fractions; bps count as whole numbers.+ "whole": kind != "ordinal" and "." not in tok,+ }++def text_features(text: str) -> dict:"""Split a title into content words, quantities, years and flags.A 1-31 number (or ordinal) right after a month name is a day of- month and goes to ``days``, not ``quantities``: "Oct 28, 2026" must- not block a pair with "October 2026" on the number gate (the- end-date gate covers it). A four-digit 1900-2100 number is a year- unless a dollar sign, a thousands comma or a comparator makes it an- amount: "above $2,000" is the quantity ``>2000``. ``actions`` and- ``directions`` are the rate verbs and above/below words the title- uses, after synonym folding.+ month and goes to ``days`` (and ``dated``, by month), not+ ``quantities``: "Oct 28, 2026" must not block a pair with "October+ 2026" on the number gate; the day gate compares days only when both+ titles state one for the same month. A four-digit 1900-2100 number+ is a year unless a dollar sign, a thousands comma, a percent sign or+ a comparator makes it an amount: "above $2,000" is the quantity+ ``>2000``. ``actions``, ``directions`` and ``poles`` are the rate+ verbs, above/below words and opposite words the title uses, after+ synonym folding. ``typed`` carries each quantity's unit and mode,+ ``order`` the quantity keys in order of first appearance, ``times``+ the clock times."""scanned = _scan(text)toks = [tok for tok, _ in scanned]+ kinds = [kind for _, kind in scanned]words: set[str] = set()quantities: set[str] = set()+ typed: list[dict] = []years: set[str] = set()days: set[str] = set()+ months: set[str] = set()+ dated: dict[str, set[str]] = {}actions: set[str] = set()directions: set[str] = set()+ poles: dict[str, set[str]] = {}negated = Falsefor i, (tok, kind) in enumerate(scanned):if kind == "word":@@ -264,6 +480,11 @@ def text_features(text: str) -> dict:actions.add(tok)if tok in _DIRECTIONS:directions.add(_DIRECTIONS[tok])+ if tok in _POLES:+ axis, pole = _POLES[tok]+ poles.setdefault(axis, set()).add(pole)+ if tok in _MONTHS and _is_month(toks, kinds, i):+ months.add(tok)if tok in _NEGATIONS:negated = Trueelif tok in _COMPARATOR_WORDS:@@ -276,18 +497,30 @@ def text_features(text: str) -> dict:day = _day_of_month(tok, kind)if kind == "year" and not comparator:years.add(tok)- elif i > 0 and toks[i - 1] in _MONTHS and day is not None:+ elif i > 0 and toks[i - 1] in _MONTHS and _is_month(toks, kinds, i - 1) and day is not None:days.add(day)+ dated.setdefault(toks[i - 1], set()).add(day)else:quantities.add(comparator + tok)+ typed.append(_typed_quantity(toks, kinds, i, comparator))+ order: list[str] = []+ for q in typed:+ if q["key"] not in order:+ order.append(q["key"])return {"words": words,"quantities": quantities,+ "typed": typed,+ "order": order,"years": years,"days": days,+ "months": months,+ "dated": dated,+ "times": clock_times(text),"negated": negated,"actions": actions,"directions": directions,+ "poles": poles,}@@ -311,27 +544,80 @@ def _text(m: dict, *keys: str) -> str:return " ".join(s for s in (m.get(k) for k in keys) if isinstance(s, str) and s)+def _alias_phrases(text: str) -> str:+ for pattern, alias in _NAME_PHRASES:+ text = pattern.sub(f" {alias} ", text)+ return text+++def _name_word(raw: str) -> str:+ word = _fold_word(raw.lower())+ return _NAME_ALIASES.get(word, word)+++def _names(*texts: str) -> set[str]:+ """Distinguishing names in the texts: capitalized words and listed+ names, after phrase and name folding. Words that only shape the+ question (stopwords, months, rate verbs, directions, opposite words,+ negations, yes/no, number words) are never names."""+ names: set[str] = set()+ for text in texts:+ for match in re.finditer(r"[A-Za-z][A-Za-z.&']*", _alias_phrases(_ascii(text))):+ raw = match.group(0)+ word = _name_word(raw)+ if (+ len(word) < 2 or word in _STOPWORDS or word in _NOT_NAMES or word in _MONTHS+ or word in _ACTIONS or word in _DIRECTIONS or word in _COMPARATOR_WORDS+ or word in _POLES or word in _NEGATIONS or word in _NUMBER_WORDS+ ):+ continue+ if word in _LISTED_NAMES or raw[0].isupper():+ names.add(word)+ return names+++def _present(text: str) -> set[str]:+ """Every word of ``text`` after phrase and name folding: what a name+ on the other venue is looked up in."""+ return {_NAME_ALIASES.get(tok, tok) for tok, kind in _scan(_alias_phrases(_ascii(text))) if kind == "word"}++def _market_features(m: dict, venue: str) -> dict:"""Everything the gates read from one market, computed once.``bag`` (question, event title and, on Kalshi, the YES side) feeds- the overlap score, quantities, years and negation. Actions and- directions come from the contract's own wording (question plus YES- side), not the event title, which can list every alternative.+ the overlap score, quantities, years, months, days, clock times and+ negation. Actions, directions, opposite words and names come from+ the contract's own wording (question plus YES side), not the event+ title, which can list every alternative; a name is looked up in the+ other venue's whole bag."""if venue == "kalshi":- bag = text_features(_text(m, "question", "context", "yes_side"))- gate = text_features(_text(m, "question", "yes_side"))+ bag_text = _text(m, "question", "context", "yes_side")+ gate_texts = [m.get("question") or "", m.get("yes_side") or ""]else:- bag = text_features(_text(m, "question", "context"))- gate = text_features(_text(m, "question"))+ bag_text = _text(m, "question", "context")+ gate_texts = [m.get("question") or ""]+ bag = text_features(bag_text)+ gate = text_features(" ".join(t for t in gate_texts if isinstance(t, str)))+ present = _present(bag_text)+ actions = set(gate["actions"])+ if "hold" in actions and not present & _RATE_CONTEXT:+ actions.discard("hold") # "hold an election", "maintain control"outcomes = [str(o) for o in (m.get("outcomes") or [])]+ question = text_features(_text(m, "question"))return {"bag": bag,- "actions": gate["actions"],+ "actions": actions,"directions": gate["directions"],- "question_words": text_features(_text(m, "question"))["words"],- "side_words": text_features(_text(m, "yes_side"))["words"],+ "poles": gate["poles"],+ "names": _names(*(t for t in gate_texts if isinstance(t, str))),+ "present": present,+ "order": question["order"],+ "question_words": question["words"],+ # "bps" is a unit, compared with its value by the quantity gates+ # ("Cut 25bps" against "cut by 0.25%").+ "side_words": text_features(_text(m, "yes_side"))["words"] - {"bps"},"outcomes": outcomes,"labels": [set(tokenize(o)) for o in outcomes],"end": _parse_time(m.get("end_time")),@@ -385,6 +671,80 @@ def _boundary_only(k_quantities: set[str], p_quantities: set[str]) -> bool:)+def _quantity_classes(typed: list[dict]) -> set[str]:+ """Keys with strictness dropped for fractional thresholds only."""+ return {_threshold_class(q["key"]) if not q["whole"] else q["key"] for q in typed}+++def _quantities_gate(kb: dict, pb: dict) -> tuple[str, str]:+ """("agree" | "boundary" | "differ", reason). Values compare in one+ unit (bps as hundredths of a percent); reasons quote the titles'+ own numbers."""+ k_raw, p_raw = sorted(kb["quantities"]), sorted(pb["quantities"])+ k_keys = {q["key"] for q in kb["typed"]}+ p_keys = {q["key"] for q in pb["typed"]}+ if k_raw == p_raw:+ return "agree", f"quantities agree {k_raw}" if k_raw else ""+ if k_keys == p_keys:+ return "agree", f"quantities agree once bps are read as percent: Kalshi {k_raw} vs Polymarket {p_raw}"+ if _boundary_only(kb["quantities"], pb["quantities"]) or (+ len(k_keys) == len(p_keys) and _quantity_classes(kb["typed"]) == _quantity_classes(pb["typed"])+ ):+ return "boundary", (+ f"quantities agree except at the boundary: Kalshi {k_raw} vs Polymarket {p_raw} "+ "(the same unless the outcome lands exactly on the threshold)"+ )+ return "differ", f"quantities differ: Kalshi {k_raw} vs Polymarket {p_raw}"+++def _by_key(typed: list[dict], field: str, key: str = "key") -> dict[str, set[str]]:+ out: dict[str, set[str]] = {}+ for q in typed:+ if q[field] is not None:+ out.setdefault(_threshold_class(q[key]), set()).add(q[field])+ return out+++def _late_gates(kf: dict, pf: dict) -> str | None:+ """The gates added after the review of 2026-10-17, in order; the first+ failing one's reason, or None. They run after the original gates so+ a pair those reject keeps its reason."""+ kb, pb = kf["bag"], pf["bag"]+ if kb["times"] and pb["times"] and kb["times"] != pb["times"]:+ return f"clock times differ: Kalshi {sorted(kb['times'])} vs Polymarket {sorted(pb['times'])}"+ if kb["months"] and pb["months"] and not kb["months"] & pb["months"]:+ return f"months differ: Kalshi {sorted(kb['months'])} vs Polymarket {sorted(pb['months'])}"+ for month in sorted(set(kb["dated"]) & set(pb["dated"])):+ k_days, p_days = kb["dated"][month], pb["dated"][month]+ if k_days != p_days:+ return (+ f"days differ in {month}: Kalshi {sorted(k_days, key=int)} vs Polymarket {sorted(p_days, key=int)}"+ )+ for field, key, what in (+ ("unit", "key", "units"), ("unit_raw", "raw", "units"), ("mode", "key", "kinds ('to' a level vs 'by' a change)"),+ ):+ k_by, p_by = _by_key(kb["typed"], field, key), _by_key(pb["typed"], field, key)+ for key in sorted(set(k_by) & set(p_by)):+ if not k_by[key] & p_by[key]:+ return (+ f"quantity {what} differ for {key}: Kalshi {sorted(k_by[key])} vs Polymarket {sorted(p_by[key])}"+ )+ k_order, p_order = kf["order"], pf["order"]+ if len(k_order) >= 2 and set(k_order) == set(p_order) and k_order != p_order:+ return f"quantities in a different order: Kalshi {k_order} vs Polymarket {p_order}"+ for axis in sorted(set(kf["poles"]) & set(pf["poles"])):+ if kf["poles"][axis] != pf["poles"][axis]:+ return f"opposite wording: Kalshi {sorted(kf['poles'][axis])} vs Polymarket {sorted(pf['poles'][axis])}"+ k_extra = kf["names"] - pf["present"]+ p_extra = pf["names"] - kf["present"]+ k_listed, p_listed = k_extra & _LISTED_NAMES, p_extra & _LISTED_NAMES+ if k_listed or p_listed:+ return f"a listed name on one venue only: Kalshi {sorted(k_listed)} vs Polymarket {sorted(p_listed)}"+ if k_extra and p_extra:+ return f"distinguishing names differ: Kalshi {sorted(k_extra)} vs Polymarket {sorted(p_extra)}"+ return None++def _score(kf: dict, pf: dict, now: datetime) -> dict:kb, pb = kf["bag"], pf["bag"]reasons: list[str] = []@@ -401,19 +761,12 @@ def _score(kf: dict, pf: dict, now: datetime) -> dict:f"{len(kb['words'])} Kalshi / {len(pb['words'])} Polymarket words)")- boundary = False- if kb["quantities"] != pb["quantities"]:- if not _boundary_only(kb["quantities"], pb["quantities"]):- return reject(- f"quantities differ: Kalshi {sorted(kb['quantities'])} vs Polymarket {sorted(pb['quantities'])}"- )- boundary = True- reasons.append(- f"quantities agree except at the boundary: Kalshi {sorted(kb['quantities'])} vs Polymarket "- f"{sorted(pb['quantities'])} (the same unless the outcome lands exactly on the threshold)"- )- elif kb["quantities"]:- reasons.append(f"quantities agree {sorted(kb['quantities'])}")+ verdict, why = _quantities_gate(kb, pb)+ if verdict == "differ":+ return reject(why)+ boundary = verdict == "boundary"+ if why:+ reasons.append(why)if kb["years"] and pb["years"] and not (kb["years"] & pb["years"]):return reject(f"years differ: Kalshi {sorted(kb['years'])} vs Polymarket {sorted(pb['years'])}")@@ -448,6 +801,10 @@ def _score(kf: dict, pf: dict, now: datetime) -> dict:return reject(side_reason)reasons.append(side_reason)+ late = _late_gates(kf, pf)+ if late:+ return reject(late)+confidence = round(dice * date_factor, 3)return {"confidence": confidence, "rejected": False, "reasons": reasons, "side": side, "boundary_differs": boundary}@@ -468,6 +825,14 @@ def score_pair(kalshi: dict, poly: dict, now: datetime | None = None) -> dict:# Matching# ---------------------------------------------------------------------------+def _prefix(words: set[str], df: dict[str, int], t: float) -> list[str]:+ """The rarest ``n - need + 1`` of a market's words, in the total order+ (document frequency, then the word): the order every market shares."""+ ordered = sorted(words, key=lambda w: (df[w], w))+ need = math.ceil(t * len(ordered) / (2 - t) - 1e-9) # rounding can only lengthen the prefix+ return ordered[: len(ordered) - max(need, 1) + 1]++def _candidate_pairs(k_feats: list[dict], p_feats: list[dict], min_dice: float) -> list[tuple[int, int]]:"""Every pair whose title-word Dice can reach ``min_dice``.@@ -477,9 +842,11 @@ def _candidate_pairs(k_feats: list[dict], p_feats: list[dict], min_dice: float)o satisfies o >= t * n / (2 - t) for either market's word count n(from 2o >= t * (n + m) and m >= o), and the first shared word sitswithin the first n - o + 1 words of each market, so both prefixes- hold it. Pairs sharing no word are never candidates. (The bound and- the prefix step were checked with z3 on 2026-10-17, the prefix step- for up to 16 words; a test compares this with a brute-force scan.)+ hold it. "First shared word" needs one total order for every+ market, so ties in rarity break on the word itself (``_prefix``).+ Pairs sharing no word are never candidates. The argument is checked+ by tests, not machine-proved: a brute-force scan comparison and a+ test of the total order."""t = min(max(float(min_dice), 0.0), 1.0)df: dict[str, int] = {}@@ -487,18 +854,13 @@ def _candidate_pairs(k_feats: list[dict], p_feats: list[dict], min_dice: float)for w in f["bag"]["words"]:df[w] = df.get(w, 0) + 1- def prefix(words: set[str]) -> list[str]:- ordered = sorted(words, key=lambda w: (df[w], w))- need = math.ceil(t * len(ordered) / (2 - t) - 1e-9) # rounding can only lengthen the prefix- return ordered[: len(ordered) - max(need, 1) + 1]-index: dict[str, list[int]] = {}for pi, f in enumerate(p_feats):- for w in prefix(f["bag"]["words"]):+ for w in _prefix(f["bag"]["words"], df, t):index.setdefault(w, []).append(pi)pairs: set[tuple[int, int]] = set()for ki, f in enumerate(k_feats):- for w in prefix(f["bag"]["words"]):+ for w in _prefix(f["bag"]["words"], df, t):for pi in index.get(w, ()):pairs.add((ki, pi))return sorted(pairs)@@ -591,15 +953,44 @@ def _brief(m: dict) -> dict:"spread": m.get("spread"),"last_price": m.get("last_price"),"last_trade_time": m.get("last_trade_time"),+ "stale": bool(m.get("stale")),+ "stale_age_seconds": m.get("stale_age_seconds") if m.get("stale") else None,"url": m.get("url"),}+def _flip(value: object) -> float | None:+ return round(1.0 - value, 4) if isinstance(value, (int, float)) and not isinstance(value, bool) else None+++def _aligned_brief(m: dict, side: str | None) -> dict:+ """The Polymarket side's brief priced on the outcome the Kalshi YES+ pays on: for an inverted pair the quotes are flipped (YES bid = 1 -+ the first outcome's ask, and so on). ``quoted_outcome`` names it."""+ brief = _brief(m)+ outcomes = brief.get("outcomes") or []+ if side == "inverted" and len(outcomes) == 2:+ brief.update({+ "probability": _flip(m.get("probability")),+ "yes_bid": _flip(m.get("yes_ask")),+ "yes_ask": _flip(m.get("yes_bid")),+ "last_price": _flip(m.get("last_price")),+ "quoted_outcome": outcomes[1],+ })+ else:+ brief["quoted_outcome"] = outcomes[0] if outcomes else None+ return brief++def _quote_flags(venue: str, m: dict) -> list[str]:"""Why one side's price is not a live two-sided quote ([] when it is)."""if m.get("probability") is None:- return [] # no price: the gap is None and the pair sorts last anyway+ return [f"{venue} price unavailable (basis: {m.get('price_basis') or 'not stated'})"]flags = []+ if m.get("stale"):+ age = m.get("stale_age_seconds")+ old = f"{age / 3600:.1f} h old" if isinstance(age, (int, float)) else "of unknown age"+ flags.append(f"{venue} price from a copy {old}, served from expired cache after a failed live fetch")basis = m.get("price_basis")if basis != "bid_ask_midpoint":flags.append(f"{venue} price is not a two-sided quote midpoint (basis: {basis or 'not stated'})")@@ -624,7 +1015,7 @@ def _pair_record(km: dict, pm: dict, result: dict) -> dict:"reasons": result["reasons"],"side": result["side"],"kalshi": _brief(km),- "polymarket": _brief(pm),+ "polymarket": _aligned_brief(pm, result["side"]),"kalshi_yes_probability": k_prob,"polymarket_yes_probability": p_aligned,"price_gap": gap,diff --git a/src/world_intel_mcp/cli.py b/src/world_intel_mcp/cli.pyindex 8c09b94..00952bb 100644--- a/src/world_intel_mcp/cli.py+++ b/src/world_intel_mcp/cli.py@@ -868,12 +868,14 @@ def predictions(ctx: click.Context, limit: int) -> None:if _bail_on_error(ctx, data):return+ _print_data_gaps(data)mkts = data.get("markets", [])if not mkts:console.print("[yellow]No prediction market data available[/yellow]")return- table = Table(title="Prediction Markets", box=box.SIMPLE_HEAVY)+ title = "Prediction Markets (expired cached copy)" if data.get("stale") else "Prediction Markets"+ table = Table(title=title, box=box.SIMPLE_HEAVY)table.add_column("Question", max_width=50)table.add_column("YES %", justify="right")table.add_column("Sentiment")@@ -899,6 +901,27 @@ def _pct(value: Any) -> str:return "-" if value is None else f"{value * 100:.1f}%"+def _stale_note(m: dict) -> str:+ """" (expired copy, N h old)" for a market priced from expired cache."""+ if not m.get("stale"):+ return ""+ age = m.get("stale_age_seconds")+ return f" (expired copy, {age / 3600:.1f} h old)" if isinstance(age, (int, float)) else " (expired copy)"+++def _last_trade_note(side: dict) -> str | None:+ """The Kalshi last-trade lookup result for the Flags column."""+ status = side.get("last_trade_lookup")+ if status is None and not side.get("last_trade_time"):+ return None+ if side.get("last_trade_time"):+ price = side.get("last_trade_price")+ at = f" at {_pct(price)}" if price is not None else ""+ copy = " (expired copy)" if status == "found_in_expired_copy" else ""+ return f"kalshi last trade {side['last_trade_time']}{at}{copy}"+ return f"kalshi last trade: {str(status).replace('_', ' ')}"++# Data gaps print through the shared _print_data_gaps defined with the# AOI commands below (module-level, resolved at call time).@@ -944,7 +967,7 @@ def kalshi(ctx: click.Context, category: str | None, series: str | None, limit:_cell(m.get("question") or ""),_cell(m.get("yes_side") or ""),_pct(m.get("probability")),- _cell(m.get("price_basis", "")),+ _cell(m.get("price_basis", "") + _stale_note(m)),f"{m.get('volume_24h', 0):,.0f}",f"{m.get('volume', 0):,.0f}",)@@ -1015,9 +1038,9 @@ def prediction_compare(gap = p.get("price_gap")flags = list(p.get("flags") or [])flagged = flagged or bool(flags)- traded = (p.get("kalshi") or {}).get("last_trade_time")+ traded = _last_trade_note(p.get("kalshi") or {})if traded:- flags.append(f"kalshi last trade {traded}")+ flags.append(traded)table.add_row( # wrapped, not sliced (see the kalshi table)_cell(p.get("kalshi", {}).get("question") or ""),_cell(p.get("polymarket", {}).get("question") or ""),diff --git a/src/world_intel_mcp/sources/kalshi.py b/src/world_intel_mcp/sources/kalshi.pyindex 24eb96e..87da815 100644--- a/src/world_intel_mcp/sources/kalshi.py+++ b/src/world_intel_mcp/sources/kalshi.py@@ -129,6 +129,23 @@ def _finite(value: Any) -> float | None:return number if number is not None and math.isfinite(number) else None+def _quote(value: Any) -> float | None:+ """A quote in dollars per $1 payout: a finite number from 0 to 1, else+ None (a cents-scaled "45" or a negative value is not a price)."""+ number = _finite(value)+ return number if number is not None and 0 <= number <= 1 else None+++_QUOTE_FIELDS = ("yes_bid_dollars", "yes_ask_dollars", "last_price_dollars")+++def bad_quote_fields(market: dict) -> list[str]:+ """Quote fields present in a raw market but not a price from 0 to 1."""+ return [f for f in _QUOTE_FIELDS if _num(market.get(f)) is not None and _quote(market.get(f)) is None] + [+ f for f in _QUOTE_FIELDS if market.get(f) not in (None, "") and _num(market.get(f)) is None+ ]++def _volume(value: Any) -> float | None:"""A contract count: absent means 0.0; unparseable, non-finite ornegative means None (unreadable, counted by the caller)."""@@ -186,9 +203,9 @@ def implied_probability(market: dict) -> tuple[float | None, str]:ask < 1); otherwise the last trade when it is a finite price in(0, 1]; otherwise None ("no price")."""- bid = _finite(market.get("yes_bid_dollars"))- ask = _finite(market.get("yes_ask_dollars"))- last = _finite(market.get("last_price_dollars"))+ bid = _quote(market.get("yes_bid_dollars"))+ ask = _quote(market.get("yes_ask_dollars"))+ last = _quote(market.get("last_price_dollars"))if bid is not None and ask is not None and 0 < bid <= ask < 1:return round((bid + ask) / 2, 4), "bid_ask_midpoint"if last is not None and 0 < last <= 1:@@ -203,6 +220,10 @@ def normalize_market(market: dict, event: dict | None = None) -> dict:unreadable (callers count such rows); ``spread`` is set only for atwo-sided book. Kalshi's market listings carry no last-trade time(``last_trade_time`` is None here; see fetch_kalshi_last_trade_times).+ Quote fields outside 0..1 are None (bad_quote_fields names them).+ ``stale`` and ``stale_age_seconds`` say whether the listing response+ this market came from was served from expired cache (the event dict+ carries the marker, set by _events_pages)."""event = event or {}prob, basis = implied_probability(market)@@ -210,9 +231,10 @@ def normalize_market(market: dict, event: dict | None = None) -> dict:series_ticker = event.get("series_ticker") or (event_ticker.split("-", 1)[0] if event_ticker else "")yes_side = market.get("yes_sub_title") or ""title = market.get("title") or ""- bid = _finite(market.get("yes_bid_dollars"))- ask = _finite(market.get("yes_ask_dollars"))+ bid = _quote(market.get("yes_bid_dollars"))+ ask = _quote(market.get("yes_ask_dollars"))two_sided = bid is not None and ask is not None and 0 < bid <= ask < 1+ stale = bool(event.get("_stale"))return {"venue": "kalshi","id": market.get("ticker", ""),@@ -227,8 +249,10 @@ def normalize_market(market: dict, event: dict | None = None) -> dict:"yes_bid": bid,"yes_ask": ask,"spread": round(ask - bid, 4) if two_sided else None,- "last_price": _finite(market.get("last_price_dollars")),+ "last_price": _quote(market.get("last_price_dollars")),"last_trade_time": None,+ "stale": stale,+ "stale_age_seconds": _stale_age(event) if stale else None,"volume": _volume(market.get("volume_fp")),"volume_24h": _volume(market.get("volume_24h_fp")),"open_interest": _finite(market.get("open_interest_fp")),@@ -280,7 +304,14 @@ async def _events_pages(f"Kalshi /events page {pages + 1} unavailable (no live or cached data)")pages += 1- events.extend(e for e in data["events"] if isinstance(e, dict))+ page = [e for e in data["events"] if isinstance(e, dict)]+ if data.get("_stale"):+ # Carried onto each market by normalize_market, so a price+ # from this page is marked where it is shown.+ for e in page:+ e["_stale"] = True+ e["_stale_age_seconds"] = _stale_age(data)+ events.extend(page)cursor = data.get("cursor") or ""if not cursor:return events, pages, True, None@@ -393,13 +424,20 @@ async def fetch_kalshi_open_markets()markets: list[dict] = []+ bad_quotes = 0for ev in events:if category and ev.get("category") and ev["category"] != category:continuefor m in ev.get("markets") or []:if isinstance(m, dict) and m.get("status") in (None, "active", "open"):markets.append(normalize_market(m, ev))+ bad_quotes += bool(bad_quote_fields(m))scan["events_scanned"] = len(events)+ if bad_quotes:+ gaps.append(+ f"{bad_quotes} market(s) with a quote field that is not a price from 0 to 1 "+ "(left null; the price falls back or is missing)"+ )if stale:scan["stale_responses"] = len(stale)scan["stale_max_age_seconds"] = max_stale_age(stale)@@ -715,17 +753,24 @@ async def fetch_kalshi_price_history(async def fetch_kalshi_last_trade_times(- fetcher: Fetcher, tickers: list[str]+ fetcher: Fetcher, tickers: list[str], *,+ prices: dict[str, float | None] | None = None,+ stale_tickers: dict[str, float | None] | None = None,) -> tuple[dict[str, str | None], list[str]]:"""When each market last traded, from the public ``/markets/trades``endpoint (``limit=1``). The market listings carry no trade time(measured 2026-10-17), so a price whose basis is the last trade has- no age without this lookup.+ no age without this lookup. ``limit=1`` returns the venue's first+ listed trade, which is the latest only while the endpoint lists+ newest first (observed live on 2026-10-17 in two samples).Returns (``{ticker: created_time or None}``, gaps): None means thevenue returned no trade; a ticker missing from the dict could not belooked up or its trade time was unreadable (named in the gaps, never- guessed).+ guessed). When the caller passes ``prices``, the same trade's+ ``yes_price_dollars`` is recorded per ticker (None when unreadable+ or outside 0..1); ``stale_tickers`` records the tickers whose answer+ was served from expired cache, with the copy's age in seconds."""times: dict[str, str | None] = {}failed = 0@@ -742,14 +787,20 @@ async def fetch_kalshi_last_trade_times(if not isinstance(trades, list):failed += 1continue+ if stale_tickers is not None and data.get("_stale"):+ stale_tickers[ticker] = _stale_age(data)if not trades:times[ticker] = None+ if prices is not None:+ prices[ticker] = Nonecontinuecreated = trades[0].get("created_time") if isinstance(trades[0], dict) else Noneif not isinstance(created, str) or _ts(created) is None:failed += 1continuetimes[ticker] = created+ if prices is not None:+ prices[ticker] = _quote(trades[0].get("yes_price_dollars"))gaps: list[str] = []if failed:gaps.append(f"Kalshi last-trade time unavailable for {failed} market(s)")diff --git a/src/world_intel_mcp/sources/prediction.py b/src/world_intel_mcp/sources/prediction.pyindex 8bde54b..4300b16 100644--- a/src/world_intel_mcp/sources/prediction.py+++ b/src/world_intel_mcp/sources/prediction.py@@ -31,6 +31,7 @@ import jsonimport loggingimport mathimport re+import timefrom datetime import datetime, timezonefrom urllib.parse import quote@@ -174,6 +175,9 @@ async def fetch_prediction_markets(fetcher: Fetcher, limit: int = 20) -> dict:{"markets": [...], "count": N, "source": "polymarket","timestamp": "<iso>"}++ A list served from expired cache adds ``stale``, ``degraded`` and a+ ``data_gaps`` entry."""params = {"limit": str(limit),@@ -256,6 +260,7 @@ async def fetch_prediction_markets(fetcher: Fetcher, limit: int = 20) -> dict:gaps = []if stale:result["stale"] = True+ result["degraded"] = Truegaps.append("served from expired cache after a failed live fetch")if unpriced:gaps.append(f"{unpriced} market(s) left out: outcome price missing, unreadable or outside 0..1")@@ -323,19 +328,24 @@ async def fetch_polymarket_open(fetcher: Fetcher, limit: int = 300) -> dict:"""Open binary Polymarket markets by 24h volume, cross-venue shape.Returns ``{"markets", "requested", "rows_read", "non_binary_skipped",- "error"?, "degraded"?, "stale"?}``.+ "error"?, "degraded"?, "stale"?, "stale_max_age_seconds"?}``. Each+ market carries ``stale`` and ``stale_age_seconds``: whether its page+ was served from expired cache, and that copy's age."""limit = max(1, min(int(limit), MAX_OPEN_MARKETS))- rows: list = []+ rows: list[tuple[dict, float | None, bool]] = []offset = 0stale_before = _stale_serves(fetcher, "polymarket")+ stale_ages: list[float | None] = []error = Nonewhile offset < limit:size = min(_GAMMA_PAGE_MAX, limit - offset)+ cache_key = f"prediction:polymarket:open:{size}:{offset}"+ page_before = _stale_serves(fetcher, "polymarket")data = await fetcher.get_json(_GAMMA_MARKETS_URL,source="polymarket",- cache_key=f"prediction:polymarket:open:{size}:{offset}",+ cache_key=cache_key,cache_ttl=300,params={"limit": str(size), "offset": str(offset), "active": "true",@@ -348,11 +358,21 @@ async def fetch_polymarket_open(fetcher: Fetcher, limit: int = 300) -> dict:if data is None else f"Unexpected Polymarket response shape: {type(data).__name__}")break- rows.extend(r for r in data if isinstance(r, dict))+ page_stale = _stale_serves(fetcher, "polymarket") > page_before+ age = _stale_copy_age(fetcher, cache_key) if page_stale else None+ if page_stale:+ stale_ages.append(age)+ rows.extend((r, age, page_stale) for r in data if isinstance(r, dict))if len(data) < size:breakoffset += size- markets = [m for m in (normalize_gamma_market(r) for r in rows) if m is not None]+ markets = []+ for row, age, page_stale in rows:+ m = normalize_gamma_market(row)+ if m is not None:+ m["stale"] = page_stale+ m["stale_age_seconds"] = age+ markets.append(m)out: dict = {"markets": markets,"requested": limit,@@ -362,11 +382,22 @@ async def fetch_polymarket_open(fetcher: Fetcher, limit: int = 300) -> dict:if error:out["error"] = errorout["degraded"] = True- if _stale_serves(fetcher, "polymarket") > stale_before:+ if stale_ages or _stale_serves(fetcher, "polymarket") > stale_before:out["stale"] = True+ known = [a for a in stale_ages if a is not None]+ out["stale_max_age_seconds"] = max(known) if known else Nonereturn out+def _stale_copy_age(fetcher: Fetcher, cache_key: str) -> float | None:+ """Age in seconds of the cached copy under ``cache_key`` (a list+ payload carries no inline age), or None when unknown."""+ meta = fetcher.cache.get_stale_meta(cache_key)+ if meta is None or not isinstance(meta[1], (int, float)):+ return None+ return round(max(0.0, time.time() - meta[1]), 1)++def _closed_ts(value) -> int | None:if not value:return Nonediff --git a/src/world_intel_mcp/sources/prediction_cross.py b/src/world_intel_mcp/sources/prediction_cross.pyindex a373195..8c0a041 100644--- a/src/world_intel_mcp/sources/prediction_cross.py+++ b/src/world_intel_mcp/sources/prediction_cross.py@@ -33,10 +33,19 @@ Gaps / not covered:- Polymarket scoring shares the ``polymarket`` circuit breaker and 1 sfloor with Gamma (see sources/prediction.py): ``limit=100`` makes upto 100 sequential CLOB calls.+- Matching runs in a worker thread so the event loop keeps serving;+ a caller's timeout (the collector's 45 s) returns on time but cannot+ stop a thread already matching, which finishes in the background.+ Pair counts are not capped beyond the venue limits.+- Kalshi last-trade lookups are paced and sequential (at most 10 within+ 10 s); the trade price and time come from the same trade, but a+ separate read from the listing, so a pair whose looked-up trade price+ differs from the listing's last price is flagged."""from __future__ import annotations+import asyncioimport loggingimport mathfrom datetime import datetime, timezone@@ -63,7 +72,12 @@ _MATCH_METHOD = ("when both titles state one; negation, rate actions (cut, raise, hold) and directions ""(above, below) must agree; end dates within max(3 days, 15% of the horizon); the ""Kalshi YES side must be named in the Polymarket question or match exactly one "- "Polymarket outcome. match_confidence = title word overlap (Dice) x date factor "+ "Polymarket outcome. When both titles state them, clock times and days of the same month "+ "must be equal and months must overlap; quantity units ($, %, bps), 'to' vs 'by' and the "+ "order of two or more amounts must agree; opposite words (rise/fall, positive/negative, "+ "before/after, win/lose, approve/reject) must not conflict; a distinguishing name "+ "(capitalized, or a listed chamber, party, country or qualifier) on both venues but not "+ "shared rejects, and a listed one on one venue only rejects. match_confidence = title word overlap (Dice) x date factor ""(1.0 same day, 0.5 at the tolerance edge): an ordinal score for ranking candidate ""pairs, not a probability that two markets are the same event. One-to-one, greedy ""by confidence."@@ -79,8 +93,24 @@ _RANKING = (MAX_PAIRS = 500# Matched pairs whose Kalshi price is a last trade get that trade's time-# looked up (one /markets/trades request each), for the first this many.+# looked up (one /markets/trades request each), for the first this many,+# within one overall deadline: the collector gives the whole comparison+# 45 s, and a trades endpoint that accepts connections and never answers+# held compare for 145.7 s before (measured in review, 2026-10-17).MAX_TRADE_LOOKUPS = 10+TRADE_LOOKUP_DEADLINE_SECONDS = 10.0+# What a Kalshi side's ``last_trade_lookup`` says about its+# ``last_trade_time`` (null for every status but "found" and+# "found_in_expired_copy").+LOOKUP_STATUSES = {+ "found": "time and price of the latest trade, read live",+ "found_in_expired_copy": "time and price of the latest trade in an expired cached copy (age in last_trade_age_seconds)",+ "no_trade": "the venue lists no trade for the market",+ "lookup_failed": "the trades endpoint could not be read or its trade time was unreadable",+ "not_looked_up_cap": f"beyond the first {MAX_TRADE_LOOKUPS} last-trade pairs shown",+ "not_looked_up_deadline": f"the lookups' {TRADE_LOOKUP_DEADLINE_SECONDS:g} s deadline passed first",+ "not_looked_up_stale_listing": "the price came from an expired listing copy, so a live trade time would not date it",+}MAX_OFFSETS = 10# A price window spans at most 8760 + 720 hours, so one market's hourly# candles stay under the candlestick endpoint's 10,000 cap.@@ -183,6 +213,8 @@ async def fetch_prediction_compare(gaps.append(f"polymarket: {p_raw['error']}")if p_raw.get("stale"):gaps.append("polymarket: served from expired cache after a failed live fetch")+ if p_raw.get("stale_max_age_seconds") is not None:+ gaps.append(f"polymarket: the oldest open-markets copy is {p_raw['stale_max_age_seconds'] / 3600:.1f} h old")gaps.append(f"polymarket: top {p_raw['requested']} open markets by 24h volume requested; "f"{p_raw['non_binary_skipped']} non-binary rows skipped"@@ -197,10 +229,13 @@ async def fetch_prediction_compare("method": _MATCH_METHOD,"price_gap": "kalshi YES probability minus polymarket probability of the same side","price_basis": (- "per side: bid_ask_midpoint (two-sided book), last_trade (Kalshi's last trade; its time is "- "looked up for the first matched pairs), gamma_outcome_price (Polymarket's listed outcome "- "price; Gamma publishes no last-trade time) or no_price"+ "per side: bid_ask_midpoint (two-sided book), last_trade (Kalshi's last trade; its time and "+ f"price are looked up for the first {MAX_TRADE_LOOKUPS} such pairs shown, and "+ "last_trade_lookup says what a null last_trade_time means), gamma_outcome_price "+ "(Polymarket's listed outcome price; Gamma publishes no last-trade time) or no_price. A "+ "side priced from an expired cached copy carries stale: true and its age, and is flagged"),+ "last_trade_lookup": LOOKUP_STATUSES,"ranking": _RANKING,"units": {"kalshi": kalshi.UNITS, "polymarket": prediction.UNITS},"timestamp": _utc_now_iso(),@@ -224,20 +259,18 @@ async def fetch_prediction_compare(})return out- result = match_markets(k_markets, p_markets, min_confidence=min_confidence)+ # CPU-bound (seconds at the documented maxima): off the event loop, so+ # other tool calls and the collector's timeout are not starved.+ result = await asyncio.to_thread(match_markets, k_markets, p_markets, min_confidence=min_confidence)matched = result["matched"]matched.sort(key=lambda r: (r["price_gap"] is None, not r["comparable"], -abs(r["price_gap"] or 0.0), -r["match_confidence"],))shown = matched[:limit]- lookups = [p for p in shown if p["kalshi"].get("price_basis") == "last_trade"][:MAX_TRADE_LOOKUPS]- if lookups:- times, trade_gaps = await kalshi.fetch_kalshi_last_trade_times(fetcher, [p["kalshi"]["id"] for p in lookups])- for p in lookups:- p["kalshi"]["last_trade_time"] = times.get(p["kalshi"]["id"])- # Supplementary: a failed lookup leaves the time null and is named- # here; the comparison itself is complete, so it is not degraded.- gaps.extend(f"kalshi: {g}" for g in trade_gaps)+ # Supplementary: a lookup that fails or is skipped leaves the time null,+ # says why per side and is named in a gap; the comparison itself is+ # complete, so it is not degraded.+ gaps.extend(f"kalshi: {g}" for g in await _last_trades(fetcher, shown))out.update({"matched": shown,"matched_count": len(matched),@@ -253,6 +286,91 @@ async def fetch_prediction_compare(return out+async def _last_trades(fetcher: Fetcher, shown: list[dict]) -> list[str]:+ """Look up the last trade behind each shown last-trade Kalshi price.++ Sets ``last_trade_time``, ``last_trade_price`` and+ ``last_trade_lookup`` (a LOOKUP_STATUSES key) on each such Kalshi+ side, flags a pair whose looked-up trade price differs from the+ listing's last price, and returns the gaps. At most+ MAX_TRADE_LOOKUPS requests, all within TRADE_LOOKUP_DEADLINE_SECONDS.+ """+ sides = [p for p in shown if p["kalshi"].get("price_basis") == "last_trade"]+ if not sides:+ return []+ live = [p for p in sides if not p["kalshi"].get("stale")]+ targets = live[:MAX_TRADE_LOOKUPS]+ target_ids = {id(p) for p in targets}+ times: dict[str, str | None] = {}+ prices: dict[str, float | None] = {}+ stale: dict[str, float | None] = {}+ attempted: set[str] = set()++ async def run() -> None:+ for p in targets:+ ticker = p["kalshi"]["id"]+ found, _ = await kalshi.fetch_kalshi_last_trade_times(+ fetcher, [ticker], prices=prices, stale_tickers=stale+ )+ times.update(found)+ attempted.add(ticker)++ timed_out = False+ try:+ await asyncio.wait_for(run(), TRADE_LOOKUP_DEADLINE_SECONDS)+ except asyncio.TimeoutError:+ timed_out = True+ counts: dict[str, int] = {}+ for p in sides:+ side = p["kalshi"]+ ticker = side["id"]+ if side.get("stale"):+ status = "not_looked_up_stale_listing"+ elif id(p) not in target_ids:+ status = "not_looked_up_cap"+ elif ticker not in attempted:+ status = "not_looked_up_deadline"+ elif ticker not in times:+ status = "lookup_failed"+ elif times[ticker] is None:+ status = "no_trade"+ else:+ status = "found_in_expired_copy" if ticker in stale else "found"+ counts[status] = counts.get(status, 0) + 1+ side["last_trade_lookup"] = status+ side["last_trade_time"] = times.get(ticker) if status.startswith("found") else None+ side["last_trade_price"] = prices.get(ticker) if status.startswith("found") else None+ if status == "found_in_expired_copy":+ side["last_trade_age_seconds"] = stale[ticker]+ listed, traded = side.get("last_price"), side["last_trade_price"]+ if traded is not None and listed is not None and abs(traded - listed) > 1e-9:+ p["flags"].append(+ f"kalshi last trade at {traded:g} (looked up separately) differs from the listing's last price {listed:g}"+ )+ p["comparable"] = False+ gaps = []+ if counts.get("lookup_failed"):+ gaps.append(f"Kalshi last-trade time unavailable for {counts['lookup_failed']} market(s)")+ if stale:+ gaps.append(kalshi.stale_gap("Kalshi last-trade times", list(stale.values())))+ if counts.get("not_looked_up_cap"):+ gaps.append(+ f"last-trade time looked up for the first {len(targets)} of {len(live)} last-trade pairs "+ f"shown; {counts['not_looked_up_cap']} not looked up (cap {MAX_TRADE_LOOKUPS})"+ )+ if timed_out:+ gaps.append(+ f"last-trade lookups stopped at the {TRADE_LOOKUP_DEADLINE_SECONDS:g} s deadline; "+ f"{counts.get('not_looked_up_deadline', 0)} pair(s) not looked up"+ )+ if counts.get("not_looked_up_stale_listing"):+ gaps.append(+ f"last-trade time not looked up for {counts['not_looked_up_stale_listing']} pair(s) priced "+ "from an expired listing copy"+ )+ return gaps++_STALE_HISTORY = "price history served from expired cache"diff --git a/src/world_intel_mcp/tests/test_cli_prediction_stale.py b/src/world_intel_mcp/tests/test_cli_prediction_stale.pynew file mode 100644index 0000000..b36eb7e--- /dev/null+++ b/src/world_intel_mcp/tests/test_cli_prediction_stale.py@@ -0,0 +1,162 @@+"""CLI marking of expired copies and last-trade lookups (re-review round,+2026-10-17): ``predictions`` titles an expired Polymarket list as such+and prints its data gaps; ``kalshi`` notes a market priced from an+expired copy; ``prediction-compare`` says what each Kalshi last-trade+lookup found.++The ``predictions`` test runs the real source against a respx-mocked+Gamma outage with an expired cached list; the others use fakes+returning the documented shapes (pinned in test_prediction_cross_lookups.py).++Gaps / not covered: terminal widths below 200 columns.+"""++import asyncio as asyncio_mod++import httpx+import pytest+import respx+from click.testing import CliRunner++from world_intel_mcp import cli+from world_intel_mcp.fetcher import Fetcher+from world_intel_mcp.sources.prediction import fetch_prediction_markets+from world_intel_mcp.tests.test_sources_kalshi_stale import TEN_DAYS, expire+from world_intel_mcp.tests.test_sources_prediction import _MARKETS++GAMMA_ANY = r".*gamma-api\.polymarket\.com.*"+STALE_GAP = "served from expired cache after a failed live fetch"++[email protected](autouse=True)+def _cli_isolation(tmp_path, monkeypatch: pytest.MonkeyPatch) -> None:+ monkeypatch.setenv("WORLD_INTEL_CACHE_DB", str(tmp_path / "cli-cache.db"))+ monkeypatch.setenv("COLUMNS", "200")+ # rich pins the width when COLUMNS is set at import; None reads it per call.+ monkeypatch.setattr(cli.console, "_width", None)+ monkeypatch.setattr(cli, "_fetcher", None)++ async def _sleep(*args, **kwargs) -> None:+ return None++ monkeypatch.setattr(asyncio_mod, "sleep", _sleep)+++def _invoke(*args: str):+ return CliRunner().invoke(cli.main, list(args), catch_exceptions=False)+++def _fake(payload):+ async def _f(*args, **kwargs):+ return payload++ return _f+++# ---------------------------------------------------------------------------+# predictions (legacy Polymarket list)+# ---------------------------------------------------------------------------++async def test_legacy_list_from_expired_cache_is_stale_and_degraded(fetcher: Fetcher) -> None:+ expire(fetcher, "prediction:polymarket:20", _MARKETS[:2])+ respx.get(url__regex=GAMMA_ANY).mock(return_value=httpx.Response(500))++ result = await fetch_prediction_markets(fetcher)++ assert result["count"] == 2+ assert result["stale"] is True and result["degraded"] is True+ assert result["data_gaps"] == [STALE_GAP]+++def test_predictions_titles_an_expired_list_and_prints_its_gaps(+ fetcher: Fetcher, monkeypatch: pytest.MonkeyPatch+) -> None:+ expire(fetcher, "prediction:polymarket:20", _MARKETS[:2])+ respx.get(url__regex=GAMMA_ANY).mock(return_value=httpx.Response(500))+ monkeypatch.setattr(cli, "_fetcher", fetcher)++ result = _invoke("predictions")++ assert result.exit_code == 0+ assert f"Gap: {STALE_GAP}" in result.output+ assert "Prediction Markets (expired cached copy)" in result.output+++def test_predictions_live_list_has_the_plain_title_and_no_gap(+ fetcher: Fetcher, monkeypatch: pytest.MonkeyPatch+) -> None:+ respx.get(url__regex=GAMMA_ANY).mock(return_value=httpx.Response(200, json=_MARKETS[:2]))+ monkeypatch.setattr(cli, "_fetcher", fetcher)++ result = _invoke("predictions")++ assert result.exit_code == 0+ assert "Gap:" not in result.output and "expired" not in result.output+ assert "Prediction Markets" in result.output+++# ---------------------------------------------------------------------------+# kalshi and prediction-compare+# ---------------------------------------------------------------------------++def test_kalshi_notes_a_market_priced_from_an_expired_copy(monkeypatch: pytest.MonkeyPatch) -> None:+ payload = {+ "markets": [+ {"question": "Will the Federal Reserve Hike rates by 25bps at their October 2026 meeting?",+ "yes_side": "Hike 25bps", "probability": 0.135, "price_basis": "bid_ask_midpoint",+ "volume_24h": 61814.96, "volume": 4066283.05, "stale": True, "stale_age_seconds": TEN_DAYS},+ {"question": "Will the Federal Reserve Hike rates by 0bps at their October 2026 meeting?",+ "yes_side": "Fed maintains rate", "probability": 0.865, "price_basis": "bid_ask_midpoint",+ "volume_24h": 64959.15, "volume": 5635368.81, "stale": False, "stale_age_seconds": None},+ ],+ "count": 2, "scan": {}, "units": {}, "data_gaps": [],+ }+ monkeypatch.setattr(cli.kalshi_src, "fetch_kalshi_markets", _fake(payload))++ result = _invoke("kalshi", "--series", "KXFEDDECISION")++ assert result.exit_code == 0+ assert result.output.count("(expired copy, 240.0 h old)") == 1+++def _pair(kalshi_side: dict) -> dict:+ return {+ "match_confidence": 0.724, "flags": [], "comparable": False,+ "kalshi": dict({"question": "Will the Federal Reserve Cut rates by 25bps at their October 2026 meeting?"},+ **kalshi_side),+ "polymarket": {"question": "Will the Fed decrease interest rates by 25 bps after the October 2026 meeting?"},+ "kalshi_yes_probability": 0.01, "polymarket_yes_probability": 0.0035, "price_gap": 0.0065,+ }+++def test_compare_flags_say_what_each_last_trade_lookup_found(monkeypatch: pytest.MonkeyPatch) -> None:+ payload = {+ "matched": [+ _pair({"last_trade_lookup": "found", "last_trade_time": "2026-10-17T17:43:15Z", "last_trade_price": 0.03}),+ _pair({"last_trade_lookup": "found_in_expired_copy", "last_trade_time": "2026-10-07T21:54:56Z",+ "last_trade_price": 0.01}),+ _pair({"last_trade_lookup": "not_looked_up_cap", "last_trade_time": None}),+ ],+ "matched_count": 3, "kalshi_markets_compared": 3, "polymarket_markets_compared": 3,+ "unmatched_kalshi": 0, "unmatched_polymarket": 0, "min_confidence": 0.6, "data_gaps": [],+ }+ monkeypatch.setattr(cli.prediction_cross, "fetch_prediction_compare", _fake(payload))++ result = _invoke("prediction-compare")++ assert result.exit_code == 0+ # The Flags column wraps (max 40 wide), so the full notes are checked+ # on the helper and their pieces in the rendered table.+ assert [cli._last_trade_note(p["kalshi"]) for p in payload["matched"]] == [+ "kalshi last trade 2026-10-17T17:43:15Z at 3.0%",+ "kalshi last trade 2026-10-07T21:54:56Z at 1.0% (expired copy)",+ "kalshi last trade: not looked up cap",+ ]+ for piece in ("2026-10-17T17:43:15Z", "at 3.0%", "2026-10-07T21:54:56Z", "(expired copy)",+ "kalshi last trade: not looked up cap"):+ assert piece in result.output+ assert cli._last_trade_note({"question": "no lookup"}) is Nonediff --git a/src/world_intel_mcp/tests/test_prediction_cross_lookups.py b/src/world_intel_mcp/tests/test_prediction_cross_lookups.pynew file mode 100644index 0000000..73f5580--- /dev/null+++ b/src/world_intel_mcp/tests/test_prediction_cross_lookups.py@@ -0,0 +1,305 @@+"""Cross-venue compare: per-pair staleness, the last-trade lookup's+bounds and statuses, quotes on the outcome the Kalshi YES pays on,+Kalshi quote range checks, and matching off the event loop (re-review+round, 2026-10-17).++Same respx fixtures as test_prediction_cross.py and+test_prediction_cross_review.py. respx keeps one route per pattern, and+a later ``.mock()`` on the same pattern replaces the earlier response.++Gaps / not covered: a real outage on either venue; the 10 s deadline+itself (the deadline test shortens it); the collector's 45 s budget is+not exercised end to end here.+"""++import asyncio as asyncio_mod+import threading+import time++import httpx+import pytest+import respx++from world_intel_mcp.analysis import prediction_matching as pm+from world_intel_mcp.fetcher import Fetcher+from world_intel_mcp.sources import kalshi, prediction_cross+from world_intel_mcp.sources.prediction_cross import fetch_prediction_compare+from world_intel_mcp.tests.test_prediction_cross import GAMMA, POLY_OPEN, _gamma+from world_intel_mcp.tests.test_sources_kalshi import EVENTS, FED_MARKETS, _event, _m+from world_intel_mcp.tests.test_sources_kalshi_stale import TEN_DAYS, TRADES, expire++C25 = "KXFEDDECISION-26OCT-C25"+H25 = "KXFEDDECISION-26OCT-H25"+LAST_TRADE = "2026-10-17T17:43:15.314853Z"+STALE_FLAG = "price from a copy 240.0 h old, served from expired cache after a failed live fetch"++[email protected](autouse=True)+def _no_sleep(monkeypatch: pytest.MonkeyPatch) -> None:+ async def _sleep(*args, **kwargs) -> None:+ return None++ monkeypatch.setattr(asyncio_mod, "sleep", _sleep)+++def _fed_events(status: int = 200) -> None:+ respx.get(url__regex=EVENTS).mock(+ return_value=httpx.Response(status, json={"events": [_event(FED_MARKETS)], "cursor": ""})+ )+++def _gamma_open(status: int = 200, rows=POLY_OPEN) -> None:+ respx.get(url__regex=GAMMA).mock(return_value=httpx.Response(status, json=rows))+++def _trades(price: str = "0.0100", status: int = 200, trades=None):+ rows = [{"created_time": LAST_TRADE, "yes_price_dollars": price, "count_fp": "935.00"}] if trades is None else trades+ return respx.get(url__regex=TRADES).mock(return_value=httpx.Response(status, json={"trades": rows, "cursor": ""}))+++def _by_kalshi(out: dict) -> dict:+ return {p["kalshi"]["id"]: p for p in out["matched"]}+++# ---------------------------------------------------------------------------+# Staleness per pair+# ---------------------------------------------------------------------------++async def test_a_pair_priced_from_an_expired_kalshi_copy_is_flagged_not_comparable(fetcher: Fetcher) -> None:+ expire(fetcher, "kalshi:events:series:KXFEDDECISION:first", {"events": [_event(FED_MARKETS)], "cursor": ""})+ _fed_events(500)+ _gamma_open()+ trades = _trades()++ out = await fetch_prediction_compare(fetcher, series="KXFEDDECISION")++ h25 = _by_kalshi(out)[H25]+ assert h25["kalshi"]["stale"] is True+ assert TEN_DAYS <= h25["kalshi"]["stale_age_seconds"] < TEN_DAYS + 60+ assert h25["flags"] == [f"kalshi {STALE_FLAG}"] and h25["comparable"] is False+ assert out["comparable_count"] == 0+ # The trade lookup is skipped for a side priced from an expired listing:+ # a live trade time beside an old price would read as a fresh price.+ c25 = _by_kalshi(out)[C25]+ assert c25["kalshi"]["last_trade_lookup"] == "not_looked_up_stale_listing"+ assert c25["kalshi"]["last_trade_time"] is None+ assert trades.calls.call_count == 0+ assert "kalshi: last-trade time not looked up for 1 pair(s) priced from an expired listing copy" in out["data_gaps"]+++async def test_a_pair_priced_from_an_expired_polymarket_copy_is_flagged_and_ranked_after_live(fetcher: Fetcher) -> None:+ _fed_events()+ expire(fetcher, "prediction:polymarket:open:5:0", POLY_OPEN)+ _gamma_open(500)+ _trades()++ out = await fetch_prediction_compare(fetcher, series="KXFEDDECISION", polymarket_limit=5)++ h25 = _by_kalshi(out)[H25]+ assert h25["polymarket"]["stale"] is True+ assert h25["flags"] == [f"polymarket {STALE_FLAG}"] and h25["comparable"] is False+ assert out["comparable_count"] == 0+ assert "polymarket: the oldest open-markets copy is 240.0 h old" in out["data_gaps"]+++# ---------------------------------------------------------------------------+# Last-trade lookups: statuses, trade price, cap and deadline+# ---------------------------------------------------------------------------++async def test_a_market_with_no_trade_says_no_trade(fetcher: Fetcher) -> None:+ _fed_events()+ _gamma_open()+ _trades(trades=[])++ out = await fetch_prediction_compare(fetcher, series="KXFEDDECISION")++ side = _by_kalshi(out)[C25]["kalshi"]+ assert (side["last_trade_lookup"], side["last_trade_time"]) == ("no_trade", None)+ assert out["last_trade_lookup"]["no_trade"] # the statuses are explained in the response+ assert "last_trade_lookup" not in _by_kalshi(out)[H25]["kalshi"] # a midpoint needs no lookup+++async def test_a_failed_lookup_says_lookup_failed(fetcher: Fetcher) -> None:+ _fed_events()+ _gamma_open()+ _trades(status=500)++ out = await fetch_prediction_compare(fetcher, series="KXFEDDECISION")++ side = _by_kalshi(out)[C25]["kalshi"]+ assert (side["last_trade_lookup"], side["last_trade_time"]) == ("lookup_failed", None)+++async def test_a_trade_price_that_differs_from_the_listing_is_flagged(fetcher: Fetcher) -> None:+ _fed_events()+ _gamma_open()+ _trades(price="0.0300")++ out = await fetch_prediction_compare(fetcher, series="KXFEDDECISION")++ c25 = _by_kalshi(out)[C25]+ assert (c25["kalshi"]["last_trade_time"], c25["kalshi"]["last_trade_price"]) == (LAST_TRADE, 0.03)+ assert c25["kalshi"]["last_trade_lookup"] == "found"+ assert "kalshi last trade at 0.03 (looked up separately) differs from the listing's last price 0.01" in c25["flags"]+ assert c25["comparable"] is False+++async def test_a_trade_from_an_expired_copy_says_so_with_its_age(fetcher: Fetcher) -> None:+ _fed_events()+ _gamma_open()+ expire(fetcher, f"kalshi:trades:last:{C25}", {"trades": [+ {"created_time": LAST_TRADE, "yes_price_dollars": "0.0100", "count_fp": "935.00"},+ ], "cursor": ""})+ _trades(status=500)++ out = await fetch_prediction_compare(fetcher, series="KXFEDDECISION")++ side = _by_kalshi(out)[C25]["kalshi"]+ assert (side["last_trade_lookup"], side["last_trade_time"]) == ("found_in_expired_copy", LAST_TRADE)+ assert TEN_DAYS <= side["last_trade_age_seconds"] < TEN_DAYS + 60+++def _ladder(n: int):+ """``n`` same-event pairs, each Kalshi side priced from a last trade."""+ markets, rows = [], []+ for i in range(n):+ level = f"{100 + i},000"+ k = _m(f"KXBTC-26OCT-B{100 + i}", f"Will Bitcoin be above ${level} at the end of October 2026?", "",+ "0.0000", "0.0300", "0.0200", "1000.00", str(100 - i))+ k["event_ticker"] = "KXBTC-26OCT"+ markets.append(k)+ rows.append(_gamma(f"77{i:02d}", f"Will Bitcoin be above ${level} at the end of October 2026?",+ '["0.02", "0.98"]', 0.01, 0.03))+ event = _event(markets, ticker="KXBTC-26OCT", series="KXBTC", title="Bitcoin at the end of October 2026?")+ return event, rows+++async def test_last_trade_lookups_stop_at_the_cap_and_say_how_many_were_skipped(fetcher: Fetcher) -> None:+ event, rows = _ladder(12)+ respx.get(url__regex=EVENTS).mock(return_value=httpx.Response(200, json={"events": [event], "cursor": ""}))+ _gamma_open(rows=rows)+ trades = _trades(price="0.0200")++ out = await fetch_prediction_compare(fetcher, series="KXBTC")++ assert out["matched_count"] == 12+ assert trades.calls.call_count == prediction_cross.MAX_TRADE_LOOKUPS == 10+ statuses = [p["kalshi"]["last_trade_lookup"] for p in out["matched"]]+ assert statuses == ["found"] * 10 + ["not_looked_up_cap"] * 2+ assert "kalshi: last-trade time looked up for the first 10 of 12 last-trade pairs shown; 2 not looked up (cap 10)" in (+ out["data_gaps"]+ )+++async def test_a_trades_endpoint_that_never_answers_is_cut_at_the_deadline(+ fetcher: Fetcher, monkeypatch: pytest.MonkeyPatch+) -> None:+ monkeypatch.setattr(prediction_cross, "TRADE_LOOKUP_DEADLINE_SECONDS", 0.2)+ _fed_events()+ _gamma_open()++ async def _hang(request):+ await asyncio_mod.Event().wait() # asyncio.sleep is patched out here++ respx.get(url__regex=TRADES).mock(side_effect=_hang)++ started = time.monotonic()+ # The outer bound turns a missing deadline into a failure, not a hung run.+ out = await asyncio_mod.wait_for(fetch_prediction_compare(fetcher, series="KXFEDDECISION"), 5)+ elapsed = time.monotonic() - started++ assert elapsed < 5+ side = _by_kalshi(out)[C25]["kalshi"]+ assert (side["last_trade_lookup"], side["last_trade_time"]) == ("not_looked_up_deadline", None)+ assert "kalshi: last-trade lookups stopped at the 0.2 s deadline; 1 pair(s) not looked up" in out["data_gaps"]+ assert out["matched_count"] == 2 and "degraded" not in out # supplementary, the comparison is complete+++# ---------------------------------------------------------------------------+# Matching runs off the event loop+# ---------------------------------------------------------------------------++async def test_matching_runs_in_a_worker_thread(fetcher: Fetcher, monkeypatch: pytest.MonkeyPatch) -> None:+ _fed_events()+ _gamma_open()+ _trades()+ seen: list[int] = []+ real = prediction_cross.match_markets++ def _spy(*args, **kwargs):+ seen.append(threading.get_ident())+ return real(*args, **kwargs)++ monkeypatch.setattr(prediction_cross, "match_markets", _spy)++ out = await fetch_prediction_compare(fetcher, series="KXFEDDECISION")++ assert out["matched_count"] == 2+ assert len(seen) == 1 and seen[0] != threading.get_ident()+++# ---------------------------------------------------------------------------+# Pair briefs: the quoted outcome, quote ranges, no-price flags+# ---------------------------------------------------------------------------++def test_an_inverted_pair_quotes_the_outcome_the_kalshi_yes_pays_on() -> None:+ from world_intel_mcp.tests.test_prediction_matching import NOW, _k, _p++ k = _k("Will Zhejiang Lions win the Zhejiang Lions vs Shanxi Loongs game?", "Shanxi Loongs",+ end="2026-10-17T13:00:00Z", prob=0.40, context="Zhejiang Lions vs Shanxi Loongs")+ p = _p("Zhejiang Lions vs. Shanxi Loongs", outcomes=("Zhejiang Lions", "Shanxi Loongs"),+ end="2026-10-17T14:00:00Z", prob=0.62)+ p.update(yes_bid=0.61, yes_ask=0.63, last_price=0.6)++ (pair,) = pm.match_markets([k], [p], min_confidence=0.5, now=NOW)["matched"]++ assert pair["side"] == "inverted"+ brief = pair["polymarket"]+ assert brief["quoted_outcome"] == "Shanxi Loongs"+ assert (brief["probability"], brief["yes_bid"], brief["yes_ask"], brief["last_price"]) == (0.38, 0.37, 0.39, 0.4)+++def test_kalshi_quotes_outside_0_to_1_are_left_null_and_the_pair_says_why() -> None:+ raw = _m("KXFEDDECISION-26OCT-X", "Will the Federal Reserve Hike rates by 25bps at their October 2026 meeting?",+ "Hike 25bps", "45", "46", "45", "10.00", "1.00")+ market = kalshi.normalize_market(raw, _event([raw]))+ assert (market["yes_bid"], market["yes_ask"], market["last_price"]) == (None, None, None)+ assert (market["probability"], market["price_basis"]) == (None, "no_price")+ assert kalshi.bad_quote_fields(raw) == ["yes_bid_dollars", "yes_ask_dollars", "last_price_dollars"]+ assert kalshi.bad_quote_fields(FED_MARKETS[0]) == []++ flags = pm._quote_flags("kalshi", market)+ assert flags == ["kalshi price unavailable (basis: no_price)"]+++async def test_kalshi_listing_counts_quotes_that_are_not_prices(fetcher: Fetcher) -> None:+ bad = _m("KXFEDDECISION-26OCT-X", "Will the Federal Reserve Hike rates by 75bps at their October 2026 meeting?",+ "Hike 75bps", "45", "46", "45", "10.00", "1.00")+ respx.get(url__regex=EVENTS).mock(+ return_value=httpx.Response(200, json={"events": [_event(FED_MARKETS + [bad])], "cursor": ""})+ )++ out = await kalshi.fetch_kalshi_open_markets(fetcher, series="KXFEDDECISION")++ assert ("1 market(s) with a quote field that is not a price from 0 to 1 (left null; the price falls back or "+ "is missing)") in out["data_gaps"]diff --git a/src/world_intel_mcp/tests/test_prediction_cross_review.py b/src/world_intel_mcp/tests/test_prediction_cross_review.pyindex c22e2bf..333884a 100644--- a/src/world_intel_mcp/tests/test_prediction_cross_review.py+++ b/src/world_intel_mcp/tests/test_prediction_cross_review.py@@ -8,8 +8,9 @@ earlier response, so each test sets up the shared venues first andthen overrides the one endpoint it fails.Gaps / not covered: a real outage on either venue; ordering of the live-trades endpoint (the lookup asks for one trade and does not depend on-it).+trades endpoint. The lookup asks for one trade (``limit=1``), so it+relies on the endpoint listing newest first, which was observed live on+2026-10-17 (two samples) and is not pinned by any test here."""import asyncio as asyncio_moddiff --git a/src/world_intel_mcp/tests/test_prediction_matching_gates.py b/src/world_intel_mcp/tests/test_prediction_matching_gates.pyindex c046302..254b9f7 100644--- a/src/world_intel_mcp/tests/test_prediction_matching_gates.py+++ b/src/world_intel_mcp/tests/test_prediction_matching_gates.py@@ -16,7 +16,8 @@ date tolerance is fixed.Gaps / not covered: wording beyond these cases; the candidate-paircompleteness check is randomized over a small vocabulary, not-exhaustive (the prefix bound itself was checked with z3).+exhaustive, and the prefix bound is not machine-proved (the total order+it relies on is tested in test_prediction_matching_late_gates.py)."""import randomdiff --git a/src/world_intel_mcp/tests/test_prediction_matching_late_gates.py b/src/world_intel_mcp/tests/test_prediction_matching_late_gates.pynew file mode 100644index 0000000..1029718--- /dev/null+++ b/src/world_intel_mcp/tests/test_prediction_matching_late_gates.py@@ -0,0 +1,271 @@+"""Gates that reject a pair from the two titles alone (re-review round,+2026-10-17): days of the same month, months, clock times, quantity+units and kinds, quantity order, opposite wording, rate actions worded+as verbs, listed and distinguishing names, negation words, dash minus+signs, number words, and a plain four-digit amount after a comparator.++One rejection test per class, each with the reason the gate gives. The+same-event cases each class must keep matching sit beside them: a day+on one venue only, a clock time on one venue only, 25 bps against+0.25%, "raised" against "increased", "hold" outside a rate question,+"between" against at least/at most, and "Fed" against "Federal+Reserve" (the live pairs keep matching in test_prediction_cross.py).++Every expected reason and score below was computed with the module+before it was written here (scratch probes); "now" is pinned so the+date tolerance is fixed.++Gaps / not covered: wording beyond these cases. Known false negatives+(documented in the module): "before Jan 1, 2027" against "in 2026",+"first" against "1st", "keep rates" against "hold rates". Names are+read from capitalization and short lists, so an unlisted lower-case+qualifier still costs only Dice.+"""++from datetime import datetime, timezone++import pytest++from world_intel_mcp.analysis import prediction_matching as pm++NOW = datetime(2026, 10, 17, 16, 0, tzinfo=timezone.utc)+END = "2026-12-10T19:00:00Z"+++def _k(question, yes="", end=END):+ return {"venue": "kalshi", "id": "K", "question": question, "context": "", "yes_side": yes,+ "outcomes": ["Yes", "No"], "end_time": end, "probability": 0.5}+++def _p(question, end=END):+ return {"venue": "polymarket", "id": "P", "question": question, "context": "", "yes_side": "",+ "outcomes": ["Yes", "No"], "end_time": end, "probability": 0.5}+++# ---------------------------------------------------------------------------+# Different events: rejected, with the gate that caught them+# ---------------------------------------------------------------------------++# case: (Kalshi question, Kalshi YES side, Kalshi end, Polymarket question, Polymarket end, reason)+REJECTED = {+ "days of the same month": (+ "Will Bitcoin be above $100,000 on Oct 20, 2026?", "", "2026-10-20T21:00:00Z",+ "Will Bitcoin be above $100,000 on October 21, 2026?", "2026-10-21T16:00:00Z",+ "days differ in october: Kalshi ['20'] vs Polymarket ['21']",+ ),+ "a deadline ladder": (+ "Will the government shutdown end before Jan 1, 2027?", "", "2027-01-01T00:00:00Z",+ "Will the government shutdown end before January 8, 2027?", "2027-01-08T00:00:00Z",+ "days differ in january: Kalshi ['1'] vs Polymarket ['8']",+ ),+ "a deadline ladder with the day on the Kalshi YES side": (+ "When will the government shutdown end?", "Before Jan 1, 2027", "2027-01-01T00:00:00Z",+ "Will the government shutdown end before January 8, 2027?", "2027-01-08T00:00:00Z",+ "days differ in january: Kalshi ['1'] vs Polymarket ['8']",+ ),+ "months": (+ "Will the US enter a recession by March 2028?", "", "2028-03-31T00:00:00Z",+ "Will the US enter a recession by April 2028?", "2028-04-30T00:00:00Z",+ "months differ: Kalshi ['march'] vs Polymarket ['april']",+ ),+ "clock times": (+ "Will Bitcoin be above $100,000 at 5pm ET on Oct 20, 2026?", "", "2026-10-20T21:00:00Z",+ "Will Bitcoin be above $100,000 at 12pm ET on Oct 20, 2026?", "2026-10-20T16:00:00Z",+ "clock times differ: Kalshi ['17:00'] vs Polymarket ['12:00']",+ ),+ "dollars vs percent": (+ "Will US GDP grow by $500 in 2026?", "", END, "Will US GDP grow by 500% in 2026?", END,+ "quantity units differ for 500: Kalshi ['$'] vs Polymarket ['%']",+ ),+ "basis points vs percent of the same number": (+ "Will the Fed cut rates by 50 bps in December 2026?", "", END,+ "Will the Fed cut rates by 50% in December 2026?", END,+ "quantity units differ for 50: Kalshi ['bps'] vs Polymarket ['%']",+ ),+ "to a level vs by a change": (+ "Will the Fed cut rates to 3.5% in December 2026?", "", END,+ "Will the Fed cut rates by 3.5% in December 2026?", END,+ "quantity kinds ('to' a level vs 'by' a change) differ for 3.5: Kalshi ['to'] vs Polymarket ['by']",+ ),+ "quantity order": (+ "Will Brazil beat Argentina 2-1 in the 2026 final?", "", END,+ "Will Brazil beat Argentina 1-2 in the 2026 final?", END,+ "quantities in a different order: Kalshi ['2', '1'] vs Polymarket ['1', '2']",+ ),+ "rise vs fall": (+ "Will oil prices rise in December 2026?", "", END, "Will oil prices fall in December 2026?", END,+ "opposite wording: Kalshi ['up'] vs Polymarket ['down']",+ ),+ "positive vs negative": (+ "Will Q4 GDP growth be positive in 2026?", "", END, "Will Q4 GDP growth be negative in 2026?", END,+ "opposite wording: Kalshi ['positive'] vs Polymarket ['negative']",+ ),+ "before vs after": (+ "Will Musk leave Tesla before the 2026 election?", "", END,+ "Will Musk leave Tesla after the 2026 election?", END,+ "opposite wording: Kalshi ['before'] vs Polymarket ['after']",+ ),+ "tighten vs ease": (+ "Will the ECB tighten policy in December 2026?", "", END, "Will the ECB ease policy in December 2026?", END,+ "actions differ: Kalshi ['raise'] vs Polymarket ['cut']",+ ),+ "boost vs reduce": (+ "Will the BoE boost rates in December 2026?", "", END, "Will the BoE reduce rates in December 2026?", END,+ "actions differ: Kalshi ['raise'] vs Polymarket ['cut']",+ ),+ "increase vs decrease": (+ "Will the Fed increase rates in December 2026?", "", END,+ "Will the Fed decrease rates in December 2026?", END,+ "actions differ: Kalshi ['raise'] vs Polymarket ['cut']",+ ),+ "chambers": (+ "Will Democrats win the House in 2026?", "", END, "Will Democrats win the Senate in 2026?", END,+ "a listed name on one venue only: Kalshi ['house'] vs Polymarket ['senate']",+ ),+ "countries": (+ "Will the US impose tariffs on China in 2026?", "", END,+ "Will the US impose tariffs on Canada in 2026?", END,+ "a listed name on one venue only: Kalshi ['china'] vs Polymarket ['canada']",+ ),+ "central banks named in full": (+ "Will the Bank of England cut rates in December 2026?", "", END,+ "Will the Bank of Japan cut rates in December 2026?", END,+ "distinguishing names differ: Kalshi ['boe'] vs Polymarket ['boj']",+ ),+ "people": (+ "Will Trump pardon Navarro in 2026?", "", END, "Will Trump pardon Bannon in 2026?", END,+ "distinguishing names differ: Kalshi ['navarro'] vs Polymarket ['bannon']",+ ),+ "Bitcoin vs Bitcoin Cash": (+ "Will Bitcoin hit $150k in 2026?", "", END, "Will Bitcoin Cash hit $150k in 2026?", END,+ "a listed name on one venue only: Kalshi [] vs Polymarket ['cash']",+ ),+ "headline vs core": (+ "Will CPI be above 3% in November 2026?", "", END, "Will core CPI be above 3% in November 2026?", END,+ "a listed name on one venue only: Kalshi [] vs Polymarket ['core']",+ ),+ "won't": (+ "Will Bitcoin hit $150k in 2026?", "", END, "Bitcoin won't hit $150k in 2026?", END,+ "negation differs (one title says not/no/never/without, the other does not)",+ ),+ "fail to": (+ "Will Congress pass the budget in 2026?", "", END, "Will Congress fail to pass the budget in 2026?", END,+ "negation differs (one title says not/no/never/without, the other does not)",+ ),+ "an en dash minus sign": (+ "Will Q4 GDP growth be below –0.5%?", "", END, "Will Q4 GDP growth be below 0.5%?", END,+ "quantities differ: Kalshi ['4', '<-0.5'] vs Polymarket ['4', '<0.5']",+ ),+ "number words": (+ "Will the Fed cut rates three times in 2026?", "", END, "Will the Fed cut rates four times in 2026?", END,+ "quantities differ: Kalshi ['3'] vs Polymarket ['4']",+ ),+ "a plain four-digit amount after a comparator (above)": (+ "Will the S&P 500 be above 2050 in 2026?", "", END, "Will the S&P 500 be above 2100 in 2026?", END,+ "quantities differ: Kalshi ['500', '>2050'] vs Polymarket ['500', '>2100']",+ ),+ "a plain four-digit amount after a comparator (at least)": (+ "Will gold be at least 2000 in 2026?", "", END, "Will gold be at least 1950 in 2026?", END,+ "quantities differ: Kalshi ['>=2000'] vs Polymarket ['>=1950']",+ ),+}++[email protected]("case", sorted(REJECTED))+def test_titles_naming_different_events_are_rejected(case: str) -> None:+ kalshi_q, yes, kalshi_end, poly_q, poly_end, reason = REJECTED[case]+ result = pm.score_pair(_k(kalshi_q, yes, kalshi_end), _p(poly_q, poly_end), now=NOW)+ assert result["rejected"] is True and result["confidence"] == 0.0+ assert result["reasons"][-1] == reason+++def test_a_day_rejection_also_keeps_the_pair_out_of_matches() -> None:+ out = pm.match_markets(+ [_k("Will Bitcoin be above $100,000 on Oct 20, 2026?", end="2026-10-20T21:00:00Z")],+ [_p("Will Bitcoin be above $100,000 on October 21, 2026?", end="2026-10-21T16:00:00Z")],+ now=NOW,+ )+ assert out["matched"] == [] and out["unmatched_kalshi"] == 1 and out["unmatched_polymarket"] == 1+++# ---------------------------------------------------------------------------+# Same events: still matched+# ---------------------------------------------------------------------------++# case: (Kalshi question, Kalshi end, Polymarket question, Polymarket end, confidence, a reason it carries)+MATCHED = {+ "a day on one venue only": (+ "Will the Fed cut rates on Oct 28, 2026?", "2026-10-28T18:00:00Z",+ "Will the Fed cut rates in October 2026?", "2026-10-28T18:00:00Z",+ 1.0, "actions agree ['cut']",+ ),+ "a clock time on one venue only": (+ "Will Bitcoin be above $100,000 at 5pm ET on Oct 20, 2026?", "2026-10-20T21:00:00Z",+ "Will Bitcoin be above $100,000 on Oct 20, 2026?", "2026-10-20T21:00:00Z",+ 1.0, "quantities agree ['>100000']",+ ),+ "25 bps vs 0.25%": (+ "Will the Fed cut rates by 25 bps in December 2026?", END,+ "Will the Fed cut rates by 0.25% in December 2026?", END,+ 0.889, "quantities agree once bps are read as percent: Kalshi ['25'] vs Polymarket ['0.25']",+ ),+ "raised vs increased": (+ "Will the Fed have raised rates in December 2026?", END,+ "Will the Fed have increased rates in December 2026?", END,+ 1.0, "actions agree ['raise']",+ ),+ "hold outside a rate question is not an action": (+ "Will Iran hold a presidential election in 2026?", END,+ "Will Iran have a presidential election in 2026?", END,+ 0.857, "both venues price YES on the question as worded",+ ),+ "between vs at least and at most": (+ "Will CPI be between 3% and 4% in November 2026?", END,+ "Will CPI be at least 3% and at most 4% in November 2026?", END,+ 1.0, "both venues price YES on the question as worded",+ ),+ "Fed vs Federal Reserve": (+ "Will the Fed hike rates in December 2026?", END,+ "Will the Federal Reserve hike rates in December 2026?", END,+ 0.667, "actions agree ['raise']",+ ),+}++[email protected]("case", sorted(MATCHED))+def test_same_events_still_match(case: str) -> None:+ kalshi_q, kalshi_end, poly_q, poly_end, confidence, reason = MATCHED[case]+ result = pm.score_pair(_k(kalshi_q, end=kalshi_end), _p(poly_q, end=poly_end), now=NOW)+ assert result["rejected"] is False and result["confidence"] == confidence+ assert confidence >= pm.DEFAULT_MIN_CONFIDENCE+ assert reason in result["reasons"]+++def test_hold_in_a_rate_question_is_still_an_action() -> None:+ result = pm.score_pair(+ _k("Will the Fed hold rates in December 2026?"), _p("Will the Fed raise rates in December 2026?"), now=NOW+ )+ assert result["reasons"][-1] == "actions differ: Kalshi ['hold'] vs Polymarket ['raise']"+++def test_clock_times_read_twelve_and_twenty_four_hour_forms() -> None:+ assert pm.clock_times("5pm ET on Oct 20") == {"17:00"}+ assert pm.clock_times("at 17:00 UTC") == {"17:00"}+ assert pm.clock_times("12:30 pm") == {"12:30"}+ assert pm.clock_times("noon") == set()+++# ---------------------------------------------------------------------------+# Candidate blocking: the total order every market shares+# ---------------------------------------------------------------------------++def test_prefix_breaks_rarity_ties_on_the_word_itself() -> None:+ """Prefix filtering finds every pair only if all markets order their+ words the same way; with equal document frequencies that order is+ alphabetical, never the set's iteration order."""+ words = set("alpha bravo charlie delta echo foxtrot golf hotel india juliet kilo lima".split())+ tied = {w: 3 for w in words}+ # 12 words at 0.75: need = ceil(0.75 * 12 / 1.25) = 8, so the prefix keeps 12 - 8 + 1 = 5.+ assert pm._prefix(words, tied, 0.75) == ["alpha", "bravo", "charlie", "delta", "echo"]+ rarer = dict(tied, lima=1, alpha=5)+ assert pm._prefix(words, rarer, 0.75) == ["lima", "bravo", "charlie", "delta", "echo"]diff --git a/src/world_intel_mcp/tools/prediction_markets.py b/src/world_intel_mcp/tools/prediction_markets.pyindex f4b97f1..a45d10d 100644--- a/src/world_intel_mcp/tools/prediction_markets.py+++ b/src/world_intel_mcp/tools/prediction_markets.py@@ -26,7 +26,8 @@ TOOLS: list[Tool] = ["$1 contract; raw bid/ask/last kept). Filter by category (e.g. 'Economics', 'Politics', ""'Elections', 'Financials', 'Climate and Weather') or series ticker (e.g. 'KXFED'). The ""response states which events were scanned and whether the scan was exhaustive; data "- "served from expired cache after a failed fetch is marked stale and degraded."+ "served from expired cache after a failed fetch is marked stale and degraded, and each "+ "market from such a copy carries stale: true and its age."),inputSchema={"type": "object",@@ -70,13 +71,16 @@ TOOLS: list[Tool] = [name="intel_prediction_compare",description=("The same event on Kalshi and Polymarket side by side: deterministic matching on normalized "- "titles, quantities, years, end dates, negation, rate actions (cut/raise/hold), directions "- "(above/below) and outcome side, with a match_confidence (an ordinal score for ranking "+ "titles, quantities (value, unit, to/by, order), years, months, days, clock times, end "+ "dates, negation, rate actions (cut/raise/hold), directions (above/below), opposite words "+ "(rise/fall, before/after, ...), distinguishing names (House/Senate, Fed/BoJ, Bitcoin/"+ "Bitcoin Cash) and outcome side, with a match_confidence (an ordinal score for ranking ""pairs, not a probability) and the reasons for each pair. Below min_confidence a pair is ""not a match. Returns matched pairs with the price gap, each side's price basis, spread "- "and last-trade time (a pair is flagged and ranked after comparable ones when either side "- "is not a two-sided quote midpoint or its spread is wide), the closest rejected pairs with "- "why, and unmatched counts per venue."+ "and last-trade time with its lookup status (a pair is flagged and ranked after comparable "+ "ones when either side is not a live two-sided quote midpoint, comes from an expired "+ "cached copy or its spread is wide), the closest rejected pairs with why, and unmatched "+ "counts per venue."),inputSchema={"type": "object",