Skip to content

160-tool MCP server for real-time global intelligence from free public APIs: markets, liquidity, prediction markets, SEC filings, conflict, military, aircraft-cohort early warning, migration and border flows, cyber, climate, news and more. Dated observation history, 29 research prompts, cited briefs, geofences, SSE dashboard. MIT, no paid API keys.

Prediction markets: re-review fixes (event gates, per-pair staleness, bounded last-trade lookups)

Phoenix Agent (mac-studio)ca9708047d08
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 2d699ca..2b1b784 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -76,6 +76,29 @@
characters. `intel_prediction_markets` names how many markets it left
out for an unusable price.
+### Fixed: prediction-market re-review findings
+- Matching rejects a pair when both titles name different days of the
+ same month, no shared month, or different clock times; when a quantity
+ differs in unit ($, %, bps) or kind (to a level, by a change) or the
+ same numbers come in a different order; on opposite wording (rise or
+ fall, positive or negative, before or after); and when a listed name
+ (chamber, party, country, a qualifier such as core or cash) or another
+ capitalized name appears on one venue only. More negation words
+ (won't, fail to), rate verbs (tighten, ease, boost, reduce and their
+ forms), number words, dash minus signs, "between" ranges and bps read
+ as hundredths of a percent. "Hold" counts as a rate action only in a
+ rate question.
+- `intel_prediction_compare`: a side priced from an expired copy carries
+ `stale` and its age and its pair is flagged and not comparable. The
+ Kalshi last-trade lookup reads the trade's price with its time, flags a
+ pair whose trade price differs from the listing, skips sides priced from
+ an expired listing, stops after 10 lookups or 10 s, and says per side
+ what a null time means (`last_trade_lookup`). Matching runs in a worker
+ thread. A pair with no price is flagged; Kalshi quotes outside 0 to 1
+ are left null and counted.
+- `intel_prediction_markets` sets `degraded` with `stale`; `intel
+ predictions` prints its data gaps and titles an expired list as such.
+
## 0.11.0 - 2026-10-12
Cross-border migration (MCP tools, CLI, collector and a dashboard
diff --git a/ROADMAP.md b/ROADMAP.md
index 16c5c50..7c41126 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -578,7 +578,7 @@ dashboard.
| 1 | Hostility timeline | Per country pair or country, daily counts of threat / sanction / force / criticism coverage plus tone; sortable by most covered, most hostile, biggest change | :red_circle: |
| 1 | Story breadth | News clusters carry a distinct-outlet count; a top-stories view ranks by how many outlets carry a story | :red_circle: |
| 1 | Liquidity and flows | Fed balance sheet, Treasury General Account, reverse repo and net liquidity (FRED keyless CSV), money supply (US M2, euro area M1/M2/M3, never summed), ECB, ICI fund flows and ETF net issuance, FINRA margin statistics, BLS series; amounts kept separate from flows | :red_circle: |
-| 1 | Kalshi + cross-venue markets | Kalshi public market data next to Polymarket; same-event comparison with an explicit match confidence; settled markets scored against pre-settlement prices (Brier) | :white_check_mark: Unreleased: `intel_kalshi_markets`, `intel_prediction_compare`, `intel_prediction_settled` (`sources/kalshi.py`, `sources/prediction_cross.py`, `analysis/prediction_matching.py`), CLI and 2 collector sources. Kalshi endpoints measured 2026-10-17, all HTTP 200 without an account: `/events` (nested markets), `/series` (`include_volume`), `/markets?status=settled`, `/markets/candlesticks` (batch, capped at 10,000 candles per call), `/historical/cutoff`. The `/events` `category` parameter is ignored by the API, so categories resolve through `/series?category=`, whose filter is loose too (7 of the 869 series it returned for Economics were other categories), so series are matched exactly; an unknown category answers `{"series": null}` and is reported as a gap, not an outage. The exchange-wide settled listing is intraday sports (1,000 rows covered about 40 minutes; none open 24h), so settled samples default to the Economics category's top series. Polymarket Gamma caps a page at 100 rows. Matching is deliberately conservative: live 2026-10-17 it matched the Fed 25 bps hike/cut pairs for October and December and rejected 25 vs 50+ bps, 2026 vs 2027, and the Fed-hold pair ("no change" vs "Hike 0bps": a known false negative). Team and person aliases are not resolved, so most sports pairs stay unmatched. Kalshi requests are spaced 0.2 s apart in-process: the two collector sources scanning the same series at once drew HTTP 429, and at 0.1 s a 26-request category scan still drew one (none at 0.2 s in one live run). Settled samples are the most recently settled markets and cluster in time (one football match supplied 11 of 25 Polymarket markets in a live run), so a Brier score describes its sample, not a venue's skill. Every number a caller passes has a range stated in the input schema (at most 25 Kalshi series or pages per scan), so one call cannot fan out into unbounded upstream requests. Review round (2026-10-17): expired-cache serves are marked on every Kalshi read and on Polymarket's settled listing and histories, and a stale history is not scored; matching adds rate-action and direction gates, keeps signs and ordinals, reads a $ amount as an amount, and its candidate blocking is complete against a brute-force scan in tests; compare carries each side's price basis, spread and Kalshi last-trade time (public `/markets/trades`, HTTP 200 without an account, measured 2026-10-17) and ranks flagged pairs after comparable ones; Polymarket is scored against the earlier of endDate and closedTime (on 20 closed markets measured 2026-10-17, closedTime came 2.7 to 5.5 h after endDate for football and 164 h before it for one early resolution, and umaEndDate equalled closedTime in all 20, so no outcome-known time is published); Kalshi pacing applies only on a cache miss. The shared `polymarket` circuit breaker (three failed history calls pause all Polymarket reads for 300 s) is documented, not changed |
+| 1 | Kalshi + cross-venue markets | Kalshi public market data next to Polymarket; same-event comparison with an explicit match confidence; settled markets scored against pre-settlement prices (Brier) | :white_check_mark: Unreleased: `intel_kalshi_markets`, `intel_prediction_compare`, `intel_prediction_settled` (`sources/kalshi.py`, `sources/prediction_cross.py`, `analysis/prediction_matching.py`), CLI and 2 collector sources. Kalshi endpoints measured 2026-10-17, all HTTP 200 without an account: `/events` (nested markets), `/series` (`include_volume`), `/markets?status=settled`, `/markets/candlesticks` (batch, capped at 10,000 candles per call), `/historical/cutoff`. The `/events` `category` parameter is ignored by the API, so categories resolve through `/series?category=`, whose filter is loose too (7 of the 869 series it returned for Economics were other categories), so series are matched exactly; an unknown category answers `{"series": null}` and is reported as a gap, not an outage. The exchange-wide settled listing is intraday sports (1,000 rows covered about 40 minutes; none open 24h), so settled samples default to the Economics category's top series. Polymarket Gamma caps a page at 100 rows. Matching is deliberately conservative: live 2026-10-17 it matched the Fed 25 bps hike/cut pairs for October and December and rejected 25 vs 50+ bps, 2026 vs 2027, and the Fed-hold pair ("no change" vs "Hike 0bps": a known false negative). Team and person aliases are not resolved, so most sports pairs stay unmatched. Kalshi requests are spaced 0.2 s apart in-process: the two collector sources scanning the same series at once drew HTTP 429, and at 0.1 s a 26-request category scan still drew one (none at 0.2 s in one live run). Settled samples are the most recently settled markets and cluster in time (one football match supplied 11 of 25 Polymarket markets in a live run), so a Brier score describes its sample, not a venue's skill. Every number a caller passes has a range stated in the input schema (at most 25 Kalshi series or pages per scan), so one call cannot fan out into unbounded upstream requests. Review round (2026-10-17): expired-cache serves are marked on every Kalshi read and on Polymarket's settled listing and histories, and a stale history is not scored; matching adds rate-action and direction gates, keeps signs and ordinals, reads a $ amount as an amount, and its candidate blocking is complete against a brute-force scan in tests; compare carries each side's price basis, spread and Kalshi last-trade time (public `/markets/trades`, HTTP 200 without an account, measured 2026-10-17) and ranks flagged pairs after comparable ones; Polymarket is scored against the earlier of endDate and closedTime (on 20 closed markets measured 2026-10-17, closedTime came 2.7 to 5.5 h after endDate for football and 164 h before it for one early resolution, and umaEndDate equalled closedTime in all 20, so no outcome-known time is published); Kalshi pacing applies only on a cache miss. The shared `polymarket` circuit breaker (three failed history calls pause all Polymarket reads for 300 s) is documented, not changed. Re-review round (2026-10-17): matching also rejects different days of the same month, disjoint months, different clock times, quantity unit or kind mismatches, reordered quantities, opposite wording and a name on one venue only (adversarial set: 42 of 46 cases right, the 4 misses are same-event pairs rejected); a side priced from an expired copy is flagged and not comparable; Kalshi last-trade lookups read the trade price, skip expired listings, stop after 10 lookups or 10 s and name each null time; matching runs in a worker thread (2,127 Kalshi x 2,000 Polymarket in 3.4 s on mac-studio, scratch benchmark) |
| 1 | News vs market reaction | Companies and countries named in story clusters mapped to tickers / country indices; same-day move against a benchmark | :red_circle: |
| 2 | Connections with timing offsets | Correlation between retained series over a shared sample, tested at leads and lags, with minimum sample size and false-discovery control; states that it does not prove causation | :red_circle: |
| 2 | Day timeline | What every retained source recorded for a given day, by observation date or collection date | :red_circle: |
diff --git a/src/world_intel_mcp/analysis/prediction_matching.py b/src/world_intel_mcp/analysis/prediction_matching.py
index 6c02b8f..e91333c 100644
--- a/src/world_intel_mcp/analysis/prediction_matching.py
+++ b/src/world_intel_mcp/analysis/prediction_matching.py
@@ -4,12 +4,16 @@ Pure functions, no I/O. Used by ``sources/prediction_cross.py`` to put
the same event on Kalshi and Polymarket side by side and to score
settled markets against their pre-settlement prices.
-Matching is deterministic: normalized title tokens, quantities, years,
-negation, rate actions (cut / raise / hold), directions (above /
-below), end dates and the outcome side must all agree before a pair is
-scored, and a pair below ``DEFAULT_MIN_CONFIDENCE`` is not a match. A
-wrong match is worse than no match, so every gate is a hard reject and
-every reject carries its reason.
+Matching is deterministic: normalized title tokens, quantities (value,
+unit, "to" vs "by" and order), years, months, days of month, clock
+times, negation, rate actions (cut / raise / hold), directions (above /
+below), opposite words (rise / fall, positive / negative, before /
+after, win / lose, approve / reject), distinguishing names, end dates
+and the outcome side must all agree before a pair is scored, and a
+pair below ``DEFAULT_MIN_CONFIDENCE`` is not a match. A wrong match is
+worse than no match, so every gate is a hard reject and every reject
+carries its reason. A gate decides from the two markets alone, never
+from what else either venue lists.
``match_confidence`` (``confidence`` in ``score_pair``) is an ordinal
similarity score for ranking candidate pairs: title-word Dice overlap
@@ -27,10 +31,11 @@ Normalized market shape (both venues) consumed here::
"price_basis": str, "spread": float | None} # quote fields (optional)
Gaps / not covered:
-- No synonym coverage beyond the small table below (month names, rate
- verbs, above/below). Pairs that word the same event differently
- ("Fed cuts" vs "FOMC lowers the target range") fall below the
- threshold and are reported unmatched, not matched.
+- No synonym coverage beyond the small tables below (month names, rate
+ verbs and their inflections, above/below, a few central-bank and
+ country names). Pairs that word the same event differently ("Fed
+ cuts" vs "FOMC lowers the target range") fall below the threshold
+ and are reported unmatched, not matched.
- Team, ticker and person aliases ("KC" vs "Kansas City Chiefs") are
not resolved, so most sports pairs stay unmatched.
- Resolution-rule equivalence is not checked: two markets can share a
@@ -38,10 +43,26 @@ Gaps / not covered:
list says what was compared; it does not claim more.
- Ordinals written as words ("first", "second") are plain words, not
quantities, and a number glued to a unit ("100mm") loses the unit.
+ Number words are read from "two" to "twelve"; "one" stays a word.
- A fractional threshold worded strict on one venue and inclusive on
the other ("above 4.5%" vs "4.5% or higher") is matched with a
boundary flag; an integer one is not, because "more than 50" means
"51 or more".
+- Distinguishing names are read from capitalization (plus a short
+ list of chambers, parties, compass points, qualifiers and
+ countries), not from a name database: an all-lowercase title names
+ nothing, and a title-case one names every capitalized word, which
+ only rejects more pairs. A name on one venue only (an insertion) is
+ a Dice penalty, except for the listed qualifiers ("core CPI" vs
+ "CPI", "Bitcoin Cash" vs "Bitcoin"), which reject.
+- The opposite-word and month/day/clock gates reject only when both
+ titles state one; a title that leaves it out ("CPI above 0.3%" vs
+ "CPI rise more than 0.3%", "October 2026" vs "Oct 28, 2026") is
+ left to the other gates and the end-date window. A day written
+ before its month ("20 October") is not read as a day.
+- A deadline worded as a date on one venue and as a calendar year on
+ the other ("before Jan 1, 2027" vs "in 2026") is rejected on the
+ year gate (a false negative, not a wrong match).
"""
from __future__ import annotations
@@ -92,32 +113,131 @@ _SYNONYMS = {
"less": "below", "wins": "win", "won": "win", "winner": "win",
# "U.S." tokenizes to "u.s" (edge dots are stripped).
"usa": "us", "u.s": "us", "u.s.a": "us", "bp": "bps", "basis": "bps",
+ # Inflections of the rate verbs, and verbs that name the same move.
+ "cutting": "cut", "lowered": "cut", "lowering": "cut", "decreased": "cut", "decreasing": "cut",
+ "reduce": "cut", "reduces": "cut", "reduced": "cut", "reducing": "cut", "trim": "cut",
+ "trims": "cut", "trimmed": "cut", "trimming": "cut", "ease": "cut",
+ "eases": "cut", "eased": "cut", "easing": "cut", "raised": "raise", "raising": "raise",
+ "hiked": "raise", "hiking": "raise", "increased": "raise", "increasing": "raise",
+ "boost": "raise", "boosts": "raise", "boosted": "raise", "boosting": "raise",
+ "tighten": "raise", "tightens": "raise", "tightened": "raise", "tightening": "raise",
+ "held": "hold", "holding": "hold", "paused": "hold", "maintained": "hold",
+ # Inflections of the opposite-word pairs (_POLES).
+ "rises": "rise", "rising": "rise", "rose": "rise", "risen": "rise", "falls": "fall",
+ "falling": "fall", "fell": "fall", "fallen": "fall", "drops": "drop", "dropped": "drop",
+ "dropping": "drop", "loses": "lose", "losing": "lose", "lost": "lose", "loser": "lose",
+ "approves": "approve", "approved": "approve", "approval": "approve", "rejects": "reject",
+ "rejected": "reject", "rejection": "reject",
}
_MONTHS = frozenset(
"january february march april may june july august september october "
"november december".split()
)
-_NEGATIONS = frozenset({"not", "no", "never", "without"})
+# "May" is a month only next to a date or after one of these words; "who
+# may win" is a verb.
+_MAY_BEFORE = frozenset("in by before after of until since end early mid late from through".split())
+_NEGATIONS = frozenset({
+ "not", "no", "never", "without", "wont", "cannot", "cant", "dont", "doesnt", "didnt",
+ "isnt", "arent", "wasnt", "fail", "fails", "failed", "neither", "nor", "none", "nobody",
+})
_YES_NO = frozenset({"yes", "no"})
# Hard gates: two titles must name the same rate action and the same
# direction (after synonym folding), or the pair is rejected.
_ACTIONS = frozenset({"cut", "raise", "hold"})
_DIRECTIONS = {"above": "above", "below": "below", "atleast": "above", "atmost": "below", "ormore": "above"}
+# "hold" is the rate action only next to one of these (after synonym and
+# name folding); "hold an election" is not a rate decision.
+_RATE_CONTEXT = frozenset(
+ "rate rates interest bps fed ecb boe boj boc rba snb pboc monetary funds".split()
+)
+# Opposite words: when both titles use a word on the same axis, the poles
+# must agree. A title that uses none is left to the other gates ("after
+# the October meeting" vs "at their October meeting" is one event).
+_POLES = {
+ "rise": ("move", "up"), "climb": ("move", "up"), "climbs": ("move", "up"), "reach": ("move", "up"),
+ "reaches": ("move", "up"), "hit": ("move", "up"), "hits": ("move", "up"), "surpass": ("move", "up"),
+ "surpasses": ("move", "up"), "fall": ("move", "down"), "drop": ("move", "down"), "dip": ("move", "down"),
+ "dips": ("move", "down"), "decline": ("move", "down"), "declines": ("move", "down"),
+ "sink": ("move", "down"), "sinks": ("move", "down"), "plunge": ("move", "down"),
+ "positive": ("sign", "positive"), "negative": ("sign", "negative"),
+ "before": ("time", "before"), "after": ("time", "after"),
+ "win": ("result", "win"), "lose": ("result", "lose"),
+ "approve": ("approval", "approve"), "reject": ("approval", "reject"),
+}
_SCALE_WORDS = {"thousand": 1e3, "million": 1e6, "billion": 1e9, "trillion": 1e12}
_MULTIPLIERS = {"": 1.0, "k": 1e3, "m": 1e6, "mn": 1e6, "b": 1e9, "bn": 1e9, "t": 1e12, "tn": 1e12}
+_NUMBER_WORDS = {
+ "once": 1, "twice": 2, "thrice": 3, "two": 2, "three": 3, "four": 4, "five": 5, "six": 6, "seven": 7, "eight": 8, "nine": 9,
+ "ten": 10, "eleven": 11, "twelve": 12,
+}
_NUM_RE = re.compile(r"^(-?)\$?(\d+(?:\.\d+)?)(k|mn|m|bn|b|tn|t)?$")
_ORDINAL_RE = re.compile(r"^(\d+)(st|nd|rd|th)$")
# Ordinal | number (a leading minus only when it is not a hyphen inside
# a word or range; never starting inside a digit run, so "100mm" cannot
-# leave a partial "10") | word.
+# leave a partial "10") | word | percent sign (marks the number before it).
_TOKEN_RE = re.compile(
r"(?<![\d.,])\d+(?:st|nd|rd|th)(?![a-z0-9])"
r"|(?:(?<![a-z0-9.,])-)?\$?(?<![\d.,])\d(?:[\d,]*\d)?(?:\.\d+)?(?:(?:k|mn|m|bn|b|tn|t)(?![a-z]))?"
r"|[a-z][a-z.&']*"
+ r"|%"
)
_QUANTITY_RE = re.compile(r"^(>=|<=|>|<)?(.+)$")
+# Dash characters that stand for a minus sign before a digit (U+2212 is
+# mapped in _prepare too).
+_DASH_RE = re.compile("[‐-―﹣-](?=\\s*\\d)")
+_TIME_ZONES = r"(?:\s*\b(?:et|edt|est|ct|cdt|cst|mt|mdt|mst|pt|pdt|pst|utc|gmt)\b)?"
+# Clock times: "11:59 pm et", "5pm", "17:00". Read into their own gate
+# (both titles state one: they must agree) and kept out of quantities.
+_CLOCK_12_RE = re.compile(r"(?<![\d.:])(\d{1,2})(?::([0-5]\d))?\s*([ap])\.?m\.?(?![a-z])" + _TIME_ZONES)
+_CLOCK_24_RE = re.compile(r"(?<![\d.:])([01]?\d|2[0-3]):([0-5]\d)(?![\d:])" + _TIME_ZONES)
+# "between 2.5% and 2.7%" is the two thresholds it names.
+_BETWEEN_RE = re.compile(r"\bbetween\s+(\$?-?\d[\d.,]*\s*%?)\s*(?:and|to|-)\s*(?=\$?-?\d)")
+
+# Distinguishing names. A capitalized word (or one in _LISTED_NAMES) that
+# one title names and the other's text lacks tells two events apart when
+# both titles have one ("House" vs "Senate", "Navarro" vs "Bannon"); a
+# listed qualifier rejects even when only one title has it ("core CPI"
+# vs "CPI"). Phrases fold to one name first so "Federal Reserve" is
+# "Fed"; these folds feed only the name gate, not the overlap score.
+_NAME_PHRASES = [
+ (re.compile(r"\breserve bank of australia\b", re.I), "RBA"),
+ (re.compile(r"\bpeople'?s bank of china\b", re.I), "PBoC"),
+ (re.compile(r"\bfederal reserve\b", re.I), "Fed"),
+ (re.compile(r"\bbank of england\b", re.I), "BoE"),
+ (re.compile(r"\bbank of japan\b", re.I), "BoJ"),
+ (re.compile(r"\bbank of canada\b", re.I), "BoC"),
+ (re.compile(r"\beuropean central bank\b", re.I), "ECB"),
+ (re.compile(r"\bunited states\b", re.I), "US"),
+ (re.compile(r"\b(?:united kingdom|great britain)\b", re.I), "UK"),
+]
+_NAME_ALIASES = {
+ "fomc": "fed", "democratic": "democrat", "democrats": "democrat", "dem": "democrat",
+ "dems": "democrat", "republicans": "republican", "gop": "republican", "btc": "bitcoin",
+ "britain": "uk", "british": "uk", "american": "us", "americans": "us", "chinese": "china",
+ "japanese": "japan", "german": "germany", "french": "france", "russian": "russia",
+ "ukrainian": "ukraine", "israeli": "israel", "iranian": "iran", "canadian": "canada",
+ "mexican": "mexico", "brazilian": "brazil", "indian": "india", "korean": "korea",
+ "taiwanese": "taiwan", "european": "europe", "eurozone": "europe", "turkish": "turkey",
+}
+_COUNTRIES = frozenset(
+ """
+ uk china japan germany france italy spain canada mexico brazil india russia ukraine israel
+ iran iraq syria gaza lebanon yemen taiwan korea australia england scotland ireland europe eu
+ argentina turkey saudi venezuela poland netherlands switzerland sweden norway egypt nigeria
+ pakistan indonesia vietnam philippines colombia chile peru greece portugal belgium austria
+ denmark finland hungary romania qatar uae cuba belarus
+ """.split()
+)
+_LISTED_NAMES = _COUNTRIES | frozenset(
+ "house senate democrat republican labour conservative tory north south east west core "
+ "headline cash classic primary runoff".split()
+)
+_NOT_NAMES = frozenset(
+ "who what which when where how why can could should would may might must yes no vs versus "
+ "monday tuesday wednesday thursday friday saturday sunday today tomorrow tonight".split()
+)
def clip_text(value: object, limit: int = MAX_TEXT_CHARS) -> str:
@@ -140,9 +260,44 @@ def _canonical_number(raw: str, scale: float = 1.0) -> str | None:
return f"{value:.12g}" if value != int(value) else str(int(value))
-def _prepare(text: str) -> str:
+def _ascii(text: str) -> str:
+ """Typographic signs folded, dashes before a digit made minus signs,
+ then ASCII; case kept."""
text = (text or "").replace("’", "'").replace("≥", ">=").replace("≤", "<=").replace("−", "-")
- text = unicodedata.normalize("NFKD", text).encode("ascii", "ignore").decode().lower()
+ text = _DASH_RE.sub("-", text)
+ return unicodedata.normalize("NFKD", text).encode("ascii", "ignore").decode()
+
+
+def _clock_minutes(hour: str, minute: str | None, meridiem: str | None) -> str | None:
+ h, m = int(hour), int(minute or 0)
+ if meridiem:
+ if not 1 <= h <= 12:
+ return None
+ h = h % 12 + (12 if meridiem == "p" else 0)
+ return f"{h:02d}:{m:02d}"
+
+
+def clock_times(text: str) -> set[str]:
+ """Clock times a title states, as 24-hour "HH:MM" ("11:59 PM ET" ->
+ "23:59", "5pm" -> "17:00")."""
+ low = _ascii(text).lower()
+ found = {_clock_minutes(*m.groups()) for m in _CLOCK_12_RE.finditer(low)}
+ found |= {_clock_minutes(m.group(1), m.group(2), None) for m in _CLOCK_24_RE.finditer(low)}
+ return {t for t in found if t is not None}
+
+
+def _prepare(text: str) -> str:
+ text = _ascii(text).lower()
+ # Clock times are read by clock_times(); here they would leave their
+ # hour and minute behind as quantities ("11:59 PM" as 11 and 59).
+ text = _CLOCK_12_RE.sub(" ", text)
+ text = _CLOCK_24_RE.sub(" ", text)
+ text = _BETWEEN_RE.sub(r" atleast \1 and atmost ", text)
+ text = re.sub(
+ r"\b(" + "|".join(_NUMBER_WORDS) + r")\b", lambda m: str(_NUMBER_WORDS[m.group(1)]), text
+ )
+ text = re.sub(r"(\d)\s*%", r"\1 %", text)
+ text = re.sub(r"\b(?:negative|minus)\s+(?=\d)", " -", text)
# Comparators become words so a quantity keeps its direction:
# ">25bps" and "25 bps" are different contracts.
text = text.replace(">=", " atleast ").replace("<=", " atmost ")
@@ -159,11 +314,24 @@ def _prepare(text: str) -> str:
return re.sub(r"(\d)(bps|bp)\b", r"\1 \2", text)
+def _fold_word(raw: str) -> str:
+ """A lowercase word token as compared: edge punctuation, possessive
+ and apostrophes dropped, synonyms folded ("" when nothing is left)."""
+ word = raw.strip(".'&")
+ if word.endswith("'s"):
+ word = word[:-2]
+ word = word.replace("'", "")
+ return _SYNONYMS.get(word, word) if word else ""
+
+
def _scan(text: str) -> list[tuple[str, str]]:
- """``(token, kind)`` pairs; kind is word, number, money, year or ordinal.
+ """``(token, kind)`` pairs; kind is word, number, money, percent, year
+ or ordinal.
``year`` is only the shape (four plain digits, 1900-2100); whether it
is a year or an amount depends on what precedes it (text_features).
+ A "%" or "percent" after a number makes it ``percent`` and is not a
+ token itself.
"""
raws = _TOKEN_RE.findall(_prepare(text))
out: list[tuple[str, str]] = []
@@ -171,6 +339,10 @@ def _scan(text: str) -> list[tuple[str, str]]:
while i < len(raws):
raw = raws[i]
i += 1
+ if raw == "%" or (raw in ("percent", "pct") and out and out[-1][1] in ("number", "year")):
+ if out and out[-1][1] in ("number", "year"):
+ out[-1] = (out[-1][0], "percent")
+ continue
ordinal = _ORDINAL_RE.match(raw)
if ordinal:
out.append((f"{int(ordinal.group(1))}{ordinal.group(2)}", "ordinal"))
@@ -191,12 +363,9 @@ def _scan(text: str) -> list[tuple[str, str]]:
kind = "number"
out.append((num, kind))
continue
- word = raw.strip(".'&")
- if word.endswith("'s"):
- word = word[:-2]
- word = word.replace("'", "")
+ word = _fold_word(raw)
if word:
- out.append((_SYNONYMS.get(word, word), "word"))
+ out.append((word, "word"))
return out
@@ -237,26 +406,73 @@ def _day_of_month(tok: str, kind: str) -> str | None:
return None
+def _is_month(toks: list[str], kinds: list[str], i: int) -> bool:
+ if toks[i] != "may":
+ return toks[i] in _MONTHS
+ after = kinds[i + 1] if i + 1 < len(kinds) else ""
+ return after in ("number", "year", "ordinal") or (i > 0 and toks[i - 1] in _MAY_BEFORE)
+
+
+def _mode(toks: list[str], i: int) -> str | None:
+ """"to" (a level) or "by" (a change) before the number at ``toks[i]``."""
+ j = i - 1
+ while j >= 0 and toks[j] in ("than", "the", "$", "above", "below", "atleast", "atmost"):
+ j -= 1
+ return toks[j] if j >= 0 and toks[j] in ("to", "by") else None
+
+
+def _typed_quantity(toks: list[str], kinds: list[str], i: int, comparator: str) -> dict:
+ """One quantity with its unit ("$", "%" with bps read as hundredths
+ of a percent, or None), its "to"/"by" mode and the key the gates
+ compare (comparator plus the value in that unit)."""
+ tok, kind = toks[i], kinds[i]
+ unit = {"money": "$", "percent": "%"}.get(kind)
+ unit_raw = unit
+ value = tok
+ if i + 1 < len(toks) and toks[i + 1] == "bps" and kind == "number":
+ unit, unit_raw = "%", "bps"
+ value = _canonical_number(str(float(tok) / 100)) or tok
+ return {
+ "raw": comparator + tok,
+ "key": comparator + value,
+ "unit": unit,
+ "unit_raw": unit_raw,
+ "mode": _mode(toks, i),
+ # Strictness matters for whole numbers ("more than 50" is 51 or
+ # more), not for fractions; bps count as whole numbers.
+ "whole": kind != "ordinal" and "." not in tok,
+ }
+
+
def text_features(text: str) -> dict:
"""Split a title into content words, quantities, years and flags.
A 1-31 number (or ordinal) right after a month name is a day of
- month and goes to ``days``, not ``quantities``: "Oct 28, 2026" must
- not block a pair with "October 2026" on the number gate (the
- end-date gate covers it). A four-digit 1900-2100 number is a year
- unless a dollar sign, a thousands comma or a comparator makes it an
- amount: "above $2,000" is the quantity ``>2000``. ``actions`` and
- ``directions`` are the rate verbs and above/below words the title
- uses, after synonym folding.
+ month and goes to ``days`` (and ``dated``, by month), not
+ ``quantities``: "Oct 28, 2026" must not block a pair with "October
+ 2026" on the number gate; the day gate compares days only when both
+ titles state one for the same month. A four-digit 1900-2100 number
+ is a year unless a dollar sign, a thousands comma, a percent sign or
+ a comparator makes it an amount: "above $2,000" is the quantity
+ ``>2000``. ``actions``, ``directions`` and ``poles`` are the rate
+ verbs, above/below words and opposite words the title uses, after
+ synonym folding. ``typed`` carries each quantity's unit and mode,
+ ``order`` the quantity keys in order of first appearance, ``times``
+ the clock times.
"""
scanned = _scan(text)
toks = [tok for tok, _ in scanned]
+ kinds = [kind for _, kind in scanned]
words: set[str] = set()
quantities: set[str] = set()
+ typed: list[dict] = []
years: set[str] = set()
days: set[str] = set()
+ months: set[str] = set()
+ dated: dict[str, set[str]] = {}
actions: set[str] = set()
directions: set[str] = set()
+ poles: dict[str, set[str]] = {}
negated = False
for i, (tok, kind) in enumerate(scanned):
if kind == "word":
@@ -264,6 +480,11 @@ def text_features(text: str) -> dict:
actions.add(tok)
if tok in _DIRECTIONS:
directions.add(_DIRECTIONS[tok])
+ if tok in _POLES:
+ axis, pole = _POLES[tok]
+ poles.setdefault(axis, set()).add(pole)
+ if tok in _MONTHS and _is_month(toks, kinds, i):
+ months.add(tok)
if tok in _NEGATIONS:
negated = True
elif tok in _COMPARATOR_WORDS:
@@ -276,18 +497,30 @@ def text_features(text: str) -> dict:
day = _day_of_month(tok, kind)
if kind == "year" and not comparator:
years.add(tok)
- elif i > 0 and toks[i - 1] in _MONTHS and day is not None:
+ elif i > 0 and toks[i - 1] in _MONTHS and _is_month(toks, kinds, i - 1) and day is not None:
days.add(day)
+ dated.setdefault(toks[i - 1], set()).add(day)
else:
quantities.add(comparator + tok)
+ typed.append(_typed_quantity(toks, kinds, i, comparator))
+ order: list[str] = []
+ for q in typed:
+ if q["key"] not in order:
+ order.append(q["key"])
return {
"words": words,
"quantities": quantities,
+ "typed": typed,
+ "order": order,
"years": years,
"days": days,
+ "months": months,
+ "dated": dated,
+ "times": clock_times(text),
"negated": negated,
"actions": actions,
"directions": directions,
+ "poles": poles,
}
@@ -311,27 +544,80 @@ def _text(m: dict, *keys: str) -> str:
return " ".join(s for s in (m.get(k) for k in keys) if isinstance(s, str) and s)
+def _alias_phrases(text: str) -> str:
+ for pattern, alias in _NAME_PHRASES:
+ text = pattern.sub(f" {alias} ", text)
+ return text
+
+
+def _name_word(raw: str) -> str:
+ word = _fold_word(raw.lower())
+ return _NAME_ALIASES.get(word, word)
+
+
+def _names(*texts: str) -> set[str]:
+ """Distinguishing names in the texts: capitalized words and listed
+ names, after phrase and name folding. Words that only shape the
+ question (stopwords, months, rate verbs, directions, opposite words,
+ negations, yes/no, number words) are never names."""
+ names: set[str] = set()
+ for text in texts:
+ for match in re.finditer(r"[A-Za-z][A-Za-z.&']*", _alias_phrases(_ascii(text))):
+ raw = match.group(0)
+ word = _name_word(raw)
+ if (
+ len(word) < 2 or word in _STOPWORDS or word in _NOT_NAMES or word in _MONTHS
+ or word in _ACTIONS or word in _DIRECTIONS or word in _COMPARATOR_WORDS
+ or word in _POLES or word in _NEGATIONS or word in _NUMBER_WORDS
+ ):
+ continue
+ if word in _LISTED_NAMES or raw[0].isupper():
+ names.add(word)
+ return names
+
+
+def _present(text: str) -> set[str]:
+ """Every word of ``text`` after phrase and name folding: what a name
+ on the other venue is looked up in."""
+ return {_NAME_ALIASES.get(tok, tok) for tok, kind in _scan(_alias_phrases(_ascii(text))) if kind == "word"}
+
+
def _market_features(m: dict, venue: str) -> dict:
"""Everything the gates read from one market, computed once.
``bag`` (question, event title and, on Kalshi, the YES side) feeds
- the overlap score, quantities, years and negation. Actions and
- directions come from the contract's own wording (question plus YES
- side), not the event title, which can list every alternative.
+ the overlap score, quantities, years, months, days, clock times and
+ negation. Actions, directions, opposite words and names come from
+ the contract's own wording (question plus YES side), not the event
+ title, which can list every alternative; a name is looked up in the
+ other venue's whole bag.
"""
if venue == "kalshi":
- bag = text_features(_text(m, "question", "context", "yes_side"))
- gate = text_features(_text(m, "question", "yes_side"))
+ bag_text = _text(m, "question", "context", "yes_side")
+ gate_texts = [m.get("question") or "", m.get("yes_side") or ""]
else:
- bag = text_features(_text(m, "question", "context"))
- gate = text_features(_text(m, "question"))
+ bag_text = _text(m, "question", "context")
+ gate_texts = [m.get("question") or ""]
+ bag = text_features(bag_text)
+ gate = text_features(" ".join(t for t in gate_texts if isinstance(t, str)))
+ present = _present(bag_text)
+ actions = set(gate["actions"])
+ if "hold" in actions and not present & _RATE_CONTEXT:
+ actions.discard("hold") # "hold an election", "maintain control"
outcomes = [str(o) for o in (m.get("outcomes") or [])]
+ question = text_features(_text(m, "question"))
return {
"bag": bag,
- "actions": gate["actions"],
+ "actions": actions,
"directions": gate["directions"],
- "question_words": text_features(_text(m, "question"))["words"],
- "side_words": text_features(_text(m, "yes_side"))["words"],
+ "poles": gate["poles"],
+ "names": _names(*(t for t in gate_texts if isinstance(t, str))),
+ "present": present,
+ "order": question["order"],
+ "question_words": question["words"],
+ # "bps" is a unit, compared with its value by the quantity gates
+ # ("Cut 25bps" against "cut by 0.25%").
+ "side_words": text_features(_text(m, "yes_side"))["words"] - {"bps"},
"outcomes": outcomes,
"labels": [set(tokenize(o)) for o in outcomes],
"end": _parse_time(m.get("end_time")),
@@ -385,6 +671,80 @@ def _boundary_only(k_quantities: set[str], p_quantities: set[str]) -> bool:
)
+def _quantity_classes(typed: list[dict]) -> set[str]:
+ """Keys with strictness dropped for fractional thresholds only."""
+ return {_threshold_class(q["key"]) if not q["whole"] else q["key"] for q in typed}
+
+
+def _quantities_gate(kb: dict, pb: dict) -> tuple[str, str]:
+ """("agree" | "boundary" | "differ", reason). Values compare in one
+ unit (bps as hundredths of a percent); reasons quote the titles'
+ own numbers."""
+ k_raw, p_raw = sorted(kb["quantities"]), sorted(pb["quantities"])
+ k_keys = {q["key"] for q in kb["typed"]}
+ p_keys = {q["key"] for q in pb["typed"]}
+ if k_raw == p_raw:
+ return "agree", f"quantities agree {k_raw}" if k_raw else ""
+ if k_keys == p_keys:
+ return "agree", f"quantities agree once bps are read as percent: Kalshi {k_raw} vs Polymarket {p_raw}"
+ if _boundary_only(kb["quantities"], pb["quantities"]) or (
+ len(k_keys) == len(p_keys) and _quantity_classes(kb["typed"]) == _quantity_classes(pb["typed"])
+ ):
+ return "boundary", (
+ f"quantities agree except at the boundary: Kalshi {k_raw} vs Polymarket {p_raw} "
+ "(the same unless the outcome lands exactly on the threshold)"
+ )
+ return "differ", f"quantities differ: Kalshi {k_raw} vs Polymarket {p_raw}"
+
+
+def _by_key(typed: list[dict], field: str, key: str = "key") -> dict[str, set[str]]:
+ out: dict[str, set[str]] = {}
+ for q in typed:
+ if q[field] is not None:
+ out.setdefault(_threshold_class(q[key]), set()).add(q[field])
+ return out
+
+
+def _late_gates(kf: dict, pf: dict) -> str | None:
+ """The gates added after the review of 2026-10-17, in order; the first
+ failing one's reason, or None. They run after the original gates so
+ a pair those reject keeps its reason."""
+ kb, pb = kf["bag"], pf["bag"]
+ if kb["times"] and pb["times"] and kb["times"] != pb["times"]:
+ return f"clock times differ: Kalshi {sorted(kb['times'])} vs Polymarket {sorted(pb['times'])}"
+ if kb["months"] and pb["months"] and not kb["months"] & pb["months"]:
+ return f"months differ: Kalshi {sorted(kb['months'])} vs Polymarket {sorted(pb['months'])}"
+ for month in sorted(set(kb["dated"]) & set(pb["dated"])):
+ k_days, p_days = kb["dated"][month], pb["dated"][month]
+ if k_days != p_days:
+ return (
+ f"days differ in {month}: Kalshi {sorted(k_days, key=int)} vs Polymarket {sorted(p_days, key=int)}"
+ )
+ for field, key, what in (
+ ("unit", "key", "units"), ("unit_raw", "raw", "units"), ("mode", "key", "kinds ('to' a level vs 'by' a change)"),
+ ):
+ k_by, p_by = _by_key(kb["typed"], field, key), _by_key(pb["typed"], field, key)
+ for key in sorted(set(k_by) & set(p_by)):
+ if not k_by[key] & p_by[key]:
+ return (
+ f"quantity {what} differ for {key}: Kalshi {sorted(k_by[key])} vs Polymarket {sorted(p_by[key])}"
+ )
+ k_order, p_order = kf["order"], pf["order"]
+ if len(k_order) >= 2 and set(k_order) == set(p_order) and k_order != p_order:
+ return f"quantities in a different order: Kalshi {k_order} vs Polymarket {p_order}"
+ for axis in sorted(set(kf["poles"]) & set(pf["poles"])):
+ if kf["poles"][axis] != pf["poles"][axis]:
+ return f"opposite wording: Kalshi {sorted(kf['poles'][axis])} vs Polymarket {sorted(pf['poles'][axis])}"
+ k_extra = kf["names"] - pf["present"]
+ p_extra = pf["names"] - kf["present"]
+ k_listed, p_listed = k_extra & _LISTED_NAMES, p_extra & _LISTED_NAMES
+ if k_listed or p_listed:
+ return f"a listed name on one venue only: Kalshi {sorted(k_listed)} vs Polymarket {sorted(p_listed)}"
+ if k_extra and p_extra:
+ return f"distinguishing names differ: Kalshi {sorted(k_extra)} vs Polymarket {sorted(p_extra)}"
+ return None
+
+
def _score(kf: dict, pf: dict, now: datetime) -> dict:
kb, pb = kf["bag"], pf["bag"]
reasons: list[str] = []
@@ -401,19 +761,12 @@ def _score(kf: dict, pf: dict, now: datetime) -> dict:
f"{len(kb['words'])} Kalshi / {len(pb['words'])} Polymarket words)"
)
- boundary = False
- if kb["quantities"] != pb["quantities"]:
- if not _boundary_only(kb["quantities"], pb["quantities"]):
- return reject(
- f"quantities differ: Kalshi {sorted(kb['quantities'])} vs Polymarket {sorted(pb['quantities'])}"
- )
- boundary = True
- reasons.append(
- f"quantities agree except at the boundary: Kalshi {sorted(kb['quantities'])} vs Polymarket "
- f"{sorted(pb['quantities'])} (the same unless the outcome lands exactly on the threshold)"
- )
- elif kb["quantities"]:
- reasons.append(f"quantities agree {sorted(kb['quantities'])}")
+ verdict, why = _quantities_gate(kb, pb)
+ if verdict == "differ":
+ return reject(why)
+ boundary = verdict == "boundary"
+ if why:
+ reasons.append(why)
if kb["years"] and pb["years"] and not (kb["years"] & pb["years"]):
return reject(f"years differ: Kalshi {sorted(kb['years'])} vs Polymarket {sorted(pb['years'])}")
@@ -448,6 +801,10 @@ def _score(kf: dict, pf: dict, now: datetime) -> dict:
return reject(side_reason)
reasons.append(side_reason)
+ late = _late_gates(kf, pf)
+ if late:
+ return reject(late)
+
confidence = round(dice * date_factor, 3)
return {"confidence": confidence, "rejected": False, "reasons": reasons, "side": side, "boundary_differs": boundary}
@@ -468,6 +825,14 @@ def score_pair(kalshi: dict, poly: dict, now: datetime | None = None) -> dict:
# Matching
# ---------------------------------------------------------------------------
+def _prefix(words: set[str], df: dict[str, int], t: float) -> list[str]:
+ """The rarest ``n - need + 1`` of a market's words, in the total order
+ (document frequency, then the word): the order every market shares."""
+ ordered = sorted(words, key=lambda w: (df[w], w))
+ need = math.ceil(t * len(ordered) / (2 - t) - 1e-9) # rounding can only lengthen the prefix
+ return ordered[: len(ordered) - max(need, 1) + 1]
+
+
def _candidate_pairs(k_feats: list[dict], p_feats: list[dict], min_dice: float) -> list[tuple[int, int]]:
"""Every pair whose title-word Dice can reach ``min_dice``.
@@ -477,9 +842,11 @@ def _candidate_pairs(k_feats: list[dict], p_feats: list[dict], min_dice: float)
o satisfies o >= t * n / (2 - t) for either market's word count n
(from 2o >= t * (n + m) and m >= o), and the first shared word sits
within the first n - o + 1 words of each market, so both prefixes
- hold it. Pairs sharing no word are never candidates. (The bound and
- the prefix step were checked with z3 on 2026-10-17, the prefix step
- for up to 16 words; a test compares this with a brute-force scan.)
+ hold it. "First shared word" needs one total order for every
+ market, so ties in rarity break on the word itself (``_prefix``).
+ Pairs sharing no word are never candidates. The argument is checked
+ by tests, not machine-proved: a brute-force scan comparison and a
+ test of the total order.
"""
t = min(max(float(min_dice), 0.0), 1.0)
df: dict[str, int] = {}
@@ -487,18 +854,13 @@ def _candidate_pairs(k_feats: list[dict], p_feats: list[dict], min_dice: float)
for w in f["bag"]["words"]:
df[w] = df.get(w, 0) + 1
- def prefix(words: set[str]) -> list[str]:
- ordered = sorted(words, key=lambda w: (df[w], w))
- need = math.ceil(t * len(ordered) / (2 - t) - 1e-9) # rounding can only lengthen the prefix
- return ordered[: len(ordered) - max(need, 1) + 1]
-
index: dict[str, list[int]] = {}
for pi, f in enumerate(p_feats):
- for w in prefix(f["bag"]["words"]):
+ for w in _prefix(f["bag"]["words"], df, t):
index.setdefault(w, []).append(pi)
pairs: set[tuple[int, int]] = set()
for ki, f in enumerate(k_feats):
- for w in prefix(f["bag"]["words"]):
+ for w in _prefix(f["bag"]["words"], df, t):
for pi in index.get(w, ()):
pairs.add((ki, pi))
return sorted(pairs)
@@ -591,15 +953,44 @@ def _brief(m: dict) -> dict:
"spread": m.get("spread"),
"last_price": m.get("last_price"),
"last_trade_time": m.get("last_trade_time"),
+ "stale": bool(m.get("stale")),
+ "stale_age_seconds": m.get("stale_age_seconds") if m.get("stale") else None,
"url": m.get("url"),
}
+def _flip(value: object) -> float | None:
+ return round(1.0 - value, 4) if isinstance(value, (int, float)) and not isinstance(value, bool) else None
+
+
+def _aligned_brief(m: dict, side: str | None) -> dict:
+ """The Polymarket side's brief priced on the outcome the Kalshi YES
+ pays on: for an inverted pair the quotes are flipped (YES bid = 1 -
+ the first outcome's ask, and so on). ``quoted_outcome`` names it."""
+ brief = _brief(m)
+ outcomes = brief.get("outcomes") or []
+ if side == "inverted" and len(outcomes) == 2:
+ brief.update({
+ "probability": _flip(m.get("probability")),
+ "yes_bid": _flip(m.get("yes_ask")),
+ "yes_ask": _flip(m.get("yes_bid")),
+ "last_price": _flip(m.get("last_price")),
+ "quoted_outcome": outcomes[1],
+ })
+ else:
+ brief["quoted_outcome"] = outcomes[0] if outcomes else None
+ return brief
+
+
def _quote_flags(venue: str, m: dict) -> list[str]:
"""Why one side's price is not a live two-sided quote ([] when it is)."""
if m.get("probability") is None:
- return [] # no price: the gap is None and the pair sorts last anyway
+ return [f"{venue} price unavailable (basis: {m.get('price_basis') or 'not stated'})"]
flags = []
+ if m.get("stale"):
+ age = m.get("stale_age_seconds")
+ old = f"{age / 3600:.1f} h old" if isinstance(age, (int, float)) else "of unknown age"
+ flags.append(f"{venue} price from a copy {old}, served from expired cache after a failed live fetch")
basis = m.get("price_basis")
if basis != "bid_ask_midpoint":
flags.append(f"{venue} price is not a two-sided quote midpoint (basis: {basis or 'not stated'})")
@@ -624,7 +1015,7 @@ def _pair_record(km: dict, pm: dict, result: dict) -> dict:
"reasons": result["reasons"],
"side": result["side"],
"kalshi": _brief(km),
- "polymarket": _brief(pm),
+ "polymarket": _aligned_brief(pm, result["side"]),
"kalshi_yes_probability": k_prob,
"polymarket_yes_probability": p_aligned,
"price_gap": gap,
diff --git a/src/world_intel_mcp/cli.py b/src/world_intel_mcp/cli.py
index 8c09b94..00952bb 100644
--- a/src/world_intel_mcp/cli.py
+++ b/src/world_intel_mcp/cli.py
@@ -868,12 +868,14 @@ def predictions(ctx: click.Context, limit: int) -> None:
if _bail_on_error(ctx, data):
return
+ _print_data_gaps(data)
mkts = data.get("markets", [])
if not mkts:
console.print("[yellow]No prediction market data available[/yellow]")
return
- table = Table(title="Prediction Markets", box=box.SIMPLE_HEAVY)
+ title = "Prediction Markets (expired cached copy)" if data.get("stale") else "Prediction Markets"
+ table = Table(title=title, box=box.SIMPLE_HEAVY)
table.add_column("Question", max_width=50)
table.add_column("YES %", justify="right")
table.add_column("Sentiment")
@@ -899,6 +901,27 @@ def _pct(value: Any) -> str:
return "-" if value is None else f"{value * 100:.1f}%"
+def _stale_note(m: dict) -> str:
+ """" (expired copy, N h old)" for a market priced from expired cache."""
+ if not m.get("stale"):
+ return ""
+ age = m.get("stale_age_seconds")
+ return f" (expired copy, {age / 3600:.1f} h old)" if isinstance(age, (int, float)) else " (expired copy)"
+
+
+def _last_trade_note(side: dict) -> str | None:
+ """The Kalshi last-trade lookup result for the Flags column."""
+ status = side.get("last_trade_lookup")
+ if status is None and not side.get("last_trade_time"):
+ return None
+ if side.get("last_trade_time"):
+ price = side.get("last_trade_price")
+ at = f" at {_pct(price)}" if price is not None else ""
+ copy = " (expired copy)" if status == "found_in_expired_copy" else ""
+ return f"kalshi last trade {side['last_trade_time']}{at}{copy}"
+ return f"kalshi last trade: {str(status).replace('_', ' ')}"
+
+
# Data gaps print through the shared _print_data_gaps defined with the
# AOI commands below (module-level, resolved at call time).
@@ -944,7 +967,7 @@ def kalshi(ctx: click.Context, category: str | None, series: str | None, limit:
_cell(m.get("question") or ""),
_cell(m.get("yes_side") or ""),
_pct(m.get("probability")),
- _cell(m.get("price_basis", "")),
+ _cell(m.get("price_basis", "") + _stale_note(m)),
f"{m.get('volume_24h', 0):,.0f}",
f"{m.get('volume', 0):,.0f}",
)
@@ -1015,9 +1038,9 @@ def prediction_compare(
gap = p.get("price_gap")
flags = list(p.get("flags") or [])
flagged = flagged or bool(flags)
- traded = (p.get("kalshi") or {}).get("last_trade_time")
+ traded = _last_trade_note(p.get("kalshi") or {})
if traded:
- flags.append(f"kalshi last trade {traded}")
+ flags.append(traded)
table.add_row( # wrapped, not sliced (see the kalshi table)
_cell(p.get("kalshi", {}).get("question") or ""),
_cell(p.get("polymarket", {}).get("question") or ""),
diff --git a/src/world_intel_mcp/sources/kalshi.py b/src/world_intel_mcp/sources/kalshi.py
index 24eb96e..87da815 100644
--- a/src/world_intel_mcp/sources/kalshi.py
+++ b/src/world_intel_mcp/sources/kalshi.py
@@ -129,6 +129,23 @@ def _finite(value: Any) -> float | None:
return number if number is not None and math.isfinite(number) else None
+def _quote(value: Any) -> float | None:
+ """A quote in dollars per $1 payout: a finite number from 0 to 1, else
+ None (a cents-scaled "45" or a negative value is not a price)."""
+ number = _finite(value)
+ return number if number is not None and 0 <= number <= 1 else None
+
+
+_QUOTE_FIELDS = ("yes_bid_dollars", "yes_ask_dollars", "last_price_dollars")
+
+
+def bad_quote_fields(market: dict) -> list[str]:
+ """Quote fields present in a raw market but not a price from 0 to 1."""
+ return [f for f in _QUOTE_FIELDS if _num(market.get(f)) is not None and _quote(market.get(f)) is None] + [
+ f for f in _QUOTE_FIELDS if market.get(f) not in (None, "") and _num(market.get(f)) is None
+ ]
+
+
def _volume(value: Any) -> float | None:
"""A contract count: absent means 0.0; unparseable, non-finite or
negative means None (unreadable, counted by the caller)."""
@@ -186,9 +203,9 @@ def implied_probability(market: dict) -> tuple[float | None, str]:
ask < 1); otherwise the last trade when it is a finite price in
(0, 1]; otherwise None ("no price").
"""
- bid = _finite(market.get("yes_bid_dollars"))
- ask = _finite(market.get("yes_ask_dollars"))
- last = _finite(market.get("last_price_dollars"))
+ bid = _quote(market.get("yes_bid_dollars"))
+ ask = _quote(market.get("yes_ask_dollars"))
+ last = _quote(market.get("last_price_dollars"))
if bid is not None and ask is not None and 0 < bid <= ask < 1:
return round((bid + ask) / 2, 4), "bid_ask_midpoint"
if last is not None and 0 < last <= 1:
@@ -203,6 +220,10 @@ def normalize_market(market: dict, event: dict | None = None) -> dict:
unreadable (callers count such rows); ``spread`` is set only for a
two-sided book. Kalshi's market listings carry no last-trade time
(``last_trade_time`` is None here; see fetch_kalshi_last_trade_times).
+ Quote fields outside 0..1 are None (bad_quote_fields names them).
+ ``stale`` and ``stale_age_seconds`` say whether the listing response
+ this market came from was served from expired cache (the event dict
+ carries the marker, set by _events_pages).
"""
event = event or {}
prob, basis = implied_probability(market)
@@ -210,9 +231,10 @@ def normalize_market(market: dict, event: dict | None = None) -> dict:
series_ticker = event.get("series_ticker") or (event_ticker.split("-", 1)[0] if event_ticker else "")
yes_side = market.get("yes_sub_title") or ""
title = market.get("title") or ""
- bid = _finite(market.get("yes_bid_dollars"))
- ask = _finite(market.get("yes_ask_dollars"))
+ bid = _quote(market.get("yes_bid_dollars"))
+ ask = _quote(market.get("yes_ask_dollars"))
two_sided = bid is not None and ask is not None and 0 < bid <= ask < 1
+ stale = bool(event.get("_stale"))
return {
"venue": "kalshi",
"id": market.get("ticker", ""),
@@ -227,8 +249,10 @@ def normalize_market(market: dict, event: dict | None = None) -> dict:
"yes_bid": bid,
"yes_ask": ask,
"spread": round(ask - bid, 4) if two_sided else None,
- "last_price": _finite(market.get("last_price_dollars")),
+ "last_price": _quote(market.get("last_price_dollars")),
"last_trade_time": None,
+ "stale": stale,
+ "stale_age_seconds": _stale_age(event) if stale else None,
"volume": _volume(market.get("volume_fp")),
"volume_24h": _volume(market.get("volume_24h_fp")),
"open_interest": _finite(market.get("open_interest_fp")),
@@ -280,7 +304,14 @@ async def _events_pages(
f"Kalshi /events page {pages + 1} unavailable (no live or cached data)"
)
pages += 1
- events.extend(e for e in data["events"] if isinstance(e, dict))
+ page = [e for e in data["events"] if isinstance(e, dict)]
+ if data.get("_stale"):
+ # Carried onto each market by normalize_market, so a price
+ # from this page is marked where it is shown.
+ for e in page:
+ e["_stale"] = True
+ e["_stale_age_seconds"] = _stale_age(data)
+ events.extend(page)
cursor = data.get("cursor") or ""
if not cursor:
return events, pages, True, None
@@ -393,13 +424,20 @@ async def fetch_kalshi_open_markets(
)
markets: list[dict] = []
+ bad_quotes = 0
for ev in events:
if category and ev.get("category") and ev["category"] != category:
continue
for m in ev.get("markets") or []:
if isinstance(m, dict) and m.get("status") in (None, "active", "open"):
markets.append(normalize_market(m, ev))
+ bad_quotes += bool(bad_quote_fields(m))
scan["events_scanned"] = len(events)
+ if bad_quotes:
+ gaps.append(
+ f"{bad_quotes} market(s) with a quote field that is not a price from 0 to 1 "
+ "(left null; the price falls back or is missing)"
+ )
if stale:
scan["stale_responses"] = len(stale)
scan["stale_max_age_seconds"] = max_stale_age(stale)
@@ -715,17 +753,24 @@ async def fetch_kalshi_price_history(
async def fetch_kalshi_last_trade_times(
- fetcher: Fetcher, tickers: list[str]
+ fetcher: Fetcher, tickers: list[str], *,
+ prices: dict[str, float | None] | None = None,
+ stale_tickers: dict[str, float | None] | None = None,
) -> tuple[dict[str, str | None], list[str]]:
"""When each market last traded, from the public ``/markets/trades``
endpoint (``limit=1``). The market listings carry no trade time
(measured 2026-10-17), so a price whose basis is the last trade has
- no age without this lookup.
+ no age without this lookup. ``limit=1`` returns the venue's first
+ listed trade, which is the latest only while the endpoint lists
+ newest first (observed live on 2026-10-17 in two samples).
Returns (``{ticker: created_time or None}``, gaps): None means the
venue returned no trade; a ticker missing from the dict could not be
looked up or its trade time was unreadable (named in the gaps, never
- guessed).
+ guessed). When the caller passes ``prices``, the same trade's
+ ``yes_price_dollars`` is recorded per ticker (None when unreadable
+ or outside 0..1); ``stale_tickers`` records the tickers whose answer
+ was served from expired cache, with the copy's age in seconds.
"""
times: dict[str, str | None] = {}
failed = 0
@@ -742,14 +787,20 @@ async def fetch_kalshi_last_trade_times(
if not isinstance(trades, list):
failed += 1
continue
+ if stale_tickers is not None and data.get("_stale"):
+ stale_tickers[ticker] = _stale_age(data)
if not trades:
times[ticker] = None
+ if prices is not None:
+ prices[ticker] = None
continue
created = trades[0].get("created_time") if isinstance(trades[0], dict) else None
if not isinstance(created, str) or _ts(created) is None:
failed += 1
continue
times[ticker] = created
+ if prices is not None:
+ prices[ticker] = _quote(trades[0].get("yes_price_dollars"))
gaps: list[str] = []
if failed:
gaps.append(f"Kalshi last-trade time unavailable for {failed} market(s)")
diff --git a/src/world_intel_mcp/sources/prediction.py b/src/world_intel_mcp/sources/prediction.py
index 8bde54b..4300b16 100644
--- a/src/world_intel_mcp/sources/prediction.py
+++ b/src/world_intel_mcp/sources/prediction.py
@@ -31,6 +31,7 @@ import json
import logging
import math
import re
+import time
from datetime import datetime, timezone
from urllib.parse import quote
@@ -174,6 +175,9 @@ async def fetch_prediction_markets(fetcher: Fetcher, limit: int = 20) -> dict:
{"markets": [...], "count": N, "source": "polymarket",
"timestamp": "<iso>"}
+
+ A list served from expired cache adds ``stale``, ``degraded`` and a
+ ``data_gaps`` entry.
"""
params = {
"limit": str(limit),
@@ -256,6 +260,7 @@ async def fetch_prediction_markets(fetcher: Fetcher, limit: int = 20) -> dict:
gaps = []
if stale:
result["stale"] = True
+ result["degraded"] = True
gaps.append("served from expired cache after a failed live fetch")
if unpriced:
gaps.append(f"{unpriced} market(s) left out: outcome price missing, unreadable or outside 0..1")
@@ -323,19 +328,24 @@ async def fetch_polymarket_open(fetcher: Fetcher, limit: int = 300) -> dict:
"""Open binary Polymarket markets by 24h volume, cross-venue shape.
Returns ``{"markets", "requested", "rows_read", "non_binary_skipped",
- "error"?, "degraded"?, "stale"?}``.
+ "error"?, "degraded"?, "stale"?, "stale_max_age_seconds"?}``. Each
+ market carries ``stale`` and ``stale_age_seconds``: whether its page
+ was served from expired cache, and that copy's age.
"""
limit = max(1, min(int(limit), MAX_OPEN_MARKETS))
- rows: list = []
+ rows: list[tuple[dict, float | None, bool]] = []
offset = 0
stale_before = _stale_serves(fetcher, "polymarket")
+ stale_ages: list[float | None] = []
error = None
while offset < limit:
size = min(_GAMMA_PAGE_MAX, limit - offset)
+ cache_key = f"prediction:polymarket:open:{size}:{offset}"
+ page_before = _stale_serves(fetcher, "polymarket")
data = await fetcher.get_json(
_GAMMA_MARKETS_URL,
source="polymarket",
- cache_key=f"prediction:polymarket:open:{size}:{offset}",
+ cache_key=cache_key,
cache_ttl=300,
params={
"limit": str(size), "offset": str(offset), "active": "true",
@@ -348,11 +358,21 @@ async def fetch_polymarket_open(fetcher: Fetcher, limit: int = 300) -> dict:
if data is None else f"Unexpected Polymarket response shape: {type(data).__name__}"
)
break
- rows.extend(r for r in data if isinstance(r, dict))
+ page_stale = _stale_serves(fetcher, "polymarket") > page_before
+ age = _stale_copy_age(fetcher, cache_key) if page_stale else None
+ if page_stale:
+ stale_ages.append(age)
+ rows.extend((r, age, page_stale) for r in data if isinstance(r, dict))
if len(data) < size:
break
offset += size
- markets = [m for m in (normalize_gamma_market(r) for r in rows) if m is not None]
+ markets = []
+ for row, age, page_stale in rows:
+ m = normalize_gamma_market(row)
+ if m is not None:
+ m["stale"] = page_stale
+ m["stale_age_seconds"] = age
+ markets.append(m)
out: dict = {
"markets": markets,
"requested": limit,
@@ -362,11 +382,22 @@ async def fetch_polymarket_open(fetcher: Fetcher, limit: int = 300) -> dict:
if error:
out["error"] = error
out["degraded"] = True
- if _stale_serves(fetcher, "polymarket") > stale_before:
+ if stale_ages or _stale_serves(fetcher, "polymarket") > stale_before:
out["stale"] = True
+ known = [a for a in stale_ages if a is not None]
+ out["stale_max_age_seconds"] = max(known) if known else None
return out
+def _stale_copy_age(fetcher: Fetcher, cache_key: str) -> float | None:
+ """Age in seconds of the cached copy under ``cache_key`` (a list
+ payload carries no inline age), or None when unknown."""
+ meta = fetcher.cache.get_stale_meta(cache_key)
+ if meta is None or not isinstance(meta[1], (int, float)):
+ return None
+ return round(max(0.0, time.time() - meta[1]), 1)
+
+
def _closed_ts(value) -> int | None:
if not value:
return None
diff --git a/src/world_intel_mcp/sources/prediction_cross.py b/src/world_intel_mcp/sources/prediction_cross.py
index a373195..8c0a041 100644
--- a/src/world_intel_mcp/sources/prediction_cross.py
+++ b/src/world_intel_mcp/sources/prediction_cross.py
@@ -33,10 +33,19 @@ Gaps / not covered:
- Polymarket scoring shares the ``polymarket`` circuit breaker and 1 s
floor with Gamma (see sources/prediction.py): ``limit=100`` makes up
to 100 sequential CLOB calls.
+- Matching runs in a worker thread so the event loop keeps serving;
+ a caller's timeout (the collector's 45 s) returns on time but cannot
+ stop a thread already matching, which finishes in the background.
+ Pair counts are not capped beyond the venue limits.
+- Kalshi last-trade lookups are paced and sequential (at most 10 within
+ 10 s); the trade price and time come from the same trade, but a
+ separate read from the listing, so a pair whose looked-up trade price
+ differs from the listing's last price is flagged.
"""
from __future__ import annotations
+import asyncio
import logging
import math
from datetime import datetime, timezone
@@ -63,7 +72,12 @@ _MATCH_METHOD = (
"when both titles state one; negation, rate actions (cut, raise, hold) and directions "
"(above, below) must agree; end dates within max(3 days, 15% of the horizon); the "
"Kalshi YES side must be named in the Polymarket question or match exactly one "
- "Polymarket outcome. match_confidence = title word overlap (Dice) x date factor "
+ "Polymarket outcome. When both titles state them, clock times and days of the same month "
+ "must be equal and months must overlap; quantity units ($, %, bps), 'to' vs 'by' and the "
+ "order of two or more amounts must agree; opposite words (rise/fall, positive/negative, "
+ "before/after, win/lose, approve/reject) must not conflict; a distinguishing name "
+ "(capitalized, or a listed chamber, party, country or qualifier) on both venues but not "
+ "shared rejects, and a listed one on one venue only rejects. match_confidence = title word overlap (Dice) x date factor "
"(1.0 same day, 0.5 at the tolerance edge): an ordinal score for ranking candidate "
"pairs, not a probability that two markets are the same event. One-to-one, greedy "
"by confidence."
@@ -79,8 +93,24 @@ _RANKING = (
MAX_PAIRS = 500
# Matched pairs whose Kalshi price is a last trade get that trade's time
-# looked up (one /markets/trades request each), for the first this many.
+# looked up (one /markets/trades request each), for the first this many,
+# within one overall deadline: the collector gives the whole comparison
+# 45 s, and a trades endpoint that accepts connections and never answers
+# held compare for 145.7 s before (measured in review, 2026-10-17).
MAX_TRADE_LOOKUPS = 10
+TRADE_LOOKUP_DEADLINE_SECONDS = 10.0
+# What a Kalshi side's ``last_trade_lookup`` says about its
+# ``last_trade_time`` (null for every status but "found" and
+# "found_in_expired_copy").
+LOOKUP_STATUSES = {
+ "found": "time and price of the latest trade, read live",
+ "found_in_expired_copy": "time and price of the latest trade in an expired cached copy (age in last_trade_age_seconds)",
+ "no_trade": "the venue lists no trade for the market",
+ "lookup_failed": "the trades endpoint could not be read or its trade time was unreadable",
+ "not_looked_up_cap": f"beyond the first {MAX_TRADE_LOOKUPS} last-trade pairs shown",
+ "not_looked_up_deadline": f"the lookups' {TRADE_LOOKUP_DEADLINE_SECONDS:g} s deadline passed first",
+ "not_looked_up_stale_listing": "the price came from an expired listing copy, so a live trade time would not date it",
+}
MAX_OFFSETS = 10
# A price window spans at most 8760 + 720 hours, so one market's hourly
# candles stay under the candlestick endpoint's 10,000 cap.
@@ -183,6 +213,8 @@ async def fetch_prediction_compare(
gaps.append(f"polymarket: {p_raw['error']}")
if p_raw.get("stale"):
gaps.append("polymarket: served from expired cache after a failed live fetch")
+ if p_raw.get("stale_max_age_seconds") is not None:
+ gaps.append(f"polymarket: the oldest open-markets copy is {p_raw['stale_max_age_seconds'] / 3600:.1f} h old")
gaps.append(
f"polymarket: top {p_raw['requested']} open markets by 24h volume requested; "
f"{p_raw['non_binary_skipped']} non-binary rows skipped"
@@ -197,10 +229,13 @@ async def fetch_prediction_compare(
"method": _MATCH_METHOD,
"price_gap": "kalshi YES probability minus polymarket probability of the same side",
"price_basis": (
- "per side: bid_ask_midpoint (two-sided book), last_trade (Kalshi's last trade; its time is "
- "looked up for the first matched pairs), gamma_outcome_price (Polymarket's listed outcome "
- "price; Gamma publishes no last-trade time) or no_price"
+ "per side: bid_ask_midpoint (two-sided book), last_trade (Kalshi's last trade; its time and "
+ f"price are looked up for the first {MAX_TRADE_LOOKUPS} such pairs shown, and "
+ "last_trade_lookup says what a null last_trade_time means), gamma_outcome_price "
+ "(Polymarket's listed outcome price; Gamma publishes no last-trade time) or no_price. A "
+ "side priced from an expired cached copy carries stale: true and its age, and is flagged"
),
+ "last_trade_lookup": LOOKUP_STATUSES,
"ranking": _RANKING,
"units": {"kalshi": kalshi.UNITS, "polymarket": prediction.UNITS},
"timestamp": _utc_now_iso(),
@@ -224,20 +259,18 @@ async def fetch_prediction_compare(
})
return out
- result = match_markets(k_markets, p_markets, min_confidence=min_confidence)
+ # CPU-bound (seconds at the documented maxima): off the event loop, so
+ # other tool calls and the collector's timeout are not starved.
+ result = await asyncio.to_thread(match_markets, k_markets, p_markets, min_confidence=min_confidence)
matched = result["matched"]
matched.sort(key=lambda r: (
r["price_gap"] is None, not r["comparable"], -abs(r["price_gap"] or 0.0), -r["match_confidence"],
))
shown = matched[:limit]
- lookups = [p for p in shown if p["kalshi"].get("price_basis") == "last_trade"][:MAX_TRADE_LOOKUPS]
- if lookups:
- times, trade_gaps = await kalshi.fetch_kalshi_last_trade_times(fetcher, [p["kalshi"]["id"] for p in lookups])
- for p in lookups:
- p["kalshi"]["last_trade_time"] = times.get(p["kalshi"]["id"])
- # Supplementary: a failed lookup leaves the time null and is named
- # here; the comparison itself is complete, so it is not degraded.
- gaps.extend(f"kalshi: {g}" for g in trade_gaps)
+ # Supplementary: a lookup that fails or is skipped leaves the time null,
+ # says why per side and is named in a gap; the comparison itself is
+ # complete, so it is not degraded.
+ gaps.extend(f"kalshi: {g}" for g in await _last_trades(fetcher, shown))
out.update({
"matched": shown,
"matched_count": len(matched),
@@ -253,6 +286,91 @@ async def fetch_prediction_compare(
return out
+async def _last_trades(fetcher: Fetcher, shown: list[dict]) -> list[str]:
+ """Look up the last trade behind each shown last-trade Kalshi price.
+
+ Sets ``last_trade_time``, ``last_trade_price`` and
+ ``last_trade_lookup`` (a LOOKUP_STATUSES key) on each such Kalshi
+ side, flags a pair whose looked-up trade price differs from the
+ listing's last price, and returns the gaps. At most
+ MAX_TRADE_LOOKUPS requests, all within TRADE_LOOKUP_DEADLINE_SECONDS.
+ """
+ sides = [p for p in shown if p["kalshi"].get("price_basis") == "last_trade"]
+ if not sides:
+ return []
+ live = [p for p in sides if not p["kalshi"].get("stale")]
+ targets = live[:MAX_TRADE_LOOKUPS]
+ target_ids = {id(p) for p in targets}
+ times: dict[str, str | None] = {}
+ prices: dict[str, float | None] = {}
+ stale: dict[str, float | None] = {}
+ attempted: set[str] = set()
+
+ async def run() -> None:
+ for p in targets:
+ ticker = p["kalshi"]["id"]
+ found, _ = await kalshi.fetch_kalshi_last_trade_times(
+ fetcher, [ticker], prices=prices, stale_tickers=stale
+ )
+ times.update(found)
+ attempted.add(ticker)
+
+ timed_out = False
+ try:
+ await asyncio.wait_for(run(), TRADE_LOOKUP_DEADLINE_SECONDS)
+ except asyncio.TimeoutError:
+ timed_out = True
+ counts: dict[str, int] = {}
+ for p in sides:
+ side = p["kalshi"]
+ ticker = side["id"]
+ if side.get("stale"):
+ status = "not_looked_up_stale_listing"
+ elif id(p) not in target_ids:
+ status = "not_looked_up_cap"
+ elif ticker not in attempted:
+ status = "not_looked_up_deadline"
+ elif ticker not in times:
+ status = "lookup_failed"
+ elif times[ticker] is None:
+ status = "no_trade"
+ else:
+ status = "found_in_expired_copy" if ticker in stale else "found"
+ counts[status] = counts.get(status, 0) + 1
+ side["last_trade_lookup"] = status
+ side["last_trade_time"] = times.get(ticker) if status.startswith("found") else None
+ side["last_trade_price"] = prices.get(ticker) if status.startswith("found") else None
+ if status == "found_in_expired_copy":
+ side["last_trade_age_seconds"] = stale[ticker]
+ listed, traded = side.get("last_price"), side["last_trade_price"]
+ if traded is not None and listed is not None and abs(traded - listed) > 1e-9:
+ p["flags"].append(
+ f"kalshi last trade at {traded:g} (looked up separately) differs from the listing's last price {listed:g}"
+ )
+ p["comparable"] = False
+ gaps = []
+ if counts.get("lookup_failed"):
+ gaps.append(f"Kalshi last-trade time unavailable for {counts['lookup_failed']} market(s)")
+ if stale:
+ gaps.append(kalshi.stale_gap("Kalshi last-trade times", list(stale.values())))
+ if counts.get("not_looked_up_cap"):
+ gaps.append(
+ f"last-trade time looked up for the first {len(targets)} of {len(live)} last-trade pairs "
+ f"shown; {counts['not_looked_up_cap']} not looked up (cap {MAX_TRADE_LOOKUPS})"
+ )
+ if timed_out:
+ gaps.append(
+ f"last-trade lookups stopped at the {TRADE_LOOKUP_DEADLINE_SECONDS:g} s deadline; "
+ f"{counts.get('not_looked_up_deadline', 0)} pair(s) not looked up"
+ )
+ if counts.get("not_looked_up_stale_listing"):
+ gaps.append(
+ f"last-trade time not looked up for {counts['not_looked_up_stale_listing']} pair(s) priced "
+ "from an expired listing copy"
+ )
+ return gaps
+
+
_STALE_HISTORY = "price history served from expired cache"
diff --git a/src/world_intel_mcp/tests/test_cli_prediction_stale.py b/src/world_intel_mcp/tests/test_cli_prediction_stale.py
new file mode 100644
index 0000000..b36eb7e
--- /dev/null
+++ b/src/world_intel_mcp/tests/test_cli_prediction_stale.py
@@ -0,0 +1,162 @@
+"""CLI marking of expired copies and last-trade lookups (re-review round,
+2026-10-17): ``predictions`` titles an expired Polymarket list as such
+and prints its data gaps; ``kalshi`` notes a market priced from an
+expired copy; ``prediction-compare`` says what each Kalshi last-trade
+lookup found.
+
+The ``predictions`` test runs the real source against a respx-mocked
+Gamma outage with an expired cached list; the others use fakes
+returning the documented shapes (pinned in test_prediction_cross_lookups.py).
+
+Gaps / not covered: terminal widths below 200 columns.
+"""
+
+import asyncio as asyncio_mod
+
+import httpx
+import pytest
+import respx
+from click.testing import CliRunner
+
+from world_intel_mcp import cli
+from world_intel_mcp.fetcher import Fetcher
+from world_intel_mcp.sources.prediction import fetch_prediction_markets
+from world_intel_mcp.tests.test_sources_kalshi_stale import TEN_DAYS, expire
+from world_intel_mcp.tests.test_sources_prediction import _MARKETS
+
+GAMMA_ANY = r".*gamma-api\.polymarket\.com.*"
+STALE_GAP = "served from expired cache after a failed live fetch"
+
+
[email protected](autouse=True)
+def _cli_isolation(tmp_path, monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setenv("WORLD_INTEL_CACHE_DB", str(tmp_path / "cli-cache.db"))
+ monkeypatch.setenv("COLUMNS", "200")
+ # rich pins the width when COLUMNS is set at import; None reads it per call.
+ monkeypatch.setattr(cli.console, "_width", None)
+ monkeypatch.setattr(cli, "_fetcher", None)
+
+ async def _sleep(*args, **kwargs) -> None:
+ return None
+
+ monkeypatch.setattr(asyncio_mod, "sleep", _sleep)
+
+
+def _invoke(*args: str):
+ return CliRunner().invoke(cli.main, list(args), catch_exceptions=False)
+
+
+def _fake(payload):
+ async def _f(*args, **kwargs):
+ return payload
+
+ return _f
+
+
+# ---------------------------------------------------------------------------
+# predictions (legacy Polymarket list)
+# ---------------------------------------------------------------------------
+
+async def test_legacy_list_from_expired_cache_is_stale_and_degraded(fetcher: Fetcher) -> None:
+ expire(fetcher, "prediction:polymarket:20", _MARKETS[:2])
+ respx.get(url__regex=GAMMA_ANY).mock(return_value=httpx.Response(500))
+
+ result = await fetch_prediction_markets(fetcher)
+
+ assert result["count"] == 2
+ assert result["stale"] is True and result["degraded"] is True
+ assert result["data_gaps"] == [STALE_GAP]
+
+
+def test_predictions_titles_an_expired_list_and_prints_its_gaps(
+ fetcher: Fetcher, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ expire(fetcher, "prediction:polymarket:20", _MARKETS[:2])
+ respx.get(url__regex=GAMMA_ANY).mock(return_value=httpx.Response(500))
+ monkeypatch.setattr(cli, "_fetcher", fetcher)
+
+ result = _invoke("predictions")
+
+ assert result.exit_code == 0
+ assert f"Gap: {STALE_GAP}" in result.output
+ assert "Prediction Markets (expired cached copy)" in result.output
+
+
+def test_predictions_live_list_has_the_plain_title_and_no_gap(
+ fetcher: Fetcher, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ respx.get(url__regex=GAMMA_ANY).mock(return_value=httpx.Response(200, json=_MARKETS[:2]))
+ monkeypatch.setattr(cli, "_fetcher", fetcher)
+
+ result = _invoke("predictions")
+
+ assert result.exit_code == 0
+ assert "Gap:" not in result.output and "expired" not in result.output
+ assert "Prediction Markets" in result.output
+
+
+# ---------------------------------------------------------------------------
+# kalshi and prediction-compare
+# ---------------------------------------------------------------------------
+
+def test_kalshi_notes_a_market_priced_from_an_expired_copy(monkeypatch: pytest.MonkeyPatch) -> None:
+ payload = {
+ "markets": [
+ {"question": "Will the Federal Reserve Hike rates by 25bps at their October 2026 meeting?",
+ "yes_side": "Hike 25bps", "probability": 0.135, "price_basis": "bid_ask_midpoint",
+ "volume_24h": 61814.96, "volume": 4066283.05, "stale": True, "stale_age_seconds": TEN_DAYS},
+ {"question": "Will the Federal Reserve Hike rates by 0bps at their October 2026 meeting?",
+ "yes_side": "Fed maintains rate", "probability": 0.865, "price_basis": "bid_ask_midpoint",
+ "volume_24h": 64959.15, "volume": 5635368.81, "stale": False, "stale_age_seconds": None},
+ ],
+ "count": 2, "scan": {}, "units": {}, "data_gaps": [],
+ }
+ monkeypatch.setattr(cli.kalshi_src, "fetch_kalshi_markets", _fake(payload))
+
+ result = _invoke("kalshi", "--series", "KXFEDDECISION")
+
+ assert result.exit_code == 0
+ assert result.output.count("(expired copy, 240.0 h old)") == 1
+
+
+def _pair(kalshi_side: dict) -> dict:
+ return {
+ "match_confidence": 0.724, "flags": [], "comparable": False,
+ "kalshi": dict({"question": "Will the Federal Reserve Cut rates by 25bps at their October 2026 meeting?"},
+ **kalshi_side),
+ "polymarket": {"question": "Will the Fed decrease interest rates by 25 bps after the October 2026 meeting?"},
+ "kalshi_yes_probability": 0.01, "polymarket_yes_probability": 0.0035, "price_gap": 0.0065,
+ }
+
+
+def test_compare_flags_say_what_each_last_trade_lookup_found(monkeypatch: pytest.MonkeyPatch) -> None:
+ payload = {
+ "matched": [
+ _pair({"last_trade_lookup": "found", "last_trade_time": "2026-10-17T17:43:15Z", "last_trade_price": 0.03}),
+ _pair({"last_trade_lookup": "found_in_expired_copy", "last_trade_time": "2026-10-07T21:54:56Z",
+ "last_trade_price": 0.01}),
+ _pair({"last_trade_lookup": "not_looked_up_cap", "last_trade_time": None}),
+ ],
+ "matched_count": 3, "kalshi_markets_compared": 3, "polymarket_markets_compared": 3,
+ "unmatched_kalshi": 0, "unmatched_polymarket": 0, "min_confidence": 0.6, "data_gaps": [],
+ }
+ monkeypatch.setattr(cli.prediction_cross, "fetch_prediction_compare", _fake(payload))
+
+ result = _invoke("prediction-compare")
+
+ assert result.exit_code == 0
+ # The Flags column wraps (max 40 wide), so the full notes are checked
+ # on the helper and their pieces in the rendered table.
+ assert [cli._last_trade_note(p["kalshi"]) for p in payload["matched"]] == [
+ "kalshi last trade 2026-10-17T17:43:15Z at 3.0%",
+ "kalshi last trade 2026-10-07T21:54:56Z at 1.0% (expired copy)",
+ "kalshi last trade: not looked up cap",
+ ]
+ for piece in ("2026-10-17T17:43:15Z", "at 3.0%", "2026-10-07T21:54:56Z", "(expired copy)",
+ "kalshi last trade: not looked up cap"):
+ assert piece in result.output
+ assert cli._last_trade_note({"question": "no lookup"}) is None
diff --git a/src/world_intel_mcp/tests/test_prediction_cross_lookups.py b/src/world_intel_mcp/tests/test_prediction_cross_lookups.py
new file mode 100644
index 0000000..73f5580
--- /dev/null
+++ b/src/world_intel_mcp/tests/test_prediction_cross_lookups.py
@@ -0,0 +1,305 @@
+"""Cross-venue compare: per-pair staleness, the last-trade lookup's
+bounds and statuses, quotes on the outcome the Kalshi YES pays on,
+Kalshi quote range checks, and matching off the event loop (re-review
+round, 2026-10-17).
+
+Same respx fixtures as test_prediction_cross.py and
+test_prediction_cross_review.py. respx keeps one route per pattern, and
+a later ``.mock()`` on the same pattern replaces the earlier response.
+
+Gaps / not covered: a real outage on either venue; the 10 s deadline
+itself (the deadline test shortens it); the collector's 45 s budget is
+not exercised end to end here.
+"""
+
+import asyncio as asyncio_mod
+import threading
+import time
+
+import httpx
+import pytest
+import respx
+
+from world_intel_mcp.analysis import prediction_matching as pm
+from world_intel_mcp.fetcher import Fetcher
+from world_intel_mcp.sources import kalshi, prediction_cross
+from world_intel_mcp.sources.prediction_cross import fetch_prediction_compare
+from world_intel_mcp.tests.test_prediction_cross import GAMMA, POLY_OPEN, _gamma
+from world_intel_mcp.tests.test_sources_kalshi import EVENTS, FED_MARKETS, _event, _m
+from world_intel_mcp.tests.test_sources_kalshi_stale import TEN_DAYS, TRADES, expire
+
+C25 = "KXFEDDECISION-26OCT-C25"
+H25 = "KXFEDDECISION-26OCT-H25"
+LAST_TRADE = "2026-10-17T17:43:15.314853Z"
+STALE_FLAG = "price from a copy 240.0 h old, served from expired cache after a failed live fetch"
+
+
[email protected](autouse=True)
+def _no_sleep(monkeypatch: pytest.MonkeyPatch) -> None:
+ async def _sleep(*args, **kwargs) -> None:
+ return None
+
+ monkeypatch.setattr(asyncio_mod, "sleep", _sleep)
+
+
+def _fed_events(status: int = 200) -> None:
+ respx.get(url__regex=EVENTS).mock(
+ return_value=httpx.Response(status, json={"events": [_event(FED_MARKETS)], "cursor": ""})
+ )
+
+
+def _gamma_open(status: int = 200, rows=POLY_OPEN) -> None:
+ respx.get(url__regex=GAMMA).mock(return_value=httpx.Response(status, json=rows))
+
+
+def _trades(price: str = "0.0100", status: int = 200, trades=None):
+ rows = [{"created_time": LAST_TRADE, "yes_price_dollars": price, "count_fp": "935.00"}] if trades is None else trades
+ return respx.get(url__regex=TRADES).mock(return_value=httpx.Response(status, json={"trades": rows, "cursor": ""}))
+
+
+def _by_kalshi(out: dict) -> dict:
+ return {p["kalshi"]["id"]: p for p in out["matched"]}
+
+
+# ---------------------------------------------------------------------------
+# Staleness per pair
+# ---------------------------------------------------------------------------
+
+async def test_a_pair_priced_from_an_expired_kalshi_copy_is_flagged_not_comparable(fetcher: Fetcher) -> None:
+ expire(fetcher, "kalshi:events:series:KXFEDDECISION:first", {"events": [_event(FED_MARKETS)], "cursor": ""})
+ _fed_events(500)
+ _gamma_open()
+ trades = _trades()
+
+ out = await fetch_prediction_compare(fetcher, series="KXFEDDECISION")
+
+ h25 = _by_kalshi(out)[H25]
+ assert h25["kalshi"]["stale"] is True
+ assert TEN_DAYS <= h25["kalshi"]["stale_age_seconds"] < TEN_DAYS + 60
+ assert h25["flags"] == [f"kalshi {STALE_FLAG}"] and h25["comparable"] is False
+ assert out["comparable_count"] == 0
+ # The trade lookup is skipped for a side priced from an expired listing:
+ # a live trade time beside an old price would read as a fresh price.
+ c25 = _by_kalshi(out)[C25]
+ assert c25["kalshi"]["last_trade_lookup"] == "not_looked_up_stale_listing"
+ assert c25["kalshi"]["last_trade_time"] is None
+ assert trades.calls.call_count == 0
+ assert "kalshi: last-trade time not looked up for 1 pair(s) priced from an expired listing copy" in out["data_gaps"]
+
+
+async def test_a_pair_priced_from_an_expired_polymarket_copy_is_flagged_and_ranked_after_live(fetcher: Fetcher) -> None:
+ _fed_events()
+ expire(fetcher, "prediction:polymarket:open:5:0", POLY_OPEN)
+ _gamma_open(500)
+ _trades()
+
+ out = await fetch_prediction_compare(fetcher, series="KXFEDDECISION", polymarket_limit=5)
+
+ h25 = _by_kalshi(out)[H25]
+ assert h25["polymarket"]["stale"] is True
+ assert h25["flags"] == [f"polymarket {STALE_FLAG}"] and h25["comparable"] is False
+ assert out["comparable_count"] == 0
+ assert "polymarket: the oldest open-markets copy is 240.0 h old" in out["data_gaps"]
+
+
+# ---------------------------------------------------------------------------
+# Last-trade lookups: statuses, trade price, cap and deadline
+# ---------------------------------------------------------------------------
+
+async def test_a_market_with_no_trade_says_no_trade(fetcher: Fetcher) -> None:
+ _fed_events()
+ _gamma_open()
+ _trades(trades=[])
+
+ out = await fetch_prediction_compare(fetcher, series="KXFEDDECISION")
+
+ side = _by_kalshi(out)[C25]["kalshi"]
+ assert (side["last_trade_lookup"], side["last_trade_time"]) == ("no_trade", None)
+ assert out["last_trade_lookup"]["no_trade"] # the statuses are explained in the response
+ assert "last_trade_lookup" not in _by_kalshi(out)[H25]["kalshi"] # a midpoint needs no lookup
+
+
+async def test_a_failed_lookup_says_lookup_failed(fetcher: Fetcher) -> None:
+ _fed_events()
+ _gamma_open()
+ _trades(status=500)
+
+ out = await fetch_prediction_compare(fetcher, series="KXFEDDECISION")
+
+ side = _by_kalshi(out)[C25]["kalshi"]
+ assert (side["last_trade_lookup"], side["last_trade_time"]) == ("lookup_failed", None)
+
+
+async def test_a_trade_price_that_differs_from_the_listing_is_flagged(fetcher: Fetcher) -> None:
+ _fed_events()
+ _gamma_open()
+ _trades(price="0.0300")
+
+ out = await fetch_prediction_compare(fetcher, series="KXFEDDECISION")
+
+ c25 = _by_kalshi(out)[C25]
+ assert (c25["kalshi"]["last_trade_time"], c25["kalshi"]["last_trade_price"]) == (LAST_TRADE, 0.03)
+ assert c25["kalshi"]["last_trade_lookup"] == "found"
+ assert "kalshi last trade at 0.03 (looked up separately) differs from the listing's last price 0.01" in c25["flags"]
+ assert c25["comparable"] is False
+
+
+async def test_a_trade_from_an_expired_copy_says_so_with_its_age(fetcher: Fetcher) -> None:
+ _fed_events()
+ _gamma_open()
+ expire(fetcher, f"kalshi:trades:last:{C25}", {"trades": [
+ {"created_time": LAST_TRADE, "yes_price_dollars": "0.0100", "count_fp": "935.00"},
+ ], "cursor": ""})
+ _trades(status=500)
+
+ out = await fetch_prediction_compare(fetcher, series="KXFEDDECISION")
+
+ side = _by_kalshi(out)[C25]["kalshi"]
+ assert (side["last_trade_lookup"], side["last_trade_time"]) == ("found_in_expired_copy", LAST_TRADE)
+ assert TEN_DAYS <= side["last_trade_age_seconds"] < TEN_DAYS + 60
+
+
+def _ladder(n: int):
+ """``n`` same-event pairs, each Kalshi side priced from a last trade."""
+ markets, rows = [], []
+ for i in range(n):
+ level = f"{100 + i},000"
+ k = _m(f"KXBTC-26OCT-B{100 + i}", f"Will Bitcoin be above ${level} at the end of October 2026?", "",
+ "0.0000", "0.0300", "0.0200", "1000.00", str(100 - i))
+ k["event_ticker"] = "KXBTC-26OCT"
+ markets.append(k)
+ rows.append(_gamma(f"77{i:02d}", f"Will Bitcoin be above ${level} at the end of October 2026?",
+ '["0.02", "0.98"]', 0.01, 0.03))
+ event = _event(markets, ticker="KXBTC-26OCT", series="KXBTC", title="Bitcoin at the end of October 2026?")
+ return event, rows
+
+
+async def test_last_trade_lookups_stop_at_the_cap_and_say_how_many_were_skipped(fetcher: Fetcher) -> None:
+ event, rows = _ladder(12)
+ respx.get(url__regex=EVENTS).mock(return_value=httpx.Response(200, json={"events": [event], "cursor": ""}))
+ _gamma_open(rows=rows)
+ trades = _trades(price="0.0200")
+
+ out = await fetch_prediction_compare(fetcher, series="KXBTC")
+
+ assert out["matched_count"] == 12
+ assert trades.calls.call_count == prediction_cross.MAX_TRADE_LOOKUPS == 10
+ statuses = [p["kalshi"]["last_trade_lookup"] for p in out["matched"]]
+ assert statuses == ["found"] * 10 + ["not_looked_up_cap"] * 2
+ assert "kalshi: last-trade time looked up for the first 10 of 12 last-trade pairs shown; 2 not looked up (cap 10)" in (
+ out["data_gaps"]
+ )
+
+
+async def test_a_trades_endpoint_that_never_answers_is_cut_at_the_deadline(
+ fetcher: Fetcher, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ monkeypatch.setattr(prediction_cross, "TRADE_LOOKUP_DEADLINE_SECONDS", 0.2)
+ _fed_events()
+ _gamma_open()
+
+ async def _hang(request):
+ await asyncio_mod.Event().wait() # asyncio.sleep is patched out here
+
+ respx.get(url__regex=TRADES).mock(side_effect=_hang)
+
+ started = time.monotonic()
+ # The outer bound turns a missing deadline into a failure, not a hung run.
+ out = await asyncio_mod.wait_for(fetch_prediction_compare(fetcher, series="KXFEDDECISION"), 5)
+ elapsed = time.monotonic() - started
+
+ assert elapsed < 5
+ side = _by_kalshi(out)[C25]["kalshi"]
+ assert (side["last_trade_lookup"], side["last_trade_time"]) == ("not_looked_up_deadline", None)
+ assert "kalshi: last-trade lookups stopped at the 0.2 s deadline; 1 pair(s) not looked up" in out["data_gaps"]
+ assert out["matched_count"] == 2 and "degraded" not in out # supplementary, the comparison is complete
+
+
+# ---------------------------------------------------------------------------
+# Matching runs off the event loop
+# ---------------------------------------------------------------------------
+
+async def test_matching_runs_in_a_worker_thread(fetcher: Fetcher, monkeypatch: pytest.MonkeyPatch) -> None:
+ _fed_events()
+ _gamma_open()
+ _trades()
+ seen: list[int] = []
+ real = prediction_cross.match_markets
+
+ def _spy(*args, **kwargs):
+ seen.append(threading.get_ident())
+ return real(*args, **kwargs)
+
+ monkeypatch.setattr(prediction_cross, "match_markets", _spy)
+
+ out = await fetch_prediction_compare(fetcher, series="KXFEDDECISION")
+
+ assert out["matched_count"] == 2
+ assert len(seen) == 1 and seen[0] != threading.get_ident()
+
+
+# ---------------------------------------------------------------------------
+# Pair briefs: the quoted outcome, quote ranges, no-price flags
+# ---------------------------------------------------------------------------
+
+def test_an_inverted_pair_quotes_the_outcome_the_kalshi_yes_pays_on() -> None:
+ from world_intel_mcp.tests.test_prediction_matching import NOW, _k, _p
+
+ k = _k("Will Zhejiang Lions win the Zhejiang Lions vs Shanxi Loongs game?", "Shanxi Loongs",
+ end="2026-10-17T13:00:00Z", prob=0.40, context="Zhejiang Lions vs Shanxi Loongs")
+ p = _p("Zhejiang Lions vs. Shanxi Loongs", outcomes=("Zhejiang Lions", "Shanxi Loongs"),
+ end="2026-10-17T14:00:00Z", prob=0.62)
+ p.update(yes_bid=0.61, yes_ask=0.63, last_price=0.6)
+
+ (pair,) = pm.match_markets([k], [p], min_confidence=0.5, now=NOW)["matched"]
+
+ assert pair["side"] == "inverted"
+ brief = pair["polymarket"]
+ assert brief["quoted_outcome"] == "Shanxi Loongs"
+ assert (brief["probability"], brief["yes_bid"], brief["yes_ask"], brief["last_price"]) == (0.38, 0.37, 0.39, 0.4)
+
+
+def test_kalshi_quotes_outside_0_to_1_are_left_null_and_the_pair_says_why() -> None:
+ raw = _m("KXFEDDECISION-26OCT-X", "Will the Federal Reserve Hike rates by 25bps at their October 2026 meeting?",
+ "Hike 25bps", "45", "46", "45", "10.00", "1.00")
+ market = kalshi.normalize_market(raw, _event([raw]))
+ assert (market["yes_bid"], market["yes_ask"], market["last_price"]) == (None, None, None)
+ assert (market["probability"], market["price_basis"]) == (None, "no_price")
+ assert kalshi.bad_quote_fields(raw) == ["yes_bid_dollars", "yes_ask_dollars", "last_price_dollars"]
+ assert kalshi.bad_quote_fields(FED_MARKETS[0]) == []
+
+ flags = pm._quote_flags("kalshi", market)
+ assert flags == ["kalshi price unavailable (basis: no_price)"]
+
+
+async def test_kalshi_listing_counts_quotes_that_are_not_prices(fetcher: Fetcher) -> None:
+ bad = _m("KXFEDDECISION-26OCT-X", "Will the Federal Reserve Hike rates by 75bps at their October 2026 meeting?",
+ "Hike 75bps", "45", "46", "45", "10.00", "1.00")
+ respx.get(url__regex=EVENTS).mock(
+ return_value=httpx.Response(200, json={"events": [_event(FED_MARKETS + [bad])], "cursor": ""})
+ )
+
+ out = await kalshi.fetch_kalshi_open_markets(fetcher, series="KXFEDDECISION")
+
+ assert ("1 market(s) with a quote field that is not a price from 0 to 1 (left null; the price falls back or "
+ "is missing)") in out["data_gaps"]
diff --git a/src/world_intel_mcp/tests/test_prediction_cross_review.py b/src/world_intel_mcp/tests/test_prediction_cross_review.py
index c22e2bf..333884a 100644
--- a/src/world_intel_mcp/tests/test_prediction_cross_review.py
+++ b/src/world_intel_mcp/tests/test_prediction_cross_review.py
@@ -8,8 +8,9 @@ earlier response, so each test sets up the shared venues first and
then overrides the one endpoint it fails.
Gaps / not covered: a real outage on either venue; ordering of the live
-trades endpoint (the lookup asks for one trade and does not depend on
-it).
+trades endpoint. The lookup asks for one trade (``limit=1``), so it
+relies on the endpoint listing newest first, which was observed live on
+2026-10-17 (two samples) and is not pinned by any test here.
"""
import asyncio as asyncio_mod
diff --git a/src/world_intel_mcp/tests/test_prediction_matching_gates.py b/src/world_intel_mcp/tests/test_prediction_matching_gates.py
index c046302..254b9f7 100644
--- a/src/world_intel_mcp/tests/test_prediction_matching_gates.py
+++ b/src/world_intel_mcp/tests/test_prediction_matching_gates.py
@@ -16,7 +16,8 @@ date tolerance is fixed.
Gaps / not covered: wording beyond these cases; the candidate-pair
completeness check is randomized over a small vocabulary, not
-exhaustive (the prefix bound itself was checked with z3).
+exhaustive, and the prefix bound is not machine-proved (the total order
+it relies on is tested in test_prediction_matching_late_gates.py).
"""
import random
diff --git a/src/world_intel_mcp/tests/test_prediction_matching_late_gates.py b/src/world_intel_mcp/tests/test_prediction_matching_late_gates.py
new file mode 100644
index 0000000..1029718
--- /dev/null
+++ b/src/world_intel_mcp/tests/test_prediction_matching_late_gates.py
@@ -0,0 +1,271 @@
+"""Gates that reject a pair from the two titles alone (re-review round,
+2026-10-17): days of the same month, months, clock times, quantity
+units and kinds, quantity order, opposite wording, rate actions worded
+as verbs, listed and distinguishing names, negation words, dash minus
+signs, number words, and a plain four-digit amount after a comparator.
+
+One rejection test per class, each with the reason the gate gives. The
+same-event cases each class must keep matching sit beside them: a day
+on one venue only, a clock time on one venue only, 25 bps against
+0.25%, "raised" against "increased", "hold" outside a rate question,
+"between" against at least/at most, and "Fed" against "Federal
+Reserve" (the live pairs keep matching in test_prediction_cross.py).
+
+Every expected reason and score below was computed with the module
+before it was written here (scratch probes); "now" is pinned so the
+date tolerance is fixed.
+
+Gaps / not covered: wording beyond these cases. Known false negatives
+(documented in the module): "before Jan 1, 2027" against "in 2026",
+"first" against "1st", "keep rates" against "hold rates". Names are
+read from capitalization and short lists, so an unlisted lower-case
+qualifier still costs only Dice.
+"""
+
+from datetime import datetime, timezone
+
+import pytest
+
+from world_intel_mcp.analysis import prediction_matching as pm
+
+NOW = datetime(2026, 10, 17, 16, 0, tzinfo=timezone.utc)
+END = "2026-12-10T19:00:00Z"
+
+
+def _k(question, yes="", end=END):
+ return {"venue": "kalshi", "id": "K", "question": question, "context": "", "yes_side": yes,
+ "outcomes": ["Yes", "No"], "end_time": end, "probability": 0.5}
+
+
+def _p(question, end=END):
+ return {"venue": "polymarket", "id": "P", "question": question, "context": "", "yes_side": "",
+ "outcomes": ["Yes", "No"], "end_time": end, "probability": 0.5}
+
+
+# ---------------------------------------------------------------------------
+# Different events: rejected, with the gate that caught them
+# ---------------------------------------------------------------------------
+
+# case: (Kalshi question, Kalshi YES side, Kalshi end, Polymarket question, Polymarket end, reason)
+REJECTED = {
+ "days of the same month": (
+ "Will Bitcoin be above $100,000 on Oct 20, 2026?", "", "2026-10-20T21:00:00Z",
+ "Will Bitcoin be above $100,000 on October 21, 2026?", "2026-10-21T16:00:00Z",
+ "days differ in october: Kalshi ['20'] vs Polymarket ['21']",
+ ),
+ "a deadline ladder": (
+ "Will the government shutdown end before Jan 1, 2027?", "", "2027-01-01T00:00:00Z",
+ "Will the government shutdown end before January 8, 2027?", "2027-01-08T00:00:00Z",
+ "days differ in january: Kalshi ['1'] vs Polymarket ['8']",
+ ),
+ "a deadline ladder with the day on the Kalshi YES side": (
+ "When will the government shutdown end?", "Before Jan 1, 2027", "2027-01-01T00:00:00Z",
+ "Will the government shutdown end before January 8, 2027?", "2027-01-08T00:00:00Z",
+ "days differ in january: Kalshi ['1'] vs Polymarket ['8']",
+ ),
+ "months": (
+ "Will the US enter a recession by March 2028?", "", "2028-03-31T00:00:00Z",
+ "Will the US enter a recession by April 2028?", "2028-04-30T00:00:00Z",
+ "months differ: Kalshi ['march'] vs Polymarket ['april']",
+ ),
+ "clock times": (
+ "Will Bitcoin be above $100,000 at 5pm ET on Oct 20, 2026?", "", "2026-10-20T21:00:00Z",
+ "Will Bitcoin be above $100,000 at 12pm ET on Oct 20, 2026?", "2026-10-20T16:00:00Z",
+ "clock times differ: Kalshi ['17:00'] vs Polymarket ['12:00']",
+ ),
+ "dollars vs percent": (
+ "Will US GDP grow by $500 in 2026?", "", END, "Will US GDP grow by 500% in 2026?", END,
+ "quantity units differ for 500: Kalshi ['$'] vs Polymarket ['%']",
+ ),
+ "basis points vs percent of the same number": (
+ "Will the Fed cut rates by 50 bps in December 2026?", "", END,
+ "Will the Fed cut rates by 50% in December 2026?", END,
+ "quantity units differ for 50: Kalshi ['bps'] vs Polymarket ['%']",
+ ),
+ "to a level vs by a change": (
+ "Will the Fed cut rates to 3.5% in December 2026?", "", END,
+ "Will the Fed cut rates by 3.5% in December 2026?", END,
+ "quantity kinds ('to' a level vs 'by' a change) differ for 3.5: Kalshi ['to'] vs Polymarket ['by']",
+ ),
+ "quantity order": (
+ "Will Brazil beat Argentina 2-1 in the 2026 final?", "", END,
+ "Will Brazil beat Argentina 1-2 in the 2026 final?", END,
+ "quantities in a different order: Kalshi ['2', '1'] vs Polymarket ['1', '2']",
+ ),
+ "rise vs fall": (
+ "Will oil prices rise in December 2026?", "", END, "Will oil prices fall in December 2026?", END,
+ "opposite wording: Kalshi ['up'] vs Polymarket ['down']",
+ ),
+ "positive vs negative": (
+ "Will Q4 GDP growth be positive in 2026?", "", END, "Will Q4 GDP growth be negative in 2026?", END,
+ "opposite wording: Kalshi ['positive'] vs Polymarket ['negative']",
+ ),
+ "before vs after": (
+ "Will Musk leave Tesla before the 2026 election?", "", END,
+ "Will Musk leave Tesla after the 2026 election?", END,
+ "opposite wording: Kalshi ['before'] vs Polymarket ['after']",
+ ),
+ "tighten vs ease": (
+ "Will the ECB tighten policy in December 2026?", "", END, "Will the ECB ease policy in December 2026?", END,
+ "actions differ: Kalshi ['raise'] vs Polymarket ['cut']",
+ ),
+ "boost vs reduce": (
+ "Will the BoE boost rates in December 2026?", "", END, "Will the BoE reduce rates in December 2026?", END,
+ "actions differ: Kalshi ['raise'] vs Polymarket ['cut']",
+ ),
+ "increase vs decrease": (
+ "Will the Fed increase rates in December 2026?", "", END,
+ "Will the Fed decrease rates in December 2026?", END,
+ "actions differ: Kalshi ['raise'] vs Polymarket ['cut']",
+ ),
+ "chambers": (
+ "Will Democrats win the House in 2026?", "", END, "Will Democrats win the Senate in 2026?", END,
+ "a listed name on one venue only: Kalshi ['house'] vs Polymarket ['senate']",
+ ),
+ "countries": (
+ "Will the US impose tariffs on China in 2026?", "", END,
+ "Will the US impose tariffs on Canada in 2026?", END,
+ "a listed name on one venue only: Kalshi ['china'] vs Polymarket ['canada']",
+ ),
+ "central banks named in full": (
+ "Will the Bank of England cut rates in December 2026?", "", END,
+ "Will the Bank of Japan cut rates in December 2026?", END,
+ "distinguishing names differ: Kalshi ['boe'] vs Polymarket ['boj']",
+ ),
+ "people": (
+ "Will Trump pardon Navarro in 2026?", "", END, "Will Trump pardon Bannon in 2026?", END,
+ "distinguishing names differ: Kalshi ['navarro'] vs Polymarket ['bannon']",
+ ),
+ "Bitcoin vs Bitcoin Cash": (
+ "Will Bitcoin hit $150k in 2026?", "", END, "Will Bitcoin Cash hit $150k in 2026?", END,
+ "a listed name on one venue only: Kalshi [] vs Polymarket ['cash']",
+ ),
+ "headline vs core": (
+ "Will CPI be above 3% in November 2026?", "", END, "Will core CPI be above 3% in November 2026?", END,
+ "a listed name on one venue only: Kalshi [] vs Polymarket ['core']",
+ ),
+ "won't": (
+ "Will Bitcoin hit $150k in 2026?", "", END, "Bitcoin won't hit $150k in 2026?", END,
+ "negation differs (one title says not/no/never/without, the other does not)",
+ ),
+ "fail to": (
+ "Will Congress pass the budget in 2026?", "", END, "Will Congress fail to pass the budget in 2026?", END,
+ "negation differs (one title says not/no/never/without, the other does not)",
+ ),
+ "an en dash minus sign": (
+ "Will Q4 GDP growth be below –0.5%?", "", END, "Will Q4 GDP growth be below 0.5%?", END,
+ "quantities differ: Kalshi ['4', '<-0.5'] vs Polymarket ['4', '<0.5']",
+ ),
+ "number words": (
+ "Will the Fed cut rates three times in 2026?", "", END, "Will the Fed cut rates four times in 2026?", END,
+ "quantities differ: Kalshi ['3'] vs Polymarket ['4']",
+ ),
+ "a plain four-digit amount after a comparator (above)": (
+ "Will the S&P 500 be above 2050 in 2026?", "", END, "Will the S&P 500 be above 2100 in 2026?", END,
+ "quantities differ: Kalshi ['500', '>2050'] vs Polymarket ['500', '>2100']",
+ ),
+ "a plain four-digit amount after a comparator (at least)": (
+ "Will gold be at least 2000 in 2026?", "", END, "Will gold be at least 1950 in 2026?", END,
+ "quantities differ: Kalshi ['>=2000'] vs Polymarket ['>=1950']",
+ ),
+}
+
+
[email protected]("case", sorted(REJECTED))
+def test_titles_naming_different_events_are_rejected(case: str) -> None:
+ kalshi_q, yes, kalshi_end, poly_q, poly_end, reason = REJECTED[case]
+ result = pm.score_pair(_k(kalshi_q, yes, kalshi_end), _p(poly_q, poly_end), now=NOW)
+ assert result["rejected"] is True and result["confidence"] == 0.0
+ assert result["reasons"][-1] == reason
+
+
+def test_a_day_rejection_also_keeps_the_pair_out_of_matches() -> None:
+ out = pm.match_markets(
+ [_k("Will Bitcoin be above $100,000 on Oct 20, 2026?", end="2026-10-20T21:00:00Z")],
+ [_p("Will Bitcoin be above $100,000 on October 21, 2026?", end="2026-10-21T16:00:00Z")],
+ now=NOW,
+ )
+ assert out["matched"] == [] and out["unmatched_kalshi"] == 1 and out["unmatched_polymarket"] == 1
+
+
+# ---------------------------------------------------------------------------
+# Same events: still matched
+# ---------------------------------------------------------------------------
+
+# case: (Kalshi question, Kalshi end, Polymarket question, Polymarket end, confidence, a reason it carries)
+MATCHED = {
+ "a day on one venue only": (
+ "Will the Fed cut rates on Oct 28, 2026?", "2026-10-28T18:00:00Z",
+ "Will the Fed cut rates in October 2026?", "2026-10-28T18:00:00Z",
+ 1.0, "actions agree ['cut']",
+ ),
+ "a clock time on one venue only": (
+ "Will Bitcoin be above $100,000 at 5pm ET on Oct 20, 2026?", "2026-10-20T21:00:00Z",
+ "Will Bitcoin be above $100,000 on Oct 20, 2026?", "2026-10-20T21:00:00Z",
+ 1.0, "quantities agree ['>100000']",
+ ),
+ "25 bps vs 0.25%": (
+ "Will the Fed cut rates by 25 bps in December 2026?", END,
+ "Will the Fed cut rates by 0.25% in December 2026?", END,
+ 0.889, "quantities agree once bps are read as percent: Kalshi ['25'] vs Polymarket ['0.25']",
+ ),
+ "raised vs increased": (
+ "Will the Fed have raised rates in December 2026?", END,
+ "Will the Fed have increased rates in December 2026?", END,
+ 1.0, "actions agree ['raise']",
+ ),
+ "hold outside a rate question is not an action": (
+ "Will Iran hold a presidential election in 2026?", END,
+ "Will Iran have a presidential election in 2026?", END,
+ 0.857, "both venues price YES on the question as worded",
+ ),
+ "between vs at least and at most": (
+ "Will CPI be between 3% and 4% in November 2026?", END,
+ "Will CPI be at least 3% and at most 4% in November 2026?", END,
+ 1.0, "both venues price YES on the question as worded",
+ ),
+ "Fed vs Federal Reserve": (
+ "Will the Fed hike rates in December 2026?", END,
+ "Will the Federal Reserve hike rates in December 2026?", END,
+ 0.667, "actions agree ['raise']",
+ ),
+}
+
+
[email protected]("case", sorted(MATCHED))
+def test_same_events_still_match(case: str) -> None:
+ kalshi_q, kalshi_end, poly_q, poly_end, confidence, reason = MATCHED[case]
+ result = pm.score_pair(_k(kalshi_q, end=kalshi_end), _p(poly_q, end=poly_end), now=NOW)
+ assert result["rejected"] is False and result["confidence"] == confidence
+ assert confidence >= pm.DEFAULT_MIN_CONFIDENCE
+ assert reason in result["reasons"]
+
+
+def test_hold_in_a_rate_question_is_still_an_action() -> None:
+ result = pm.score_pair(
+ _k("Will the Fed hold rates in December 2026?"), _p("Will the Fed raise rates in December 2026?"), now=NOW
+ )
+ assert result["reasons"][-1] == "actions differ: Kalshi ['hold'] vs Polymarket ['raise']"
+
+
+def test_clock_times_read_twelve_and_twenty_four_hour_forms() -> None:
+ assert pm.clock_times("5pm ET on Oct 20") == {"17:00"}
+ assert pm.clock_times("at 17:00 UTC") == {"17:00"}
+ assert pm.clock_times("12:30 pm") == {"12:30"}
+ assert pm.clock_times("noon") == set()
+
+
+# ---------------------------------------------------------------------------
+# Candidate blocking: the total order every market shares
+# ---------------------------------------------------------------------------
+
+def test_prefix_breaks_rarity_ties_on_the_word_itself() -> None:
+ """Prefix filtering finds every pair only if all markets order their
+ words the same way; with equal document frequencies that order is
+ alphabetical, never the set's iteration order."""
+ words = set("alpha bravo charlie delta echo foxtrot golf hotel india juliet kilo lima".split())
+ tied = {w: 3 for w in words}
+ # 12 words at 0.75: need = ceil(0.75 * 12 / 1.25) = 8, so the prefix keeps 12 - 8 + 1 = 5.
+ assert pm._prefix(words, tied, 0.75) == ["alpha", "bravo", "charlie", "delta", "echo"]
+ rarer = dict(tied, lima=1, alpha=5)
+ assert pm._prefix(words, rarer, 0.75) == ["lima", "bravo", "charlie", "delta", "echo"]
diff --git a/src/world_intel_mcp/tools/prediction_markets.py b/src/world_intel_mcp/tools/prediction_markets.py
index f4b97f1..a45d10d 100644
--- a/src/world_intel_mcp/tools/prediction_markets.py
+++ b/src/world_intel_mcp/tools/prediction_markets.py
@@ -26,7 +26,8 @@ TOOLS: list[Tool] = [
"$1 contract; raw bid/ask/last kept). Filter by category (e.g. 'Economics', 'Politics', "
"'Elections', 'Financials', 'Climate and Weather') or series ticker (e.g. 'KXFED'). The "
"response states which events were scanned and whether the scan was exhaustive; data "
- "served from expired cache after a failed fetch is marked stale and degraded."
+ "served from expired cache after a failed fetch is marked stale and degraded, and each "
+ "market from such a copy carries stale: true and its age."
),
inputSchema={
"type": "object",
@@ -70,13 +71,16 @@ TOOLS: list[Tool] = [
name="intel_prediction_compare",
description=(
"The same event on Kalshi and Polymarket side by side: deterministic matching on normalized "
- "titles, quantities, years, end dates, negation, rate actions (cut/raise/hold), directions "
- "(above/below) and outcome side, with a match_confidence (an ordinal score for ranking "
+ "titles, quantities (value, unit, to/by, order), years, months, days, clock times, end "
+ "dates, negation, rate actions (cut/raise/hold), directions (above/below), opposite words "
+ "(rise/fall, before/after, ...), distinguishing names (House/Senate, Fed/BoJ, Bitcoin/"
+ "Bitcoin Cash) and outcome side, with a match_confidence (an ordinal score for ranking "
"pairs, not a probability) and the reasons for each pair. Below min_confidence a pair is "
"not a match. Returns matched pairs with the price gap, each side's price basis, spread "
- "and last-trade time (a pair is flagged and ranked after comparable ones when either side "
- "is not a two-sided quote midpoint or its spread is wide), the closest rejected pairs with "
- "why, and unmatched counts per venue."
+ "and last-trade time with its lookup status (a pair is flagged and ranked after comparable "
+ "ones when either side is not a live two-sided quote midpoint, comes from an expired "
+ "cached copy or its spread is wide), the closest rejected pairs with why, and unmatched "
+ "counts per venue."
),
inputSchema={
"type": "object",