Skip to content

160-tool MCP server for real-time global intelligence from free public APIs: markets, liquidity, prediction markets, SEC filings, conflict, military, aircraft-cohort early warning, migration and border flows, cyber, climate, news and more. Dated observation history, 29 research prompts, cited briefs, geofences, SSE dashboard. MIT, no paid API keys.

Add lagged connections between stored series (Phase 29 wave 2)

Phoenix Agent (mac-studio)4bc1d0efa7a2
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c12802f..76b3fe5 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -229,6 +229,56 @@ source here, and no proxy is substituted for any of these measures.
history capture is an error.
- Tools 139 -> 143.
+### Added: connections with timing offsets (Phase 29)
+- `intel_lagged_connections` (CLI: `intel history connections`): Pearson
+ correlation between 2 to 8 series in the history store (keys as
+ `intel_series_history` lists them), aligned on a UTC day, week (Monday to
+ Sunday) or month grid and tested at every lag from `-max_lag` to
+ `+max_lag` (default 5, at most 30, reduced so one call runs at most 1,000
+ tests). Lag +k compares A at period t with B at t+k, so A leads B by k.
+- Default transform: period-over-period changes, never taken across a gap;
+ `levels` on request. Responses say which ran and why changes are the
+ default (levels of trending series correlate spuriously).
+- Per pair: the lag with the smallest p, with r, n (overlapping periods
+ after the transform and shift), a two-sided p from Student's t (computed
+ from the regularized incomplete beta function, standard library only) and
+ the Benjamini-Hochberg q across every test in the call, with the
+ Benjamini-Yekutieli `q_by` alongside. Significant at q <= 0.05 by default
+ (allowed 0.001 to 0.25); the other significant lags are listed too.
+- A pair and lag with fewer than 30 overlapping periods (callers may go down
+ to 20), or a constant side, is listed as not tested with its n, never
+ dropped. 30 points give about 80% power for a true |r| of 0.5 at alpha
+ 0.05 (module docstring).
+- One vintage, the latest collected value for each `observed_at`. Rows
+ without `observed_at` (Yahoo quotes) are not used, with the reason: rows
+ are stored only on change, so a missing day cannot be told from an
+ unchanged one. A period takes its last observation; nothing is averaged.
+- Serial correlation: each series' lag-1 autocorrelation after the
+ transform, and per strongest result an AR(1) effective sample size and
+ the p recomputed with it, as a diagnostic outside q. Nothing is
+ prewhitened.
+- `cannot_prove` in every response: correlation is not causation, a lead is
+ not a causal direction, serial correlation inflates significance, a
+ shared driver produces correlation, the store holds only what the
+ collector recorded, BH under dependence is approximate and covers one
+ call, and a small q is not a large effect.
+- An unknown grid or transform is an error naming the allowed values;
+ numeric inputs are clamped and each clamp is reported.
+- Measured 2026-10-17 over a live MCP stdio session and the CLI, on four
+ real FRED series (WALCL, WDTGAL, WTREGEN, RRPONTSYD) loaded into an
+ isolated store as one capture; illustrative, not findings. Week grid,
+ changes: 26 of 66 tests significant; the Wednesday TGA level against its
+ week average gives r 0.61 at lag 0 and 0.59 at lag +1, the averaging
+ window rather than a cause. Levels: Fed total assets against either TGA
+ series gives r 0.74 at lag 0 (n 1,244) with an AR(1) effective sample
+ size of 5.3 and 3.4. Day grid: the weekly series have no adjacent-day
+ changes, so nothing is tested and the response says why.
+- With the default 365-day observation retention
+ (`WORLD_INTEL_HISTORY_DAYS`), a store filled only by the collector spans
+ about a year, so the month grid cannot reach the 20-point floor without a
+ longer retention or backfilled history.
+- Tools 152 -> 153.
+
### Added: news signals (ROADMAP Phase 29)
- `intel_hostility_timeline`: per UTC day, GDELT DOC 2.0 coverage of a
country or a pair of countries (`timelinevolraw`): matching articles,
diff --git a/README.md b/README.md
index 0638fe3..1b291d5 100644
--- a/README.md
+++ b/README.md
@@ -6,7 +6,7 @@
[![Python 3.11+](https://img.shields.io/badge/Python-3.11%2B-green)](https://python.org)
[![License](https://img.shields.io/badge/License-MIT-yellow)](LICENSE)
-Real-time global intelligence across **30+ domains** with **152 MCP tools**, a live ops-center dashboard, a CLI, and a **Qdrant vector store** for enterprise-grade semantic search across accumulated intelligence. All data comes from free, public APIs: no paid subscriptions required.
+Real-time global intelligence across **30+ domains** with **153 MCP tools**, a live ops-center dashboard, a CLI, and a **Qdrant vector store** for enterprise-grade semantic search across accumulated intelligence. All data comes from free, public APIs: no paid subscriptions required.
Built for AI agents that need world awareness: market conditions, geopolitical risk, military posture, aircraft-cohort early warning, refugee and migration flows, supply chain disruptions, cyber threats, and more — all queryable via the Model Context Protocol. The vector store enables natural language queries like *"military activity near Taiwan"* or *"cyber threats targeting healthcare"* across all historical data.
@@ -67,8 +67,9 @@ Built for AI agents that need world awareness: market conditions, geopolitical r
| **Tropical Cyclones** | 2 | NHC (Atlantic, E/C Pacific) + JTWC (NW Pacific, N Indian, S Hemisphere), with opt-in forecast tracks: positions, UTC valid times and intensities, plus NHC's cone of uncertainty |
| **Situation Brief** | 1 | Cited situational awareness brief over MCP: bounded server-side overview synthesized via local Ollama, with a mechanically-cited fallback |
| **Collection History** | 4 | Collector run log (per-source health, overdue and failing sources), saved captures with change detection and replay, dated observation series, revisions |
+| **Connections Between Series** | 1 | Correlation between stored series at leads and lags, with a minimum sample, false-discovery control and what it cannot prove |
-**Total: 152 tools** across 30+ intelligence domains.
+**Total: 153 tools** across 30+ intelligence domains.
---
@@ -142,6 +143,8 @@ intel top-stories -m 4 # stories carried by 4+ outlets
intel status # cache + circuit breaker health
intel guide # the research guide (also served as intel://guide)
intel history health # per-source collection health from the collector's run log
+intel history connections central_bank_rates:federal_funds_rate central_bank_rates:boe_bank_rate_sonia
+ # correlation at leads and lags between stored series (day grid by default)
```
---
@@ -606,6 +609,13 @@ History accrues only while the collector (`intel-collector`) runs; nothing befor
| `intel_series_history` | One series' observations with all three dates, or the list of available series |
| `intel_revisions` | Values for the same `observed_at` that a later capture changed (USGS magnitude revisions, NOAA Kp updates) |
+### Connections Between Series (1)
+Reads the history store only, so the sample is what the collector saved, not each series' full history. A connection is a correlation in that sample: it does not show causation, and a lead does not show causal direction.
+
+| Tool | Description |
+|------|-------------|
+| `intel_lagged_connections` | Correlation between 2 to 8 stored series (keys as `intel_series_history` lists them) on a UTC day, week or month grid, tested at every lag from `-max_lag` to `+max_lag` (default 5, at most 30, at most 1,000 tests a call). Default transform is period-over-period changes, because levels of trending series correlate spuriously; `levels` on request. Per pair: the lag with the smallest p, with Pearson r, overlap n, two-sided p from Student's t and the Benjamini-Hochberg q across every test in the call (significant at q <= 0.05 by default). Lag +k compares A at period t with B at t+k, so A leads B by k. Pair/lag overlaps under 30 points are listed as not tested with their n; revised points use the latest vintage; rows without `observed_at` (Yahoo quotes) are not used. Each strongest result also carries an AR(1) effective sample size as a serial-correlation diagnostic |
+
---
## Watching your own area (geofences/AOIs)
diff --git a/ROADMAP.md b/ROADMAP.md
index 259a553..29f1c5b 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -580,7 +580,7 @@ dashboard.
| 1 | Liquidity and flows | Fed balance sheet, Treasury General Account, reverse repo and net liquidity (FRED keyless CSV), money supply (US M2, euro area M1/M2/M3, never summed), ECB, ICI fund flows and ETF net issuance, FINRA margin statistics, BLS series; amounts kept separate from flows | :yellow_circle: Built, unreleased: 5 tools (`intel_liquidity_board`, `intel_net_liquidity`, `intel_money_supply`, `intel_fed_liquidity_series`, `intel_ecb_series`), 5 CLI commands, 2 collector sources. FRED graph CSV and ECB Data Portal live-verified 2026-10-17. The TGA term is WDTGAL (Wednesday level, USD millions): WTREGEN is a week average; the net liquidity `method` block states each term's units, conversion and how RRPONTSYD (daily, USD billions) is aligned to the Wednesday. Blocked, measured 2026-10-17, and listed in the board's `blocked` field; no proxy is substituted, so fund flows and ETF net issuance have no source: ICI (money market fund assets at /research/stats/mmf, ETF net issuance at /research/stats/etf) answers 403 Akamai "Access Denied" on every path, robots.txt included; FINRA margin statistics XLSX answers 200, but the finra.org Terms of Use (Restrictions d and e) forbid building a database from the site and data mining, scraping or harvesting tools, robots included, and the page says data feeds are not available; BLS public API v1 answers 200 without a key (CPI-U, payrolls), but api.bls.gov robots.txt is `Disallow: /` and the terms page on www.bls.gov answers 403 to this server's User-Agent, so it is not used |
| 1 | Kalshi + cross-venue markets | Kalshi public market data next to Polymarket; same-event comparison with an explicit match confidence; settled markets scored against pre-settlement prices (Brier) | :red_circle: |
| 1 | News vs market reaction | Companies and countries named in story clusters mapped to tickers / country indices; same-day move against a benchmark | :white_check_mark: Built 2026-10-17, unreleased: `intel_news_market_reaction` and `intel news-reaction`. Company rows: stock, benchmark (SPY, or the exchange country's index) and excess return on the story's session; country rows: index return with a trailing z-score. Session rule in the exchange's timezone (after close = next session; weekends and holidays read from the listing's own bars). Open: `^TASI` and `^TA125` (Saudi and Israel in the country-index table) return 404 on Yahoo, so those benchmarks are data gaps; story times inherit `sources/news.py`'s RSS timestamp offset until that is fixed |
-| 2 | Connections with timing offsets | Correlation between retained series over a shared sample, tested at leads and lags, with minimum sample size and false-discovery control; states that it does not prove causation | :red_circle: |
+| 2 | Connections with timing offsets | Correlation between retained series over a shared sample, tested at leads and lags, with minimum sample size and false-discovery control; states that it does not prove causation | :white_check_mark: Built 2026-10-17, unreleased: `intel_lagged_connections` and `intel history connections`. Pearson r on a UTC day, week or month grid at every lag from -max_lag to +max_lag (default 5, cap 30, at most 1,000 tests a call), period-over-period changes by default, p from Student's t (regularized incomplete beta, standard library), Benjamini-Hochberg q across the call (default 0.05), a 30-point minimum with every untested pair and lag listed with its n, latest vintage, rows without `observed_at` refused with the reason, an AR(1) effective sample size as a diagnostic, and a `cannot_prove` list. p-values match scipy to 3.3e-13 relative error. Verified 2026-10-17 over a live MCP stdio session and the CLI on four real FRED series loaded into an isolated store. Not done: rank correlation, prewhitening |
| 2 | Day timeline | What every retained source recorded for a given day, by observation date or collection date | :red_circle: |
| 3 | Dashboard views | Liquidity board, situations, top stories, collection health panels | :red_circle: |
diff --git a/src/world_intel_mcp/analysis/lagged_connections.py b/src/world_intel_mcp/analysis/lagged_connections.py
new file mode 100644
index 0000000..94c2a35
--- /dev/null
+++ b/src/world_intel_mcp/analysis/lagged_connections.py
@@ -0,0 +1,845 @@
+"""Connections with timing offsets between series in the history store.
+
+For 2 to ``MAX_SERIES`` series the collector saved (``history.py``; keys as
+``intel_series_history`` lists them), the series are aligned on a UTC
+calendar grid, transformed, and their Pearson correlation is tested at every
+lag from ``-max_lag`` to ``+max_lag``, with a Benjamini-Hochberg
+false-discovery adjustment across every test run in the call. Standard
+library only.
+
+Alignment:
+- Grid periods are UTC calendar days, ISO weeks (Monday to Sunday, labelled
+ by the Monday's date) or calendar months. ``observed_at`` decides the
+ period: a ``YYYY-MM-DD`` date or an instant (stored in UTC) falls in the
+ day, week and month containing it; a ``YYYY-MM`` month fits only the month
+ grid and a ``YYYY`` year fits none. Rows coarser than the grid are not
+ used, and are counted.
+- Vintage: when a later capture revised the value for one ``observed_at``,
+ the latest collected value is used (ties broken by insertion order).
+ Earlier vintages are ignored; the number of revised observation dates is
+ reported per series.
+- Sampling: a period takes the value of its last observation by
+ ``observed_at``, so a daily series on the week grid is its last day of the
+ week. Values inside a period are not averaged: averaging a random walk
+ makes its period-to-period changes serially correlated.
+- Rows without ``observed_at`` (Yahoo quotes) are not used. Their only date
+ is ``collected_at``, when this server saw a changed value, and rows are
+ stored only when the value changes, so a day with no row cannot be told
+ from a day with an unchanged value. A series with no dated rows is
+ excluded with that reason.
+- ``start`` / ``end`` select whole grid periods (the ones containing them and
+ every period between). Without them the window is the series' combined
+ span, cut to the most recent ``MAX_PERIODS`` periods.
+
+Transform:
+- ``changes`` (default): a period's value minus the immediately preceding
+ period's, only when both exist; a change is never taken across a gap.
+ Levels of trending series correlate spuriously (two independent random
+ walks routinely reach |r| above 0.5), which is why changes are the default.
+- ``levels``: the sampled values.
+
+Tests:
+- Pairs are (A, B) in the order the series were given. Lag +k compares A at
+ period t with B at period t+k: a connection at +k means A's moves come k
+ periods before B's (A leads B by k); lag -k means B leads A by k.
+- n counts the periods where both transformed values exist after the shift
+ (pairwise complete). A (pair, lag) with n below the minimum, or with no
+ variation on either side, is not tested and is listed with its n.
+- p is two-sided, from Student's t with n - 2 degrees of freedom. With
+ t^2 = r^2 (n - 2) / (1 - r^2), df / (df + t^2) = 1 - r^2, so
+ P(|T| >= |t|) = I_{1 - r^2}((n - 2) / 2, 1 / 2), the regularized incomplete
+ beta function, evaluated by its continued fraction (DLMF 8.17.22) with the
+ modified Lentz method. No permutation test is run.
+- q is the Benjamini-Hochberg adjusted p over every test run in the call;
+ ``q_by`` is the Benjamini-Yekutieli version, valid under any dependence
+ between tests. A test is significant when q <= the threshold (default
+ ``DEFAULT_Q`` = 0.05).
+
+Minimum sample, ``DEFAULT_MIN_OVERLAP`` = 30 overlapping points (callers may
+raise it, or lower it to ``MIN_OVERLAP_FLOOR`` = 20): at n = 30 a two-sided
+test at alpha 0.05 detects a true |rho| of 0.5 with about 80% power before
+any multiple-testing correction (Fisher z approximation: n = ((1.960 +
+0.842) / atanh 0.5)^2 + 3 = 29.0; simulated with bivariate normal draws:
+0.826 at n = 30, 0.643 at n = 20). Below that, a moderate real correlation is
+more often missed than found, and one that passes rests on few points. The
+floor is 20 because below it a single outlying pair can dominate r and the t
+reference distribution leans on near-normal values.
+
+Serial correlation: p and q treat periods as independent. Each series'
+lag-1 autocorrelation after the transform is reported, and each strongest
+result carries ``n_effective_ar1`` and ``p_ar1``: n scaled by
+(1 - phi_a phi_b) / (1 + phi_a phi_b), the large-sample variance inflation
+of r for two AR(1) series (Bartlett), capped at n. That is a diagnostic only:
+q uses p, and nothing is prewhitened.
+
+Gaps / not covered:
+- Pearson only: no rank correlation, so a few extreme changes can drive r.
+- No prewhitening and no correction for autocorrelation beyond lag 1 or for
+ volatility clustering; ``p_ar1`` is an AR(1) approximation outside q.
+- Benjamini-Hochberg assumes independent or positively dependent tests;
+ neighbouring lags of one pair share most of their data, so q is
+ approximate (``q_by`` is the any-dependence bound).
+- The adjustment covers one call. Other calls with other series, grids or
+ windows are further tests it cannot see.
+- Business-day series lose their Monday changes on the day grid (Friday to
+ Monday is not adjacent). Within one week or month, two series' last
+ observations can fall on different days (Wednesday levels against a
+ Friday close), so "same period" is not "same day".
+- One vintage only (latest seen); there is no first-release view.
+- Reference p-values in the tests come from scipy in a throwaway venv, not
+ from an independent derivation at run time.
+"""
+
+from __future__ import annotations
+
+import math
+import re
+from datetime import date
+from typing import Any, Iterable, Sequence
+
+from ..history import HISTORY_ACCRUAL_NOTE, HistoryStore
+
+GRIDS = ("day", "week", "month")
+TRANSFORMS = ("changes", "levels")
+DEFAULT_GRID = "day"
+DEFAULT_TRANSFORM = "changes"
+DEFAULT_MAX_LAG = 5
+MAX_LAG_CAP = 30
+DEFAULT_MIN_OVERLAP = 30
+MIN_OVERLAP_FLOOR = 20
+DEFAULT_Q = 0.05
+Q_FLOOR = 0.001
+Q_CAP = 0.25
+MAX_SERIES = 8
+MAX_TESTS = 1000
+MAX_PERIODS = 3000
+# Fewer adjacent pairs than this and a lag-1 autocorrelation is not reported.
+_AUTOCORR_MIN_PAIRS = 10
+
+_UNIT = {"day": "day", "week": "week", "month": "month"}
+
+GRID_RULES = {
+ "day": "UTC calendar days",
+ "week": "ISO weeks in UTC, Monday to Sunday, labelled by the Monday's date",
+ "month": "UTC calendar months, labelled YYYY-MM",
+}
+SAMPLING_RULE = (
+ "Each period takes the value of its last observation by observed_at (a daily "
+ "series on the week grid is its last day of the week). Values inside a period "
+ "are not averaged."
+)
+VINTAGE_RULE = (
+ "Latest vintage: where later captures revised the value for an observed_at, the "
+ "latest collected value is used and earlier values are ignored."
+)
+UNDATED_RULE = (
+ "Rows without observed_at (for example Yahoo quotes, saved with collected_at "
+ "only) are not used: collected_at is when this server saw a changed value, not "
+ "when it was observed, and rows are stored only on change, so a day with no row "
+ "cannot be told from a day with an unchanged value."
+)
+TRANSFORM_RULES = {
+ "changes": (
+ "Period-over-period changes: a period's value minus the immediately "
+ "preceding period's, only when both exist; never across a gap."
+ ),
+ "levels": "Levels: the sampled values, untransformed.",
+}
+WHY_CHANGES = (
+ "Changes are the default because levels of trending series correlate "
+ "spuriously: two independent random walks routinely reach |r| above 0.5 in "
+ "levels and near 0 in changes."
+)
+LAG_CONVENTION = (
+ "Pairs are (A, B) in the order the series were given. Lag +k: A's value at "
+ "period t is compared with B's value at period t+k, so a connection at lag +k "
+ "means A's moves come k periods before B's (A leads B by k). Lag -k: B leads A "
+ "by k. Lag 0: the same period."
+)
+P_RULE = (
+ "Two-sided p from Student's t with n - 2 degrees of freedom, computed exactly "
+ "as I_(1 - r^2)((n - 2)/2, 1/2), the regularized incomplete beta function."
+)
+MIN_OVERLAP_REASON = (
+ "30 overlapping points give about 80% power to detect a true |r| of 0.5 at "
+ "alpha 0.05 (two-sided, before multiple-testing correction); fewer miss "
+ "moderate real correlations more often than they find them. The floor is 20: "
+ "below it one outlying pair can dominate r."
+)
+AR1_RULE = (
+ "p and q treat periods as independent. Done: each series' lag-1 "
+ "autocorrelation after the transform is reported, and each strongest result "
+ "carries n_effective_ar1 (n scaled by (1 - phi_a phi_b) / (1 + phi_a phi_b), "
+ "capped at n) and p_ar1 (p recomputed with it). Not done: prewhitening, "
+ "autocorrelation beyond lag 1, volatility clustering; p_ar1 does not enter q."
+)
+NOT_TESTED_REASONS = {
+ "below_min_overlap": "fewer overlapping periods than min_overlap",
+ "no_variation": "one side is constant over the overlap, so r is undefined",
+}
+CANNOT_PROVE = [
+ "Correlation is not causation: a strong r at any lag shows the two series moved "
+ "together in this sample, not that one moved the other.",
+ "A lead is not evidence of causal direction: when A's moves come k periods "
+ "before B's, B can still drive A (through expectations or release timing), or "
+ "a third factor can reach A first.",
+ "Serial correlation inflates significance even after differencing: p and q "
+ "treat periods as independent. What was done: lag-1 autocorrelations are "
+ "reported and p_ar1 recomputes the strongest results with an AR(1) effective "
+ "sample size. What was not: prewhitening, autocorrelation beyond lag 1, "
+ "volatility clustering. p_ar1 does not enter q.",
+ "A shared common driver produces correlation: two series that both respond to "
+ "a third (the business cycle, a policy date, a holiday calendar) correlate "
+ "with no link between them.",
+ "The store holds only what the collector recorded. " + HISTORY_ACCRUAL_NOTE
+ + " The sample is the collection period, not the series' full history, and "
+ "periods the collector missed are absent, not zero.",
+ "Benjamini-Hochberg controls the false discovery rate for independent or "
+ "positively dependent tests; neighbouring lags of one pair share most of "
+ "their data, so q is approximate (q_by holds under any dependence). The "
+ "adjustment covers this call only, not other calls with other series, grids "
+ "or windows.",
+]
+
+
+# ---------------------------------------------------------------------------
+# Statistics (standard library only)
+# ---------------------------------------------------------------------------
+
+_CF_EPS = 1e-15
+_CF_TINY = 1e-300
+_CF_MAX_ITER = 10_000
+
+
+def _beta_continued_fraction(a: float, b: float, x: float) -> float:
+ """Denominator f of I_x(a, b) = x^a (1-x)^b / (a B(a, b)) / f, where
+ f = 1 + d1 / (1 + d2 / (1 + ...)) with DLMF 8.17.22's coefficients
+ d_(2m+1) = -(a+m)(a+b+m)x / ((a+2m)(a+2m+1)) and
+ d_(2m) = m(b-m)x / ((a+2m-1)(a+2m)), by the modified Lentz method."""
+ f = c = 1.0
+ d = 0.0
+ for j in range(1, _CF_MAX_ITER):
+ m = j // 2
+ if j % 2:
+ coef = -(a + m) * (a + b + m) * x / ((a + 2 * m) * (a + 2 * m + 1))
+ else:
+ coef = m * (b - m) * x / ((a + 2 * m - 1) * (a + 2 * m))
+ d = 1.0 + coef * d
+ if abs(d) < _CF_TINY:
+ d = _CF_TINY
+ d = 1.0 / d
+ c = 1.0 + coef / c
+ if abs(c) < _CF_TINY:
+ c = _CF_TINY
+ delta = c * d
+ f *= delta
+ if abs(delta - 1.0) < _CF_EPS:
+ return f
+ raise ArithmeticError(f"incomplete beta did not converge (a={a}, b={b}, x={x})")
+
+
+def _ibeta(a: float, b: float, x: float, y: float) -> float:
+ """I_x(a, b), with y = 1 - x supplied by the caller so that whichever of
+ the two is small keeps its precision."""
+ if x <= 0.0:
+ return 0.0
+ if y <= 0.0:
+ return 1.0
+ if x > (a + 1.0) / (a + b + 2.0):
+ # The fraction converges fast only below this point; use
+ # I_x(a, b) = 1 - I_(1-x)(b, a).
+ return 1.0 - _ibeta(b, a, y, x)
+ log_front = (
+ a * math.log(x) + b * math.log(y) - math.log(a)
+ - (math.lgamma(a) + math.lgamma(b) - math.lgamma(a + b))
+ )
+ return math.exp(log_front) / _beta_continued_fraction(a, b, x)
+
+
+def regularized_incomplete_beta(a: float, b: float, x: float) -> float:
+ """I_x(a, b) for a, b > 0 and 0 <= x <= 1."""
+ if not (a > 0 and b > 0) or not 0.0 <= x <= 1.0:
+ raise ValueError(f"need a, b > 0 and 0 <= x <= 1, got a={a}, b={b}, x={x}")
+ return _ibeta(a, b, x, 1.0 - x)
+
+
+def correlation_p_value(r: float, n: float) -> float:
+ """Two-sided p of H0 rho = 0 from Student's t with n - 2 degrees of
+ freedom. ``n`` may be fractional (an effective sample size)."""
+ df = n - 2.0
+ if not df > 0:
+ raise ValueError(f"need n > 2, got {n}")
+ r = abs(r)
+ if r >= 1.0:
+ return 0.0
+ # t^2 = r^2 df / (1 - r^2), so df / (df + t^2) = 1 - r^2 exactly.
+ return min(1.0, _ibeta(df / 2.0, 0.5, (1.0 - r) * (1.0 + r), r * r))
+
+
+# Quotes a p from correlation_p_value, so it joins the list once that exists.
+CANNOT_PROVE.append(
+ "A small q is not a large effect: q says r is unlikely to be zero in this "
+ "sample, not that it is big. With 1,000 overlapping periods an |r| of 0.1 "
+ f"(1% of the variance shared) already gives p = {correlation_p_value(0.1, 1000):.2g}; "
+ "read r and n alongside q."
+)
+
+
+def pearson_r(xs: Sequence[float], ys: Sequence[float]) -> float | None:
+ """Pearson correlation, or None when either side does not vary."""
+ n = len(xs)
+ if n != len(ys):
+ raise ValueError("xs and ys differ in length")
+ if n < 2 or min(xs) == max(xs) or min(ys) == max(ys):
+ return None
+ mx = math.fsum(xs) / n
+ my = math.fsum(ys) / n
+ dx = [x - mx for x in xs]
+ dy = [y - my for y in ys]
+ sxx = math.fsum(d * d for d in dx)
+ syy = math.fsum(d * d for d in dy)
+ if sxx <= 0.0 or syy <= 0.0:
+ return None
+ r = math.fsum(u * v for u, v in zip(dx, dy)) / (math.sqrt(sxx) * math.sqrt(syy))
+ return max(-1.0, min(1.0, r))
+
+
+def benjamini_hochberg(pvalues: Sequence[float]) -> list[float]:
+ """BH adjusted p-values (q), in the input order: q at rank i is the
+ minimum over ranks j >= i of m p_(j) / j, capped at 1."""
+ m = len(pvalues)
+ order = sorted(range(m), key=lambda i: pvalues[i])
+ adjusted = [1.0] * m
+ running = 1.0
+ for rank in range(m, 0, -1):
+ i = order[rank - 1]
+ running = min(running, pvalues[i] * m / rank)
+ adjusted[i] = running
+ return adjusted
+
+
+def benjamini_yekutieli(pvalues: Sequence[float]) -> list[float]:
+ """BY adjusted p-values: BH scaled by the harmonic sum 1 + 1/2 + ... + 1/m,
+ which keeps the false discovery rate under any dependence."""
+ harmonic = math.fsum(1.0 / i for i in range(1, len(pvalues) + 1))
+ return [min(1.0, q * harmonic) for q in benjamini_hochberg(pvalues)]
+
+
+# ---------------------------------------------------------------------------
+# Grid
+# ---------------------------------------------------------------------------
+
+_DAY = re.compile(r"(\d{4})-(\d{2})-(\d{2})(?:T\d{2}:\d{2}:\d{2}Z)?")
+_MONTH = re.compile(r"(\d{4})-(\d{2})")
+_YEAR = re.compile(r"\d{4}")
+_ISO_DATE = re.compile(r"\d{4}-\d{2}-\d{2}")
+
+
+def period_of(observed_at: str, grid: str) -> tuple[int | None, str | None]:
+ """Integer period index of a stored ``observed_at`` on ``grid`` (adjacent
+ periods differ by 1), or ``(None, "coarser" | "unreadable")``."""
+ text = str(observed_at)
+ m = _DAY.fullmatch(text)
+ if m:
+ try:
+ day = date(int(m[1]), int(m[2]), int(m[3]))
+ except ValueError:
+ return None, "unreadable"
+ if grid == "day":
+ return day.toordinal(), None
+ if grid == "week":
+ # Ordinal 1 (0001-01-01) is a Monday.
+ return (day.toordinal() - 1) // 7, None
+ return day.year * 12 + day.month - 1, None
+ m = _MONTH.fullmatch(text)
+ if m:
+ if not 1 <= int(m[2]) <= 12:
+ return None, "unreadable"
+ if grid == "month":
+ return int(m[1]) * 12 + int(m[2]) - 1, None
+ return None, "coarser"
+ if _YEAR.fullmatch(text):
+ return None, "coarser"
+ return None, "unreadable"
+
+
+def period_label(index: int, grid: str) -> str:
+ if grid == "day":
+ return date.fromordinal(index).isoformat()
+ if grid == "week":
+ return date.fromordinal(7 * index + 1).isoformat()
+ return f"{index // 12:04d}-{index % 12 + 1:02d}"
+
+
+def apply_transform(levels: dict[int, float], transform: str) -> dict[int, float]:
+ if transform == "levels":
+ return dict(levels)
+ return {p: v - levels[p - 1] for p, v in levels.items() if p - 1 in levels}
+
+
+def lag1_autocorrelation(values: dict[int, float]) -> tuple[float | None, int]:
+ """Pearson r between each value and the next period's, with the number of
+ adjacent pairs; None below ``_AUTOCORR_MIN_PAIRS`` pairs or without
+ variation."""
+ xs = [values[p] for p in sorted(values) if p + 1 in values]
+ ys = [values[p + 1] for p in sorted(values) if p + 1 in values]
+ if len(xs) < _AUTOCORR_MIN_PAIRS:
+ return None, len(xs)
+ return pearson_r(xs, ys), len(xs)
+
+
+def overlap_at_lag(
+ a: dict[int, float], b: dict[int, float], lag: int
+) -> tuple[list[float], list[float]]:
+ """A at t paired with B at t + lag, over every t where both exist."""
+ xs: list[float] = []
+ ys: list[float] = []
+ for p in sorted(a):
+ other = b.get(p + lag)
+ if other is not None:
+ xs.append(a[p])
+ ys.append(other)
+ return xs, ys
+
+
+# ---------------------------------------------------------------------------
+# Inputs
+# ---------------------------------------------------------------------------
+
+
+class _InputError(ValueError):
+ pass
+
+
+def _integer(name: str, value: Any, default: int) -> int:
+ if value is None:
+ return default
+ if isinstance(value, bool):
+ raise _InputError(f"{name} must be an integer, got {value!r}")
+ if isinstance(value, int):
+ return value
+ if isinstance(value, float) and value.is_integer():
+ return int(value)
+ raise _InputError(f"{name} must be an integer, got {value!r}")
+
+
+def _clamp(name: str, value: Any, lo: Any, hi: Any, clamps: list[dict]) -> Any:
+ used = max(lo, min(hi, value))
+ if used != value:
+ clamps.append({"input": name, "requested": value, "used": used, "allowed": [lo, hi]})
+ return used
+
+
+def _iso_date(name: str, value: Any) -> date | None:
+ if value is None or value == "":
+ return None
+ if not isinstance(value, str) or not _ISO_DATE.fullmatch(value.strip()):
+ raise _InputError(f"{name} must be a date YYYY-MM-DD, got {value!r}")
+ try:
+ return date.fromisoformat(value.strip())
+ except ValueError as exc:
+ raise _InputError(f"{name} must be a date YYYY-MM-DD, got {value!r}") from exc
+
+
+def _series_keys(value: Any) -> list[str]:
+ if isinstance(value, str) or not isinstance(value, (list, tuple)):
+ raise _InputError(
+ "series_keys must be a list of 2 to "
+ f"{MAX_SERIES} series keys (as intel_series_history lists them)"
+ )
+ keys = []
+ for item in value:
+ if not isinstance(item, str) or not item.strip():
+ raise _InputError(f"series_keys entries must be non-empty strings, got {item!r}")
+ keys.append(item.strip())
+ if len(keys) < 2:
+ raise _InputError(f"series_keys needs at least 2 series, got {len(keys)}")
+ if len(keys) > MAX_SERIES:
+ raise _InputError(f"series_keys takes at most {MAX_SERIES} series per call, got {len(keys)}")
+ duplicates = sorted({k for k in keys if keys.count(k) > 1})
+ if duplicates:
+ raise _InputError(f"series_keys repeats {duplicates}")
+ return keys
+
+
+# ---------------------------------------------------------------------------
+# Store reads and per-series preparation
+# ---------------------------------------------------------------------------
+
+
+def _read_rows(store: HistoryStore, series_key: str) -> list[tuple]:
+ # HistoryStore has no bulk read for one series' every vintage, so this
+ # reads its observations table directly (parameterized, read-only).
+ return store._conn.execute(
+ "SELECT observed_at, value, source, unit, label FROM history_observations "
+ "WHERE series_key = ? ORDER BY observed_at, collected_at, id",
+ (series_key,),
+ ).fetchall()
+
+
+def _sample_on_grid(key: str, rows: list[tuple], grid: str) -> dict[str, Any]:
+ latest: dict[str, float] = {}
+ distinct: dict[str, set[float]] = {}
+ undated = 0
+ for observed_at, value, _source, _unit, _label in rows:
+ if observed_at is None:
+ undated += 1
+ continue
+ latest[observed_at] = float(value) # rows run oldest vintage first
+ distinct.setdefault(observed_at, set()).add(float(value))
+ coarser = unreadable = 0
+ coarser_kinds: set[str] = set()
+ levels: dict[int, float] = {}
+ for observed_at in sorted(latest):
+ period, why = period_of(observed_at, grid)
+ if period is None:
+ if why == "coarser":
+ coarser += 1
+ coarser_kinds.add("month" if len(observed_at) == 7 else "year")
+ else:
+ unreadable += 1
+ continue
+ levels[period] = latest[observed_at] # sorted, so the last one wins
+ last_row = rows[-1] if rows else (None, None, None, None, None)
+ return {
+ "series_key": key,
+ "source": last_row[2],
+ "unit": next((r[3] for r in reversed(rows) if r[3]), None),
+ "label": next((r[4] for r in reversed(rows) if r[4]), None),
+ "rows": len(rows),
+ "rows_without_observed_at": undated,
+ "rows_coarser_than_grid": coarser,
+ "rows_unreadable_observed_at": unreadable,
+ "revised_observed_at": sum(1 for values in distinct.values() if len(values) > 1),
+ "dated_observations": len(latest),
+ "_levels": levels,
+ "_coarser_kinds": coarser_kinds,
+ }
+
+
+def _exclusion_reason(info: dict, grid: str) -> str | None:
+ if info["dated_observations"] == 0:
+ return (
+ f"no row has observed_at ({info['rows_without_observed_at']} rows carry "
+ "collected_at only). " + UNDATED_RULE
+ )
+ if not info["_levels"]:
+ if info["rows_coarser_than_grid"]:
+ kinds = " and ".join(sorted(info["_coarser_kinds"]))
+ hint = " Use grid=month." if info["_coarser_kinds"] == {"month"} else ""
+ return (
+ f"every observation describes a {kinds}, coarser than the {grid} grid."
+ + hint
+ )
+ return "no observed_at could be read as a date"
+ return None
+
+
+def _median_gap(periods: Iterable[int]) -> float | None:
+ ordered = sorted(periods)
+ gaps = sorted(b - a for a, b in zip(ordered, ordered[1:]))
+ if not gaps:
+ return None
+ mid = len(gaps) // 2
+ return float(gaps[mid]) if len(gaps) % 2 else (gaps[mid - 1] + gaps[mid]) / 2.0
+
+
+def _sig(value: float) -> float:
+ return float(f"{value:.3g}")
+
+
+# ---------------------------------------------------------------------------
+# The analysis
+# ---------------------------------------------------------------------------
+
+
+def lagged_connections(
+ store: HistoryStore,
+ series_keys: Any,
+ grid: Any = None,
+ transform: Any = None,
+ max_lag: Any = None,
+ min_overlap: Any = None,
+ q: Any = None,
+ start: Any = None,
+ end: Any = None,
+) -> dict:
+ """Correlation between stored series at leads and lags (module docstring)."""
+ clamps: list[dict] = []
+ try:
+ keys = _series_keys(series_keys)
+ grid = DEFAULT_GRID if grid is None else grid
+ if grid not in GRIDS:
+ raise _InputError(f"unknown grid {grid!r}; allowed: {', '.join(GRIDS)}")
+ transform = DEFAULT_TRANSFORM if transform is None else transform
+ if transform not in TRANSFORMS:
+ raise _InputError(
+ f"unknown transform {transform!r}; allowed: {', '.join(TRANSFORMS)}"
+ )
+ requested_lag = _clamp(
+ "max_lag", _integer("max_lag", max_lag, DEFAULT_MAX_LAG), 0, MAX_LAG_CAP, clamps
+ )
+ min_n = _clamp(
+ "min_overlap",
+ _integer("min_overlap", min_overlap, DEFAULT_MIN_OVERLAP),
+ MIN_OVERLAP_FLOOR, MAX_PERIODS, clamps,
+ )
+ if q is None:
+ q = DEFAULT_Q
+ if isinstance(q, bool) or not isinstance(q, (int, float)) or not math.isfinite(q):
+ raise _InputError(f"q must be a number, got {q!r}")
+ q_threshold = _clamp("q", float(q), Q_FLOOR, Q_CAP, clamps)
+ start_d = _iso_date("start", start)
+ end_d = _iso_date("end", end)
+ if start_d and end_d and start_d > end_d:
+ raise _InputError(f"start {start_d} is after end {end_d}")
+ except _InputError as exc:
+ return {"error": str(exc)}
+
+ rows = {key: _read_rows(store, key) for key in keys}
+ unknown = [key for key in keys if not rows[key]]
+ if unknown:
+ return {
+ "error": f"no observations stored for {unknown}; call intel_series_history "
+ "without series_key to list available series",
+ "unknown_series": unknown,
+ }
+
+ infos = [_sample_on_grid(key, rows[key], grid) for key in keys]
+ excluded = []
+ usable = []
+ for info in infos:
+ reason = _exclusion_reason(info, grid)
+ if reason:
+ excluded.append({"series_key": info["series_key"], "reason": reason})
+ else:
+ usable.append(info)
+
+ # One shared window of whole grid periods.
+ window: dict[str, Any] = {"start": None, "end": None, "limited_to_recent": False}
+ lo = period_of(start_d.isoformat(), grid)[0] if start_d else None
+ hi = period_of(end_d.isoformat(), grid)[0] if end_d else None
+ if usable:
+ first = min(min(i["_levels"]) for i in usable)
+ last = max(max(i["_levels"]) for i in usable)
+ lo = first if lo is None else lo
+ hi = last if hi is None else hi
+ if hi - lo + 1 > MAX_PERIODS:
+ lo = hi - MAX_PERIODS + 1
+ window["limited_to_recent"] = True
+ window.update(start=period_label(lo, grid), end=period_label(hi, grid),
+ periods=max(0, hi - lo + 1))
+
+ still_usable = []
+ for info in usable:
+ inside = {p: v for p, v in info["_levels"].items() if lo <= p <= hi}
+ info["periods_outside_window"] = len(info["_levels"]) - len(inside)
+ info["_levels"] = inside
+ if not inside:
+ excluded.append({"series_key": info["series_key"],
+ "reason": "no observation inside the window"})
+ continue
+ info["_values"] = apply_transform(inside, transform)
+ still_usable.append(info)
+ usable = still_usable
+
+ pair_index = [(i, j) for i in range(len(usable)) for j in range(i + 1, len(usable))]
+ lag = requested_lag
+ if pair_index and len(pair_index) * (2 * lag + 1) > MAX_TESTS:
+ lag = max(0, (MAX_TESTS // len(pair_index) - 1) // 2)
+ clamps.append({
+ "input": "max_lag", "requested": requested_lag, "used": lag,
+ "reason": f"{len(pair_index)} pairs x {2 * lag + 1} lags keeps the call "
+ f"within {MAX_TESTS} tests",
+ })
+
+ tests: list[dict] = []
+ pairs: list[dict] = []
+ for i, j in pair_index:
+ a, b = usable[i], usable[j]
+ pair = {"a": a["series_key"], "b": b["series_key"], "not_tested": [], "_tests": []}
+ for k in range(-lag, lag + 1):
+ xs, ys = overlap_at_lag(a["_values"], b["_values"], k)
+ n = len(xs)
+ if n < min_n:
+ pair["not_tested"].append({"lag": k, "n": n, "reason": "below_min_overlap"})
+ continue
+ r = pearson_r(xs, ys)
+ if r is None:
+ pair["not_tested"].append({"lag": k, "n": n, "reason": "no_variation"})
+ continue
+ t_abs = math.inf if abs(r) >= 1.0 else abs(r) * math.sqrt((n - 2) / (1 - r * r))
+ test = {"lag": k, "r": r, "n": n, "p": correlation_p_value(r, n), "_t": t_abs}
+ tests.append(test)
+ pair["_tests"].append(test)
+ pairs.append(pair)
+
+ pvalues = [t["p"] for t in tests]
+ for test, q_bh, q_by in zip(tests, benjamini_hochberg(pvalues), benjamini_yekutieli(pvalues)):
+ test["q"], test["q_by"] = q_bh, q_by
+ test["significant"] = q_bh <= q_threshold
+
+ for info in usable:
+ info["_phi"], info["_phi_pairs"] = lag1_autocorrelation(info["_values"])
+
+ unit = _UNIT[grid]
+ gaps: list[str] = []
+
+ def _entry(test: dict, a: dict, b: dict, full: bool) -> dict:
+ k = test["lag"]
+ plural = "" if abs(k) == 1 else "s"
+ timing = (
+ f"A leads B by {k} {unit}{plural}" if k > 0
+ else f"B leads A by {-k} {unit}{plural}" if k < 0
+ else f"same {unit}"
+ )
+ out = {
+ "lag": k,
+ "timing": timing,
+ "r": round(test["r"], 4),
+ "direction": "same" if test["r"] > 0 else "opposite" if test["r"] < 0 else "none",
+ "n": test["n"],
+ "p": _sig(test["p"]),
+ "q": _sig(test["q"]),
+ "q_by": _sig(test["q_by"]),
+ "significant": test["significant"],
+ }
+ if full:
+ phi_a, phi_b = a["_phi"], b["_phi"]
+ if phi_a is None or phi_b is None:
+ out["n_effective_ar1"] = out["p_ar1"] = None
+ else:
+ prod = phi_a * phi_b
+ if prod >= 1.0:
+ n_eff = 0.0
+ elif prod <= -1.0:
+ n_eff = float(test["n"])
+ else:
+ n_eff = min(float(test["n"]), test["n"] * (1.0 - prod) / (1.0 + prod))
+ out["n_effective_ar1"] = round(n_eff, 1)
+ out["p_ar1"] = _sig(correlation_p_value(test["r"], n_eff)) if n_eff >= 3 else None
+ return out
+
+ pair_out = []
+ by_key = {info["series_key"]: info for info in usable}
+ for pair in pairs:
+ a, b = by_key[pair["a"]], by_key[pair["b"]]
+ ranked = sorted(pair["_tests"], key=lambda t: (t["p"], -t["_t"]))
+ strongest = [t for t in ranked if (t["p"], t["_t"]) == (ranked[0]["p"], ranked[0]["_t"])] if ranked else []
+ pair_out.append({
+ "a": pair["a"],
+ "b": pair["b"],
+ "lags_tested": len(pair["_tests"]),
+ "lags_not_tested": len(pair["not_tested"]),
+ "strongest": [_entry(t, a, b, full=True) for t in strongest],
+ "significant": [_entry(t, a, b, full=False) for t in ranked if t["significant"]],
+ "not_tested": pair["not_tested"],
+ })
+ if not ranked:
+ biggest = max((nt["n"] for nt in pair["not_tested"]), default=0)
+ gaps.append(
+ f"{pair['a']} / {pair['b']}: no lag could be tested (largest overlap "
+ f"{biggest} periods; minimum {min_n})"
+ )
+
+ series_out = []
+ for info in usable:
+ values = info["_values"]
+ series_out.append({
+ "series_key": info["series_key"],
+ "source": info["source"],
+ "label": info["label"],
+ "unit": info["unit"],
+ "rows": info["rows"],
+ "rows_without_observed_at": info["rows_without_observed_at"],
+ "rows_coarser_than_grid": info["rows_coarser_than_grid"],
+ "rows_unreadable_observed_at": info["rows_unreadable_observed_at"],
+ "revised_observed_at": info["revised_observed_at"],
+ "dated_observations": info["dated_observations"],
+ "periods": len(info["_levels"]),
+ "periods_outside_window": info["periods_outside_window"],
+ "first_period": period_label(min(info["_levels"]), grid),
+ "last_period": period_label(max(info["_levels"]), grid),
+ "median_period_gap": _median_gap(info["_levels"]),
+ "points_after_transform": len(values),
+ "lag1_autocorrelation": None if info["_phi"] is None else round(info["_phi"], 4),
+ "lag1_pairs": info["_phi_pairs"],
+ })
+ key = info["series_key"]
+ if info["rows_without_observed_at"]:
+ gaps.append(f"{key}: {info['rows_without_observed_at']} rows without observed_at not used")
+ if info["rows_coarser_than_grid"]:
+ gaps.append(f"{key}: {info['rows_coarser_than_grid']} rows coarser than the {grid} grid not used")
+ if info["rows_unreadable_observed_at"]:
+ gaps.append(f"{key}: {info['rows_unreadable_observed_at']} rows with an unreadable observed_at not used")
+ if info["revised_observed_at"]:
+ gaps.append(
+ f"{key}: {info['revised_observed_at']} observation dates were revised between "
+ "captures; the latest collected value is used"
+ )
+ if len(values) < min_n:
+ what = "period-over-period changes" if transform == "changes" else "periods"
+ gap = (f"{key}: {len(values)} {what} on the {grid} grid, fewer than the "
+ f"{min_n}-point minimum, so no test involving it can run")
+ spacing = _median_gap(info["_levels"])
+ if transform == "changes" and spacing is not None and spacing > 1:
+ gap += (f"; its observations are a median {spacing:g} {unit}s apart, "
+ "and changes need adjacent periods, so a coarser grid may fit")
+ gaps.append(gap)
+ if window["limited_to_recent"]:
+ gaps.append(f"window limited to the most recent {MAX_PERIODS} {unit}s")
+ if len(usable) < 2:
+ gaps.append("fewer than two series could be used, so nothing was tested")
+
+ significant = sum(1 for t in tests if t["significant"])
+ with_signal = sum(1 for p in pair_out if p["significant"])
+ not_run = sum(p["lags_not_tested"] for p in pair_out)
+ return {
+ "summary": (
+ f"{with_signal} of {len(pair_out)} pairs have a lag with q <= {q_threshold:g} "
+ f"({significant} of {len(tests)} tests; {not_run} pair/lag combinations not tested)"
+ ),
+ "series": series_out,
+ "excluded_series": excluded,
+ "pairs": pair_out,
+ "tests_run": len(tests),
+ "tests_not_run": not_run,
+ "significant_tests": significant,
+ "method": {
+ "grid": grid,
+ "grid_rule": GRID_RULES[grid],
+ "window": window,
+ "sampling": SAMPLING_RULE,
+ "vintage": VINTAGE_RULE,
+ "undated_rows": UNDATED_RULE,
+ "transform": transform,
+ "transform_rule": TRANSFORM_RULES[transform],
+ "why_changes_default": WHY_CHANGES,
+ "lags": [-lag, lag],
+ "lag_convention": LAG_CONVENTION,
+ "overlap": "Pairwise: each (pair, lag) test uses every period where both "
+ "transformed values exist; n is reported per test.",
+ "correlation": "Pearson r",
+ "p_value": P_RULE,
+ "multiple_testing": f"Benjamini-Hochberg q across all {len(tests)} tests run in "
+ "this call; q_by is the Benjamini-Yekutieli version, valid under any dependence.",
+ "q_threshold": q_threshold,
+ "q_threshold_default": DEFAULT_Q,
+ "strongest": "Per pair, the tested lag(s) with the smallest p (ties broken by "
+ "the larger |t|).",
+ "min_overlap": min_n,
+ "min_overlap_default": DEFAULT_MIN_OVERLAP,
+ "min_overlap_reason": MIN_OVERLAP_REASON,
+ "not_tested_reasons": dict(NOT_TESTED_REASONS),
+ "serial_correlation": AR1_RULE,
+ "limits": {
+ "max_series": MAX_SERIES, "max_lag": MAX_LAG_CAP,
+ "max_tests": MAX_TESTS, "max_periods": MAX_PERIODS,
+ "min_overlap_floor": MIN_OVERLAP_FLOOR, "q_range": [Q_FLOOR, Q_CAP],
+ },
+ },
+ "inputs_clamped": clamps,
+ "cannot_prove": list(CANNOT_PROVE),
+ "data_gaps": gaps,
+ "note": HISTORY_ACCRUAL_NOTE,
+ }
diff --git a/src/world_intel_mcp/cli.py b/src/world_intel_mcp/cli.py
index 23e13a5..6af7f9b 100644
--- a/src/world_intel_mcp/cli.py
+++ b/src/world_intel_mcp/cli.py
@@ -3318,5 +3318,105 @@ def history_revisions(
console.print(f"[yellow]Cannot prove:[/yellow] {escape(line)}")
+from .analysis import lagged_connections as _lagged # noqa: E402
+
+
+def _p_text(value: float) -> str:
+ return "<1e-300" if value == 0 else f"{value:.2g}"
+
+
+@history_group.command(name="connections")
[email protected]("series_keys", nargs=-1, required=True)
[email protected]("--grid", type=click.Choice(_lagged.GRIDS), default=_lagged.DEFAULT_GRID,
+ show_default=True, help="UTC calendar grid (weeks run Monday to Sunday)")
[email protected]("--transform", type=click.Choice(_lagged.TRANSFORMS),
+ default=_lagged.DEFAULT_TRANSFORM, show_default=True,
+ help="changes (period-over-period) or levels")
[email protected]("--max-lag", type=int, default=_lagged.DEFAULT_MAX_LAG, show_default=True,
+ help=f"Largest lead or lag in grid periods (0 to {_lagged.MAX_LAG_CAP})")
[email protected]("--min-overlap", type=int, default=_lagged.DEFAULT_MIN_OVERLAP, show_default=True,
+ help=f"Fewest overlapping periods a test needs (at least {_lagged.MIN_OVERLAP_FLOOR})")
[email protected]("--q", "q", type=float, default=_lagged.DEFAULT_Q, show_default=True,
+ help="False discovery rate threshold (Benjamini-Hochberg)")
[email protected]("--start", default=None, help="First date YYYY-MM-DD")
[email protected]("--end", default=None, help="Last date YYYY-MM-DD")
+def history_connections(
+ ctx: click.Context, series_keys: tuple[str, ...], grid: str, transform: str,
+ max_lag: int, min_overlap: int, q: float, start: str | None, end: str | None,
+) -> None:
+ """Correlation between stored series at leads and lags.
+
+ Lag +k compares the first series of a pair at period t with the second
+ at t+k, so the first leads by k. A connection is not a cause.
+ """
+ data = _with_history_store(
+ lambda s: _lagged.lagged_connections(
+ s, list(series_keys), grid=grid, transform=transform, max_lag=max_lag,
+ min_overlap=min_overlap, q=q, start=start, end=end,
+ )
+ )
+ if _bail_on_error(ctx, data):
+ return
+
+ method = data["method"]
+ lo, hi = method["lags"]
+ table = Table(
+ title=f"Connections ({method['grid']} grid, {method['transform']}, lags {lo:+d} to "
+ f"{hi:+d}, {data['tests_run']} tests, BH q <= {method['q_threshold']:g})",
+ box=box.SIMPLE_HEAVY,
+ )
+ for name in ("A", "B", "Lag", "Timing", "r", "n", "n_eff", "p", "q", "Significant"):
+ justify = "right" if name in ("Lag", "r", "n", "n_eff", "p", "q") else "left"
+ table.add_column(name, justify=justify)
+ for pair in data["pairs"]:
+ if not pair["strongest"]:
+ table.add_row(_cell(pair["a"]), _cell(pair["b"]), "-",
+ Text("not tested", style="dim"), "-", "-", "-", "-", "-", "-")
+ for entry in pair["strongest"]:
+ n_eff = entry.get("n_effective_ar1")
+ table.add_row(
+ _cell(pair["a"]), _cell(pair["b"]), f"{entry['lag']:+d}", entry["timing"],
+ f"{entry['r']:+.3f}", str(entry["n"]), "-" if n_eff is None else f"{n_eff:g}",
+ _p_text(entry["p"]), _p_text(entry["q"]),
+ Text("yes", style="green") if entry["significant"] else Text("no"),
+ )
+ console.print(table)
+ console.print(escape(data["summary"]))
+ console.print(
+ "n_eff: AR(1) effective sample size from the two series' lag-1 autocorrelations, "
+ "a diagnostic only: p and q use n."
+ )
+ console.print(f"Lag convention: {escape(method['lag_convention'])}")
+ console.print(
+ f"Transform: {escape(method['transform_rule'])} {escape(method['why_changes_default'])}"
+ )
+ for pair in data["pairs"]:
+ best = {e["lag"] for e in pair["strongest"]}
+ others = [e for e in pair["significant"] if e["lag"] not in best]
+ if others:
+ listed = ", ".join(
+ f"lag {e['lag']:+d} (r {e['r']:+.3f}, q {_p_text(e['q'])})" for e in others
+ )
+ console.print(f"Also significant: {escape(pair['a'])} / {escape(pair['b'])}: {listed}")
+ if pair["not_tested"]:
+ listed = ", ".join(
+ f"lag {nt['lag']:+d} (n {nt['n']}"
+ + (", no variation)" if nt["reason"] == "no_variation" else ")")
+ for nt in pair["not_tested"]
+ )
+ console.print(
+ f"[yellow]Not tested[/yellow] (minimum {method['min_overlap']} overlapping "
+ f"periods): {escape(pair['a'])} / {escape(pair['b'])}: {listed}"
+ )
+ for item in data["excluded_series"]:
+ console.print(
+ f"[yellow]Excluded:[/yellow] {escape(item['series_key'])}: {escape(item['reason'])}"
+ )
+ _print_data_gaps(data)
+ for line in data["cannot_prove"]:
+ console.print(f"[yellow]Cannot prove:[/yellow] {escape(line)}")
+
+
if __name__ == "__main__":
main()
diff --git a/src/world_intel_mcp/tests/test_lagged_connections.py b/src/world_intel_mcp/tests/test_lagged_connections.py
new file mode 100644
index 0000000..e203f41
--- /dev/null
+++ b/src/world_intel_mcp/tests/test_lagged_connections.py
@@ -0,0 +1,714 @@
+"""Tests for analysis/lagged_connections.py, the intel_lagged_connections
+tool and ``intel history connections``.
+
+Reference values (P_CASES, BETA_CASES, PEARSON_*) were computed once with
+scipy 1.18.1 and numpy 2.5.4 in a throwaway venv outside the repo and are
+hardcoded here; nothing in the repo imports scipy:
+
+- P_CASES: ``2 * scipy.stats.t.sf(abs(r) * sqrt(df / (1 - r**2)), df)`` with
+ ``df = n - 2`` (fractional n stands in for an effective sample size).
+- BETA_CASES: ``scipy.special.betainc(a, b, x)``.
+- PEARSON_R, PEARSON_P: ``scipy.stats.pearsonr(PEARSON_X, PEARSON_Y)``.
+
+The Benjamini-Hochberg examples are worked by hand in their test's
+docstring. Synthetic series come from ``random.Random(seed).random()``
+through Box-Muller, so they are identical on every supported Python (the
+module promises ``random()`` stays stable across versions; ``gauss`` does
+not). Series are written through ``HistoryStore.record_cycle`` with a
+test-only extractor, so the analysis reads what the production write path
+stored.
+"""
+
+import importlib
+import json
+import math
+import os
+import random
+import statistics
+import types
+from datetime import date, timedelta
+
+import pytest
+from click.testing import CliRunner
+
+from world_intel_mcp import cli, history
+from world_intel_mcp.analysis import lagged_connections as lc
+from world_intel_mcp.history import HistoryStore, Observation, SourceOutcome
+
+from .test_server_registry import _TMP_CACHE
+
+T0 = 1_760_000_000.0 # 2025-10-09T08:53:20Z
+D0 = date(2025, 1, 6) # a Monday
+
+P_CASES = [
+ (0.0, 30, 1.0),
+ (0.05, 30, 0.7930219578171515),
+ (0.3, 30, 0.10724594805795433),
+ (0.5, 30, 0.004899933667068084),
+ (0.9, 30, 1.316606070026386e-11),
+ (-0.5, 30, 0.004899933667068084),
+ (0.2, 100, 0.04603628646005413),
+ (0.1, 1000, 0.0015441161074010938),
+ (0.999, 50, 1.9009987456813015e-66),
+ (0.7, 3, 0.5063666222132699),
+ (0.5, 4, 0.5),
+ (0.3, 3000, 1.9407172086957736e-63),
+ (0.08, 3000, 1.1493171414190742e-05),
+ (0.5, 21.5, 0.019331374409111864),
+ (0.35, 7.25, 0.42927655756801303),
+ (0.62, 12, 0.031508181347306076),
+]
+BETA_CASES = [
+ (0.5, 0.5, 0.3, 0.36901011956554536),
+ (2.5, 0.5, 0.9, 0.4895897445644278),
+ (10.0, 3.0, 0.7, 0.25281534785499993),
+ (100.0, 0.5, 0.99, 0.15677586542444089),
+ (0.5, 50.0, 0.01, 0.6826956021258024),
+ (14.0, 0.5, 0.75, 0.004899933667068084),
+ (3.0, 7.0, 0.2, 0.26180249600000016),
+ (1.0, 1.0, 0.42, 0.42),
+]
+PEARSON_X = [1.2, 2.3, 2.9, 4.1, 5.0, 6.2, 6.8, 8.1, 9.0, 9.9, 11.3, 12.0]
+PEARSON_Y = [2.0, 1.1, 3.9, 3.0, 6.1, 4.2, 7.9, 6.0, 8.1, 11.9, 9.0, 10.2]
+PEARSON_R = 0.9036537837015148
+PEARSON_P = 5.551095396255053e-05
+
+
+def _normals(seed: int, n: int) -> list[float]:
+ rng = random.Random(seed)
+ out: list[float] = []
+ while len(out) < n:
+ u1, u2 = rng.random(), rng.random()
+ radius = math.sqrt(-2.0 * math.log(1.0 - u1))
+ out += [radius * math.cos(2 * math.pi * u2), radius * math.sin(2 * math.pi * u2)]
+ return out[:n]
+
+
+def _walk(seed: int, n: int) -> list[float]:
+ level, out = 0.0, []
+ for step in _normals(seed, n):
+ level += step
+ out.append(level)
+ return out
+
+
+def _days(values, offset: int = 0) -> list[tuple[str, float]]:
+ return [((D0 + timedelta(days=i + offset)).isoformat(), v) for i, v in enumerate(values)]
+
+
+def store(tmp_path, monkeypatch):
+ monkeypatch.setattr(history, "_EXTRACTORS", dict(history._EXTRACTORS))
+ history.register_extractor(
+ "lagtest",
+ lambda payload: [
+ Observation(p["key"], p["value"], observed_at=p["observed_at"])
+ for p in payload["points"]
+ ],
+ )
+ s = HistoryStore(tmp_path / "history.db")
+ yield s
+ s.close()
+
+
+def _write(store, series: dict, at: float = T0) -> None:
+ """One collector capture of source 'lagtest' holding every given point."""
+ points = [
+ {"key": key, "observed_at": observed_at, "value": value}
+ for key, rows in series.items()
+ for observed_at, value in rows
+ ]
+ store.record_cycle([SourceOutcome("lagtest", at, 1.0, {"points": points})], now=at)
+
+
+def _run(store, keys, **kw):
+ return lc.lagged_connections(store, [f"lagtest:{k}" if ":" not in k else k for k in keys], **kw)
+
+
+def _planted(store, at: float = T0) -> None:
+ """x is a random walk; y_t = x_(t-3) + noise, 200 days."""
+ x = _walk(11, 200)
+ noise = _normals(12, 200)
+ y = [x[t - 3] + 0.3 * noise[t] for t in range(3, 200)]
+ _write(store, {"x": _days(x), "y": _days(y, offset=3)}, at)
+
+
+# ---------------------------------------------------------------------------
+# Statistics against independent reference values
+# ---------------------------------------------------------------------------
+
+
[email protected]("r,n,expected", P_CASES)
+def test_p_value_matches_scipy_reference(r, n, expected) -> None:
+ assert lc.correlation_p_value(r, n) == pytest.approx(expected, rel=1e-9, abs=1e-300)
+
+
[email protected]("a,b,x,expected", BETA_CASES)
+def test_incomplete_beta_matches_scipy_reference(a, b, x, expected) -> None:
+ assert lc.regularized_incomplete_beta(a, b, x) == pytest.approx(expected, rel=1e-12)
+
+
+def test_incomplete_beta_edges_and_bad_arguments() -> None:
+ assert lc.regularized_incomplete_beta(2.0, 3.0, 0.0) == 0.0
+ assert lc.regularized_incomplete_beta(2.0, 3.0, 1.0) == 1.0
+ for bad in [(0.0, 1.0, 0.5), (1.0, -1.0, 0.5), (1.0, 1.0, 1.5)]:
+ with pytest.raises(ValueError):
+ lc.regularized_incomplete_beta(*bad)
+ with pytest.raises(ValueError):
+ lc.correlation_p_value(0.5, 2)
+ assert lc.correlation_p_value(1.0, 10) == 0.0
+ assert lc.correlation_p_value(-1.0, 10) == 0.0
+
+
+def test_pearson_matches_scipy_reference() -> None:
+ r = lc.pearson_r(PEARSON_X, PEARSON_Y)
+ assert r == pytest.approx(PEARSON_R, rel=1e-12)
+ assert lc.correlation_p_value(r, len(PEARSON_X)) == pytest.approx(PEARSON_P, rel=1e-9)
+ # statistics.correlation is a second, independent Pearson implementation.
+ assert r == pytest.approx(statistics.correlation(PEARSON_X, PEARSON_Y), rel=1e-12)
+ assert lc.pearson_r([1.0, 1.0, 1.0], [1.0, 2.0, 3.0]) is None
+ assert lc.pearson_r([0.1, 0.1, 0.1, 0.1], [1.0, 2.0, 3.0, 4.0]) is None
+ with pytest.raises(ValueError):
+ lc.pearson_r([1.0, 2.0], [1.0])
+
+
+def test_benjamini_hochberg_hand_worked() -> None:
+ """m = 4, p = (0.02, 0.03, 0.035, 0.5). Sorted ranks give m p / rank of
+ 0.08, 0.06, 0.04667 and 0.5; stepping down from the largest rank keeps
+ the running minimum, so q = (0.04667, 0.04667, 0.04667, 0.5). The first
+ p alone (0.08 > 0.05) would not pass; the step-up rescues it.
+
+ m = 5, p = (0.01, 0.04, 0.03, 0.005, 0.2): m p / rank in rank order is
+ 0.025, 0.025, 0.05, 0.05, 0.2, already non-decreasing, so q in input
+ order is (0.025, 0.05, 0.05, 0.025, 0.2).
+
+ BY multiplies BH by 1 + 1/2 + 1/3 + 1/4 = 25/12 for m = 4: 0.04667 x
+ 25/12 = 0.09722, and 0.5 x 25/12 caps at 1."""
+ assert lc.benjamini_hochberg([0.02, 0.03, 0.035, 0.5]) == pytest.approx(
+ [0.14 / 3, 0.14 / 3, 0.14 / 3, 0.5]
+ )
+ assert lc.benjamini_hochberg([0.01, 0.04, 0.03, 0.005, 0.2]) == pytest.approx(
+ [0.025, 0.05, 0.05, 0.025, 0.2]
+ )
+ assert lc.benjamini_yekutieli([0.02, 0.03, 0.035, 0.5]) == pytest.approx(
+ [0.14 / 3 * 25 / 12] * 3 + [1.0]
+ )
+ assert lc.benjamini_hochberg([0.9, 0.95]) == pytest.approx([0.95, 0.95])
+ assert lc.benjamini_hochberg([]) == []
+
+
+# ---------------------------------------------------------------------------
+# Grid, sampling, transform
+# ---------------------------------------------------------------------------
+
+
+def test_period_rules_for_each_grid() -> None:
+ day = date(2026, 10, 12).toordinal()
+ assert lc.period_of("2026-10-12", "day") == (day, None)
+ assert lc.period_of("2026-10-12T23:59:59Z", "day") == (day, None)
+ assert lc.period_of("2026-10-13", "day") == (day + 1, None)
+ # Weeks run Monday (2026-10-12) to Sunday (2026-10-18).
+ monday = lc.period_of("2026-10-12", "week")[0]
+ assert lc.period_of("2026-10-18T22:00:00Z", "week")[0] == monday
+ assert lc.period_of("2026-10-19", "week")[0] == monday + 1
+ assert lc.period_label(lc.period_of("2026-10-15", "week")[0], "week") == "2026-10-12"
+ december = lc.period_of("2025-12-31", "month")[0]
+ assert lc.period_of("2026-01-01", "month")[0] == december + 1
+ assert lc.period_of("2026-01", "month")[0] == december + 1
+ assert lc.period_label(december + 1, "month") == "2026-01"
+ assert lc.period_label(day, "day") == "2026-10-12"
+ assert lc.period_of("2026-01", "day") == (None, "coarser")
+ assert lc.period_of("2026-01", "week") == (None, "coarser")
+ assert lc.period_of("2026", "month") == (None, "coarser")
+ for bad in ("2026-13", "2026-02-30", "yesterday", "2026-10-12T03:00:00+02:00"):
+ assert lc.period_of(bad, "day")[1] == "unreadable"
+
+
+def test_changes_are_never_taken_across_a_gap() -> None:
+ levels = {1: 10.0, 2: 12.0, 3: 11.0, 5: 20.0, 6: 26.0}
+ assert lc.apply_transform(levels, "changes") == {2: 2.0, 3: -1.0, 6: 6.0}
+ assert lc.apply_transform(levels, "levels") == levels
+
+
+def test_lag1_autocorrelation_uses_adjacent_periods_only() -> None:
+ values = {p: v for p, v in enumerate([3.0, 1.0, 4.0, 1.0, 5.0, 9.0, 2.0, 6.0, 5.0, 3.0, 5.0, 8.0])}
+ values[40] = 100.0 # isolated: no neighbour, so it joins no pair
+ phi, pairs = lc.lag1_autocorrelation(values)
+ seq = [values[p] for p in range(12)]
+ assert pairs == 11
+ assert phi == pytest.approx(statistics.correlation(seq[:-1], seq[1:]), rel=1e-12)
+ assert lc.lag1_autocorrelation({p: float(p % 3) for p in range(5)}) == (None, 4)
+
+
+def test_week_grid_takes_each_weeks_last_observation_and_latest_vintage(store) -> None:
+ # Business days 2025-01-06 (Mon) to 2025-01-16 (Thu), valued 0..8.
+ weekdays = [D0 + timedelta(days=i) for i in range(11) if (D0 + timedelta(days=i)).weekday() < 5]
+ _write(store, {"w": [(d.isoformat(), float(i)) for i, d in enumerate(weekdays)]})
+ friday = weekdays[4].isoformat() # 2025-01-10
+ _write(store, {"w": [(friday, 99.0)]}, at=T0 + 600) # a later capture revises Friday
+ info = lc._sample_on_grid("lagtest:w", lc._read_rows(store, "lagtest:w"), "week")
+ first_week = lc.period_of(D0.isoformat(), "week")[0]
+ # Week 1 ends on Friday, read at its revised value; week 2 ends on Thursday (8).
+ assert info["_levels"] == {first_week: 99.0, first_week + 1: 8.0}
+ assert info["revised_observed_at"] == 1
+ assert (info["rows"], info["dated_observations"]) == (10, 9)
+
+
+# ---------------------------------------------------------------------------
+# What the tests find
+# ---------------------------------------------------------------------------
+
+
+def test_planted_lag_is_found_at_the_predicted_lag_and_survives_fdr(store) -> None:
+ _planted(store)
+ result = _run(store, ["x", "y"])
+ (pair,) = result["pairs"]
+ (best,) = pair["strongest"]
+ # y_t = x_(t-3): x's value at t is compared with y's at t+3, so +3.
+ assert best["lag"] == 3
+ assert best["timing"] == "A leads B by 3 days"
+ assert best["direction"] == "same"
+ assert best["r"] > 0.9
+ # 199 changes of x (days 1-199), 196 of y (days 4-199); at +3 they meet on 196.
+ assert best["n"] == 196
+ assert best["q"] < 1e-50 and best["significant"] is True
+ assert [e["lag"] for e in pair["significant"]] == [3]
+ assert result["tests_run"] == 11 and result["significant_tests"] == 1
+ assert result["method"]["q_threshold"] == 0.05
+ assert "A leads B by k" in result["method"]["lag_convention"]
+ assert result["method"]["transform"] == "changes"
+
+
+def test_lag_sign_follows_the_order_the_series_are_given(store) -> None:
+ """B_t = A_(t-2) exactly: given as (A, B) the connection sits at +2 with
+ r = 1; given as (B, A) it sits at -2, read as B leads A."""
+ a = _walk(5, 60)
+ _write(store, {"a": _days(a), "b": _days(a[:-2], offset=2)})
+ forward = _run(store, ["a", "b"])["pairs"][0]["strongest"][0]
+ assert (forward["lag"], forward["timing"]) == (2, "A leads B by 2 days")
+ assert forward["r"] == pytest.approx(1.0) and forward["p"] == 0.0
+ backward = _run(store, ["b", "a"])["pairs"][0]["strongest"][0]
+ assert (backward["lag"], backward["timing"]) == (-2, "B leads A by 2 days")
+
+
+def test_independent_series_give_no_fdr_significant_result(store) -> None:
+ """Four independent random walks (seeds 101-104), 250 days, default
+ settings: 6 pairs x 11 lags = 66 tests, none at q <= 0.05 (the smallest
+ q measured is 0.699)."""
+ _write(store, {f"ind{i}": _days(_walk(100 + i, 250)) for i in range(1, 5)})
+ result = _run(store, [f"ind{i}" for i in range(1, 5)])
+ assert result["tests_run"] == 66
+ assert result["significant_tests"] == 0
+ assert all(not p["significant"] for p in result["pairs"])
+ assert min(e["q"] for p in result["pairs"] for e in p["strongest"]) > 0.05
+ assert result["summary"].startswith("0 of 6 pairs")
+
+
+def test_random_walks_correlate_in_levels_but_not_in_changes(store) -> None:
+ """Independent random walks are the textbook spurious correlation. Seeds
+ 1000/1001: |r| = 0.87 in levels (q ~ 1e-59), nothing in changes. Over 20
+ seed pairs (measured: 14 with an FDR-significant level lag, 1 with an
+ FDR-significant change lag, the nominal 5% rate) the default protects.
+ The AR(1) diagnostic flags the level result: its effective n is ~1."""
+ _write(store, {"rwa": _days(_walk(1000, 200)), "rwb": _days(_walk(1001, 200))})
+ levels = _run(store, ["rwa", "rwb"], transform="levels")
+ best = levels["pairs"][0]["strongest"][0]
+ assert abs(best["r"]) > 0.8 and best["significant"] is True
+ assert best["n_effective_ar1"] < 0.1 * best["n"]
+ assert best["p_ar1"] is None or best["p_ar1"] > 100 * best["p"]
+ assert all(s["lag1_autocorrelation"] > 0.9 for s in levels["series"])
+ changes = _run(store, ["rwa", "rwb"])
+ assert changes["significant_tests"] == 0
+ assert abs(changes["pairs"][0]["strongest"][0]["r"]) < 0.2
+
+ level_hits = change_hits = 0
+ for s in range(1, 20):
+ _write(store, {f"a{s}": _days(_walk(1000 + 2 * s, 200)),
+ f"b{s}": _days(_walk(1001 + 2 * s, 200))}, at=T0 + s)
+ level_hits += _run(store, [f"a{s}", f"b{s}"], transform="levels")["significant_tests"] > 0
+ change_hits += _run(store, [f"a{s}", f"b{s}"])["significant_tests"] > 0
+ assert level_hits >= 12
+ assert change_hits <= 2
+
+
+def test_weekly_grid_finds_a_two_week_lead_in_daily_data(store) -> None:
+ """y(d) = x(d - 14) daily; on the week grid each week takes its Sunday,
+ so Y_w = X_(w-2) and the weekly changes match exactly at +2 weeks."""
+ x = _walk(21, 7 * 60)
+ _write(store, {"x": _days(x), "y": _days(x[:-14], offset=14)})
+ result = _run(store, ["x", "y"], grid="week", max_lag=3)
+ best = result["pairs"][0]["strongest"][0]
+ assert (best["lag"], best["timing"]) == (2, "A leads B by 2 weeks")
+ assert best["r"] == pytest.approx(1.0)
+ assert result["method"]["grid_rule"].startswith("ISO weeks in UTC, Monday to Sunday")
+ assert {s["series_key"]: s["first_period"] for s in result["series"]} == {
+ "lagtest:x": D0.isoformat(), "lagtest:y": (D0 + timedelta(days=14)).isoformat()
+ }
+
+
+def test_month_periods_fit_the_month_grid_and_are_coarser_than_days(store) -> None:
+ a = _walk(31, 41)
+ months = [f"{2020 + m // 12}-{m % 12 + 1:02d}" for m in range(41)]
+ _write(store, {"ma": list(zip(months, a)), "mb": list(zip(months[1:], a[:-1]))})
+ monthly = _run(store, ["ma", "mb"], grid="month", max_lag=2)
+ best = monthly["pairs"][0]["strongest"][0]
+ assert (best["lag"], best["timing"]) == (1, "A leads B by 1 month")
+ daily = _run(store, ["ma", "mb"])
+ assert daily["pairs"] == []
+ reasons = {e["series_key"]: e["reason"] for e in daily["excluded_series"]}
+ assert "coarser than the day grid" in reasons["lagtest:ma"]
+ assert "Use grid=month" in reasons["lagtest:ma"]
+ assert "fewer than two series could be used, so nothing was tested" in daily["data_gaps"]
+
+
+# ---------------------------------------------------------------------------
+# Minimum sample, revisions, undated rows
+# ---------------------------------------------------------------------------
+
+
+def test_minimum_overlap_excludes_and_reports_each_lag_with_its_n(store) -> None:
+ """33 days each, so 32 changes on the same days: lag k overlaps 32 - |k|.
+ At the default minimum of 30, lags -2..+2 run and the rest are listed."""
+ _write(store, {"s1": _days(_walk(41, 33)), "s2": _days(_walk(42, 33))})
+ result = _run(store, ["s1", "s2"])
+ (pair,) = result["pairs"]
+ assert result["method"]["min_overlap"] == lc.DEFAULT_MIN_OVERLAP == 30
+ assert "80% power" in result["method"]["min_overlap_reason"]
+ assert pair["not_tested"] == [
+ {"lag": -5, "n": 27, "reason": "below_min_overlap"},
+ {"lag": -4, "n": 28, "reason": "below_min_overlap"},
+ {"lag": -3, "n": 29, "reason": "below_min_overlap"},
+ {"lag": 3, "n": 29, "reason": "below_min_overlap"},
+ {"lag": 4, "n": 28, "reason": "below_min_overlap"},
+ {"lag": 5, "n": 27, "reason": "below_min_overlap"},
+ ]
+ assert (pair["lags_tested"], pair["lags_not_tested"]) == (5, 6)
+ assert (result["tests_run"], result["tests_not_run"]) == (5, 6)
+ assert abs(pair["strongest"][0]["lag"]) <= 2
+ lowered = _run(store, ["s1", "s2"], min_overlap=20)
+ assert (lowered["tests_run"], lowered["tests_not_run"]) == (11, 0)
+
+
+def test_pairs_without_variation_and_short_series_are_reported(store) -> None:
+ _write(store, {
+ "flat": _days([4.33] * 40),
+ "moving": _days(_walk(51, 40)),
+ "short": _days(_walk(52, 10)),
+ })
+ result = _run(store, ["flat", "moving", "short"], max_lag=1)
+ by_pair = {(p["a"], p["b"]): p for p in result["pairs"]}
+ flat = by_pair[("lagtest:flat", "lagtest:moving")]
+ assert [nt["reason"] for nt in flat["not_tested"]] == ["no_variation"] * 3
+ assert [nt["n"] for nt in flat["not_tested"]] == [38, 39, 38]
+ assert flat["strongest"] == []
+ gaps = "\n".join(result["data_gaps"])
+ assert "lagtest:short: 9 period-over-period changes on the day grid" in gaps
+ assert "lagtest:flat / lagtest:moving: no lag could be tested" in gaps
+
+
+def test_revisions_use_the_latest_vintage(store) -> None:
+ """First capture: a is unrelated noise. A later capture revises every one
+ of a's 60 points to b two days later. The tool must read the revision."""
+ b = _walk(61, 62)
+ _write(store, {"a": _days(_normals(62, 60)), "b": _days(b)})
+ before = _run(store, ["a", "b"], max_lag=3)
+ assert before["significant_tests"] == 0
+ _write(store, {"a": _days(b[2:])}, at=T0 + 3600)
+ after = _run(store, ["a", "b"], max_lag=3)
+ best = after["pairs"][0]["strongest"][0]
+ assert (best["lag"], best["r"]) == (2, pytest.approx(1.0))
+ a_info = next(s for s in after["series"] if s["series_key"] == "lagtest:a")
+ assert a_info["revised_observed_at"] == 60
+ assert a_info["rows"] == 120 and a_info["dated_observations"] == 60
+ assert "latest collected value" in after["method"]["vintage"]
+ assert any("60 observation dates were revised" in g for g in after["data_gaps"])
+
+
+def test_series_without_observed_at_are_excluded_with_the_stated_rule(store) -> None:
+ for i, price in enumerate(_walk(71, 40)):
+ store.record_cycle(
+ [SourceOutcome("market_quotes", T0 + 300 * i, 1.0,
+ {"quotes": [{"symbol": "^GSPC", "price": 5000 + price}]})],
+ now=T0 + 300 * i,
+ )
+ _planted(store)
+ result = _run(store, ["market_quotes:^GSPC:price", "x", "y"])
+ (excluded,) = result["excluded_series"]
+ assert excluded["series_key"] == "market_quotes:^GSPC:price"
+ assert "no row has observed_at (40 rows carry collected_at only)" in excluded["reason"]
+ assert "stored only on change" in excluded["reason"]
+ assert [(p["a"], p["b"]) for p in result["pairs"]] == [("lagtest:x", "lagtest:y")]
+ assert "collected_at" in result["method"]["undated_rows"]
+ alone = _run(store, ["market_quotes:^GSPC:price", "x"])
+ assert alone["pairs"] == [] and alone["tests_run"] == 0
+ assert "fewer than two series could be used, so nothing was tested" in alone["data_gaps"]
+
+
+def test_window_selects_whole_periods(store) -> None:
+ _planted(store)
+ result = _run(store, ["x", "y"], start="2025-03-01", end="2025-05-31")
+ window = result["method"]["window"]
+ assert (window["start"], window["end"], window["periods"]) == ("2025-03-01", "2025-05-31", 92)
+ x_info = result["series"][0]
+ assert x_info["periods"] == 92 and x_info["periods_outside_window"] == 108
+ assert result["pairs"][0]["strongest"][0]["lag"] == 3
+ weekly = _run(store, ["x", "y"], grid="week", start="2025-03-05", end="2025-03-05", min_overlap=20)
+ assert weekly["method"]["window"]["start"] == "2025-03-03"
+ empty = _run(store, ["x", "y"], start="2030-01-01")
+ assert empty["pairs"] == []
+ assert {e["reason"] for e in empty["excluded_series"]} == {"no observation inside the window"}
+
+
+def test_window_is_capped_to_the_most_recent_periods(store, monkeypatch) -> None:
+ monkeypatch.setattr(lc, "MAX_PERIODS", 120)
+ _planted(store)
+ result = _run(store, ["x", "y"])
+ window = result["method"]["window"]
+ assert window["limited_to_recent"] is True and window["periods"] == 120
+ assert window["end"] == (D0 + timedelta(days=199)).isoformat()
+ assert "window limited to the most recent 120 days" in result["data_gaps"]
+
+
+# ---------------------------------------------------------------------------
+# Serial-correlation diagnostic
+# ---------------------------------------------------------------------------
+
+
+def test_ar1_effective_sample_size_follows_the_reported_autocorrelations(store) -> None:
+ _planted(store)
+ result = _run(store, ["x", "y"], transform="levels")
+ best = result["pairs"][0]["strongest"][0]
+ phi_a, phi_b = (s["lag1_autocorrelation"] for s in result["series"])
+ expected = min(best["n"], best["n"] * (1 - phi_a * phi_b) / (1 + phi_a * phi_b))
+ assert best["n_effective_ar1"] == pytest.approx(expected, abs=0.5)
+ assert best["n_effective_ar1"] < best["n"]
+ assert "does not enter q" in result["method"]["serial_correlation"]
+ assert any("Serial correlation inflates significance" in c for c in result["cannot_prove"])
+
+
+def test_cannot_prove_says_a_small_q_is_not_a_large_effect(store) -> None:
+ """The effect-size line quotes the module's own p for |r| 0.1 at n 1,000
+ (P_CASES pins that p to scipy's 0.00154). The list a call returns is a
+ copy: clearing it leaves the module's list whole."""
+ _planted(store)
+ result = _run(store, ["x", "y"])
+ line = next(c for c in result["cannot_prove"] if c.startswith("A small q is not a large effect"))
+ assert "an |r| of 0.1 (1% of the variance shared) already gives p = 0.0015;" in line
+ result["cannot_prove"].clear()
+ result["method"]["not_tested_reasons"].clear()
+ assert len(lc.CANNOT_PROVE) == 7
+ assert set(lc.NOT_TESTED_REASONS) == {"below_min_overlap", "no_variation"}
+
+
+# ---------------------------------------------------------------------------
+# Inputs: clamps and errors
+# ---------------------------------------------------------------------------
+
+
+def test_numeric_inputs_are_clamped_and_reported(store) -> None:
+ _planted(store)
+ result = _run(store, ["x", "y"], max_lag=500, min_overlap=5, q=0.9)
+ assert result["method"]["lags"] == [-30, 30]
+ assert result["method"]["min_overlap"] == 20
+ assert result["method"]["q_threshold"] == 0.25
+ assert {c["input"]: (c["requested"], c["used"]) for c in result["inputs_clamped"]} == {
+ "max_lag": (500, 30), "min_overlap": (5, 20), "q": (0.9, 0.25)
+ }
+ low = _run(store, ["x", "y"], max_lag=-4, q=0)
+ assert low["method"]["lags"] == [0, 0] and low["tests_run"] == 1
+ assert low["method"]["q_threshold"] == 0.001
+ assert _run(store, ["x", "y"], max_lag=4.0)["method"]["lags"] == [-4, 4]
+
+
+def test_total_tests_are_capped_by_reducing_max_lag(store) -> None:
+ _write(store, {f"c{i}": _days(_walk(80 + i, 40)) for i in range(8)})
+ result = _run(store, [f"c{i}" for i in range(8)], max_lag=30, min_overlap=20)
+ assert result["method"]["lags"] == [-17, 17]
+ assert len(result["pairs"]) == 28
+ assert result["tests_run"] + result["tests_not_run"] == 28 * 35 <= lc.MAX_TESTS
+ reduced = [c for c in result["inputs_clamped"] if c.get("reason")]
+ assert reduced == [{
+ "input": "max_lag", "requested": 30, "used": 17,
+ "reason": "28 pairs x 35 lags keeps the call within 1000 tests",
+ }]
+
+
+ "kwargs,message",
+ [
+ ({"series_keys": ["lagtest:x"]}, "at least 2 series"),
+ ({"series_keys": [f"lagtest:k{i}" for i in range(9)]}, "at most 8 series"),
+ ({"series_keys": ["lagtest:x", "lagtest:x"]}, "repeats ['lagtest:x']"),
+ ({"series_keys": "lagtest:x"}, "must be a list"),
+ ({"series_keys": None}, "must be a list"),
+ ({"series_keys": ["lagtest:x", ""]}, "non-empty strings"),
+ ({"grid": "hourly"}, "unknown grid 'hourly'; allowed: day, week, month"),
+ ({"transform": "logs"}, "unknown transform 'logs'; allowed: changes, levels"),
+ ({"max_lag": "5"}, "max_lag must be an integer"),
+ ({"max_lag": True}, "max_lag must be an integer"),
+ ({"min_overlap": 25.5}, "min_overlap must be an integer"),
+ ({"q": "0.05"}, "q must be a number"),
+ ({"q": float("nan")}, "q must be a number"),
+ ({"start": "2025/01/01"}, "start must be a date YYYY-MM-DD"),
+ ({"end": "2025-02-30"}, "end must be a date YYYY-MM-DD"),
+ ({"start": "2025-06-01", "end": "2025-05-01"}, "is after end"),
+ ],
+)
+def test_input_errors_name_what_is_allowed(store, kwargs, message) -> None:
+ _planted(store)
+ args = {"series_keys": ["lagtest:x", "lagtest:y"], **kwargs}
+ keys = args.pop("series_keys")
+ result = lc.lagged_connections(store, keys, **args)
+ assert set(result) == {"error"}
+ assert message in result["error"]
+
+
+def test_unknown_series_is_an_error_with_a_listing_hint(store) -> None:
+ _planted(store)
+ result = _run(store, ["x", "nope"])
+ assert result["unknown_series"] == ["lagtest:nope"]
+ assert "intel_series_history" in result["error"]
+
+
+def test_user_facing_text_has_no_em_dashes() -> None:
+ from world_intel_mcp.tools import lagged_connections as tool_mod
+
+ texts = [tool_mod.TOOLS[0].description, json.dumps(tool_mod.TOOLS[0].inputSchema),
+ cli.history_connections.help or ""]
+ texts += [v for k, v in vars(lc).items() if k.isupper() and isinstance(v, str)]
+ texts += lc.CANNOT_PROVE + list(lc.GRID_RULES.values()) + list(lc.TRANSFORM_RULES.values())
+ texts += [p.help or "" for p in cli.history_connections.params if hasattr(p, "help")]
+ assert not [t for t in texts if "—" in t]
+
+
+# ---------------------------------------------------------------------------
+# MCP tool through the real dispatcher
+# ---------------------------------------------------------------------------
+
+
[email protected](scope="module")
+def server():
+ """server.py imported under test_server_registry's temp cache path, the
+ shared runtime rebound while the override is in force (that file's
+ fixture pattern)."""
+ prior = os.environ.get("WORLD_INTEL_CACHE_DB")
+ os.environ["WORLD_INTEL_CACHE_DB"] = str(_TMP_CACHE)
+ try:
+ module = importlib.import_module("world_intel_mcp.server")
+ importlib.import_module("world_intel_mcp.runtime")._initialize()
+ yield module
+ finally:
+ if prior is None:
+ os.environ.pop("WORLD_INTEL_CACHE_DB", None)
+ else:
+ os.environ["WORLD_INTEL_CACHE_DB"] = prior
+
+
+def tool_db(server, store, monkeypatch):
+ _planted(store)
+ runtime = importlib.import_module("world_intel_mcp.runtime")
+ monkeypatch.setattr(runtime, "cache", types.SimpleNamespace(db_path=store.db_path))
+ return store.db_path
+
+
+def test_tool_is_registered_with_its_schema(server) -> None:
+ (tool,) = [t for t in server.TOOLS if t.name == "intel_lagged_connections"]
+ props = tool.inputSchema["properties"]
+ assert tool.inputSchema["required"] == ["series_keys"]
+ assert props["grid"]["enum"] == ["day", "week", "month"]
+ assert props["transform"]["enum"] == ["changes", "levels"]
+ assert props["transform"]["default"] == "changes"
+ assert "accrues only while the collector" in tool.description
+ assert "correlation is not causation" in tool.description
+ assert "A leads B by k" in tool.description
+
+
+async def test_tool_through_the_real_dispatcher(server, tool_db) -> None:
+ args = {"series_keys": ["lagtest:x", "lagtest:y"], "max_lag": 4}
+ result = await server._dispatch("intel_lagged_connections", args)
+ best = result["pairs"][0]["strongest"][0]
+ assert (best["lag"], best["n"], best["significant"]) == (3, 196, True)
+ assert result["method"]["lags"] == [-4, 4] and result["tests_run"] == 9
+
+ (content,) = await server.call_tool("intel_lagged_connections", args)
+ assert json.loads(content.text)["pairs"][0]["strongest"][0]["lag"] == 3
+
+ bad = await server._dispatch(
+ "intel_lagged_connections", {"series_keys": ["lagtest:x", "lagtest:y"], "grid": "hourly"}
+ )
+ assert bad == {"error": "unknown grid 'hourly'; allowed: day, week, month"}
+ missing = await server._dispatch("intel_lagged_connections", {})
+ assert "must be a list" in missing["error"]
+
+
+# ---------------------------------------------------------------------------
+# CLI
+# ---------------------------------------------------------------------------
+
+
+def cli_db(tmp_path, monkeypatch):
+ monkeypatch.setattr(history, "_EXTRACTORS", dict(history._EXTRACTORS))
+ history.register_extractor(
+ "lagtest",
+ lambda payload: [
+ Observation(p["key"], p["value"], observed_at=p["observed_at"])
+ for p in payload["points"]
+ ],
+ )
+ db = tmp_path / "cli-cache.db"
+ monkeypatch.setenv("WORLD_INTEL_CACHE_DB", str(db))
+ monkeypatch.setenv("COLUMNS", "240")
+ monkeypatch.setattr(cli, "_fetcher", None)
+ s = HistoryStore(db)
+ _planted(s)
+ _write(s, {"[red]odd[/red]": _days(_walk(91, 33))})
+ s.close()
+ yield db
+ # The CLI's shared fetcher opened a Cache on this db; close it before
+ # monkeypatch restores the previous fetcher.
+ if cli._fetcher is not None:
+ cli._fetcher.cache.close()
+
+
+def _invoke(*args):
+ return CliRunner().invoke(cli.main, list(args), catch_exceptions=False)
+
+
+def test_cli_connections_table_states_convention_and_limits(cli_db) -> None:
+ out = _invoke("history", "connections", "lagtest:x", "lagtest:y").output
+ assert "A leads B by 3 days" in out and "+3" in out
+ assert "Lag convention: Pairs are (A, B)" in out
+ assert "Transform: Period-over-period changes" in out
+ assert "Cannot prove: Correlation is not causation" in out
+ assert "1 of 1 pairs have a lag with q <= 0.05" in out
+
+
+def test_cli_connections_lists_untested_lags_and_renders_keys_literally(cli_db) -> None:
+ out = _invoke("history", "connections", "lagtest:x", "lagtest:[red]odd[/red]").output
+ assert "lagtest:[red]odd[/red]" in out
+ assert "Not tested (minimum 30 overlapping periods)" in out
+ assert "lag +3 (n 29)" in out
+
+
+def test_cli_connections_shows_the_ar1_effective_sample_size(cli_db) -> None:
+ args = ("history", "connections", "lagtest:x", "lagtest:y", "--transform", "levels")
+ best = json.loads(_invoke("--json-output", *args).output)["pairs"][0]["strongest"][0]
+ assert best["n_effective_ar1"] < best["n"] / 2
+ out = _invoke(*args).output
+ assert f" {best['n_effective_ar1']:g} " in out and f" {best['n']} " in out
+ assert "n_eff: AR(1) effective sample size" in out and "p and q use n" in out
+
+
+def test_cli_connections_json_and_errors(cli_db) -> None:
+ data = json.loads(
+ _invoke("--json-output", "history", "connections", "lagtest:x", "lagtest:y",
+ "--grid", "week", "--min-overlap", "20").output
+ )
+ assert data["method"]["grid"] == "week"
+ assert data["method"]["min_overlap"] == 20
+ out = _invoke("history", "connections", "lagtest:x", "lagtest:nope").output
+ assert out.startswith("Error:") and "lagtest:nope" in out
+ bad = CliRunner().invoke(cli.main, ["history", "connections", "a", "b", "--grid", "hour"])
+ assert bad.exit_code == 2 and "day" in bad.output and "month" in bad.output
diff --git a/src/world_intel_mcp/tools/__init__.py b/src/world_intel_mcp/tools/__init__.py
index fc0bb4d..8a59c12 100644
--- a/src/world_intel_mcp/tools/__init__.py
+++ b/src/world_intel_mcp/tools/__init__.py
@@ -31,6 +31,7 @@ from . import ( # noqa: E402
history,
infrastructure,
intelligence,
+ lagged_connections,
liquidity,
markets,
news_market,
@@ -59,6 +60,7 @@ _MODULES = [
vector,
aoi,
history,
+ lagged_connections,
system,
]
diff --git a/src/world_intel_mcp/tools/lagged_connections.py b/src/world_intel_mcp/tools/lagged_connections.py
new file mode 100644
index 0000000..44c5298
--- /dev/null
+++ b/src/world_intel_mcp/tools/lagged_connections.py
@@ -0,0 +1,131 @@
+"""Lagged connections: correlation between stored series at leads and lags.
+
+Reads the history store (``world_intel_mcp.history``) on the same
+``db_path`` the runtime cache resolved and runs
+``analysis/lagged_connections.py`` over it; nothing here fetches upstream
+data or writes history.
+"""
+
+import asyncio
+from typing import Any
+
+from mcp.types import Tool
+
+from .. import runtime
+from ..analysis import lagged_connections as lc
+from ..history import HistoryStore
+
+TOOLS: list[Tool] = [
+ Tool(
+ name="intel_lagged_connections",
+ description=(
+ f"Correlation between 2 to {lc.MAX_SERIES} series in the history store (keys as "
+ "intel_series_history lists them), aligned on a UTC day, week or month grid and "
+ "tested at every lag from -max_lag to +max_lag. Default transform is period-over-"
+ "period changes, because levels of trending series correlate spuriously; 'levels' "
+ "on request. Per pair: the lag with the strongest evidence (smallest p) with "
+ "Pearson r, overlap n, two-sided p from Student's t, and the Benjamini-Hochberg q "
+ f"across every test in the call (significant at q <= {lc.DEFAULT_Q:g} by default). "
+ "Lag +k compares A at period t with B at t+k, so A leads B by k. Pair/lag "
+ f"overlaps under {lc.DEFAULT_MIN_OVERLAP} points are listed as not tested. Revised "
+ "points use the latest vintage; rows without observed_at (Yahoo quotes) are not "
+ "used. States what it cannot prove: correlation is not causation and a lead is not "
+ "a causal direction. History accrues only while the collector (intel-collector, "
+ "or intel_collect) runs; nothing before its first recorded cycle exists."
+ ),
+ inputSchema={
+ "type": "object",
+ "properties": {
+ "series_keys": {
+ "type": "array",
+ "items": {"type": "string"},
+ "description": (
+ f"2 to {lc.MAX_SERIES} series keys, e.g. ['space_weather:kp', "
+ "'central_bank_rates:federal_funds_rate']; pairs are formed in this order"
+ ),
+ },
+ "grid": {
+ "type": "string",
+ "enum": list(lc.GRIDS),
+ "description": "UTC calendar grid (default day; weeks run Monday to Sunday)",
+ "default": lc.DEFAULT_GRID,
+ },
+ "transform": {
+ "type": "string",
+ "enum": list(lc.TRANSFORMS),
+ "description": "changes (default, period-over-period differences) or levels",
+ "default": lc.DEFAULT_TRANSFORM,
+ },
+ "max_lag": {
+ "type": "integer",
+ "description": (
+ f"Largest lead or lag in grid periods (default {lc.DEFAULT_MAX_LAG}, "
+ f"clamped to 0..{lc.MAX_LAG_CAP}, reduced to keep a call within "
+ f"{lc.MAX_TESTS:,} tests)"
+ ),
+ "default": lc.DEFAULT_MAX_LAG,
+ },
+ "min_overlap": {
+ "type": "integer",
+ "description": (
+ f"Fewest overlapping periods a pair/lag needs to be tested (default "
+ f"{lc.DEFAULT_MIN_OVERLAP}, clamped to {lc.MIN_OVERLAP_FLOOR}.."
+ f"{lc.MAX_PERIODS})"
+ ),
+ "default": lc.DEFAULT_MIN_OVERLAP,
+ },
+ "q": {
+ "type": "number",
+ "description": (
+ f"False discovery rate threshold (default {lc.DEFAULT_Q:g}, clamped to "
+ f"{lc.Q_FLOOR:g}..{lc.Q_CAP:g})"
+ ),
+ "default": lc.DEFAULT_Q,
+ },
+ "start": {
+ "type": "string",
+ "description": "First date YYYY-MM-DD; its grid period and later are used",
+ },
+ "end": {
+ "type": "string",
+ "description": "Last date YYYY-MM-DD; its grid period and earlier are used",
+ },
+ },
+ "required": ["series_keys"],
+ },
+ ),
+]
+
+
+def _run(db_path: Any, arguments: dict[str, Any]) -> dict:
+ # Opened and closed in the worker thread: sqlite3 connections stay on
+ # the thread that created them.
+ store = HistoryStore(db_path)
+ try:
+ return lc.lagged_connections(
+ store,
+ arguments.get("series_keys"),
+ grid=arguments.get("grid"),
+ transform=arguments.get("transform"),
+ max_lag=arguments.get("max_lag"),
+ min_overlap=arguments.get("min_overlap"),
+ q=arguments.get("q"),
+ start=arguments.get("start"),
+ end=arguments.get("end"),
+ )
+ finally:
+ store.close()
+
+
+async def _lagged_connections(arguments: dict[str, Any]) -> Any:
+ # Up to 1,000 tests over up to 3,000 periods is CPU-bound for a moment,
+ # so it runs off the event loop. The path is resolved here, on the loop
+ # thread: a first touch of runtime.cache opens the shared Cache, whose
+ # sqlite connection must not be born in a worker thread.
+ db_path = runtime.cache.db_path
+ return await asyncio.to_thread(_run, db_path, arguments)
+
+
+HANDLERS = {
+ "intel_lagged_connections": _lagged_connections,
+}