Skip to content

Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.

feat(mcp): record counterfactual token savings per tool into the unified ledger (#421)

Raghav Chamadiyab541905cd320
diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md
index 2a8bdba6b..b2e06872f 100644
--- a/docs/CHANGELOG.md
+++ b/docs/CHANGELOG.md
@@ -12,6 +12,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
+- **MCP counterfactual token savings.** Every MCP tool call now records what its
+ curated answer *replaced* — the raw file exploration the agent would have done
+ otherwise — into the unified savings ledger as a `mcp:<tool>` row. `get_symbol`
+ reports the whole file it sliced one symbol from, `get_context` the full files
+ its skeletons stood in for, `search_codebase` a conservative floor per cited
+ file; estimates undersell by design. The Costs hero now reads "N MCP queries
+ answered" and grows per call, and `repowise saved --by source` surfaces the
+ per-tool `mcp:*` breakdown. Recording is best-effort and never alters a tool's
+ user-facing response.
- **Costs page savings hero.** The Costs page now leads with a results card
showing every token and dollar repowise saved your coding agent, combining
the `repowise distill` ledger with MCP tool-response savings that were
diff --git a/docs/DISTILL.md b/docs/DISTILL.md
index 63ad91c8e..86e9c445a 100644
--- a/docs/DISTILL.md
+++ b/docs/DISTILL.md
@@ -198,9 +198,11 @@ was silent. Now every drop goes through the same omission store:
the optional `query` parameter searches within the stored content. Tool count
stays at nine. See [MCP_TOOLS.md](MCP_TOOLS.md).
-> **Note:** MCP truncation is *not* counted in the savings ledger — those
-> responses were always capped, so nothing was "saved" relative to before.
-> `repowise saved` covers the distill command/hook path only.
+> **Note:** MCP tool calls now record a *counterfactual* saving — what raw file
+> exploration the curated answer replaced — as `mcp:<tool>` rows in the same
+> ledger (see [`repowise saved`](#repowise-saved--the-savings-report)). Response
+> truncation is folded into that delta (the delivered size is measured after the
+> budget cap), so it is never double-counted.
---
@@ -226,7 +228,7 @@ inside a repowise repo.
```bash
repowise saved # per-filter rollup + totals + est. dollars
repowise saved --by day # daily rollup
-repowise saved --by source # cli vs hook-bash vs hook-powershell
+repowise saved --by source # cli vs hook-* vs mcp:<tool>
repowise saved --since 2026-06-01
repowise saved --model claude-opus-4-6 # price the estimate differently
repowise saved --missed # savings raw commands left on the table
@@ -237,8 +239,10 @@ Dollar estimates price saved tokens at the chosen model's *input* rate (saved
tokens are input the agent never had to read) using the same pricing table as
`repowise costs`. Token counts are chars/4 estimates.
-The report covers the **distill command/hook path only** — MCP response
-truncation is intentionally not in the ledger (see above).
+The report covers both surfaces of the ledger: the **distill command/hook
+path** (`cli` / `hook-*` sources) and **MCP counterfactual savings** (`mcp:<tool>`
+sources — each tool answer priced against the raw exploration it replaced).
+`repowise saved --by source` separates them.
### Missed savings — `repowise saved --missed`
@@ -264,9 +268,12 @@ The local dashboard goes further. The Costs page leads with a **savings hero
card** that combines two surfaces into one honest number:
- **Distill** — the `repowise distill` command/hook ledger above.
-- **MCP tool savings** — tokens trimmed from oversized MCP tool responses,
- read straight from the omission store (`source='mcp:*'`). These were always
- saved but never showed up in the ledger before.
+- **MCP tool savings** — what each tool answer replaced: the raw file
+ exploration the agent would have done without it (`source='mcp:*'`).
+ `get_symbol` stands in for reading the whole file, `get_context` for the files
+ its skeletons summarise, `search_codebase` for opening each cited file; the
+ estimate undersells. Tools without a counterfactual estimator still contribute
+ their response-budget truncation drops.
The dollar figure is **priced at the coding agent's actual model**, not a flat
guess: saved tokens are *input* the agent never read, so they are worth that
diff --git a/packages/cli/src/repowise/cli/commands/saved_cmd.py b/packages/cli/src/repowise/cli/commands/saved_cmd.py
index ea7ba2edd..fafc63b25 100644
--- a/packages/cli/src/repowise/cli/commands/saved_cmd.py
+++ b/packages/cli/src/repowise/cli/commands/saved_cmd.py
@@ -2,9 +2,10 @@
Reads the savings ledger in the omissions sidecar
(``.repowise/omissions/omissions.db``). The ledger covers the
-``repowise distill`` path only — both direct invocations and hook rewrites.
-MCP response truncation is deliberately not recorded: those responses were
-always budget-capped, so nothing was "saved" relative to before.
+``repowise distill`` path (direct invocations and hook rewrites) plus MCP
+counterfactual savings — each tool answer priced against the raw file
+exploration it replaced, recorded under ``source='mcp:<tool>'``. Group by
+source to split the two surfaces.
Named ``saved`` rather than ``distill --stats`` because ``repowise distill``
captures everything after it as the command to run (``ignore_unknown_options``)
@@ -77,8 +78,8 @@ def saved_command(
PATH defaults to the current directory; the report covers that repo's
omission store (or the user-level fallback store when the repo has no
- ``.repowise/``). Covers the distill command/hook path only — MCP response
- truncation is not part of this ledger.
+ ``.repowise/``). Covers the distill command/hook path plus MCP
+ counterfactual savings (``source='mcp:<tool>'``); ``--by source`` splits them.
"""
from repowise.core.distill.store import OmissionStore, default_store_path
@@ -122,8 +123,8 @@ def saved_command(
border_style="dim",
show_footer=True,
caption=(
- "Covers the 'repowise distill' command/hook path only; "
- "MCP response truncation is not counted."
+ "Covers the 'repowise distill' command/hook path plus MCP "
+ "counterfactual savings (mcp:<tool>); group by source to split them."
),
)
table.add_column(group_by.capitalize(), style="cyan", footer="[bold]TOTAL[/bold]")
diff --git a/packages/core/src/repowise/core/distill/session_model.py b/packages/core/src/repowise/core/distill/session_model.py
index 539cfdb2b..0af0a6ea7 100644
--- a/packages/core/src/repowise/core/distill/session_model.py
+++ b/packages/core/src/repowise/core/distill/session_model.py
@@ -33,6 +33,8 @@ import re
from dataclasses import dataclass
from pathlib import Path
+from repowise.core.fs_walk import iter_glob
+
#: Pricing fallback when no agent transcript is detectable.
DEFAULT_MODEL = "claude-sonnet-4-6"
@@ -178,7 +180,7 @@ def _codex_sighting(repo_root: Path, sessions_root: Path | None) -> _Sighting |
return None
repo_prefix = str(repo_root).lower().rstrip("\\/")
best: _Sighting | None = None
- for path in _by_mtime_desc(root.rglob("*.jsonl")):
+ for path in _by_mtime_desc(iter_glob(root, "*.jsonl")):
try:
mtime = path.stat().st_mtime
except OSError:
diff --git a/packages/core/src/repowise/core/distill/tracking.py b/packages/core/src/repowise/core/distill/tracking.py
index 3a5d809ca..4fbe2e803 100644
--- a/packages/core/src/repowise/core/distill/tracking.py
+++ b/packages/core/src/repowise/core/distill/tracking.py
@@ -69,6 +69,114 @@ def savings_summary(
}
+def distill_summary(
+ conn: sqlite3.Connection, *, since: float | None = None
+) -> dict[str, Any]:
+ """Ledger totals for the **distill** surface only (excludes ``mcp:*``).
+
+ Identical in shape to :func:`savings_summary` but scoped to non-MCP
+ sources, so the hero card can report a clean ``repowise distill`` figure now
+ that Phase 2 also writes counterfactual ``mcp:<tool>`` rows into the same
+ ``savings`` ledger. Per-filter buckets likewise drop MCP tool rows.
+
+ *since* is a Unix timestamp lower bound on ``created_at``.
+ """
+ where = "source NOT LIKE 'mcp:%'"
+ params: tuple[float, ...] = ()
+ if since is not None:
+ where += " AND created_at >= ?"
+ params = (since,)
+ total_raw, total_distilled, events = conn.execute(
+ "SELECT COALESCE(SUM(raw_tokens),0), COALESCE(SUM(distilled_tokens),0),"
+ f" COUNT(*) FROM savings WHERE {where}",
+ params,
+ ).fetchone()
+ per_filter = {
+ row[0]: {
+ "events": row[1],
+ "raw_tokens": row[2],
+ "distilled_tokens": row[3],
+ "saved_tokens": row[2] - row[3],
+ }
+ for row in conn.execute(
+ "SELECT filter, COUNT(*), SUM(raw_tokens), SUM(distilled_tokens)"
+ f" FROM savings WHERE {where} GROUP BY filter ORDER BY SUM(raw_tokens) DESC",
+ params,
+ )
+ }
+ return {
+ "events": events,
+ "raw_tokens": total_raw,
+ "distilled_tokens": total_distilled,
+ "saved_tokens": total_raw - total_distilled,
+ "per_filter": per_filter,
+ }
+
+
+def mcp_savings_summary(
+ conn: sqlite3.Connection, *, since: float | None = None
+) -> dict[str, Any]:
+ """Unified MCP savings view — counterfactual ledger, truncation as fallback.
+
+ Two MCP signals live in the sidecar:
+
+ * **Counterfactual** rows in the ``savings`` ledger (``source='mcp:<tool>'``)
+ written by the Phase 2 instrumentation: ``saved = replaced - delivered``.
+ Because ``delivered`` is measured *after* response-budget truncation, the
+ truncation saving is already folded into this delta.
+ * **Truncation drops** in the ``omissions`` table (also ``source='mcp:<tool>'``)
+ from :func:`mcp_drops_summary` — the only signal for tools that have no
+ counterfactual estimator yet.
+
+ Merging per tool with **counterfactual precedence** avoids double counting:
+ a tool with counterfactual rows reports its ledger ``saved_tokens`` (which
+ subsumes truncation); a tool with only drops reports its dropped tokens.
+ Each ``per_tool`` row is tagged ``kind`` = ``"counterfactual"`` | ``"truncation"``.
+
+ Returns ``{events, tokens, queries, per_tool}`` where ``tokens`` is total
+ saved, ``queries`` counts counterfactual tool calls (the "N MCP queries
+ answered" headline), and ``events`` counts every contributing event.
+ """
+ ledger = {
+ _strip_mcp_prefix(row["group"]): row
+ for row in savings_rollup(conn, by="source", since=since)
+ if row["group"].startswith("mcp:")
+ }
+ drops = mcp_drops_summary(conn, since=since)["per_tool"]
+
+ per_tool: list[dict[str, Any]] = []
+ queries = 0
+ for tool, row in ledger.items():
+ per_tool.append(
+ {
+ "tool": tool,
+ "events": row["events"],
+ "tokens": row["saved_tokens"],
+ "kind": "counterfactual",
+ }
+ )
+ queries += row["events"]
+ for tool, stats in drops.items():
+ if tool in ledger:
+ continue # counterfactual already subsumes this tool's truncation
+ per_tool.append(
+ {
+ "tool": tool,
+ "events": stats["events"],
+ "tokens": stats["tokens"],
+ "kind": "truncation",
+ }
+ )
+
+ per_tool.sort(key=lambda r: r["tokens"], reverse=True)
+ return {
+ "events": sum(r["events"] for r in per_tool),
+ "tokens": sum(r["tokens"] for r in per_tool),
+ "queries": queries,
+ "per_tool": per_tool,
+ }
+
+
def mcp_drops_summary(
conn: sqlite3.Connection, *, since: float | None = None
) -> dict[str, Any]:
diff --git a/packages/core/src/repowise/core/registry/mcp_tool_registry.py b/packages/core/src/repowise/core/registry/mcp_tool_registry.py
index be03e2d5c..4bc39f442 100644
--- a/packages/core/src/repowise/core/registry/mcp_tool_registry.py
+++ b/packages/core/src/repowise/core/registry/mcp_tool_registry.py
@@ -73,18 +73,29 @@ class MCPToolRegistry:
# with no other changes — the ``@mcp.tool()`` call still works.
tool = register
- def apply(self, mcp: Any) -> None:
+ def apply(
+ self,
+ mcp: Any,
+ middleware: Callable[[Callable[..., Any]], Callable[..., Any]] | None = None,
+ ) -> None:
"""Attach every registered tool to *mcp* via ``mcp.tool()``.
Calling :meth:`apply` multiple times with the same server is a
no-op on subsequent calls; calling it with a different server
registers everything against that server too (useful for tests
that spin up isolated :class:`FastMCP` instances).
+
+ *middleware*, when given, wraps each tool function before
+ registration — the server passes its savings instrumentation in
+ this way so the registry stays decoupled from it. A signature-
+ preserving wrapper is the caller's responsibility (FastMCP reads
+ each tool's signature to build its schema). Defaults to identity.
"""
if mcp in self._applied_to:
return
for fn in self._tools:
- mcp.tool()(fn)
+ wrapped = middleware(fn) if middleware is not None else fn
+ mcp.tool()(wrapped)
self._applied_to.append(mcp)
def reset(self) -> None:
diff --git a/packages/server/src/repowise/server/mcp_server/__init__.py b/packages/server/src/repowise/server/mcp_server/__init__.py
index b3cc1a947..7b403eced 100644
--- a/packages/server/src/repowise/server/mcp_server/__init__.py
+++ b/packages/server/src/repowise/server/mcp_server/__init__.py
@@ -14,6 +14,11 @@ from __future__ import annotations
import sys
from typing import Any
+# Attach every tool that registered itself through the shared registry to
+# the FastMCP instance. Idempotent per server, so a second call (e.g. when
+# tests build an isolated mcp) is a no-op against the original mcp.
+from repowise.core.registry import mcp_tool_registry as _mcp_tool_registry
+
# --- Import submodules in dependency order (triggers tool registration) ---
from repowise.server.mcp_server import _state
from repowise.server.mcp_server._graph_utils import ( # used by routers/graph.py
@@ -25,6 +30,7 @@ from repowise.server.mcp_server._helpers import (
_get_repo,
_is_path,
)
+from repowise.server.mcp_server._savings import instrument as _savings_instrument
from repowise.server.mcp_server._server import (
create_mcp_server,
mcp,
@@ -40,12 +46,10 @@ from repowise.server.mcp_server.tool_search import search_codebase
from repowise.server.mcp_server.tool_symbol import get_symbol
from repowise.server.mcp_server.tool_why import get_why
-# Attach every tool that registered itself through the shared registry to
-# the FastMCP instance. Idempotent per server, so a second call (e.g. when
-# tests build an isolated mcp) is a no-op against the original mcp.
-from repowise.core.registry import mcp_tool_registry as _mcp_tool_registry # noqa: E402
-
-_mcp_tool_registry.apply(mcp)
+# ``middleware`` wraps each tool with savings instrumentation: every call records
+# the counterfactual raw-exploration tokens its answer replaced into the unified
+# ledger. The wrapper is signature-preserving, so tool schemas are unchanged.
+_mcp_tool_registry.apply(mcp, middleware=_savings_instrument)
# ---------------------------------------------------------------------------
# Backward-compatible access to _state globals.
diff --git a/packages/server/src/repowise/server/mcp_server/_savings/__init__.py b/packages/server/src/repowise/server/mcp_server/_savings/__init__.py
new file mode 100644
index 000000000..e0ba5e519
--- /dev/null
+++ b/packages/server/src/repowise/server/mcp_server/_savings/__init__.py
@@ -0,0 +1,21 @@
+"""MCP savings instrumentation — counterfactual token-savings per tool.
+
+Keeps the savings concern (measuring what each tool answer replaced, recording
+``mcp:<tool>`` rows into the unified ledger) out of the generic
+:class:`~repowise.core.registry.mcp_tool_registry.MCPToolRegistry`. The server
+passes :func:`instrument` into ``apply(mcp, middleware=instrument)`` at boot;
+core stays decoupled from this package.
+"""
+
+from __future__ import annotations
+
+from .counterfactual import replaced_tokens_for
+from .recorder import record_mcp_saving
+from .wrapper import declare_replaced, instrument
+
+__all__ = [
+ "declare_replaced",
+ "instrument",
+ "record_mcp_saving",
+ "replaced_tokens_for",
+]
diff --git a/packages/server/src/repowise/server/mcp_server/_savings/counterfactual.py b/packages/server/src/repowise/server/mcp_server/_savings/counterfactual.py
new file mode 100644
index 000000000..58efa925c
--- /dev/null
+++ b/packages/server/src/repowise/server/mcp_server/_savings/counterfactual.py
@@ -0,0 +1,95 @@
+"""Per-tool counterfactual estimators — "what raw exploration did this replace?"
+
+Each MCP tool answer stands in for some amount of raw file reading the agent
+would otherwise have done. ``replaced_tokens_for`` returns a **conservative
+undersell** of that raw cost, computed purely from fields already on the tool's
+result dict — no disk reads, no DB queries. When a tool can't be estimated from
+its dict alone it returns 0, and the recorder writes no row (undersell by
+omission). High-value tools that hold an exact artifact (``get_symbol`` has the
+whole source file in hand) instead *declare* their counterfactual via
+:func:`repowise.server.mcp_server._savings.wrapper.declare_replaced`; the
+wrapper prefers a declared value over anything computed here.
+
+Floors mirror :data:`repowise.core.distill.missed.RATIO_FLOOR` philosophy:
+estimates undersell rather than oversell, so the savings number on the Costs
+page is always defensible.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Callable
+from typing import Any
+
+#: Conservative per-cited-file floor for ``search_codebase``. A semantic hit
+#: stands in for the agent grepping then opening that file; we have no file
+#: size on the result dict (search never loads bodies), so we undersell each
+#: distinct cited path to a small fraction of a typical source file rather than
+#: re-reading disk. Deliberately low — the goal is never to overstate.
+SEARCH_FLOOR_PER_HIT = 400
+
+
+def _estimate_get_context(result: dict[str, Any]) -> int:
+ """Σ full-file tokens the skeleton/doc replaced, over file targets.
+
+ ``get_context`` auto-upgrades file targets to a body-elided *skeleton*; each
+ such target carries ``skeleton.full_tokens`` — the exact size of the file
+ the agent would have Read instead. Summing those is a precise, dict-only
+ counterfactual. Targets without a skeleton (small-file cards, module/symbol
+ targets) contribute nothing.
+ """
+ targets = result.get("targets")
+ if not isinstance(targets, dict):
+ return 0
+ total = 0
+ for target in targets.values():
+ if not isinstance(target, dict):
+ continue
+ skeleton = target.get("skeleton")
+ if isinstance(skeleton, dict):
+ full = skeleton.get("full_tokens")
+ if isinstance(full, int) and full > 0:
+ total += full
+ return total
+
+
+def _estimate_search_codebase(result: dict[str, Any]) -> int:
+ """``len(distinct cited files) x SEARCH_FLOOR_PER_HIT`` — a conservative floor.
+
+ Each result points the agent at a file it would otherwise have had to find
+ and open by hand. We undersell to a per-hit floor because the result dict
+ carries only the path (and a snippet), never the file's size.
+ """
+ results = result.get("results")
+ if not isinstance(results, list):
+ return 0
+ paths = {
+ r["target_path"]
+ for r in results
+ if isinstance(r, dict) and r.get("target_path")
+ }
+ return len(paths) * SEARCH_FLOOR_PER_HIT
+
+
+#: Tool name → estimator. Tools absent here emit no counterfactual (skip).
+_ESTIMATORS: dict[str, Callable[[dict[str, Any]], int]] = {
+ "get_context": _estimate_get_context,
+ "search_codebase": _estimate_search_codebase,
+}
+
+
+def replaced_tokens_for(tool: str, result: Any) -> int:
+ """Conservative raw-exploration tokens *tool*'s answer replaced.
+
+ Returns 0 for unknown tools, non-dict results, or anything the per-tool
+ estimator can't ground in the result dict. Never raises.
+ """
+ if not isinstance(result, dict):
+ return 0
+ estimator = _ESTIMATORS.get(tool)
+ if estimator is None:
+ return 0
+ try:
+ value = estimator(result)
+ except Exception:
+ return 0
+ return value if isinstance(value, int) and value > 0 else 0
diff --git a/packages/server/src/repowise/server/mcp_server/_savings/recorder.py b/packages/server/src/repowise/server/mcp_server/_savings/recorder.py
new file mode 100644
index 000000000..4392ea7c3
--- /dev/null
+++ b/packages/server/src/repowise/server/mcp_server/_savings/recorder.py
@@ -0,0 +1,74 @@
+"""Record MCP counterfactual savings into the unified ledger.
+
+A thin, best-effort bridge from a measured tool call to a ``mcp:<tool>`` row in
+the same ``savings`` ledger the CLI/hook surfaces use. Mirrors
+:class:`~repowise.server.mcp_server._budget.collector.OmissionCollector`'s
+lifecycle: the SQLite handle is opened per event and closed immediately, so a
+long-running MCP server never holds a WAL handle that would contend with
+hook-side writers between the (rare, small) writes.
+
+Failure posture matches the rest of distill: a failed write degrades to a
+silent no-op. Recording savings must never perturb a tool response.
+"""
+
+from __future__ import annotations
+
+import contextlib
+import logging
+from pathlib import Path
+
+from repowise.core.distill.store import (
+ OMISSIONS_DB_FILENAME,
+ OMISSIONS_DIRNAME,
+ OmissionStore,
+)
+
+logger = logging.getLogger(__name__)
+
+
+def record_mcp_saving(
+ repo_root: str | Path | None,
+ tool: str,
+ replaced_tokens: int,
+ delivered_tokens: int,
+) -> bool:
+ """Append one ``mcp:<tool>`` counterfactual row. Returns True on write.
+
+ ``raw_tokens`` is the counterfactual raw-exploration cost the answer
+ replaced; ``distilled_tokens`` is what the agent actually received (measured
+ after response-budget truncation, so the truncation saving is folded into
+ the delta). Rows with no net saving (``replaced <= delivered``) are skipped —
+ they would not have earned a marker either.
+
+ The store is resolved **repo-locally** (``<repo>/.repowise/omissions``) — the
+ exact sidecar the Costs endpoint reads — and only when it already exists.
+ We deliberately do *not* fall back to the ``~/.repowise`` home store: a row
+ landing there would be invisible to the dashboard and would pollute a global
+ store across unrelated repos. Never raises.
+ """
+ if replaced_tokens <= delivered_tokens or not repo_root:
+ return False
+ db_path = Path(repo_root) / ".repowise" / OMISSIONS_DIRNAME / OMISSIONS_DB_FILENAME
+ if not db_path.is_file():
+ # No repo-local sidecar → repo never opted in via `repowise init`.
+ return False
+ try:
+ store = OmissionStore(db_path)
+ except Exception:
+ logger.debug("mcp savings store open failed", exc_info=True)
+ return False
+ try:
+ store.record_saving(
+ filter_name=tool,
+ source=f"mcp:{tool}",
+ command=None,
+ raw_tokens=replaced_tokens,
+ distilled_tokens=delivered_tokens,
+ )
+ return True
+ except Exception:
+ logger.debug("mcp savings write failed; dropping silently", exc_info=True)
+ return False
+ finally:
+ with contextlib.suppress(Exception):
+ store.close()
diff --git a/packages/server/src/repowise/server/mcp_server/_savings/wrapper.py b/packages/server/src/repowise/server/mcp_server/_savings/wrapper.py
new file mode 100644
index 000000000..c1742649e
--- /dev/null
+++ b/packages/server/src/repowise/server/mcp_server/_savings/wrapper.py
@@ -0,0 +1,129 @@
+"""Instrumentation middleware — measure & record per-tool MCP savings.
+
+:func:`instrument` wraps a registered MCP tool so that every call, after the
+tool produces its (already budget-trimmed) response, measures the delivered
+token count, derives the counterfactual raw-exploration cost it replaced, and
+records a ``mcp:<tool>`` row in the unified savings ledger. It optionally stamps
+``_meta.tokens_saved`` / ``_meta.replaced_tokens`` onto the response for
+transparency.
+
+Two non-negotiables:
+
+* **Byte-identical output.** The wrapped tool returns exactly what the tool
+ returned, save for the optional additive ``_meta`` savings fields. The whole
+ savings path is wrapped in a ``try`` that degrades to returning the untouched
+ result on any failure.
+* **Signature-preserving.** FastMCP introspects each tool's signature to build
+ its input schema, so the wrapper copies ``functools.wraps`` metadata *and*
+ the original ``__signature__`` — a bare ``*args, **kwargs`` wrapper would
+ erase the tool's parameters from the MCP schema.
+
+The counterfactual comes from one of two places, in order of trust:
+ 1. a value the tool *declared* via :func:`declare_replaced` (it held the exact
+ artifact, e.g. ``get_symbol`` with the whole source file);
+ 2. otherwise the conservative estimator in :mod:`.counterfactual`.
+"""
+
+from __future__ import annotations
+
+import contextlib
+import functools
+import inspect
+import json
+import logging
+from collections.abc import Callable
+from typing import Any
+
+from repowise.core.distill.budget import estimate_tokens
+
+from . import counterfactual
+from .recorder import record_mcp_saving
+
+logger = logging.getLogger(__name__)
+
+#: ``_meta`` key a tool sets to declare its own counterfactual (see
+#: :func:`declare_replaced`). The wrapper reads and then leaves it in place as a
+#: transparency annotation.
+_DECLARED_KEY = "replaced_tokens"
+
+
+def declare_replaced(result: dict[str, Any], tokens: int) -> None:
+ """Let a tool declare an exact counterfactual the estimator can't compute.
+
+ Writes ``result["_meta"]["replaced_tokens"]``; the wrapper prefers this over
+ the generic estimator. Used by tools that already hold the replaced artifact
+ in memory (e.g. ``get_symbol`` knows the full file it sliced one symbol out
+ of). Best-effort and additive — never raises, only mutates ``_meta``.
+ """
+ if not isinstance(result, dict) or not isinstance(tokens, int) or tokens <= 0:
+ return
+ meta = result.setdefault("_meta", {})
+ if isinstance(meta, dict):
+ meta[_DECLARED_KEY] = tokens
+
+
+def _declared_tokens(result: Any) -> int | None:
+ """Return a tool-declared counterfactual from ``_meta``, if present."""
+ if not isinstance(result, dict):
+ return None
+ meta = result.get("_meta")
+ if not isinstance(meta, dict):
+ return None
+ value = meta.get(_DECLARED_KEY)
+ return value if isinstance(value, int) and value > 0 else None
+
+
+def _delivered_tokens(result: Any) -> int:
+ """Estimate tokens the agent actually received for *result*."""
+ try:
+ text = json.dumps(result, default=str)
+ except Exception:
+ return 0
+ return estimate_tokens(text)
+
+
+def _record(tool: str, result: Any) -> None:
+ """Measure, derive the counterfactual, and record — all best-effort."""
+ declared = _declared_tokens(result)
+ replaced = declared if declared is not None else counterfactual.replaced_tokens_for(tool, result)
+ if replaced <= 0:
+ return
+
+ delivered = _delivered_tokens(result)
+
+ # Resolve the repo the MCP server is scoped to. Lazy import keeps this
+ # module free of package import-ordering coupling.
+ from repowise.server.mcp_server import _state
+
+ repo_root = getattr(_state, "_repo_path", None)
+ if record_mcp_saving(repo_root, tool, replaced, delivered) and isinstance(result, dict):
+ meta = result.setdefault("_meta", {})
+ if isinstance(meta, dict):
+ meta["replaced_tokens"] = replaced
+ meta["tokens_saved"] = max(0, replaced - delivered)
+
+
+def instrument(fn: Callable[..., Any]) -> Callable[..., Any]:
+ """Wrap an async MCP tool *fn* to record its savings. Signature-preserving.
+
+ Non-coroutine callables are returned unchanged — every OSS MCP tool is
+ ``async``, and a sync tool has no measured-response hook here.
+ """
+ if not inspect.iscoroutinefunction(fn):
+ return fn
+
+ tool = getattr(fn, "__name__", "tool")
+
+ @functools.wraps(fn)
+ async def _wrapped(*args: Any, **kwargs: Any) -> Any:
+ result = await fn(*args, **kwargs)
+ try:
+ _record(tool, result)
+ except Exception: # pragma: no cover - defensive; savings never break a tool
+ logger.debug("mcp savings instrumentation failed for %s", tool, exc_info=True)
+ return result
+
+ # Preserve the original signature so FastMCP builds the correct tool schema.
+ with contextlib.suppress(ValueError, TypeError): # pragma: no cover - exotic callables
+ _wrapped.__signature__ = inspect.signature(fn) # type: ignore[attr-defined]
+ return _wrapped
diff --git a/packages/server/src/repowise/server/mcp_server/tool_symbol.py b/packages/server/src/repowise/server/mcp_server/tool_symbol.py
index 2cae27a93..f3086e3e3 100644
--- a/packages/server/src/repowise/server/mcp_server/tool_symbol.py
+++ b/packages/server/src/repowise/server/mcp_server/tool_symbol.py
@@ -326,12 +326,17 @@ async def _resolve_symbol(
def _slice_source(
repo_path: Path, file_path: str, start_line: int, end_line: int, context_lines: int
-) -> tuple[str, int, int, int | None]:
- """Read the file and return (source, actual_start, actual_end, total_lines).
-
- Returns ("", start, end, None) on read failure. Lines are 1-indexed in the
- inputs and outputs to match WikiSymbol storage. Honors _MAX_SOURCE_LINES.
+) -> tuple[str, int, int, int | None, int]:
+ """Read the file and return (source, actual_start, actual_end, total_lines, full_tokens).
+
+ ``full_tokens`` is the token cost of the *whole* file — the counterfactual
+ a single-symbol slice replaces (the agent would have Read the file to find
+ it). Returns ("", start, end, None, 0) on read failure. Lines are 1-indexed
+ in the inputs and outputs to match WikiSymbol storage. Honors
+ _MAX_SOURCE_LINES.
"""
+ from repowise.core.distill.budget import estimate_tokens
+
abs_path = (repo_path / file_path).resolve()
# Defense in depth: never read outside the repo root, even if the
# WikiSymbol.file_path was somehow tampered with.
@@ -339,13 +344,14 @@ def _slice_source(
abs_path.relative_to(repo_path.resolve())
except ValueError:
_log.warning("get_symbol path escape attempt: %s", file_path)
- return "", start_line, end_line, None
+ return "", start_line, end_line, None, 0
try:
text = abs_path.read_text(encoding="utf-8", errors="replace")
except OSError as exc:
_log.warning("get_symbol read failed for %s: %s", abs_path, exc)
- return "", start_line, end_line, None
+ return "", start_line, end_line, None, 0
+ full_tokens = estimate_tokens(text)
lines = text.splitlines()
total = len(lines)
@@ -364,7 +370,7 @@ def _slice_source(
span = _MAX_SOURCE_LINES
sliced = "\n".join(lines[s - 1 : e])
- return sliced, s, e, total
+ return sliced, s, e, total, full_tokens
@mcp.tool()
@@ -445,7 +451,7 @@ async def get_symbol(
}
repo_root = Path(str(ctx.path))
- source, start, end, _total = _slice_source(
+ source, start, end, _total, full_tokens = _slice_source(
repo_root, row.file_path, row.start_line, row.end_line, context_lines
)
@@ -470,7 +476,7 @@ async def get_symbol(
row.end_line - row.start_line + 1 + 2 * context_lines
) > _MAX_SOURCE_LINES
- return {
+ response = {
"symbol_id": row.symbol_id,
"file": row.file_path,
"name": row.name,
@@ -490,3 +496,9 @@ async def get_symbol(
repository=repository,
),
}
+ # Declare the exact counterfactual: serving one symbol replaced Reading the
+ # whole file. The savings instrumentation prefers this over its estimator.
+ from repowise.server.mcp_server._savings import declare_replaced
+
+ declare_replaced(response, full_tokens)
+ return response
diff --git a/packages/server/src/repowise/server/routers/costs.py b/packages/server/src/repowise/server/routers/costs.py
index b1b2e949a..be3b3e2df 100644
--- a/packages/server/src/repowise/server/routers/costs.py
+++ b/packages/server/src/repowise/server/routers/costs.py
@@ -155,15 +155,22 @@ async def get_distill_savings(
except sqlite3.Error:
return DistillSavingsResponse(available=False)
try:
- summary = tracking.savings_summary(conn, since=since_ts)
- per_filter = tracking.savings_rollup(conn, by="filter", since=since_ts)
+ # Distill figures exclude the ``mcp:*`` counterfactual rows Phase 2 now
+ # writes into the same ledger; the MCP block reports those separately.
+ summary = tracking.distill_summary(conn, since=since_ts)
per_day = tracking.savings_rollup(conn, by="day", since=since_ts)
- mcp = tracking.mcp_drops_summary(conn, since=since_ts)
+ # Unified MCP view: counterfactual ledger merged with truncation drops,
+ # counterfactual taking precedence per tool (no double counting).
+ mcp = tracking.mcp_savings_summary(conn, since=since_ts)
except sqlite3.Error:
return DistillSavingsResponse(available=False)
finally:
conn.close()
+ per_filter = [
+ {"group": name, **stats} for name, stats in summary["per_filter"].items()
+ ]
+
# Missed savings: best-effort scan of local agent transcripts; the module
# degrades to an empty report on any failure, never raises.
from repowise.core.distill.missed import scan_missed_savings
@@ -189,9 +196,15 @@ async def get_distill_savings(
per_day=[DistillSavingsGroup(**row) for row in per_day],
mcp_events=mcp["events"],
mcp_tokens=mcp["tokens"],
+ mcp_queries=mcp["queries"],
mcp_per_tool=[
- McpDropGroup(tool=tool, events=stats["events"], tokens=stats["tokens"])
- for tool, stats in mcp["per_tool"].items()
+ McpDropGroup(
+ tool=row["tool"],
+ events=row["events"],
+ tokens=row["tokens"],
+ kind=row["kind"],
+ )
+ for row in mcp["per_tool"]
],
missed_events=missed["events"],
missed_tokens_est=missed["est_saved_tokens"],
diff --git a/packages/server/src/repowise/server/schemas/ui_helpers.py b/packages/server/src/repowise/server/schemas/ui_helpers.py
index db4aee696..bb2fc6948 100644
--- a/packages/server/src/repowise/server/schemas/ui_helpers.py
+++ b/packages/server/src/repowise/server/schemas/ui_helpers.py
@@ -44,11 +44,18 @@ class DistillSavingsGroup(BaseModel):
class McpDropGroup(BaseModel):
- """Per-tool MCP truncation drops (``tool`` with the ``mcp:`` prefix stripped)."""
+ """Per-tool MCP savings (``tool`` with the ``mcp:`` prefix stripped).
+
+ ``kind`` distinguishes a ``"counterfactual"`` saving (the answer replaced raw
+ file exploration — recorded in the savings ledger) from a ``"truncation"``
+ drop (content trimmed past the response budget — the only signal for tools
+ without a counterfactual estimator yet).
+ """
tool: str
events: int
tokens: int
+ kind: str = "truncation"
class DistillSavingsResponse(BaseModel):
@@ -75,10 +82,13 @@ class DistillSavingsResponse(BaseModel):
pricing_source: str = "default"
per_filter: list[DistillSavingsGroup] = []
per_day: list[DistillSavingsGroup] = []
- # MCP truncation drops already on disk (omissions store, source='mcp:*'),
- # never recorded in the distill ledger.
+ # Unified MCP savings: counterfactual ledger rows (each answer replaced raw
+ # file exploration) merged with truncation drops for tools that have no
+ # counterfactual estimator. ``mcp_queries`` counts counterfactual tool calls
+ # ("N MCP queries answered"); ``mcp_tokens`` is total saved.
mcp_events: int = 0
mcp_tokens: int = 0
+ mcp_queries: int = 0
mcp_per_tool: list[McpDropGroup] = []
# Missed savings — raw (non-distilled) agent commands a filter would have
# caught, scanned best-effort from local Claude Code transcripts.
diff --git a/packages/ui/__tests__/costs/distill-savings-card.test.tsx b/packages/ui/__tests__/costs/distill-savings-card.test.tsx
index 3ea4dcbf4..9eb990a2a 100644
--- a/packages/ui/__tests__/costs/distill-savings-card.test.tsx
+++ b/packages/ui/__tests__/costs/distill-savings-card.test.tsx
@@ -59,6 +59,26 @@ describe("DistillSavingsCard", () => {
expect(link).toHaveAttribute("href", expect.stringContaining("DISTILL.md"));
});
+ it("frames MCP savings as queries answered when counterfactuals exist", () => {
+ render(
+ <DistillSavingsCard
+ data={makeData({
+ mcp_queries: 8,
+ mcp_per_tool: [
+ { tool: "get_context", events: 6, tokens: 30_000, kind: "counterfactual" },
+ { tool: "get_risk", events: 2, tokens: 9_000, kind: "truncation" },
+ ],
+ })}
+ />,
+ );
+ expect(screen.getByText(/8 queries answered/)).toBeInTheDocument();
+ });
+
+ it("falls back to the drop count when no counterfactual queries exist", () => {
+ render(<DistillSavingsCard data={makeData({ mcp_queries: 0 })} />);
+ expect(screen.getByText(/5 drops/)).toBeInTheDocument();
+ });
+
it("shows the empty state when nothing is saved", () => {
render(<DistillSavingsCard data={makeData({ available: false, saved_tokens: 0, mcp_tokens: 0 })} />);
expect(screen.getByText(/No agent token savings recorded yet/)).toBeInTheDocument();
diff --git a/packages/ui/src/costs/distill-savings-card.tsx b/packages/ui/src/costs/distill-savings-card.tsx
index 93193d07b..3875d24bd 100644
--- a/packages/ui/src/costs/distill-savings-card.tsx
+++ b/packages/ui/src/costs/distill-savings-card.tsx
@@ -16,6 +16,8 @@ export interface McpDropGroup {
tool: string;
events: number;
tokens: number;
+ /** "counterfactual" (answer replaced raw exploration) or "truncation" (budget drop). */
+ kind?: string;
}
export interface DistillSavingsData {
@@ -32,6 +34,8 @@ export interface DistillSavingsData {
per_day: DistillSavingsGroup[];
mcp_events?: number;
mcp_tokens?: number;
+ /** Count of counterfactual MCP queries answered ("N MCP queries answered"). */
+ mcp_queries?: number;
mcp_per_tool?: McpDropGroup[];
/** Raw (non-distilled) agent commands a filter would have caught. */
missed_events?: number;
@@ -56,8 +60,8 @@ function agentLabel(agent: string | undefined): string {
/**
* Hero results card for the Costs page: every token & dollar repowise saved the
* coding agent, across the `repowise distill` ledger (CLI + hook) and the MCP
- * tools whose oversized responses were trimmed. Priced at the agent's *real*
- * model — saved tokens are input the agent never had to read.
+ * tools — each curated answer replacing raw file exploration. Priced at the
+ * agent's *real* model — saved tokens are input the agent never had to read.
*/
export function DistillSavingsCard({ data }: DistillSavingsCardProps) {
const distillSaved = data?.saved_tokens ?? 0;
@@ -105,6 +109,15 @@ export function DistillSavingsCard({ data }: DistillSavingsCardProps) {
const detected = data.pricing_source && data.pricing_source !== "default";
const missed = (data.missed_tokens_est ?? 0) > 0;
+ // Prefer the counterfactual framing ("N queries answered") when any MCP call
+ // recorded a real saving; fall back to the truncation-drop count otherwise.
+ const mcpQueries = data.mcp_queries ?? 0;
+ const mcpEvents = data.mcp_events ?? 0;
+ const mcpCaption =
+ mcpQueries > 0
+ ? `MCP · ${mcpQueries.toLocaleString()} quer${mcpQueries === 1 ? "y" : "ies"} answered`
+ : `MCP · ${mcpEvents.toLocaleString()} drop${mcpEvents === 1 ? "" : "s"}`;
+
return (
<Card>
<CardContent className="py-6">
@@ -148,10 +161,7 @@ export function DistillSavingsCard({ data }: DistillSavingsCardProps) {
<div className="text-lg font-semibold tabular-nums text-[var(--color-text-primary)]">
{formatTokens(mcpSaved)}
</div>
- <div className="text-[11px] text-[var(--color-text-tertiary)]">
- MCP · {(data.mcp_events ?? 0).toLocaleString()} drop
- {(data.mcp_events ?? 0) === 1 ? "" : "s"}
- </div>
+ <div className="text-[11px] text-[var(--color-text-tertiary)]">{mcpCaption}</div>
</div>
</div>
</div>
@@ -226,9 +236,10 @@ export function DistillSavingsCard({ data }: DistillSavingsCardProps) {
)}
<p className="mt-3 text-[10px] leading-snug text-[var(--color-text-tertiary)]">
- Distill counts <code>repowise distill</code> command/hook savings; MCP counts tokens
- trimmed from oversized tool responses already on disk. Saved tokens are agent input,
- priced at the agent&apos;s input rate. Transcript scans stay on this machine.
+ Distill counts <code>repowise distill</code> command/hook savings; MCP counts the raw
+ file exploration each tool answer replaced (plus any over-budget content trimmed). Saved
+ tokens are agent input, priced at the agent&apos;s input rate. Everything stays on this
+ machine.
</p>
</CardContent>
</Card>
diff --git a/packages/web/src/lib/api/costs.ts b/packages/web/src/lib/api/costs.ts
index 5365e0d4d..b301bc874 100644
--- a/packages/web/src/lib/api/costs.ts
+++ b/packages/web/src/lib/api/costs.ts
@@ -32,6 +32,8 @@ export interface McpDropGroup {
tool: string;
events: number;
tokens: number;
+ /** "counterfactual" (answer replaced raw exploration) or "truncation" (budget drop). */
+ kind?: string;
}
export interface DistillSavings {
@@ -47,9 +49,11 @@ export interface DistillSavings {
pricing_source: string;
per_filter: DistillSavingsGroup[];
per_day: DistillSavingsGroup[];
- /** MCP truncation drops already on disk (omissions store, source mcp:*). */
+ /** Unified MCP savings — counterfactual ledger + truncation drops. */
mcp_events: number;
mcp_tokens: number;
+ /** Count of counterfactual MCP queries answered ("N MCP queries answered"). */
+ mcp_queries: number;
mcp_per_tool: McpDropGroup[];
/** Raw (non-distilled) agent commands a filter would have caught. */
missed_events: number;
diff --git a/tests/integration/mcp/__init__.py b/tests/integration/mcp/__init__.py
new file mode 100644
index 000000000..e69de29bb
diff --git a/tests/integration/mcp/test_instrument.py b/tests/integration/mcp/test_instrument.py
new file mode 100644
index 000000000..b90b934a7
--- /dev/null
+++ b/tests/integration/mcp/test_instrument.py
@@ -0,0 +1,103 @@
+"""instrument() — records a ledger row without changing user-facing output.
+
+Exercises the full savings seam end to end against a real omission sidecar: a
+wrapped tool's response is byte-identical (save for additive ``_meta`` fields)
+and produces a ``mcp:<tool>`` ledger row; an unwrapped tool produces none.
+"""
+
+from __future__ import annotations
+
+import copy
+import inspect
+from pathlib import Path
+
+import pytest
+
+from repowise.core.distill.store import OmissionStore
+from repowise.server.mcp_server import _state
+from repowise.server.mcp_server._savings import declare_replaced, instrument
+
+
+def _repo_db(repo: Path) -> Path:
+ return repo / ".repowise" / "omissions" / "omissions.db"
+
+
+def repo(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
+ """A repo with an opt-in sidecar, set as the MCP server's scoped repo."""
+ OmissionStore(_repo_db(tmp_path)).close()
+ monkeypatch.setattr(_state, "_repo_path", str(tmp_path), raising=False)
+ return tmp_path
+
+
+def _ledger(repo: Path) -> dict[str, dict]:
+ store = OmissionStore(_repo_db(repo))
+ try:
+ return {r["group"]: r for r in store.savings_rollup(by="source")}
+ finally:
+ store.close()
+
+
+async def get_context(targets: list[str], compact: bool = True) -> dict:
+ """A get_context-shaped tool whose skeleton drives the generic estimator."""
+ return {
+ "targets": {targets[0]: {"skeleton": {"tokens": 200, "full_tokens": 4000}}},
+ "_meta": {"timing_ms": 1.0},
+ }
+
+
+async def get_symbol(symbol_id: str) -> dict:
+ """A get_symbol-shaped tool that declares an exact counterfactual."""
+ response = {"symbol_id": symbol_id, "source": "def f(): ...", "_meta": {}}
+ declare_replaced(response, 5000)
+ return response
+
+
+async def test_wrapped_tool_records_and_preserves_output(repo: Path) -> None:
+ raw = await get_context(["a.py"])
+ expected = copy.deepcopy(raw)
+
+ wrapped = instrument(get_context)
+ out = await wrapped(["a.py"])
+
+ # User-facing payload unchanged; only additive _meta savings fields appear.
+ assert out["targets"] == expected["targets"]
+ assert out["_meta"]["timing_ms"] == 1.0
+ assert out["_meta"]["replaced_tokens"] == 4000
+ assert out["_meta"]["tokens_saved"] > 0
+
+ row = _ledger(repo)["mcp:get_context"]
+ assert row["raw_tokens"] == 4000
+ assert row["events"] == 1
+
+
+async def test_declared_counterfactual_wins(repo: Path) -> None:
+ wrapped = instrument(get_symbol)
+ out = await wrapped("a.py::f")
+ assert out["_meta"]["replaced_tokens"] == 5000
+ assert _ledger(repo)["mcp:get_symbol"]["raw_tokens"] == 5000
+
+
+async def test_unwrapped_tool_records_nothing(repo: Path) -> None:
+ await get_context(["a.py"]) # called directly, no middleware
+ assert _ledger(repo) == {}
+
+
+async def test_instrument_preserves_signature() -> None:
+ wrapped = instrument(get_context)
+ assert inspect.signature(wrapped) == inspect.signature(get_context)
+ assert wrapped.__name__ == "get_context"
+
+
+async def test_no_saving_writes_no_row(repo: Path) -> None:
+ async def search_codebase(query: str) -> dict:
+ return {"results": [], "_meta": {}} # nothing cited → no counterfactual
+
+ out = await instrument(search_codebase)("x")
+ assert "replaced_tokens" not in out["_meta"]
+ assert _ledger(repo) == {}
diff --git a/tests/unit/distill/test_saved_cmd.py b/tests/unit/distill/test_saved_cmd.py
index 645442c4f..2239aafdc 100644
--- a/tests/unit/distill/test_saved_cmd.py
+++ b/tests/unit/distill/test_saved_cmd.py
@@ -121,10 +121,10 @@ def test_saved_reports_totals_and_per_filter(repo_cwd: Path) -> None:
assert "TOTAL" in result.output
assert "13,800" in result.output # total saved tokens
assert "Estimated saved" in result.output
- # The scope caveat must be visible in the report itself (normalize
+ # The scope caption must be visible in the report itself (normalize
# whitespace — rich wraps the caption at terminal width).
flat = " ".join(result.output.split())
- assert "MCP response truncation is not counted" in flat
+ assert "MCP counterfactual savings (mcp:<tool>)" in flat
def test_saved_by_day(repo_cwd: Path) -> None:
diff --git a/tests/unit/distill/test_store.py b/tests/unit/distill/test_store.py
index 831796f33..cf13f5b76 100644
--- a/tests/unit/distill/test_store.py
+++ b/tests/unit/distill/test_store.py
@@ -144,6 +144,85 @@ def test_mcp_drops_summary_empty(store: OmissionStore) -> None:
assert summary == {"events": 0, "tokens": 0, "per_tool": {}}
+def test_distill_summary_excludes_mcp_rows(store: OmissionStore) -> None:
+ from repowise.core.distill.tracking import distill_summary
+
+ # Distill surface rows…
+ store.record_saving(
+ filter_name="test_output", source="cli", command="pytest",
+ raw_tokens=1000, distilled_tokens=100,
+ )
+ store.record_saving(
+ filter_name="git_log", source="hook-bash", command="git log",
+ raw_tokens=500, distilled_tokens=50,
+ )
+ # …and an MCP counterfactual row in the same ledger, which distill must skip.
+ store.record_saving(
+ filter_name="get_context", source="mcp:get_context", command=None,
+ raw_tokens=4000, distilled_tokens=400,
+ )
+
+ summary = distill_summary(store._conn)
+ assert summary["events"] == 2
+ assert summary["raw_tokens"] == 1500
+ assert summary["saved_tokens"] == 1350
+ assert set(summary["per_filter"]) == {"test_output", "git_log"}
+ assert "get_context" not in summary["per_filter"]
+
+
+def test_mcp_savings_summary_counterfactual_precedence(store: OmissionStore) -> None:
+ from repowise.core.distill.tracking import mcp_savings_summary
+
+ # Counterfactual ledger rows for two tools (these subsume their own
+ # truncation, since delivered is measured post-truncation).
+ store.record_saving(
+ filter_name="get_symbol", source="mcp:get_symbol", command=None,
+ raw_tokens=3000, distilled_tokens=300,
+ )
+ store.record_saving(
+ filter_name="get_symbol", source="mcp:get_symbol", command=None,
+ raw_tokens=1000, distilled_tokens=200,
+ )
+ store.record_saving(
+ filter_name="get_context", source="mcp:get_context", command=None,
+ raw_tokens=2000, distilled_tokens=500,
+ )
+ # Truncation drops: get_symbol also has drops (must NOT be added on top —
+ # counterfactual wins); get_risk has ONLY drops (its sole signal).
+ store.put("x" * 400, source="mcp:get_symbol", original_tokens=900, kept_tokens=0)
+ store.put("y" * 400, source="mcp:get_risk", original_tokens=700, kept_tokens=0)
+ # A distill omission must never leak into the MCP view.
+ store.put("z" * 400, source="cli:logs", original_tokens=999, kept_tokens=0)
+
+ summary = mcp_savings_summary(store._conn)
+ by_tool = {row["tool"]: row for row in summary["per_tool"]}
+
+ # get_symbol → counterfactual saved 2700+800=3500, drops ignored.
+ assert by_tool["get_symbol"] == {
+ "tool": "get_symbol", "events": 2, "tokens": 3500, "kind": "counterfactual",
+ }
+ # get_context → counterfactual saved 1500.
+ assert by_tool["get_context"]["tokens"] == 1500
+ assert by_tool["get_context"]["kind"] == "counterfactual"
+ # get_risk → truncation only.
+ assert by_tool["get_risk"] == {
+ "tool": "get_risk", "events": 1, "tokens": 700, "kind": "truncation",
+ }
+ # queries counts counterfactual events only; tokens is the merged total.
+ assert summary["queries"] == 3
+ assert summary["tokens"] == 3500 + 1500 + 700
+ assert summary["events"] == 2 + 1 + 1
+ # Ordered by tokens desc.
+ assert [r["tool"] for r in summary["per_tool"]] == ["get_symbol", "get_context", "get_risk"]
+
+
+def test_mcp_savings_summary_empty(store: OmissionStore) -> None:
+ from repowise.core.distill.tracking import mcp_savings_summary
+
+ summary = mcp_savings_summary(store._conn)
+ assert summary == {"events": 0, "tokens": 0, "queries": 0, "per_tool": []}
+
+
def test_default_store_path_finds_repowise_dir(tmp_path: Path) -> None:
repo = tmp_path / "repo"
nested = repo / "src" / "deep"
diff --git a/tests/unit/server/mcp/test_counterfactual.py b/tests/unit/server/mcp/test_counterfactual.py
new file mode 100644
index 000000000..5cfd68543
--- /dev/null
+++ b/tests/unit/server/mcp/test_counterfactual.py
@@ -0,0 +1,61 @@
+"""Per-tool counterfactual estimators — conservative, dict-only, never raise."""
+
+from __future__ import annotations
+
+from repowise.server.mcp_server._savings import counterfactual as cf
+
+
+def test_get_context_sums_skeleton_full_tokens() -> None:
+ result = {
+ "targets": {
+ "a.py": {"target": "a.py", "skeleton": {"tokens": 200, "full_tokens": 1500}},
+ "b.py": {"target": "b.py", "skeleton": {"tokens": 80, "full_tokens": 900}},
+ # No skeleton (small-file card / symbol target) → contributes nothing.
+ "c.py": {"target": "c.py", "docs": {"summary": "x"}},
+ },
+ "_meta": {},
+ }
+ assert cf.replaced_tokens_for("get_context", result) == 2400
+
+
+def test_get_context_no_skeletons_returns_zero() -> None:
+ result = {"targets": {"a.py": {"target": "a.py"}}, "_meta": {}}
+ assert cf.replaced_tokens_for("get_context", result) == 0
+
+
+def test_search_codebase_floors_per_distinct_cited_path() -> None:
+ result = {
+ "results": [
+ {"target_path": "src/a.py"},
+ {"target_path": "src/b.py"},
+ {"target_path": "src/a.py"}, # duplicate path — counted once
+ {"target_path": ""}, # empty path — ignored
+ {"title": "no path key"}, # missing path — ignored
+ ],
+ "_meta": {},
+ }
+ assert cf.replaced_tokens_for("search_codebase", result) == 2 * cf.SEARCH_FLOOR_PER_HIT
+
+
+def test_search_codebase_no_results_returns_zero() -> None:
+ assert cf.replaced_tokens_for("search_codebase", {"results": [], "_meta": {}}) == 0
+
+
+def test_unknown_tool_returns_zero() -> None:
+ assert cf.replaced_tokens_for("get_risk", {"anything": 1}) == 0
+
+
+def test_non_dict_result_returns_zero() -> None:
+ assert cf.replaced_tokens_for("get_context", None) == 0
+ assert cf.replaced_tokens_for("get_context", "oops") == 0
+
+
+def test_malformed_skeleton_does_not_raise() -> None:
+ result = {
+ "targets": {
+ "a.py": {"skeleton": {"full_tokens": "not-an-int"}},
+ "b.py": {"skeleton": "not-a-dict"},
+ "c.py": "not-a-dict",
+ }
+ }
+ assert cf.replaced_tokens_for("get_context", result) == 0
diff --git a/tests/unit/server/mcp/test_savings_recorder.py b/tests/unit/server/mcp/test_savings_recorder.py
new file mode 100644
index 000000000..66c587be5
--- /dev/null
+++ b/tests/unit/server/mcp/test_savings_recorder.py
@@ -0,0 +1,51 @@
+"""record_mcp_saving — writes a mcp:<tool> ledger row, degrades silently."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+from repowise.core.distill.store import OmissionStore
+from repowise.server.mcp_server._savings.recorder import record_mcp_saving
+
+
+def _repo_db(repo: Path) -> Path:
+ return repo / ".repowise" / "omissions" / "omissions.db"
+
+
+def _init_repo(tmp_path: Path) -> Path:
+ """Create a repo with an opt-in repo-local omission sidecar; return its root."""
+ OmissionStore(_repo_db(tmp_path)).close() # materialise the file
+ return tmp_path
+
+
+def test_records_mcp_row(tmp_path: Path) -> None:
+ repo = _init_repo(tmp_path)
+ assert record_mcp_saving(repo, "get_symbol", replaced_tokens=3000, delivered_tokens=300)
+
+ store = OmissionStore(_repo_db(repo))
+ try:
+ rollup = {r["group"]: r for r in store.savings_rollup(by="source")}
+ finally:
+ store.close()
+ assert rollup["mcp:get_symbol"]["saved_tokens"] == 2700
+ assert rollup["mcp:get_symbol"]["events"] == 1
+
+
+def test_skips_when_no_net_saving(tmp_path: Path) -> None:
+ repo = _init_repo(tmp_path)
+ # replaced <= delivered → no row.
+ assert not record_mcp_saving(repo, "get_context", replaced_tokens=200, delivered_tokens=200)
+ assert not record_mcp_saving(repo, "get_context", replaced_tokens=100, delivered_tokens=500)
+
+ store = OmissionStore(_repo_db(repo))
+ try:
+ assert store.savings_summary()["events"] == 0
+ finally:
+ store.close()
+
+
+def test_no_store_degrades_silently(tmp_path: Path) -> None:
+ # A repo that never ran `repowise init` has no sidecar; we must not create
+ # one from the read-side, and must not raise.
+ assert not record_mcp_saving(tmp_path, "get_symbol", replaced_tokens=3000, delivered_tokens=10)
+ assert not (tmp_path / ".repowise").exists()