Skip to content

Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.

feat(telemetry): MCP tool-call events, interrupted status, index-shape outcomes (#820)

Raghav Chamadiya4b7ddc577289
diff --git a/docs/TELEMETRY.md b/docs/TELEMETRY.md
index 3286a6cb2..ef14bc4e6 100644
--- a/docs/TELEMETRY.md
+++ b/docs/TELEMETRY.md
@@ -16,17 +16,19 @@ A tiny amount of anonymous data answers questions we otherwise have to guess at:
- Which CLI versions and Python versions are in use? (what we can drop support for)
- What share of runs fail, and on which command? (where the bugs are)
- Which OSes matter? (what to test on)
+- Which MCP tools do coding agents reach for, and how often do answers come
+ back stale? (where the retrieval work pays off)
That is the whole purpose. We do not sell it, and we do not use it to identify
anyone.
## What is collected
-One event, `command_run`, is sent once per CLI invocation. Every event carries:
+Every event, whatever its type, carries the same anonymous envelope:
| Field | Example | Purpose |
|---|---|---|
-| `event` | `command_run` | Event type |
+| `event` | `command_run` / `mcp_tool_call` | Event type |
| `anon_id` | `194639…d48a` | Random per-install id (see below) |
| `session_id` | `e2ecb2…1158` | Random per-process id, groups one run |
| `cli_version` | `0.21.0` | Version adoption |
@@ -34,14 +36,50 @@ One event, `command_run`, is sent once per CLI invocation. Every event carries:
| `arch` | `arm64` / `x86_64` | Architecture priorities |
| `python_version` | `3.12` | **major.minor only** |
| `is_ci` | `true` / `false` | Separate automation from humans |
+
+### `command_run` — once per CLI invocation
+
+| Field | Example | Purpose |
+|---|---|---|
| `properties.command` | `init`, `update`, `health` | Which command ran |
| `properties.subcommand` | `decision.add`, `telemetry.status` | A known subcommand name only |
| `properties.flags` | `["--resume", "--provider"]` | Option **names only**, never values |
-| `properties.status` | `ok` / `error` | Success rate |
+| `properties.status` | `ok` / `error` / `interrupted` | Success, failure, or user-cancelled (Ctrl-C) |
| `properties.error_type` | `LLMProviderError` | Exception **class name only** |
| `properties.duration_ms` | `71840` | Performance in the wild |
-Example payload:
+For `init` and `update`, a few coarse **buckets and enums** describe the shape
+of the run so we can see the size of repos people index and whether docs
+generation is used. Never exact counts, never repo or file names:
+
+| Field | Example | Purpose |
+|---|---|---|
+| `properties.file_count_bucket` | `500-999` | Repo size distribution (a range, not a count) |
+| `properties.top_language` | `python` | Which languages to prioritise |
+| `properties.docs_mode` | `true` / `false` | AI docs vs index-only adoption |
+| `properties.provider` / `properties.model` | `anthropic` / `claude-opus` | Which models generate docs |
+| `properties.embedder` | `openai` | Which embedders are in use |
+| `properties.pages_bucket` | `100-499` | Docs volume (a range) |
+
+### `mcp_tool_call` — once per MCP tool call
+
+Emitted by the MCP server (`repowise mcp` / `repowise serve`) so we can see
+which tools coding agents actually use and how often answers come back stale or
+degraded. Only the tool name and coarse enums/booleans — **never the question,
+query, results, paths, or repo/symbol names**:
+
+| Field | Example | Purpose |
+|---|---|---|
+| `properties.tool` | `get_answer`, `search_codebase` | Which tool was called |
+| `properties.status` | `ok` / `error` | Tool error rate |
+| `properties.duration_ms` | `820` | Tool latency in the wild |
+| `properties.confidence` | `high` / `medium` / `low` | Answer confidence distribution |
+| `properties.retrieval_quality` | `strong` / `weak` | Retrieval quality distribution |
+| `properties.results_bucket` | `1-3`, `4-10` | Result count for searches (a range) |
+| `properties.index_behind` | `true` / `false` | How often served content is stale |
+| `properties.embedder_degraded` | `true` / `false` | How often search runs on mock vectors |
+
+Example `command_run` payload:
```json
{
diff --git a/packages/cli/src/repowise/cli/_instrumented_group.py b/packages/cli/src/repowise/cli/_instrumented_group.py
index 055d377d0..e00035a06 100644
--- a/packages/cli/src/repowise/cli/_instrumented_group.py
+++ b/packages/cli/src/repowise/cli/_instrumented_group.py
@@ -61,11 +61,33 @@ class InstrumentedGroup(click.Group):
error_type: str | None = None
try:
return super().invoke(ctx)
+ except click.exceptions.Exit as exc:
+ # click's ctx.exit(): code 0 is a clean success (must NOT read as an
+ # error), 130 is a Ctrl-C exit, anything else is a real failure.
+ code = getattr(exc, "exit_code", 0)
+ if code in (0, None):
+ status = "ok"
+ elif code == 130:
+ status = "interrupted"
+ else:
+ status = "error"
+ error_type = "Exit"
+ raise
except SystemExit as exc:
- if exc.code not in (0, None):
+ if exc.code in (0, None):
+ status = "ok"
+ elif exc.code == 130:
+ status = "interrupted"
+ else:
status = "error"
raise
- except (click.ClickException, click.exceptions.Abort) as exc:
+ except (KeyboardInterrupt, click.exceptions.Abort):
+ # User cancelled (Ctrl-C / declined a prompt). Not a failure — long-
+ # running commands (serve/watch/init) are routinely Ctrl-C'd, and
+ # counting that as "error" made success rates uninterpretable.
+ status = "interrupted"
+ raise
+ except click.ClickException as exc:
status = "error"
error_type = type(exc).__name__ # class name only, never the message
raise
@@ -88,6 +110,7 @@ class InstrumentedGroup(click.Group):
status=status,
error_type=error_type,
duration_ms=duration_ms,
+ extra=telemetry.drain_command_outcome(),
)
def _subcommand_and_flags(
diff --git a/packages/cli/src/repowise/cli/commands/init_cmd/command.py b/packages/cli/src/repowise/cli/commands/init_cmd/command.py
index 2efd61756..d0bf4229f 100644
--- a/packages/cli/src/repowise/cli/commands/init_cmd/command.py
+++ b/packages/cli/src/repowise/cli/commands/init_cmd/command.py
@@ -83,6 +83,54 @@ from .reporting import show_analysis_summary, show_completion
from .workspace import _workspace_init
+def _record_init_outcome(
+ *,
+ result: Any,
+ effective_index_only: bool,
+ run_mode: str,
+ provider: Any,
+ embedder_name_resolved: str,
+) -> None:
+ """Attach an anonymous shape-of-the-index outcome to the ``command_run`` event.
+
+ Coarse buckets + enums only (file-count bucket, docs mode, run mode, top
+ language, provider/embedder names) — no repo names, paths, or exact counts.
+ Best-effort: never let telemetry break the command's happy path.
+ """
+ try:
+ from repowise.cli.platform import telemetry
+
+ outcome: dict[str, Any] = {
+ "outcome": "success",
+ "index_only": bool(effective_index_only),
+ "run_mode": run_mode,
+ "file_count_bucket": telemetry.bucket_count(getattr(result, "file_count", 0) or 0),
+ "symbol_count_bucket": telemetry.bucket_count(getattr(result, "symbol_count", 0) or 0),
+ }
+
+ lang_dist = getattr(getattr(result, "repo_structure", None), "root_language_distribution", None)
+ if isinstance(lang_dist, dict) and lang_dist:
+ outcome["top_language"] = max(lang_dist.items(), key=lambda kv: kv[1])[0]
+
+ if not effective_index_only and provider is not None:
+ outcome["docs_mode"] = True
+ outcome["provider"] = getattr(provider, "provider_name", None)
+ outcome["model"] = getattr(provider, "model_name", None)
+ pages = getattr(result, "generated_pages", None) or []
+ outcome["pages_bucket"] = telemetry.bucket_count(len(pages))
+ det = sum(1 for p in pages if getattr(p, "provider_name", "") == "template")
+ outcome["deterministic_pages_bucket"] = telemetry.bucket_count(det)
+ else:
+ outcome["docs_mode"] = False
+
+ if embedder_name_resolved:
+ outcome["embedder"] = embedder_name_resolved
+
+ telemetry.add_command_outcome(**{k: v for k, v in outcome.items() if v is not None})
+ except Exception:
+ return
+
+
def _run_generation_phase(
*,
repo_path: Path,
@@ -1109,6 +1157,14 @@ def init_command(
provider=provider,
)
+ _record_init_outcome(
+ result=result,
+ effective_index_only=effective_index_only,
+ run_mode=run_mode,
+ provider=provider,
+ embedder_name_resolved=embedder_name_resolved,
+ )
+
# Offer to install post-commit hook (both index-only and full modes)
offer_hook_install(console, [repo_path], yes=yes)
diff --git a/packages/cli/src/repowise/cli/commands/update_cmd/command.py b/packages/cli/src/repowise/cli/commands/update_cmd/command.py
index 0f2d8780b..050fac1bf 100644
--- a/packages/cli/src/repowise/cli/commands/update_cmd/command.py
+++ b/packages/cli/src/repowise/cli/commands/update_cmd/command.py
@@ -64,6 +64,36 @@ from .workspace import _workspace_update
log = structlog.get_logger(__name__)
+def _record_update_outcome(
+ *,
+ index_only: bool,
+ changed_count: int,
+ provider: Any = None,
+ generated_pages: list | None = None,
+) -> None:
+ """Attach an anonymous update-shape outcome to the ``command_run`` event.
+
+ Coarse buckets + enums only (changed-files bucket, docs mode, provider,
+ pages bucket). Best-effort; never breaks the command.
+ """
+ try:
+ from repowise.cli.platform import telemetry
+
+ outcome: dict[str, Any] = {
+ "outcome": "success",
+ "index_only": bool(index_only),
+ "docs_mode": not index_only and provider is not None,
+ "changed_files_bucket": telemetry.bucket_count(changed_count),
+ }
+ if not index_only and provider is not None:
+ outcome["provider"] = getattr(provider, "provider_name", None)
+ outcome["model"] = getattr(provider, "model_name", None)
+ outcome["pages_bucket"] = telemetry.bucket_count(len(generated_pages or []))
+ telemetry.add_command_outcome(**{k: v for k, v in outcome.items() if v is not None})
+ except Exception:
+ return
+
+
def _refresh_editor_stamp(
repo_path: Any, agents_md: bool | None, degraded: list[str] | None = None
) -> None:
@@ -724,6 +754,7 @@ def run_update(
emitter.error(str(exc))
raise
_refresh_editor_stamp(repo_path, agents_md, degraded)
+ _record_update_outcome(index_only=True, changed_count=len(file_diffs))
if emitter is not None:
emitter.done(
ok=True,
@@ -1202,6 +1233,12 @@ def run_update(
degraded.append(f"Cross-repo analysis: {exc}")
elapsed = time.monotonic() - start
+ _record_update_outcome(
+ index_only=False,
+ changed_count=len(file_diffs),
+ provider=provider,
+ generated_pages=generated_pages,
+ )
if emitter is not None:
emitter.done(
ok=True,
diff --git a/packages/cli/src/repowise/cli/platform/telemetry/__init__.py b/packages/cli/src/repowise/cli/platform/telemetry/__init__.py
index f30a978b3..9a35762aa 100644
--- a/packages/cli/src/repowise/cli/platform/telemetry/__init__.py
+++ b/packages/cli/src/repowise/cli/platform/telemetry/__init__.py
@@ -32,6 +32,8 @@ __all__ = [
"TELEMETRY_DOC_URL",
"CommandRunEvent",
"TelemetryEvent",
+ "add_command_outcome",
+ "drain_command_outcome",
"maybe_show_notice",
"record",
"record_command_run",
@@ -41,6 +43,52 @@ __all__ = [
]
+#: Anonymous outcome fields a command stashes for THIS invocation's
+#: ``command_run`` event. A CLI process runs exactly one command, so a plain
+#: module global is sufficient (no contextvar needed); the root wrapper drains
+#: it once when it records the event.
+_command_outcome: dict[str, Any] = {}
+
+
+def bucket_count(n: int) -> str:
+ """Bucket a count into a coarse, non-identifying range for telemetry.
+
+ Exact file/page counts can help fingerprint a specific repo; buckets give
+ the size distribution we actually want without that risk.
+ """
+ if n <= 0:
+ return "0"
+ if n < 10:
+ return "1-9"
+ if n < 100:
+ return "10-99"
+ if n < 500:
+ return "100-499"
+ if n < 1000:
+ return "500-999"
+ if n < 5000:
+ return "1k-5k"
+ return "5k+"
+
+
+def add_command_outcome(**fields: Any) -> None:
+ """Attach anonymous outcome fields to this invocation's ``command_run`` event.
+
+ For coarse, non-identifying context a command wants reported alongside the
+ generic outcome: bucketed counts, coarse enums, booleans (e.g. an index's
+ file-count bucket or configured provider). Same privacy contract as events —
+ never pass source, paths, repo/symbol names, or raw values.
+ """
+ _command_outcome.update(fields)
+
+
+def drain_command_outcome() -> dict[str, Any]:
+ """Return and clear the stashed outcome fields (called once by the wrapper)."""
+ outcome = dict(_command_outcome)
+ _command_outcome.clear()
+ return outcome
+
+
def record_command_run(
command: str,
*,
diff --git a/packages/cli/src/repowise/cli/platform/telemetry/consent.py b/packages/cli/src/repowise/cli/platform/telemetry/consent.py
index 4ac68609e..628dee96b 100644
--- a/packages/cli/src/repowise/cli/platform/telemetry/consent.py
+++ b/packages/cli/src/repowise/cli/platform/telemetry/consent.py
@@ -19,7 +19,8 @@ TELEMETRY_DOC_URL = "https://repowise.dev/telemetry"
_NOTICE = (
"Repowise collects completely anonymous usage telemetry (commands run, "
- "versions, OS, success/duration) to help prioritize what to build.\n"
+ "MCP tools used, versions, OS, success/duration) to help prioritize what to "
+ "build.\n"
"It never sees your code, file paths, or repo names. Opt out anytime with "
"'repowise telemetry disable' or DO_NOT_TRACK=1.\n"
f"Details: {TELEMETRY_DOC_URL}"
diff --git a/packages/core/src/repowise/core/platform/__init__.py b/packages/core/src/repowise/core/platform/__init__.py
new file mode 100644
index 000000000..f6a58c946
--- /dev/null
+++ b/packages/core/src/repowise/core/platform/__init__.py
@@ -0,0 +1,5 @@
+"""Shared platform primitives usable from both the CLI and the server layers.
+
+Lives in ``repowise-core`` (the lowest layer) so ``repowise.server`` can emit
+anonymous telemetry without importing ``repowise.cli`` (which it must never do).
+"""
diff --git a/packages/core/src/repowise/core/platform/telemetry.py b/packages/core/src/repowise/core/platform/telemetry.py
new file mode 100644
index 000000000..a242b812b
--- /dev/null
+++ b/packages/core/src/repowise/core/platform/telemetry.py
@@ -0,0 +1,227 @@
+"""Anonymous, opt-out telemetry for the core + server layers.
+
+A small, self-contained emitter so ``repowise.server`` (which never imports
+``repowise.cli``) can record events. It shares the on-disk and wire contract
+with the richer CLI telemetry in ``repowise.cli.platform.telemetry``:
+
+* the same ``~/.repowise/platform.json`` anonymous install id,
+* the same ``DO_NOT_TRACK`` / ``REPOWISE_TELEMETRY_DISABLED`` consent env vars,
+* the same ``POST https://api.repowise.dev/telemetry/events`` endpoint,
+
+so CLI ``command_run`` events and server ``mcp_tool_call`` events land in one
+table and group by the same install. The CLI substrate could later delegate to
+this module; it is kept separate for now to avoid churning the shipped,
+well-tested CLI path (upgrade path, not duplication of a subsystem).
+
+Privacy contract (identical to the CLI's): never put source code, file paths,
+repo or symbol names, query text, flag values, or anything user-identifiable
+into an event's ``properties``. Coarse enums, bucketed counts, and booleans
+only. Reads are best-effort and this module never raises into a caller.
+"""
+
+from __future__ import annotations
+
+import atexit
+import contextlib
+import json
+import os
+import platform
+import threading
+import uuid
+from pathlib import Path
+from typing import Any
+
+#: Production ingest endpoint (same as the CLI's PLATFORM_BASE_URL + path).
+#: Intentionally not configurable from the shipped package.
+_INGEST_URL = "https://api.repowise.dev/telemetry/events"
+
+#: Best-effort: telemetry must never stall or break a tool call.
+_TIMEOUT = 2.0
+
+_TRUTHY = {"1", "true", "yes", "on"}
+
+#: Common CI signals (kept in sync with the CLI's environment module).
+_CI_ENV_VARS = (
+ "CI",
+ "GITHUB_ACTIONS",
+ "GITLAB_CI",
+ "BUILDKITE",
+ "JENKINS_URL",
+ "TEAMCITY_VERSION",
+ "TF_BUILD",
+)
+
+#: Groups the events emitted by this process (e.g. one MCP server session).
+_SESSION_ID = uuid.uuid4().hex
+
+#: In-flight send threads, joined briefly at exit so short runs still deliver.
+_pending: list[threading.Thread] = []
+
+#: Cache the platform.json read so a burst of tool calls doesn't re-hit disk on
+#: every event. The file (anon_id + consent) changes rarely; a short TTL keeps a
+#: mid-session ``repowise telemetry disable`` effective within seconds while
+#: keeping steady-state reads off disk. Only ever read on the background thread.
+_STATE_TTL = 30.0
+_state_cache: tuple[float, dict[str, Any]] | None = None
+
+
+def _platform_json_path() -> Path:
+ """The user-global ``~/.repowise/platform.json`` the CLI already maintains."""
+ return Path.home() / ".repowise" / "platform.json"
+
+
+def _load_state() -> dict[str, Any]:
+ global _state_cache
+ import time
+
+ now = time.monotonic()
+ if _state_cache is not None and (now - _state_cache[0]) < _STATE_TTL:
+ return _state_cache[1]
+ try:
+ state = json.loads(_platform_json_path().read_text(encoding="utf-8"))
+ except Exception:
+ state = {}
+ _state_cache = (now, state)
+ return state
+
+
+def _env_truthy(name: str) -> bool:
+ return os.environ.get(name, "").strip().lower() in _TRUTHY
+
+
+def is_enabled() -> bool:
+ """Return whether telemetry may currently be sent.
+
+ Mirrors the CLI's opt-out precedence: ``DO_NOT_TRACK`` and
+ ``REPOWISE_TELEMETRY_DISABLED`` are hard offs; otherwise enabled unless the
+ user explicitly stored a disable via ``repowise telemetry disable``.
+ """
+ if _env_truthy("DO_NOT_TRACK"):
+ return False
+ if _env_truthy("REPOWISE_TELEMETRY_DISABLED"):
+ return False
+ return _load_state().get("telemetry_enabled") is not False
+
+
+def debug_mode() -> bool:
+ """``REPOWISE_TELEMETRY_DEBUG`` prints the payload to stderr instead of sending."""
+ return _env_truthy("REPOWISE_TELEMETRY_DEBUG")
+
+
+def get_anonymous_id() -> str | None:
+ """Return the persistent anonymous install id, or ``None`` if unset.
+
+ Read-only: the CLI owns creation of ``anon_id`` (on first command), so a
+ server that somehow runs before any CLI command simply sends ``None`` rather
+ than minting a competing id that would fragment install counts.
+ """
+ anon = _load_state().get("anon_id")
+ return anon if isinstance(anon, str) and anon else None
+
+
+def _is_ci() -> bool:
+ return any(os.environ.get(var) for var in _CI_ENV_VARS)
+
+
+def _version() -> str:
+ try:
+ from repowise.core import __version__
+
+ return __version__
+ except Exception:
+ return "unknown"
+
+
+def _base_facts() -> dict[str, object]:
+ info = platform.python_version_tuple()
+ return {
+ "os": platform.system().lower() or "unknown",
+ "arch": platform.machine().lower() or "unknown",
+ "python_version": f"{info[0]}.{info[1]}", # major.minor only, no patch leak
+ "is_ci": _is_ci(),
+ }
+
+
+def build_envelope(event: str, properties: dict[str, Any]) -> dict[str, object]:
+ """Assemble the anonymous wire envelope (same shape as the CLI's)."""
+ envelope: dict[str, object] = {
+ "event": event,
+ "anon_id": get_anonymous_id(),
+ "session_id": _SESSION_ID,
+ "cli_version": _version(),
+ "properties": properties,
+ }
+ envelope.update(_base_facts())
+ return envelope
+
+
+def _post(envelope: dict[str, object]) -> None:
+ with contextlib.suppress(Exception):
+ import httpx
+
+ httpx.post(_INGEST_URL, json=envelope, timeout=_TIMEOUT)
+
+
+def _deliver(event: str, properties: dict[str, Any]) -> None:
+ """Consent-check (incl. the disk-backed stored disable), build, and send.
+
+ Runs on the background thread so no disk read or network call touches the
+ caller's hot path.
+ """
+ with contextlib.suppress(Exception):
+ if _load_state().get("telemetry_enabled") is False:
+ return # stored opt-out; env hard-offs were already handled inline
+ _post(build_envelope(event, properties))
+
+
+def record_event(event: str, properties: dict[str, Any] | None = None) -> None:
+ """Record an anonymous event: respect consent, then debug-print or send.
+
+ Performance: the caller's hot path does only two env-var reads and a thread
+ hand-off. All disk I/O (anon_id + stored-consent) and the network POST run on
+ a daemon thread, so a slow or unreachable backend never delays a tool call.
+ Never raises.
+ """
+ try:
+ # Cheap synchronous gate: env hard-offs need neither disk nor a thread.
+ if _env_truthy("DO_NOT_TRACK") or _env_truthy("REPOWISE_TELEMETRY_DISABLED"):
+ return
+ props = dict(properties or {})
+
+ if debug_mode():
+ import sys
+
+ print(
+ "[repowise telemetry] would send:\n"
+ + json.dumps(build_envelope(event, props), indent=2),
+ file=sys.stderr,
+ )
+ return
+
+ thread = threading.Thread(target=_deliver, args=(event, props), daemon=True)
+ thread.start()
+ _pending[:] = [t for t in _pending if t.is_alive()]
+ _pending.append(thread)
+ except Exception:
+ # Telemetry must never break a tool call.
+ return
+
+
+def _flush() -> None:
+ """Give in-flight sends a brief moment to complete on process exit.
+
+ Bounded to ~2s total so exit is never noticeably delayed even if the backend
+ is hung. A hard-killed MCP server (watchdog) may skip this, which is fine:
+ per-call sends are best-effort and each call already dispatched immediately.
+ """
+ import time
+
+ remaining = 2.0
+ for thread in _pending:
+ if remaining <= 0:
+ break
+ start = time.monotonic()
+ with contextlib.suppress(Exception):
+ thread.join(timeout=remaining)
+ remaining -= time.monotonic() - start
diff --git a/packages/server/src/repowise/server/mcp_server/_savings/wrapper.py b/packages/server/src/repowise/server/mcp_server/_savings/wrapper.py
index 6f75ee96c..c2eeb3524 100644
--- a/packages/server/src/repowise/server/mcp_server/_savings/wrapper.py
+++ b/packages/server/src/repowise/server/mcp_server/_savings/wrapper.py
@@ -41,6 +41,69 @@ from .recorder import record_mcp_dead_end, record_mcp_saving
logger = logging.getLogger(__name__)
+#: Coarse, non-identifying result fields worth reporting per tool call. All are
+#: enums or booleans (confidence tier, retrieval quality, staleness) — never
+#: query text, paths, or repo/symbol names. See the telemetry privacy contract.
+_META_FLAGS = ("index_behind", "embedder_degraded")
+_RESULT_ENUMS = ("confidence", "retrieval_quality", "grounding")
+
+
+def _results_count_bucket(result: Any) -> str | None:
+ """Bucket a search-shaped result count (never the results themselves)."""
+ if not isinstance(result, dict):
+ return None
+ items = result.get("results")
+ if not isinstance(items, list):
+ return None
+ n = len(items)
+ if n == 0:
+ return "0"
+ if n <= 3:
+ return "1-3"
+ if n <= 10:
+ return "4-10"
+ return "10+"
+
+
+def _telemetry_properties(tool: str, result: Any, duration_ms: int) -> dict[str, Any]:
+ """Build the anonymous ``mcp_tool_call`` properties for *result*.
+
+ Only coarse enums / booleans / bucketed counts — no user-identifying data.
+ """
+ is_error = isinstance(result, dict) and bool(result.get("error"))
+ props: dict[str, Any] = {
+ "tool": tool,
+ "status": "error" if is_error else "ok",
+ "duration_ms": duration_ms,
+ }
+ if isinstance(result, dict):
+ for key in _RESULT_ENUMS:
+ value = result.get(key)
+ if isinstance(value, str) and value:
+ props[key] = value
+ meta = result.get("_meta")
+ if isinstance(meta, dict):
+ for key in _META_FLAGS:
+ if isinstance(meta.get(key), bool):
+ props[key] = meta[key]
+ bucket = _results_count_bucket(result)
+ if bucket is not None:
+ props["results_bucket"] = bucket
+ return props
+
+
+def _emit_telemetry(tool: str, result: Any, duration_ms: int) -> None:
+ """Emit one anonymous ``mcp_tool_call`` event. Best-effort, never raises.
+
+ This is the field-visibility counterpart to the local savings ledger: it
+ tells us which tools agents actually reach for, at what confidence, and how
+ often results come back stale/degraded — the adoption signal the local
+ ledger can't aggregate across installs.
+ """
+ from repowise.core.platform import telemetry
+
+ telemetry.record_event("mcp_tool_call", _telemetry_properties(tool, result, duration_ms))
+
#: ``_meta`` key a tool sets to declare its own counterfactual (see
#: :func:`declare_replaced`). The wrapper reads and then leaves it in place as a
#: transparency annotation.
@@ -126,11 +189,19 @@ def instrument(fn: Callable[..., Any]) -> Callable[..., Any]:
@functools.wraps(fn)
async def _wrapped(*args: Any, **kwargs: Any) -> Any:
+ import time
+
+ _t0 = time.perf_counter()
result = await fn(*args, **kwargs)
+ duration_ms = int((time.perf_counter() - _t0) * 1000)
try:
_record(tool, result)
except Exception: # pragma: no cover - defensive; savings never break a tool
logger.debug("mcp savings instrumentation failed for %s", tool, exc_info=True)
+ try:
+ _emit_telemetry(tool, result, duration_ms)
+ except Exception: # pragma: no cover - defensive; telemetry never breaks a tool
+ logger.debug("mcp telemetry emit failed for %s", tool, exc_info=True)
return result
# Preserve the original signature so FastMCP builds the correct tool schema.
diff --git a/tests/conftest.py b/tests/conftest.py
index 038f9808f..52a2ea6c6 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -7,6 +7,33 @@ from pathlib import Path
import pytest
[email protected](scope="session", autouse=True)
+def _no_telemetry_network():
+ """Guarantee no test emits core-layer telemetry over the network.
+
+ The MCP instrument seam emits an ``mcp_tool_call`` event via the core
+ emitter's ``_post``; a test that drives the real wrapper with consent
+ enabled would otherwise POST to the production ingest endpoint. Patch that
+ sink to a no-op. Tests that assert emit behaviour re-patch it at function
+ scope and still never touch the network.
+
+ The CLI's ``command_run`` path is intentionally left alone: its tests patch
+ ``PlatformClient.post`` at the class level, so patching the ``default_client``
+ instance here would shadow those patches.
+ """
+ from _pytest.monkeypatch import MonkeyPatch
+
+ mp = MonkeyPatch()
+ try:
+ from repowise.core.platform import telemetry as _core_telemetry
+
+ mp.setattr(_core_telemetry, "_post", lambda envelope: None, raising=False)
+ except Exception:
+ pass
+ yield
+ mp.undo()
+
+
@pytest.fixture(scope="session")
def repo_root() -> Path:
"""Absolute path to the repository root."""
diff --git a/tests/unit/cli/test_telemetry.py b/tests/unit/cli/test_telemetry.py
index 5faeb520f..b669af804 100644
--- a/tests/unit/cli/test_telemetry.py
+++ b/tests/unit/cli/test_telemetry.py
@@ -198,3 +198,95 @@ class TestCommandWrapper:
res = CliRunner().invoke(cli, ["--help"])
assert res.exit_code == 0
assert recorded == []
+
+
+class TestOutcomeClassification:
+ """Ctrl-C / clean-exit must not read as failure (status='error')."""
+
+ def _run(self, monkeypatch: pytest.MonkeyPatch, body) -> list[dict]:
+ import click
+
+ from repowise.cli._instrumented_group import InstrumentedGroup
+
+ recorded: list[dict] = []
+ monkeypatch.setattr(
+ emitter.default_client, "post", lambda path, payload, **k: recorded.append(payload) or True
+ )
+ # Force synchronous delivery so the assert doesn't race the daemon send.
+ monkeypatch.setattr(
+ emitter.threading,
+ "Thread",
+ lambda target, args, daemon: type(
+ "T", (), {"start": lambda s: target(*args), "join": lambda s, timeout=None: None}
+ )(),
+ )
+
+ @click.group(cls=InstrumentedGroup)
+ def root() -> None: ...
+
+ @root.command()
+ def sub() -> None:
+ body()
+
+ CliRunner().invoke(root, ["sub"])
+ return recorded
+
+ def test_abort_is_interrupted(self, monkeypatch: pytest.MonkeyPatch):
+ import click
+
+ def body() -> None:
+ raise click.exceptions.Abort()
+
+ rec = self._run(monkeypatch, body)
+ assert rec and rec[0]["properties"]["status"] == "interrupted"
+
+ def test_sigint_exit_code_is_interrupted(self, monkeypatch: pytest.MonkeyPatch):
+ def body() -> None:
+ raise SystemExit(130)
+
+ rec = self._run(monkeypatch, body)
+ assert rec and rec[0]["properties"]["status"] == "interrupted"
+
+ def test_clean_exit_is_ok(self, monkeypatch: pytest.MonkeyPatch):
+ def body() -> None:
+ raise SystemExit(0)
+
+ rec = self._run(monkeypatch, body)
+ assert rec and rec[0]["properties"]["status"] == "ok"
+
+ def test_real_failure_is_error(self, monkeypatch: pytest.MonkeyPatch):
+ def body() -> None:
+ raise SystemExit(2)
+
+ rec = self._run(monkeypatch, body)
+ assert rec and rec[0]["properties"]["status"] == "error"
+
+ def test_command_outcome_rides_the_event(self, monkeypatch: pytest.MonkeyPatch):
+ from repowise.cli.platform import telemetry
+
+ def body() -> None:
+ telemetry.add_command_outcome(file_count_bucket="500-999", docs_mode=False)
+
+ rec = self._run(monkeypatch, body)
+ assert rec
+ props = rec[0]["properties"]
+ assert props["file_count_bucket"] == "500-999"
+ assert props["docs_mode"] is False
+
+
+class TestBucketCount:
+ def test_buckets_are_coarse_ranges(self):
+ from repowise.cli.platform import telemetry
+
+ assert telemetry.bucket_count(0) == "0"
+ assert telemetry.bucket_count(5) == "1-9"
+ assert telemetry.bucket_count(742) == "500-999"
+ assert telemetry.bucket_count(20000) == "5k+"
+
+ def test_outcome_is_drained_once(self):
+ from repowise.cli.platform import telemetry
+
+ telemetry.add_command_outcome(a=1)
+ assert telemetry.drain_command_outcome() == {"a": 1}
+ # A second drain is empty — the field belongs to one invocation only.
+ assert telemetry.drain_command_outcome() == {}
diff --git a/tests/unit/server/mcp/test_tool_telemetry.py b/tests/unit/server/mcp/test_tool_telemetry.py
new file mode 100644
index 000000000..e42159f77
--- /dev/null
+++ b/tests/unit/server/mcp/test_tool_telemetry.py
@@ -0,0 +1,86 @@
+"""The MCP instrument seam emits one anonymous ``mcp_tool_call`` per call.
+
+Guards the two things that matter: the reported properties are coarse and
+non-identifying (enums/booleans/buckets, never query text or paths), and the
+emit is best-effort so it can never break or slow a tool response.
+"""
+
+from __future__ import annotations
+
+import pytest
+
+from repowise.server.mcp_server._savings import wrapper
+
+
+class TestTelemetryProperties:
+ def test_answer_shape_extracts_enums_and_flags(self):
+ result = {
+ "answer": "…prose the agent reads…", # must NOT be reported
+ "confidence": "high",
+ "retrieval_quality": "strong",
+ "grounding": "exact_symbol",
+ "_meta": {"index_behind": True, "embedder_degraded": False, "timing_ms": 12.3},
+ }
+ props = wrapper._telemetry_properties("get_answer", result, 42)
+ assert props == {
+ "tool": "get_answer",
+ "status": "ok",
+ "duration_ms": 42,
+ "confidence": "high",
+ "retrieval_quality": "strong",
+ "grounding": "exact_symbol",
+ "index_behind": True,
+ "embedder_degraded": False,
+ }
+
+ def test_error_result_is_status_error(self):
+ props = wrapper._telemetry_properties("get_symbol", {"error": "boom"}, 3)
+ assert props["status"] == "error"
+
+ def test_search_result_reports_count_bucket_not_content(self):
+ result = {"results": [{"path": "secret/a.py"}, {"path": "secret/b.py"}]}
+ props = wrapper._telemetry_properties("search_codebase", result, 5)
+ assert props["results_bucket"] == "1-3"
+ # Only coarse keys — no results/paths reach the wire.
+ assert set(props) == {"tool", "status", "duration_ms", "results_bucket"}
+
+ def test_non_dict_result_is_safe(self):
+ props = wrapper._telemetry_properties("get_overview", "unexpected", 1)
+ assert props == {"tool": "get_overview", "status": "ok", "duration_ms": 1}
+
+
+async def test_instrument_emits_one_event(monkeypatch: pytest.MonkeyPatch):
+ from repowise.core.platform import telemetry
+
+ calls: list[tuple[str, dict]] = []
+ monkeypatch.setattr(telemetry, "record_event", lambda event, props: calls.append((event, props)))
+
+ async def get_answer(question: str) -> dict:
+ return {"answer": "x", "confidence": "medium", "_meta": {}}
+
+ out = await wrapper.instrument(get_answer)("how does X work")
+ assert out["confidence"] == "medium" # response unchanged
+ assert len(calls) == 1
+ event, props = calls[0]
+ assert event == "mcp_tool_call"
+ assert props["tool"] == "get_answer"
+ assert props["confidence"] == "medium"
+ assert "duration_ms" in props
+
+
+async def test_telemetry_failure_never_breaks_the_tool(monkeypatch: pytest.MonkeyPatch):
+ from repowise.core.platform import telemetry
+
+ def boom(*a, **k):
+ raise RuntimeError("telemetry backend down")
+
+ monkeypatch.setattr(telemetry, "record_event", boom)
+
+ async def get_overview() -> dict:
+ return {"ok": True, "_meta": {}}
+
+ # The tool result must survive a telemetry emit that raises.
+ out = await wrapper.instrument(get_overview)()
+ assert out == {"ok": True, "_meta": {}}
diff --git a/tests/unit/test_platform_telemetry.py b/tests/unit/test_platform_telemetry.py
new file mode 100644
index 000000000..7c767dabc
--- /dev/null
+++ b/tests/unit/test_platform_telemetry.py
@@ -0,0 +1,110 @@
+"""Tests for the core-layer anonymous telemetry emitter.
+
+This is the self-contained substrate the MCP server uses (it may not import the
+CLI). It must honour the same opt-out consent, produce the same anonymous
+envelope shape, and never touch disk or the network on the caller's hot path.
+"""
+
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+import pytest
+
+from repowise.core.platform import telemetry
+
+
[email protected](autouse=True)
+def isolated(tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
+ """Point platform.json at a temp file and reset the process caches."""
+ state_path = tmp_path / "platform.json"
+ monkeypatch.setattr(telemetry, "_platform_json_path", lambda: state_path)
+ monkeypatch.setattr(telemetry, "_state_cache", None, raising=False)
+ for var in ("DO_NOT_TRACK", "REPOWISE_TELEMETRY_DISABLED", "REPOWISE_TELEMETRY_DEBUG"):
+ monkeypatch.delenv(var, raising=False)
+ telemetry._pending.clear()
+ yield telemetry, state_path
+ telemetry._pending.clear()
+
+
+def _write_state(path: Path, **fields) -> None:
+ path.write_text(json.dumps(fields), encoding="utf-8")
+
+
+def _capture(monkeypatch: pytest.MonkeyPatch) -> list[dict]:
+ """Patch the sender + force synchronous delivery; return captured envelopes."""
+ sent: list[dict] = []
+ monkeypatch.setattr(telemetry, "_post", lambda envelope: sent.append(envelope))
+ monkeypatch.setattr(telemetry, "_state_cache", None, raising=False)
+ monkeypatch.setattr(
+ telemetry.threading,
+ "Thread",
+ lambda target, args, daemon: type(
+ "T", (), {"start": lambda s: target(*args), "is_alive": lambda s: False}
+ )(),
+ )
+ return sent
+
+
+class TestConsent:
+ def test_enabled_by_default(self, isolated):
+ _, path = isolated
+ _write_state(path, anon_id="abc")
+ assert telemetry.is_enabled() is True
+
+ def test_do_not_track_blocks_send(self, isolated, monkeypatch: pytest.MonkeyPatch):
+ monkeypatch.setenv("DO_NOT_TRACK", "1")
+ sent = _capture(monkeypatch)
+ telemetry.record_event("mcp_tool_call", {"tool": "get_answer"})
+ assert sent == []
+
+ def test_stored_disable_blocks_send(self, isolated, monkeypatch: pytest.MonkeyPatch):
+ _, path = isolated
+ _write_state(path, anon_id="abc", telemetry_enabled=False)
+ sent = _capture(monkeypatch)
+ telemetry.record_event("mcp_tool_call", {"tool": "get_answer"})
+ assert sent == []
+
+
+class TestEnvelope:
+ def test_shape_and_anon_id(self, isolated, monkeypatch: pytest.MonkeyPatch):
+ _, path = isolated
+ _write_state(path, anon_id="install-xyz")
+ sent = _capture(monkeypatch)
+
+ telemetry.record_event("mcp_tool_call", {"tool": "get_answer", "status": "ok"})
+
+ assert len(sent) == 1
+ env = sent[0]
+ assert env["event"] == "mcp_tool_call"
+ assert env["anon_id"] == "install-xyz" # shared with the CLI's install id
+ assert {"session_id", "cli_version", "os", "arch", "python_version", "is_ci"} <= set(env)
+ assert env["properties"]["tool"] == "get_answer"
+
+ def test_no_patch_version_leak(self, isolated, monkeypatch: pytest.MonkeyPatch):
+ _, path = isolated
+ _write_state(path, anon_id="abc")
+ sent = _capture(monkeypatch)
+ telemetry.record_event("mcp_tool_call", {"tool": "x"})
+ assert sent[0]["python_version"].count(".") == 1 # major.minor only
+
+ def test_missing_platform_json_sends_null_anon(self, isolated, monkeypatch: pytest.MonkeyPatch):
+ # No CLI has run — no platform.json. We still send, with anon_id=None,
+ # rather than minting a competing id.
+ sent = _capture(monkeypatch)
+ telemetry.record_event("mcp_tool_call", {"tool": "x"})
+ assert sent and sent[0]["anon_id"] is None
+
+
+class TestDebug:
+ def test_debug_prints_does_not_send(
+ self, isolated, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture
+ ):
+ _, path = isolated
+ _write_state(path, anon_id="abc")
+ sent = _capture(monkeypatch)
+ monkeypatch.setenv("REPOWISE_TELEMETRY_DEBUG", "1")
+ telemetry.record_event("mcp_tool_call", {"tool": "get_answer"})
+ assert "would send" in capsys.readouterr().err
+ assert sent == []