Skip to content

Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.

feat(hooks): measure whether the agent acts on what a hook says (#1272)

Raghav Chamadiya481fe2616a9b
diff --git a/docs/agent/HOOKS.md b/docs/agent/HOOKS.md
index 19396d70b..192b54ca7 100644
--- a/docs/agent/HOOKS.md
+++ b/docs/agent/HOOKS.md
@@ -25,6 +25,9 @@ crashes or blocks your agent.
| **Command-rewrite (distill)** | Claude Code | `repowise hook rewrite install` (opt-in) | `Bash` / `PowerShell` | Rewrites noisy commands to `repowise distill <cmd>`; auto-allowed by default, set `permission: ask` to approve each one |
| **Codex context + staleness** | Codex | `repowise init --codex` | SessionStart / UserPromptSubmit / edit / Bash | Reminds Codex to use the MCP tools and flags stale context after edits |
+Every agent hook records what it said and whether the agent acted on it — see
+[`repowise hook stats`](#is-any-of-this-actually-helping--repowise-hook-stats).
+
---
## Git hook: post-commit auto-sync
@@ -192,6 +195,32 @@ Full Codex setup: [CODEX.md](CODEX.md).
---
+## Hook efficacy: `repowise hook stats`
+
+The agent hooks keep a local ledger in `.repowise/sessions/sessions.db`: what
+each hook said, and whether the agent went on to do what it pointed at.
+
+```sh
+repowise hook stats # per-surface firing counts and action rates
+repowise hook backfill --all-projects # seed it from your existing transcripts
+```
+
+The verdict comes from your own Claude Code transcripts — a firing is paired
+with the tool calls that followed it — so the numbers are yours, not a
+benchmark. `repowise update` classifies recent sessions; `hook backfill` covers
+history. Nothing leaves the machine.
+
+Notices that ask for nothing (the stale-read warning, the silent
+read-after-served measurement) report `n/a` rather than a rate. `hook stats`
+also reports hook invocation counts and wall time, including the calls that
+returned silence.
+
+> Upgrading from a release before firings were keyed by their text: run
+> `repowise hook backfill --reset` once, or older rows are counted separately
+> from the replayed ones. It never touches decisions.
+
+---
+
## What gets written where
`repowise init` writes these entries into `~/.claude/settings.json` (Claude Code)
diff --git a/docs/reference/CLI_REFERENCE.md b/docs/reference/CLI_REFERENCE.md
index f4678f5c3..cd6c05c3a 100644
--- a/docs/reference/CLI_REFERENCE.md
+++ b/docs/reference/CLI_REFERENCE.md
@@ -955,6 +955,39 @@ repowise hook uninstall --workspace
See [Auto-Sync](../scale/AUTO_SYNC.md) for all sync methods (hooks, file watcher, webhooks, polling).
+### `repowise hook stats`
+
+Show what the Claude Code agent hooks said and whether the agent acted on it,
+per surface, plus hook invocation counts and wall time. Reads the local ledger
+in `.repowise/sessions/sessions.db`.
+
+```bash
+repowise hook stats
+repowise hook stats --json # raw per-surface rows
+```
+
+Notices that ask for nothing (stale-read, the silent read-after-served
+measurement) report `n/a` rather than a rate.
+
+### `repowise hook backfill`
+
+Replay Claude Code transcripts into the ledger, so `hook stats` starts with
+history instead of only what has fired since you upgraded. Local, single-pass,
+and safe to re-run: a firing is keyed by a hash of its own text, so a replay
+settles the row it already owns.
+
+```bash
+repowise hook backfill # this checkout's transcripts
+repowise hook backfill --all-projects # include this repo's worktrees
+repowise hook backfill --days 30 # only recent transcripts
+repowise hook backfill --reset # rebuild the hook surfaces from scratch
+```
+
+`repowise update` classifies recent sessions automatically, so a backfill is
+normally a one-time catch-up. Use `--reset` once when upgrading from a release
+that keyed rows by hook input rather than by emitted text; it clears only the
+hook surfaces, never decisions.
+
### `repowise hook rewrite install|uninstall|status`
Manage the Distill command-rewrite hooks (Claude Code + Codex PreToolUse).
diff --git a/packages/cli/src/repowise/cli/commands/augment_cmd/_shared.py b/packages/cli/src/repowise/cli/commands/augment_cmd/_shared.py
index 45d61e2ee..a6f6f36e9 100644
--- a/packages/cli/src/repowise/cli/commands/augment_cmd/_shared.py
+++ b/packages/cli/src/repowise/cli/commands/augment_cmd/_shared.py
@@ -8,8 +8,42 @@ depend on ``_shared``, never the other way around.
from __future__ import annotations
import tempfile
+import time
+from functools import lru_cache
from pathlib import Path
+#: Wall clock at the first moment repowise code runs in this hook process.
+#: Every ledger row carries the elapsed time to its own write, which is the
+#: part of hook latency repowise controls. It is a *lower bound* on what the
+#: user waits: the interpreter start and the click import happen before this
+#: module loads, and the response is written after. Claude Code records the
+#: true end-to-end figure as ``attachment.durationMs``, which the transcript
+#: pass in :mod:`repowise.core.sessions.efficacy` writes over the top.
+_T0 = time.perf_counter()
+
+
+def _elapsed_ms() -> int:
+ """Milliseconds of in-process hook work so far (see :data:`_T0`)."""
+ return int((time.perf_counter() - _T0) * 1000)
+
+
+def _ledger_key(surface: str, category: str, text: str) -> str:
+ """Efficacy-ledger id for one emission, keyed on the text itself.
+
+ The text is the one part of a firing that survives verbatim into the
+ transcript, so keying on it lets the update-time classifier
+ (:func:`repowise.core.sessions.efficacy.ledger_key`, which must stay
+ identical to this) find the very row the hook wrote and settle whether the
+ agent acted on it. Keying on the inputs instead would not work: the flood
+ digest's input is the whole grep output, which the transcript does not
+ keep. Doubling as the once-per-session dedup key is free — the same
+ enrichment worded the same way has nothing new to say.
+ """
+ import hashlib
+
+ digest = hashlib.sha1(text.encode("utf-8", "replace")).hexdigest()[:12]
+ return f"{surface}:{category}:{digest}"
+
def _extract_output_text(tool_output: object) -> str:
"""Pull the textual portion of a Claude Code tool_output, defensively.
@@ -58,6 +92,7 @@ def _relativize(file_path: str, repo_path: Path) -> str | None:
return rel.as_posix()
+@lru_cache(maxsize=8)
def _find_repo_root(cwd: Path) -> Path | None:
"""Walk up from cwd to find a directory with .repowise/.
@@ -65,6 +100,10 @@ def _find_repo_root(cwd: Path) -> Path | None:
system temp ROOT is always a stray artifact (a tool that indexed with
cwd=$TMP), so neither counts as a repo opt-in; repos legitimately created
UNDER either directory still match.
+
+ Memoized: one hook invocation asks several times with the same cwd, and
+ the walk costs up to 20 ``resolve()``/``is_dir()`` syscall pairs. The
+ cache lives only as long as the hook process, so it cannot go stale.
"""
current = Path(cwd).resolve()
try:
diff --git a/packages/cli/src/repowise/cli/commands/augment_cmd/command.py b/packages/cli/src/repowise/cli/commands/augment_cmd/command.py
index b356a5ada..cce956c99 100644
--- a/packages/cli/src/repowise/cli/commands/augment_cmd/command.py
+++ b/packages/cli/src/repowise/cli/commands/augment_cmd/command.py
@@ -125,22 +125,22 @@ def _run_augment(*, client: str | None = None) -> None:
if client == "codex" and event in ("SessionStart", "UserPromptSubmit"):
session_id = payload.get("session_id", "")
- result = _handle_codex_context_event(
- event, cwd, session_id if isinstance(session_id, str) else ""
- )
+ session_id = session_id if isinstance(session_id, str) else ""
+ result = _handle_codex_context_event(event, cwd, session_id)
if result:
_emit_response(event, result)
+ _count_run(cwd, session_id, event, "", emitted=bool(result))
return
if event == "SessionStart":
# Claude Code lifecycle hook: live index-freshness + trust context,
# plus the relevance-ranked standing-decisions block.
session_id = payload.get("session_id", "")
- result = _handle_claude_session_start(
- cwd, session_id if isinstance(session_id, str) else ""
- )
+ session_id = session_id if isinstance(session_id, str) else ""
+ result = _handle_claude_session_start(cwd, session_id)
if result:
_emit_response(event, result)
+ _count_run(cwd, session_id, event, "", emitted=bool(result))
return
if event != "PostToolUse":
@@ -148,16 +148,38 @@ def _run_augment(*, client: str | None = None) -> None:
tool_output = payload.get("tool_response", payload.get("tool_output", {}))
session_id = payload.get("session_id", "")
+ session_id = session_id if isinstance(session_id, str) else ""
result = _handle_post_tool_use(
tool_name,
tool_input,
tool_output,
cwd,
client=client,
- session_id=session_id if isinstance(session_id, str) else "",
+ session_id=session_id,
)
if result:
_emit_response(event, result)
+ _count_run(cwd, session_id, event, tool_name, emitted=bool(result))
+
+
+def _count_run(cwd: str, session_id: str, event: str, tool: str, *, emitted: bool) -> None:
+ """Bill this invocation to the hook-latency counter; never fails the hook.
+
+ Deliberately after the response is written: the agent must not wait on
+ bookkeeping, and a broken sidecar must not cost an enrichment that was
+ already computed. The reported time therefore excludes this write itself.
+ """
+ if not session_id:
+ return
+ try:
+ from ._shared import _find_repo_root
+ from .decision_inject import _record_hook_run
+
+ repo_path = _find_repo_root(Path(cwd))
+ if repo_path is not None:
+ _record_hook_run(repo_path, session_id, event, tool, emitted=emitted)
+ except Exception:
+ return
def _emit_response(event: str, context: str) -> None:
diff --git a/packages/cli/src/repowise/cli/commands/augment_cmd/decision_inject.py b/packages/cli/src/repowise/cli/commands/augment_cmd/decision_inject.py
index cc7965d73..96b47bff9 100644
--- a/packages/cli/src/repowise/cli/commands/augment_cmd/decision_inject.py
+++ b/packages/cli/src/repowise/cli/commands/augment_cmd/decision_inject.py
@@ -623,10 +623,12 @@ def _edit_fix_history_notice(repo_path: Path, rel: str, session_id: str) -> str
line += "."
if session_id:
+ from ._shared import _ledger_key
+
claimed, shown = _claim_ledger(
repo_path,
session_id,
- f"fix_history:{rel}",
+ _ledger_key("fix_history", "edit_notice", line),
node_id=rel,
surface="fix_history",
category="edit_notice",
@@ -689,6 +691,8 @@ _INJECTIONS_TABLE_SQL = (
"surface TEXT NOT NULL DEFAULT '', "
"category TEXT NOT NULL DEFAULT '', "
"chars INTEGER NOT NULL DEFAULT 0, "
+ "duration_ms INTEGER NOT NULL DEFAULT 0, "
+ "acted INTEGER NOT NULL DEFAULT 0, "
"PRIMARY KEY (session_id, decision_id))"
)
@@ -698,10 +702,29 @@ _LEDGER_COLUMNS = (
("surface", "TEXT NOT NULL DEFAULT ''"),
("category", "TEXT NOT NULL DEFAULT ''"),
("chars", "INTEGER NOT NULL DEFAULT 0"),
+ ("duration_ms", "INTEGER NOT NULL DEFAULT 0"),
+ ("acted", "INTEGER NOT NULL DEFAULT 0"),
)
+#: One ledger connection per hook process, opened lazily. A single invocation
+#: can write several rows (two notices plus the run counter), and connect +
+#: WAL pragma + the CREATE/PRAGMA/ALTER migration is the expensive part, not
+#: the INSERT. The process is short-lived and single-threaded, so caching is
+#: safe; every writer commits, and process exit closes the handle.
+#: Keyed by repo path, not global: a hook process only ever sees one repo, but
+#: the test suite drives many through one interpreter, and a cache that
+#: ignored the path would hand the second repo the first one's database.
+_CONN: dict[Path, sqlite3.Connection | None] = {}
+
+
def _open_injections(repo_path: Path) -> sqlite3.Connection | None:
+ """The shared ledger connection, migrated and ready. None if unusable.
+
+ Callers must NOT close the returned connection — see :data:`_CONN`.
+ """
+ if repo_path in _CONN:
+ return _CONN[repo_path]
db_path = repo_path / ".repowise" / "sessions" / "sessions.db"
try:
db_path.parent.mkdir(parents=True, exist_ok=True)
@@ -709,15 +732,60 @@ def _open_injections(repo_path: Path) -> sqlite3.Connection | None:
conn.execute("PRAGMA journal_mode=WAL")
conn.execute("PRAGMA busy_timeout=1000")
conn.execute(_INJECTIONS_TABLE_SQL)
+ conn.execute(_HOOK_RUNS_TABLE_SQL)
existing = {row[1] for row in conn.execute("PRAGMA table_info(injections)")}
for name, decl in _LEDGER_COLUMNS:
if name not in existing:
conn.execute(f"ALTER TABLE injections ADD COLUMN {name} {decl}")
+ _CONN[repo_path] = conn
return conn
except (sqlite3.Error, OSError):
+ _CONN[repo_path] = None
return None
+_HOOK_RUNS_TABLE_SQL = (
+ "CREATE TABLE IF NOT EXISTS hook_runs ("
+ "session_id TEXT NOT NULL, event TEXT NOT NULL, tool TEXT NOT NULL, "
+ "calls INTEGER NOT NULL DEFAULT 0, emitted INTEGER NOT NULL DEFAULT 0, "
+ "total_ms INTEGER NOT NULL DEFAULT 0, PRIMARY KEY (session_id, event, tool))"
+)
+
+
+def _record_hook_run(
+ repo_path: Path, session_id: str, event: str, tool: str, *, emitted: bool
+) -> None:
+ """Count one hook invocation and what it cost. Best-effort, never raises.
+
+ Every invocation lands here, including the large majority that return
+ nothing: the matcher covers Read/Edit/Write/Grep/Glob/Bash/PowerShell and
+ the repowise MCP tools, and a silent run still pays the import cost. The
+ harness only writes a transcript record for hooks that *emitted*, so
+ without this counter the silent invocations — the bulk of the bill — are
+ invisible. Aggregated per (session, event, tool) rather than one row per
+ call, so the table stays small and the write stays a single upsert.
+ """
+ if not session_id:
+ return
+ from ._shared import _elapsed_ms
+
+ conn = _open_injections(repo_path)
+ if conn is None:
+ return
+ try:
+ conn.execute(
+ "INSERT INTO hook_runs (session_id, event, tool, calls, emitted, total_ms) "
+ "VALUES (?, ?, ?, 1, ?, ?) "
+ "ON CONFLICT(session_id, event, tool) DO UPDATE SET "
+ "calls = calls + 1, emitted = emitted + excluded.emitted, "
+ "total_ms = total_ms + excluded.total_ms",
+ (session_id, event, tool, 1 if emitted else 0, _elapsed_ms()),
+ )
+ conn.commit()
+ except sqlite3.Error:
+ pass
+
+
def _record_injections(
repo_path: Path, session_id: str, decision_ids: list[str], *, node_id: str
) -> None:
@@ -733,18 +801,19 @@ def _record_injections(
if conn is None:
return
try:
+ from ._shared import _elapsed_ms
+
now = time.time()
+ elapsed = _elapsed_ms()
conn.executemany(
"INSERT OR IGNORE INTO injections "
- "(session_id, decision_id, node_id, shown_at, surface, category) "
- "VALUES (?, ?, ?, ?, 'decision', 'session_start')",
- [(session_id, did, node_id, now) for did in decision_ids],
+ "(session_id, decision_id, node_id, shown_at, surface, category, duration_ms) "
+ "VALUES (?, ?, ?, ?, 'decision', 'session_start', ?)",
+ [(session_id, did, node_id, now, elapsed) for did in decision_ids],
)
conn.commit()
except sqlite3.Error:
pass
- finally:
- conn.close()
def _claim_ledger(
@@ -765,16 +834,31 @@ def _claim_ledger(
surface_injection_count)`` where the count covers only rows that actually
carried text (``chars > 0``) on *surface* — pure measurement rows must not
eat into an injection cap. Fail-closed: any error reports unclaimed.
+
+ ``duration_ms`` records what this firing has cost so far (see
+ :func:`_shared._elapsed_ms`); it is what ``repowise hook stats`` reports
+ until a transcript pass replaces it with the harness-measured total.
"""
+ from ._shared import _elapsed_ms
+
conn = _open_injections(repo_path)
if conn is None:
return False, 0
try:
cur = conn.execute(
"INSERT OR IGNORE INTO injections "
- "(session_id, decision_id, node_id, shown_at, surface, category, chars) "
- "VALUES (?, ?, ?, ?, ?, ?, ?)",
- (session_id, key, node_id, time.time(), surface, category, chars),
+ "(session_id, decision_id, node_id, shown_at, surface, category, chars, "
+ "duration_ms) VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
+ (
+ session_id,
+ key,
+ node_id,
+ time.time(),
+ surface,
+ category,
+ chars,
+ _elapsed_ms(),
+ ),
)
claimed = cur.rowcount > 0
count = conn.execute(
@@ -785,8 +869,6 @@ def _claim_ledger(
return claimed, int(count)
except sqlite3.Error:
return False, 0
- finally:
- conn.close()
def _claim_injection(
@@ -800,15 +882,17 @@ def _claim_injection(
the strict per-session notice cap. Fail-closed: any error reports
unclaimed, so a sidecar glitch degrades to silence, never to spam.
"""
+ from ._shared import _elapsed_ms
+
conn = _open_injections(repo_path)
if conn is None:
return False, 0
try:
cur = conn.execute(
"INSERT OR IGNORE INTO injections "
- "(session_id, decision_id, node_id, shown_at, surface, category) "
- "VALUES (?, ?, ?, ?, 'decision', 'edit_notice')",
- (session_id, decision_id, node_id, time.time()),
+ "(session_id, decision_id, node_id, shown_at, surface, category, duration_ms) "
+ "VALUES (?, ?, ?, ?, 'decision', 'edit_notice', ?)",
+ (session_id, decision_id, node_id, time.time(), _elapsed_ms()),
)
claimed = cur.rowcount > 0
# Surface-scoped: read/search enrichment rows also carry a node_id and
@@ -822,5 +906,3 @@ def _claim_injection(
return claimed, int(count)
except sqlite3.Error:
return False, 0
- finally:
- conn.close()
diff --git a/packages/cli/src/repowise/cli/commands/augment_cmd/read_state.py b/packages/cli/src/repowise/cli/commands/augment_cmd/read_state.py
index 2e2ba4359..aeba9d338 100644
--- a/packages/cli/src/repowise/cli/commands/augment_cmd/read_state.py
+++ b/packages/cli/src/repowise/cli/commands/augment_cmd/read_state.py
@@ -149,6 +149,7 @@ def _handle_read_post(
state = _load_session_state(repo_path, session_id)
notices: list[str] = []
+ fired: list[tuple[str, str]] = [] # (category, text) for the efficacy ledger
# Stale-read: this session Read the file, then Edited/Wrote it, and is
# now Reading it again. The fresh Read is fine — the flag is about any
@@ -162,6 +163,7 @@ def _handle_read_post(
f"[repowise] {rel} changed (Edit/Write) after your previous read of it — "
"excerpts from before that edit are stale."
)
+ fired.append(("stale_read", notices[-1]))
# Re-read: already read this session, unchanged since (the read→edit→read
# case is the stale notice above), and this is a full re-read of a
@@ -185,6 +187,7 @@ def _handle_read_post(
"its content is still in context. For a specific symbol use "
f'get_symbol("{rel}::Name") or a line-range read instead of re-reading the file.'
)
+ fired.append(("reread", notices[-1]))
state["seq"] += 1
state["reads"][rel] = state["seq"]
@@ -192,11 +195,41 @@ def _handle_read_post(
nudge = _skeleton_nudge(repo_path, rel, tool_output, state)
if nudge:
notices.append(nudge)
+ fired.append(("skeleton_nudge", nudge))
_save_session_state(repo_path, state)
+ for category, text in fired:
+ _log_read_firing(repo_path, session_id, category, rel, text)
return "\n".join(notices) if notices else None
+def _log_read_firing(
+ repo_path: Path, session_id: str, category: str, rel: str, text: str
+) -> None:
+ """Record one Read-surface firing in the shared efficacy ledger.
+
+ Measurement only — never changes what the agent sees, and the once-per-
+ file-per-session gate stays the state file above, not this row. The key is
+ the shared text hash (:func:`_shared._ledger_key`) so the transcript
+ classifier in :mod:`repowise.core.sessions.efficacy` settles *this* row
+ rather than inserting a second one for the same firing.
+ """
+ if not session_id:
+ return
+ from ._shared import _ledger_key
+ from .decision_inject import _claim_ledger
+
+ _claim_ledger(
+ repo_path,
+ session_id,
+ _ledger_key("read", category, text),
+ node_id=rel,
+ surface="read",
+ category=category,
+ chars=len(text),
+ )
+
+
def _read_output_line_count(tool_output: object) -> int:
"""Line count of a Read result across the hook payload shapes we see."""
if isinstance(tool_output, dict):
diff --git a/packages/cli/src/repowise/cli/commands/augment_cmd/search.py b/packages/cli/src/repowise/cli/commands/augment_cmd/search.py
index b5b8ad310..38397c806 100644
--- a/packages/cli/src/repowise/cli/commands/augment_cmd/search.py
+++ b/packages/cli/src/repowise/cli/commands/augment_cmd/search.py
@@ -4,10 +4,16 @@ from __future__ import annotations
from pathlib import Path
-from repowise.core.persistence.sql import LIKE_ESCAPE, escape_like
-
from ._shared import _extract_output_text, _find_repo_root
+# NOTE: repowise.core.persistence.sql is imported inside _rescue/_triage, not
+# here. At module scope it costs ~790ms on every single hook invocation — it
+# pulls persistence -> crud -> analysis -> dead_code.analyzer — and command.py
+# imports this module unconditionally, so a Read or Bash hook that never runs
+# a search was paying the whole bill. Both users are already async functions
+# that defer their sqlalchemy imports; this is the same rule applied one line
+# further. Keep it deferred.
+
# Tunables — fixed thresholds keep the fire pattern predictable across
# repos. If these ever need to vary, derive them from indexed-row counts
# rather than exposing knobs (every knob is a way for the hook to drift).
@@ -43,7 +49,7 @@ def _handle_search_post(
if result_count >= _DIGEST_THRESHOLD:
digest = _grep_flood_digest(repo_path, output_text)
if digest:
- _log_search_firing(repo_path, session_id, "digest", output_text, digest)
+ _log_search_firing(repo_path, session_id, "digest", digest)
return digest
# Unparseable output (e.g. Glob path lists): fall through to triage.
@@ -75,31 +81,30 @@ def _handle_search_post(
enrichment = asyncio.run(_search_enrich(repo_path, pattern, mode, result_count))
if enrichment:
- _log_search_firing(repo_path, session_id, mode, pattern, enrichment)
+ _log_search_firing(repo_path, session_id, mode, enrichment)
return enrichment
def _log_search_firing(
- repo_path: Path, session_id: str, category: str, keyed_on: str, text: str
-) -> None:
+ repo_path: Path, session_id: str, category: str, text: str) -> None:
"""Record one search enrichment in the shared ledger; measurement only.
All hook surfaces share the sessions.db efficacy ledger so the miner can
- classify used vs ignored firings in one pass. Keyed on the category plus a
- content hash — the same rescue repeated in one session logs once. Never
- changes what the agent sees; any failure is silent.
+ classify used vs ignored firings in one pass. Keyed on a hash of the
+ emitted text (:func:`_shared._ledger_key`) — the same enrichment repeated
+ in one session logs once, and the transcript classifier can recompute the
+ id from what the agent saw. Never changes what the agent sees; any failure
+ is silent.
"""
if not session_id:
return
- import hashlib
-
+ from ._shared import _ledger_key
from .decision_inject import _claim_ledger
- digest = hashlib.sha1(keyed_on.encode("utf-8", "replace")).hexdigest()[:12]
_claim_ledger(
repo_path,
session_id,
- f"search:{category}:{digest}",
+ _ledger_key("search", category, text),
node_id="",
surface="search",
category=category,
@@ -439,6 +444,8 @@ async def _rescue(
if not clean:
return None
+ from repowise.core.persistence.sql import LIKE_ESCAPE, escape_like
+
# Build a small set of token variants. Cheap; helps catch case-style
# drift without a heavy similarity index.
variants = _name_variants(clean)
@@ -515,6 +522,7 @@ async def _triage(
from sqlalchemy import select
from repowise.core.persistence import GraphNode, WikiSymbol
+ from repowise.core.persistence.sql import LIKE_ESCAPE, escape_like
if not clean:
return None
diff --git a/packages/cli/src/repowise/cli/commands/hook_cmd.py b/packages/cli/src/repowise/cli/commands/hook_cmd.py
index e46747901..970255971 100644
--- a/packages/cli/src/repowise/cli/commands/hook_cmd.py
+++ b/packages/cli/src/repowise/cli/commands/hook_cmd.py
@@ -375,6 +375,197 @@ def rewrite_status(path: str | None, workspace: bool, no_workspace: bool) -> Non
console.print(f" {icon} AGENTS.md distill section: {state} ({repo_path})")
+@hook_group.command("stats")
[email protected]("path", required=False, default=None)
+ "--json",
+ "as_json",
+ is_flag=True,
+ default=False,
+ help="Emit the raw per-surface rows as JSON instead of a table.",
+)
+def hook_stats(path: str | None, as_json: bool) -> None:
+ """Show what the agent hooks fired and whether the agent acted on it.
+
+ Reads the efficacy ledger in .repowise/sessions/sessions.db. The live
+ hooks write a row the moment they fire; `repowise hook backfill` (and the
+ update-time pass) replay transcripts to settle whether each firing was
+ acted on. A surface showing firings but no verdicts has not been
+ classified yet — run the backfill.
+ """
+ import json as json_mod
+
+ from repowise.core.sessions.efficacy import CLASSIFIED_SURFACES, NO_ACTION_EXPECTED
+ from repowise.core.sessions.staging import SessionStagingStore, default_store_path
+
+ target = resolve_command_target(path=path, workspace_flag=False, no_workspace_flag=True)
+ assert target.repo_path is not None
+ if not default_store_path(target.repo_path).exists():
+ console.print("[yellow]No hook ledger yet.[/yellow] Run `repowise hook backfill`.")
+ return
+
+ store = SessionStagingStore.open_default(target.repo_path)
+ try:
+ rows = store.efficacy_rows()
+ session_totals = store.session_duration_totals()
+ runs = store.hook_run_totals()
+ by_tool = store.hook_run_by_tool()
+ finally:
+ store.close()
+ if not rows:
+ console.print("[yellow]Hook ledger is empty.[/yellow] Run `repowise hook backfill`.")
+ return
+
+ if as_json:
+ console.print_json(json_mod.dumps({"surfaces": rows, "runs": by_tool}))
+ return
+
+ from rich.table import Table
+
+ table = Table(title="Agent hook efficacy", header_style="bold")
+ table.add_column("surface/category")
+ table.add_column("fired", justify="right")
+ table.add_column("sessions", justify="right")
+ table.add_column("acted", justify="right")
+ table.add_column("rate", justify="right")
+ table.add_column("cost", justify="right")
+ table.add_column("median ms", justify="right")
+
+ for row in rows:
+ pair = (row["surface"], row["category"])
+ classified = row["evaluated"]
+ if row["surface"] not in CLASSIFIED_SURFACES:
+ # Decision rows are judged as followed-vs-contradicted against the
+ # decision records, not as acted-on; read_enrich never emits.
+ acted, rate = "-", "[dim]n/a[/dim]"
+ elif pair in NO_ACTION_EXPECTED:
+ acted, rate = "-", "[dim]n/a[/dim]"
+ elif not classified:
+ acted, rate = "-", "[dim]unclassified[/dim]"
+ else:
+ acted = str(row["acted"])
+ pct = 100.0 * row["acted"] / classified
+ colour = "green" if pct >= 20 else ("yellow" if pct >= 5 else "red")
+ rate = f"[{colour}]{pct:.1f}%[/{colour}]"
+ n = row["duration_ms_count"]
+ table.add_row(
+ f"{row['surface']}/{row['category']}" if row["category"] else row["surface"],
+ str(row["firings"]),
+ str(row["sessions"]),
+ acted,
+ rate,
+ f"~{row['chars'] // 4}t",
+ f"{row['duration_ms_total'] // n}" if n else "[dim]-[/dim]",
+ )
+ console.print(table)
+ console.print(
+ " [dim]rate is over classified firings only; 'n/a' marks a notice with no "
+ "action to take, and read/reread counts as respected-not-re-offended.[/dim]"
+ )
+
+ if session_totals:
+ session_totals.sort()
+ median = session_totals[len(session_totals) // 2]
+ worst = session_totals[-1]
+ console.print(
+ f" [dim]emitting firings cost, per session: median {median / 1000:.1f}s, "
+ f"worst {worst / 1000:.1f}s across {len(session_totals)} sessions.[/dim]"
+ )
+
+ if not runs:
+ console.print(
+ "\n [dim]No invocation counts yet — silent hook runs are only recorded "
+ "going forward, and they are most of the cost. Re-check after a session "
+ "or two.[/dim]"
+ )
+ return
+
+ calls = sorted(r["calls"] for r in runs)
+ spent = sorted(r["total_ms"] for r in runs)
+ total_calls = sum(calls)
+ total_emitted = sum(r["emitted"] for r in runs)
+ console.print(
+ f"\n[bold]Hook invocations[/bold] across {len(runs)} session(s): "
+ f"{total_calls} calls, {total_emitted} of them said something "
+ f"({100.0 * total_emitted / total_calls:.0f}%)."
+ )
+ console.print(
+ f" per session: median {calls[len(calls) // 2]} calls / "
+ f"{spent[len(spent) // 2] / 1000:.1f}s, worst {calls[-1]} calls / "
+ f"{spent[-1] / 1000:.1f}s"
+ )
+ console.print(
+ " [dim]in-process time only: the interpreter start before repowise loads "
+ "is not counted, so this is a floor.[/dim]"
+ )
+ for row in by_tool[:6]:
+ label = f"{row['event']}:{row['tool']}" if row["tool"] else row["event"]
+ console.print(
+ f" {label:28} {row['calls']:5} calls "
+ f"{row['total_ms'] / 1000:7.1f}s "
+ f"[dim]{row['calls'] - row['emitted']} silent[/dim]"
+ )
+
+
+@hook_group.command("backfill")
[email protected]("path", required=False, default=None)
+ "--all-projects",
+ is_flag=True,
+ default=False,
+ help="Also replay transcripts from this repo's worktrees, not just this checkout.",
+)
+ "--days",
+ type=int,
+ default=None,
+ help="Only replay transcripts modified in the last N days (default: all history).",
+)
+ "--reset",
+ is_flag=True,
+ default=False,
+ help=(
+ "Clear the read/search/fix_history rows before replaying. Run this once "
+ "after upgrading: rows written under the older ledger keys cannot be "
+ "matched to a transcript firing and would be counted twice."
+ ),
+)
+def hook_backfill(path: str | None, all_projects: bool, days: int | None, reset: bool) -> None:
+ """Replay agent transcripts into the hook efficacy ledger.
+
+ Every firing is keyed by a hash of its own text, so this is safe to re-run:
+ a replayed firing settles the row it already owns instead of adding a new
+ one. Scanning is single-pass and local; nothing leaves the machine.
+ """
+ import time
+
+ from repowise.core.sessions.efficacy import discover_transcripts, ingest_transcript_efficacy
+
+ target = resolve_command_target(path=path, workspace_flag=False, no_workspace_flag=True)
+ assert target.repo_path is not None
+
+ found = discover_transcripts(target.repo_path, all_projects=all_projects)
+ if not found:
+ console.print(
+ "[yellow]No Claude Code transcripts found for this repo.[/yellow] "
+ "Hook efficacy is measured from them, so there is nothing to backfill."
+ )
+ return
+
+ since = time.time() - days * 86400.0 if days else None
+ console.print(f"Replaying {len(found)} transcript(s)...")
+ counts = ingest_transcript_efficacy(
+ target.repo_path, all_projects=all_projects, since=since, reset=reset
+ )
+ if not counts:
+ console.print(" [dim]No repowise hook firings in range.[/dim]")
+ return
+ for key in sorted(counts, key=lambda k: -counts[k]):
+ console.print(f" {key:28} {counts[key]}")
+ console.print("\nRun `repowise hook stats` to see action rates.")
+
+
@hook_group.command("status")
@click.argument("path", required=False, default=None)
@click.option(
diff --git a/packages/cli/src/repowise/cli/commands/update_cmd/command.py b/packages/cli/src/repowise/cli/commands/update_cmd/command.py
index 135bd54ba..0bf610ed6 100644
--- a/packages/cli/src/repowise/cli/commands/update_cmd/command.py
+++ b/packages/cli/src/repowise/cli/commands/update_cmd/command.py
@@ -66,6 +66,13 @@ from .workspace import _workspace_update
log = structlog.get_logger(__name__)
+#: How far back the hook-efficacy classifier re-reads transcripts on an update.
+#: Wider than the update cadence on purpose: a firing near the end of a session
+#: has no following tool calls yet, so re-running over the same transcript later
+#: is what upgrades an early "ignored" to the truth. Re-classification is
+#: idempotent (the ledger id is a hash of the firing's own text).
+_EFFICACY_LOOKBACK_SECONDS = 7 * 86400.0
+
def _docs_provider_prompt_allowed(emitter: Any) -> bool:
"""Whether a docs run may block on an interactive provider prompt.
@@ -1360,6 +1367,25 @@ def run_update(
if verbose:
console.print(f"[yellow]Injection feedback skipped: {exc}[/yellow]")
+ # The same feedback question for the non-decision hook surfaces. Those
+ # firings point at a file or a search result rather than a decision record,
+ # so they are judged by what the agent did next — which only the transcript
+ # knows. Scoped to transcripts touched since the last update; the whole
+ # history is a one-off `repowise hook backfill`.
+ try:
+ from repowise.core.sessions.efficacy import ingest_transcript_efficacy
+
+ if session_mining_enabled(cfg):
+ classified = ingest_transcript_efficacy(
+ repo_path, since=time.time() - _EFFICACY_LOOKBACK_SECONDS
+ )
+ if verbose and classified:
+ console.print(f"Hook firings classified: [green]{sum(classified.values())}[/green]")
+ except Exception as exc:
+ degraded.append(f"Hook efficacy: {exc}")
+ if verbose:
+ console.print(f"[yellow]Hook efficacy classification skipped: {exc}[/yellow]")
+
# Count of decision records touched by evolution, surfaced in the panel.
decisions_evolved = 0
diff --git a/packages/core/src/repowise/core/sessions/efficacy.py b/packages/core/src/repowise/core/sessions/efficacy.py
new file mode 100644
index 000000000..c0d538601
--- /dev/null
+++ b/packages/core/src/repowise/core/sessions/efficacy.py
@@ -0,0 +1,565 @@
+"""Did the agent act on what the hook said? — the efficacy classifier.
+
+The augment hooks write one ledger row per firing (see
+:class:`~repowise.core.sessions.staging.SessionStagingStore`), but a firing's
+*outcome* is not visible from the hook: it lives in whatever the agent did on
+the next few tool calls. Only the transcript has both halves, so this module
+replays transcripts, pairs each emission with the tool calls that followed it,
+and settles the ``acted`` column.
+
+Modeled on ``augment_cmd/served_reads.py``, which asks the same shape of
+question (did the agent Read content the MCP tools already served?) one event
+at a time. The difference is the window: adoption of a *pointer* cannot be
+judged from the next single event, so each firing is scored against the next
+:data:`ACTION_WINDOW` tool calls.
+
+Where the emissions live
+------------------------
+Claude Code records a hook firing as a ``type: "attachment"`` line whose
+``attachment.type`` is ``hook_success``; ``stdout`` holds the JSON the hook
+printed and ``durationMs`` the harness-measured wall time. It writes a second
+``hook_additional_context`` line for the same firing, carrying the rendered
+text — which is why a naive grep for ``[repowise]`` over raw transcript lines
+counts every firing twice. Only ``hook_success`` is parsed here, so counts
+here are per-emission.
+
+What counts as acting, per surface
+----------------------------------
+============= =============== ==================================================
+surface category acted when, within the window
+============= =============== ==================================================
+read skeleton_nudge a skeleton/structure call on the named file
+read reread *respected*: the file is not re-read in full again
+read stale_read n/a — a warning, not a pointer
+search triage a named file is touched
+search rescue the named file or symbol is touched
+search digest a file the digest ranked is touched
+fix_history edit_notice a test is run, or the file's history is inspected
+============= =============== ==================================================
+
+A ranged Read after a skeleton nudge is deliberately **not** acting: reading a
+range of a file you just read in full is ordinary edit-prep and cannot be
+attributed to the nudge. It is tracked separately as
+:data:`AMBIGUOUS` evidence so the generous bound stays reportable.
+
+Surfaces with no recommended action (``stale_read``) and surfaces that emit
+nothing at all (``read_enrich``'s read-after-served KPI) are never marked
+acted and are excluded from action rates rather than counted as failures.
+Decision-surface rows are judged elsewhere, against the decision records
+themselves — see
+:func:`~repowise.core.sessions.miners.decisions.apply_injection_feedback`.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import re
+from collections.abc import Iterator
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Any
+
+#: Tool calls after a firing that are still credited to it. Generous on
+#: purpose — the measured action-distance histogram puts almost all of the
+#: signal in the first three calls, so a wide window costs little and proves
+#: the tail is genuinely empty rather than merely unmeasured.
+ACTION_WINDOW = 40
+
+#: Evidence string for a follow-up that is consistent with the hook but not
+#: attributable to it. Recorded, never counted as acted.
+AMBIGUOUS = "ranged_read"
+
+#: Surfaces this module judges. ``decision`` rows key on decision-record ids
+#: and are judged by the decision miner; ``read_enrich`` is a silent KPI with
+#: no emission to act on.
+CLASSIFIED_SURFACES = ("read", "search", "fix_history")
+
+#: (surface, category) pairs that carry no recommended action, so an unacted
+#: firing is not a failure. Reported as a count, excluded from rates. The
+#: stale-read notice belongs here because the behavior it asks for — stop
+#: reasoning from a pre-edit excerpt — leaves no trace in the transcript.
+NO_ACTION_EXPECTED = frozenset({("read", "stale_read"), ("read_enrich", "read_after_served")})
+
+#: Firings whose recommended outcome is a *non*-action, scored as compliance
+#: (did the agent avoid re-offending?) rather than adoption. Same ``acted``
+#: column, and ``repowise hook stats`` labels these "respected" so the two are
+#: never read as the same measurement.
+COMPLIANCE_CATEGORIES = frozenset({("read", "reread")})
+
+
+def ledger_key(surface: str, category: str, text: str) -> str:
+ """Efficacy-ledger id for one emission.
+
+ Must stay identical to ``augment_cmd/_shared.py``'s ``_ledger_key`` — the
+ hook path cannot import this package, so the two are mirrors. Keying on
+ the emitted text is what lets a transcript firing find the row the live
+ hook wrote for it.
+ """
+ digest = hashlib.sha1(text.encode("utf-8", "replace")).hexdigest()[:12]
+ return f"{surface}:{category}:{digest}"
+
+
+# ---------------------------------------------------------------------------
+# Emission parsing
+# ---------------------------------------------------------------------------
+
+# One pattern per emission the hooks produce, matched against a single line of
+# the emitted text. Each capture group named below feeds the classifier:
+# ``target`` is the file or symbol the hook pointed at.
+_PATTERNS: tuple[tuple[str, str, re.Pattern[str]], ...] = (
+ (
+ "read",
+ "skeleton_nudge",
+ re.compile(r"^\[repowise\] A skeleton of (?P<target>\S+) is ~\d+ tokens vs ~\d+"),
+ ),
+ (
+ "read",
+ "reread",
+ re.compile(r"^\[repowise\] You already read (?P<target>\S+) this session"),
+ ),
+ (
+ "read",
+ "stale_read",
+ re.compile(r"^\[repowise\] (?P<target>\S+) changed \(Edit/Write\) after your previous"),
+ ),
+ (
+ "fix_history",
+ "edit_notice",
+ re.compile(r"^\[repowise\] (?P<target>\S+) has been bug-fixed \d+x"),
+ ),
+ (
+ "search",
+ "rescue",
+ re.compile(
+ r"^\[repowise\] No literal match for `(?P<pattern>[^`]+)`\. "
+ r"Closest indexed symbol: \S+ `(?P<symbol>[^`]+)` in (?P<target>[^\s:]+)"
+ ),
+ ),
+ (
+ "search",
+ "rescue",
+ re.compile(
+ r"^\[repowise\] No literal match for `(?P<pattern>[^`]+)`\. "
+ r"Wiki suggests `(?P<target>[^`]+)`"
+ ),
+ ),
+ (
+ "search",
+ "triage",
+ re.compile(r"^\[repowise\] \d+\+ matches for `(?P<pattern>[^`]+)`\. Top files by"),
+ ),
+ (
+ "search",
+ "digest",
+ re.compile(r"^\[repowise\] Search flood — compact digest"),
+ ),
+)
+
+#: A path-shaped token, for harvesting the file lists that follow a triage
+#: header or ride inside a digest body. Both separators are accepted: triage
+#: renders repo-relative POSIX node ids, but the digest groups whatever
+#: spelling the grep output used, which on Windows is backslashed.
+_FILEISH = re.compile(r"[\w.\-]+(?:[/\\][\w.\-]+)+\.\w{1,6}")
+
+
+@dataclass(slots=True)
+class Firing:
+ """One hook emission, with what the agent did next."""
+
+ surface: str
+ category: str
+ #: The emitted text, verbatim — the ledger key is its hash.
+ text: str
+ #: Files/symbols the emission pointed at, best first.
+ targets: list[str] = field(default_factory=list)
+ #: Search pattern the firing was about, when it names one.
+ pattern: str = ""
+ session_id: str = ""
+ ts: float | None = None
+ #: Harness-measured end-to-end wall time for the whole hook process. One
+ #: process can emit several firings, so this is the cost of the firing's
+ #: *batch*, not of the firing alone.
+ duration_ms: int = 0
+ #: Settled by :func:`classify`: True/False once judged, None when the
+ #: surface has no action to take.
+ acted: bool | None = None
+ #: What settled it — an action name, :data:`AMBIGUOUS`, or "".
+ evidence: str = ""
+ #: Tool calls between the firing and the action.
+ distance: int | None = None
+
+ @property
+ def key(self) -> str:
+ return ledger_key(self.surface, self.category, self.text)
+
+
+def parse_emission(text: str) -> list[Firing]:
+ """Split one hook emission into its firings.
+
+ A single response can carry several notices joined by newlines (a file
+ that is both governed and much-fixed emits two), so each line is matched
+ independently. A firing's ``text`` is the block it owns: its own line plus
+ any indented continuation, which is where triage and the digest keep their
+ file lists.
+ """
+ lines = text.splitlines()
+ out: list[Firing] = []
+ for i, line in enumerate(lines):
+ for surface, category, pattern in _PATTERNS:
+ m = pattern.match(line)
+ if m is None:
+ continue
+ block = [line]
+ for follow in lines[i + 1 :]:
+ if follow.startswith("[repowise]") or not follow.strip():
+ break
+ block.append(follow)
+ groups = m.groupdict()
+ targets = [t for t in (groups.get("target"), groups.get("symbol")) if t]
+ targets.extend(
+ p for p in _FILEISH.findall("\n".join(block[1:])) if p not in targets
+ )
+ out.append(
+ Firing(
+ surface=surface,
+ category=category,
+ text="\n".join(block),
+ targets=[_normalize(t) for t in targets],
+ pattern=groups.get("pattern") or "",
+ )
+ )
+ break
+ return out
+
+
+def _normalize(path: str) -> str:
+ """Repo-relative POSIX spelling, as the ledger's node ids use."""
+ return path.replace("\\\\", "/").replace("\\", "/").lstrip("./").rstrip(".,;:")
+
+
+# ---------------------------------------------------------------------------
+# Classification
+# ---------------------------------------------------------------------------
+
+#: Tools that surface a file's structure instead of its bytes — acting on a
+#: skeleton nudge means reaching for one of these.
+_STRUCTURE_TOOLS = ("get_context", "get_symbol", "search_codebase")
+
+#: Tools that answer "what does history say about this file" — acting on a
+#: fix-history notice means reaching for one of these, or running a test.
+_HISTORY_TOOLS = ("get_risk", "get_why", "get_change_risk", "get_health", "get_context")
+
+_TEST_CMD = re.compile(r"\b(pytest|jest|vitest|go test|cargo test|npm (run )?test|tox|nose)\b")
+_HISTORY_CMD = re.compile(r"\bgit (log|blame|show|bisect)\b")
+
+
+def classify(firing: Firing, following: list[tuple[str, str]]) -> Firing:
+ """Settle ``acted`` for one firing against the tool calls that followed.
+
+ *following* is ``(tool_name, serialized_input)`` in transcript order. It is
+ clipped to :data:`ACTION_WINDOW` here rather than trusted from the caller —
+ the window *is* the attribution claim, and silently widening it is how a
+ hook gets credit for something forty tool calls of unrelated work later.
+ Mutates and returns *firing* so callers can classify in place.
+ """
+ following = following[:ACTION_WINDOW]
+ if (firing.surface, firing.category) in NO_ACTION_EXPECTED:
+ firing.acted = None
+ return firing
+
+ if (firing.surface, firing.category) in COMPLIANCE_CATEGORIES:
+ return _classify_compliance(firing, following)
+
+ judge = {
+ ("read", "skeleton_nudge"): _acted_skeleton,
+ ("search", "triage"): _acted_target,
+ ("search", "rescue"): _acted_rescue,
+ ("search", "digest"): _acted_target,
+ ("fix_history", "edit_notice"): _acted_fix_history,
+ }.get((firing.surface, firing.category))
+ if judge is None:
+ firing.acted = None
+ return firing
+
+ for distance, (name, raw) in enumerate(following):
+ evidence = judge(firing, name, _normalize(raw), raw)
+ if not evidence:
+ continue
+ firing.evidence = evidence
+ if evidence == AMBIGUOUS:
+ # Keep looking: a real action later in the window still counts,
+ # and the ambiguous hit is only reported as the generous bound.
+ continue
+ firing.acted = True
+ firing.distance = distance
+ return firing
+ firing.acted = False
+ return firing
+
+
+def _acted_skeleton(firing: Firing, name: str, norm: str, raw: str) -> str:
+ """Structure call on the nudged file; a ranged re-read is ambiguous."""
+ target = firing.targets[0] if firing.targets else ""
+ if not target:
+ return ""
+ base = target.rsplit("/", 1)[-1]
+ if "get_context" in name and "skeleton" in raw and base in norm:
+ return "skeleton_call"
+ if any(t in name for t in _STRUCTURE_TOOLS) and base in norm:
+ return "structure_call"
+ if name == "Read" and target in norm and ('"offset"' in raw or '"limit"' in raw):
+ return AMBIGUOUS
+ return ""
+
+
+def _classify_compliance(firing: Firing, following: list[tuple[str, str]]) -> Firing:
+ """Score a non-action notice: respected unless the agent re-offends.
+
+ The re-read notice asks the agent to stop re-reading a file it already has
+ in context. There is no positive action to detect — reaching for
+ ``get_symbol`` instead is one compliant outcome, but so is simply moving
+ on, which is what usually happens. So the measurement is the offense:
+ another *unbounded* Read of the same file inside the window. Scoring this
+ as adoption would report a flat 0% for a notice that is in fact obeyed.
+ """
+ target = firing.targets[0] if firing.targets else ""
+ firing.acted = True
+ firing.evidence = "respected"
+ if not target:
+ return firing
+ for distance, (name, raw) in enumerate(following):
+ if name != "Read" or target not in _normalize(raw):
+ continue
+ if '"offset"' in raw or '"limit"' in raw:
+ firing.evidence = "ranged_read"
+ continue
+ firing.acted = False
+ firing.evidence = "reread_again"
+ firing.distance = distance
+ return firing
+ return firing
+
+
+def _acted_target(firing: Firing, name: str, norm: str, raw: str) -> str:
+ """Any named file showing up in a later tool call counts."""
+ for rank, target in enumerate(firing.targets):
+ if target and target in norm:
+ return f"touched_rank{rank}"
+ return ""
+
+
+def _acted_rescue(firing: Firing, name: str, norm: str, raw: str) -> str:
+ """The rescued file, or the symbol name it offered."""
+ hit = _acted_target(firing, name, norm, raw)
+ if hit:
+ return hit
+ symbol = firing.targets[1] if len(firing.targets) > 1 else ""
+ return "symbol_used" if symbol and symbol in raw else ""
+
+
+def _acted_fix_history(firing: Firing, name: str, norm: str, raw: str) -> str:
+ """Ran a test, or asked history about the file."""
+ if name in ("Bash", "PowerShell"):
+ if _TEST_CMD.search(raw):
+ return "ran_test"
+ if _HISTORY_CMD.search(raw):
+ return "read_history"
+ return ""
+ target = firing.targets[0] if firing.targets else ""
+ if any(t in name for t in _HISTORY_TOOLS) and target and target in norm:
+ return "history_tool"
+ if name == "Task" and _TEST_CMD.search(raw):
+ return "ran_test"
+ return ""
+
+
+# ---------------------------------------------------------------------------
+# Transcript replay
+# ---------------------------------------------------------------------------
+
+
+def iter_transcript_firings(path: Path, *, window: int = ACTION_WINDOW) -> Iterator[Firing]:
+ """Every classified repowise hook firing in one Claude Code transcript.
+
+ Reads the file once into two indexes — hook emissions and tool calls, both
+ by line number — then scores each emission against the calls that follow
+ it. A transcript that cannot be read at all yields nothing; individual
+ unparseable lines are skipped, never guessed at.
+ """
+ emissions: list[tuple[int, str, float | None, int, str]] = []
+ tool_uses: list[tuple[int, str, str]] = []
+ try:
+ with path.open(encoding="utf-8", errors="replace") as fh:
+ for index, line in enumerate(fh):
+ if '"attachment"' in line and "[repowise]" in line:
+ parsed = _hook_emission(line)
+ if parsed is not None:
+ emissions.append((index, *parsed))
+ elif '"tool_use"' in line:
+ tool_uses.extend(_tool_uses(line, index))
+ except OSError:
+ return
+
+ for index, text, ts, duration_ms, session_id in emissions:
+ following = [
+ (name, raw) for (j, name, raw) in tool_uses if index < j <= index + window
+ ][:window]
+ for firing in parse_emission(text):
+ firing.ts = ts
+ firing.duration_ms = duration_ms
+ firing.session_id = session_id
+ yield classify(firing, following)
+
+
+def _hook_emission(line: str) -> tuple[str, float | None, int, str] | None:
+ """``(text, ts, duration_ms, session_id)`` from a hook_success attachment.
+
+ The paired ``hook_additional_context`` line carries the same text and is
+ skipped here — counting both is what makes every firing look like two.
+ """
+ try:
+ entry = json.loads(line)
+ except ValueError:
+ return None
+ if not isinstance(entry, dict) or entry.get("type") != "attachment":
+ return None
+ attachment = entry.get("attachment")
+ if not isinstance(attachment, dict) or attachment.get("type") != "hook_success":
+ return None
+ stdout = attachment.get("stdout")
+ if not isinstance(stdout, str) or "[repowise]" not in stdout:
+ return None
+ try:
+ payload = json.loads(stdout)
+ text = payload["hookSpecificOutput"]["additionalContext"]
+ except (ValueError, KeyError, TypeError):
+ return None
+ if not isinstance(text, str):
+ return None
+ duration = attachment.get("durationMs")
+ session_id = entry.get("sessionId") or entry.get("session_id") or ""
+ return (
+ text,
+ _parse_ts(entry.get("timestamp")),
+ int(duration) if isinstance(duration, int | float) else 0,
+ session_id if isinstance(session_id, str) else "",
+ )
+
+
+def _tool_uses(line: str, index: int) -> list[tuple[int, str, str]]:
+ """``(line_index, tool_name, serialized_input)`` for one assistant line."""
+ try:
+ entry = json.loads(line)
+ except ValueError:
+ return []
+ content = ((entry or {}).get("message") or {}).get("content")
+ if not isinstance(content, list):
+ return []
+ out = []
+ for block in content:
+ if isinstance(block, dict) and block.get("type") == "tool_use":
+ name = block.get("name")
+ if isinstance(name, str):
+ out.append(
+ (index, name, json.dumps(block.get("input") or {}, ensure_ascii=False))
+ )
+ return out
+
+
+def _parse_ts(value: Any) -> float | None:
+ from repowise.core.sessions.adapters.claude_code import parse_timestamp
+
+ return parse_timestamp(value)
+
+
+# ---------------------------------------------------------------------------
+# Ledger ingest
+# ---------------------------------------------------------------------------
+
+
+def discover_transcripts(
+ repo_path: Path, *, projects_root: Path | None = None, all_projects: bool = False
+) -> list[Path]:
+ """Claude Code transcripts to replay for *repo_path*.
+
+ ``all_projects`` widens the sweep from this checkout's own transcript
+ directory to every sibling directory whose munged name shares the repo's
+ final path segment — the worktrees. A hook fired in a worktree is the same
+ hook on the same index, and leaving them out undercounts by roughly the
+ number of parallel workstreams.
+ """
+ from repowise.core.sessions.adapters.claude_code import transcript_dir_for
+
+ own = transcript_dir_for(Path(repo_path).resolve(), projects_root)
+ if not all_projects:
+ return sorted(own.glob("*.jsonl")) if own.is_dir() else []
+ root = own.parent
+ if not root.is_dir():
+ return []
+ needle = Path(repo_path).resolve().name.lower()
+ return sorted(
+ p for d in root.iterdir() if d.is_dir() and needle in d.name.lower()
+ for p in d.glob("*.jsonl")
+ )
+
+
+def ingest_transcript_efficacy(
+ repo_path: Path,
+ *,
+ projects_root: Path | None = None,
+ all_projects: bool = False,
+ since: float | None = None,
+ reset: bool = False,
+) -> dict[str, int]:
+ """Replay transcripts into the efficacy ledger. Returns per-surface counts.
+
+ Idempotent: a firing's ledger id is the hash of its own text, so replaying
+ the same transcript settles the same rows again rather than duplicating
+ them. Rows the live hook already wrote keep their first-hand ``shown_at``
+ and ``chars`` and gain the two columns only a replay can fill —
+ ``acted`` and the harness-measured ``duration_ms``.
+
+ *since* skips transcripts untouched since that epoch second, which is what
+ keeps the update-time pass off the whole 1.3 GB of history.
+
+ *reset* clears the classified surfaces before replaying. Needed once, to
+ retire rows written under the pre-text-hash ledger keys: those ids cannot
+ be recomputed from a transcript, so they would sit beside the replayed row
+ for the same firing and double the count. It never touches decision rows.
+ """
+ from repowise.core.sessions.staging import SessionStagingStore
+
+ counts: dict[str, int] = {}
+ store = SessionStagingStore.open_default(Path(repo_path).resolve())
+ try:
+ if reset:
+ store.clear_surfaces(CLASSIFIED_SURFACES)
+ for transcript in discover_transcripts(
+ repo_path, projects_root=projects_root, all_projects=all_projects
+ ):
+ try:
+ if since is not None and transcript.stat().st_mtime < since:
+ continue
+ except OSError:
+ continue
+ session_id = transcript.stem
+ for firing in iter_transcript_firings(transcript):
+ store.record_firing(
+ session_id=firing.session_id or session_id,
+ key=firing.key,
+ surface=firing.surface,
+ category=firing.category,
+ node_id=firing.targets[0] if firing.targets else "",
+ chars=len(firing.text),
+ shown_at=firing.ts or 0.0,
+ duration_ms=firing.duration_ms,
+ acted=firing.acted,
+ )
+ counts[f"{firing.surface}/{firing.category}"] = (
+ counts.get(f"{firing.surface}/{firing.category}", 0) + 1
+ )
+ store.commit()
+ finally:
+ store.close()
+ return counts
diff --git a/packages/core/src/repowise/core/sessions/staging.py b/packages/core/src/repowise/core/sessions/staging.py
index 0f9dc414c..237a43d24 100644
--- a/packages/core/src/repowise/core/sessions/staging.py
+++ b/packages/core/src/repowise/core/sessions/staging.py
@@ -68,6 +68,15 @@ CREATE TABLE IF NOT EXISTS decisions (
promoted_at REAL,
emitted_sessions INTEGER NOT NULL DEFAULT 0
);
+CREATE TABLE IF NOT EXISTS hook_runs (
+ session_id TEXT NOT NULL,
+ event TEXT NOT NULL,
+ tool TEXT NOT NULL,
+ calls INTEGER NOT NULL DEFAULT 0,
+ emitted INTEGER NOT NULL DEFAULT 0,
+ total_ms INTEGER NOT NULL DEFAULT 0,
+ PRIMARY KEY (session_id, event, tool)
+);
CREATE TABLE IF NOT EXISTS cursors (
file TEXT PRIMARY KEY,
offset INTEGER NOT NULL,
@@ -82,6 +91,8 @@ CREATE TABLE IF NOT EXISTS injections (
surface TEXT NOT NULL DEFAULT '',
category TEXT NOT NULL DEFAULT '',
chars INTEGER NOT NULL DEFAULT 0,
+ duration_ms INTEGER NOT NULL DEFAULT 0,
+ acted INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (session_id, decision_id)
);
CREATE INDEX IF NOT EXISTS idx_raw_pending ON raw_candidates(structured_key)
@@ -96,6 +107,10 @@ INJECTIONS_LEDGER_COLUMNS = (
("surface", "TEXT NOT NULL DEFAULT ''"),
("category", "TEXT NOT NULL DEFAULT ''"),
("chars", "INTEGER NOT NULL DEFAULT 0"),
+ # Wall-clock cost of the firing, and whether the agent acted on it. See
+ # :mod:`repowise.core.sessions.efficacy` for how both are filled in.
+ ("duration_ms", "INTEGER NOT NULL DEFAULT 0"),
+ ("acted", "INTEGER NOT NULL DEFAULT 0"),
)
@@ -359,23 +374,174 @@ class SessionStagingStore:
# sqlite3 (the hook path never imports repowise.core); these methods are
# the update-time reader side.
- def unevaluated_injections(self, *, before: float) -> list[dict[str, Any]]:
- """Shown-decision rows not yet judged, old enough that the showing
+ #: Surfaces whose ledger ids are ``decision_records`` primary keys, and so
+ #: can be judged by looking the record up. Every other surface is judged by
+ #: the transcript classifier in :mod:`repowise.core.sessions.efficacy`.
+ #: Pre-column rows (surface '') are all decision injections.
+ DECISION_SURFACES = ("", "decision")
+
+ def unevaluated_injections(
+ self, *, before: float, surfaces: tuple[str, ...] | None = None
+ ) -> list[dict[str, Any]]:
+ """Shown-hook rows not yet judged, old enough that the showing
session has plausibly moved past the guidance (see *before*).
- Scoped to the decision surface: the ledger also records read/search
- enrichments whose ids are not decision_records keys, and the follow-up
- classifier for those rides a separate mining pass. Pre-column rows
- (surface '') are all decision injections.
+ *surfaces* defaults to :data:`DECISION_SURFACES`, the only rows whose
+ ``decision_id`` resolves to a decision record. Pass ``()`` for every
+ surface — what the transcript classifier does, since it judges a firing
+ by what the agent did next rather than by looking anything up.
+ """
+ scope = self.DECISION_SURFACES if surfaces is None else surfaces
+ where = "evaluated = 0 AND shown_at < ?"
+ params: list[Any] = [before]
+ if scope:
+ where += f" AND surface IN ({','.join('?' * len(scope))})"
+ params.extend(scope)
+ rows = self._conn.execute(
+ "SELECT session_id, decision_id, node_id, shown_at, surface, category "
+ f"FROM injections WHERE {where} ORDER BY shown_at ASC",
+ params,
+ ).fetchall()
+ return [
+ {
+ "session_id": r[0],
+ "decision_id": r[1],
+ "node_id": r[2],
+ "shown_at": r[3],
+ "surface": r[4],
+ "category": r[5],
+ }
+ for r in rows
+ ]
+
+ def record_firing(
+ self,
+ *,
+ session_id: str,
+ key: str,
+ surface: str,
+ category: str,
+ node_id: str = "",
+ chars: int = 0,
+ shown_at: float,
+ duration_ms: int = 0,
+ acted: bool | None = None,
+ ) -> None:
+ """Upsert one classified hook firing (the transcript-side writer).
+
+ The live hooks insert their own row the moment they fire; this both
+ backfills firings that predate the ledger and settles the columns the
+ hook could not know — ``acted`` (which needs the following tool calls)
+ and the true end-to-end ``duration_ms`` (which only the harness
+ measures). An existing row keeps its ``shown_at`` and ``chars``: the
+ hook recorded those first-hand.
+
+ Every row written here is ``evaluated``, including the surfaces whose
+ verdict is "no action was called for" (*acted* ``None``, which stores
+ as 0). Those are told apart by ``(surface, category)`` at report time
+ via :data:`~repowise.core.sessions.efficacy.NO_ACTION_EXPECTED`, so an
+ unjudgeable firing is not re-examined on every update forever.
+ """
+ self._conn.execute(
+ "INSERT INTO injections "
+ "(session_id, decision_id, node_id, shown_at, surface, category, chars, "
+ "duration_ms, acted, evaluated) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, 1) "
+ "ON CONFLICT(session_id, decision_id) DO UPDATE SET "
+ "duration_ms = MAX(excluded.duration_ms, injections.duration_ms), "
+ "acted = excluded.acted, evaluated = 1",
+ (
+ session_id,
+ key,
+ node_id,
+ shown_at,
+ surface,
+ category,
+ chars,
+ duration_ms,
+ 1 if acted else 0,
+ ),
+ )
+
+ def clear_surfaces(self, surfaces: tuple[str, ...]) -> int:
+ """Drop every ledger row on *surfaces*. Returns the rows removed.
+
+ Only ever used to rebuild transcript-derived surfaces from scratch,
+ where the transcripts are the source of truth and the rows are
+ reconstructed in the same command. Decision rows carry state that has
+ no other home (a promotion's emit bookkeeping) and are never passed
+ here.
+ """
+ if not surfaces:
+ return 0
+ cur = self._conn.execute(
+ f"DELETE FROM injections WHERE surface IN ({','.join('?' * len(surfaces))})",
+ surfaces,
+ )
+ return cur.rowcount
+
+ def efficacy_rows(self) -> list[dict[str, Any]]:
+ """Per (surface, category) firing counts, action rates and latency."""
+ rows = self._conn.execute(
+ "SELECT surface, category, COUNT(*), SUM(acted), SUM(evaluated), "
+ "COUNT(DISTINCT session_id), SUM(chars), "
+ "SUM(CASE WHEN duration_ms > 0 THEN duration_ms ELSE 0 END), "
+ "SUM(CASE WHEN duration_ms > 0 THEN 1 ELSE 0 END) "
+ "FROM injections GROUP BY surface, category ORDER BY COUNT(*) DESC"
+ ).fetchall()
+ return [
+ {
+ "surface": r[0] or "decision",
+ "category": r[1] or "",
+ "firings": r[2],
+ "acted": r[3] or 0,
+ "evaluated": r[4] or 0,
+ "sessions": r[5],
+ "chars": r[6] or 0,
+ "duration_ms_total": r[7] or 0,
+ "duration_ms_count": r[8] or 0,
+ }
+ for r in rows
+ ]
+
+ def session_duration_totals(self) -> list[int]:
+ """Total hook wall-time per session, in ms, for sessions that have it."""
+ rows = self._conn.execute(
+ "SELECT SUM(duration_ms) FROM injections WHERE duration_ms > 0 "
+ "GROUP BY session_id"
+ ).fetchall()
+ return [int(r[0]) for r in rows if r[0]]
+
+ def hook_run_totals(self) -> list[dict[str, Any]]:
+ """Per-session hook invocation counts and in-process wall time.
+
+ The counterpart to :meth:`efficacy_rows`, and the only source for what
+ the *silent* invocations cost: a hook that returns nothing leaves no
+ transcript record at all, so the emissions ledger cannot see the calls
+ that make up most of the bill.
"""
rows = self._conn.execute(
- "SELECT session_id, decision_id, node_id, shown_at FROM injections "
- "WHERE evaluated = 0 AND shown_at < ? AND surface IN ('', 'decision') "
- "ORDER BY shown_at ASC",
- (before,),
+ "SELECT session_id, SUM(calls), SUM(emitted), SUM(total_ms) "
+ "FROM hook_runs GROUP BY session_id"
).fetchall()
return [
- {"session_id": r[0], "decision_id": r[1], "node_id": r[2], "shown_at": r[3]}
+ {"session_id": r[0], "calls": r[1] or 0, "emitted": r[2] or 0, "total_ms": r[3] or 0}
+ for r in rows
+ ]
+
+ def hook_run_by_tool(self) -> list[dict[str, Any]]:
+ """Invocation counts and in-process wall time per hook event and tool."""
+ rows = self._conn.execute(
+ "SELECT event, tool, SUM(calls), SUM(emitted), SUM(total_ms) "
+ "FROM hook_runs GROUP BY event, tool ORDER BY SUM(total_ms) DESC"
+ ).fetchall()
+ return [
+ {
+ "event": r[0],
+ "tool": r[1],
+ "calls": r[2] or 0,
+ "emitted": r[3] or 0,
+ "total_ms": r[4] or 0,
+ }
for r in rows
]
diff --git a/tests/unit/cli/test_augment_served_reads.py b/tests/unit/cli/test_augment_served_reads.py
index 322f517a5..e09a1bc26 100644
--- a/tests/unit/cli/test_augment_served_reads.py
+++ b/tests/unit/cli/test_augment_served_reads.py
@@ -170,9 +170,9 @@ def test_served_ranges_parser_shapes():
def test_search_firing_logs_to_ledger(tmp_path):
repo = _repo(tmp_path)
- _log_search_firing(repo, "sess-1", "rescue", "parse_yaml", "[repowise] rescue text")
- _log_search_firing(repo, "sess-1", "rescue", "parse_yaml", "[repowise] rescue text") # dedup
- _log_search_firing(repo, "", "rescue", "parse_yaml", "text") # no session id: skipped
+ _log_search_firing(repo, "sess-1", "rescue", "[repowise] rescue text")
+ _log_search_firing(repo, "sess-1", "rescue", "[repowise] rescue text") # dedup
+ _log_search_firing(repo, "", "rescue", "text") # no session id: skipped
rows = _ledger_rows(repo)
assert len(rows) == 1
@@ -182,6 +182,22 @@ def test_search_firing_logs_to_ledger(tmp_path):
assert chars == len("[repowise] rescue text")
+def test_search_firing_key_matches_the_transcript_classifier(tmp_path):
+ """The live row and the replayed firing must be the same row, not two.
+
+ The classifier recomputes a firing's ledger id from the text it finds in
+ the transcript; if that drifts from what the hook wrote, every firing is
+ counted twice and none of them is ever marked acted.
+ """
+ from repowise.core.sessions.efficacy import ledger_key
+
+ repo = _repo(tmp_path)
+ text = "[repowise] No literal match for `parse_yaml`. Closest indexed symbol: func `x` in a.py"
+ _log_search_firing(repo, "sess-1", "rescue", text)
+
+ assert _ledger_rows(repo)[0][1] == ledger_key("search", "rescue", text)
+
+
def test_claim_ledger_migrates_legacy_sidecar(tmp_path):
repo = _repo(tmp_path)
db_path = repo / ".repowise" / "sessions" / "sessions.db"
diff --git a/tests/unit/sessions/test_efficacy.py b/tests/unit/sessions/test_efficacy.py
new file mode 100644
index 000000000..e25921964
--- /dev/null
+++ b/tests/unit/sessions/test_efficacy.py
@@ -0,0 +1,314 @@
+"""Hook efficacy: emission parsing, per-surface classification, ledger ingest.
+
+The numbers this module produces are the input to every later decision about
+which hook surfaces to keep, so the tests here are mostly about the ways a
+classifier can lie: counting one firing twice, crediting an action it cannot
+attribute, or reporting 0% for a notice whose success looks like silence.
+"""
+
+from __future__ import annotations
+
+import json
+
+import pytest
+
+from repowise.core.sessions.efficacy import (
+ ACTION_WINDOW,
+ AMBIGUOUS,
+ classify,
+ ingest_transcript_efficacy,
+ iter_transcript_firings,
+ ledger_key,
+ parse_emission,
+)
+from repowise.core.sessions.staging import SessionStagingStore
+
+NUDGE = (
+ "[repowise] A skeleton of pkg/core/thing.py is ~200 tokens vs ~4000 for the "
+ 'full file. For structure-level questions use get_context(["pkg/core/thing.py"], '
+ 'include=["skeleton"]).'
+)
+TRIAGE = (
+ "[repowise] 40+ matches for `parse_yaml`. Top files by graph centrality:\n"
+ " pkg/core/loader.py\n"
+ " pkg/core/schema.py"
+)
+RESCUE = (
+ "[repowise] No literal match for `parseYaml`. Closest indexed symbol: "
+ "function `parse_yaml` in pkg/core/loader.py:12"
+)
+FIXES = "[repowise] pkg/core/loader.py has been bug-fixed 9x in the last 6 months, last 3 days ago."
+REREAD = (
+ "[repowise] You already read pkg/core/thing.py this session and it is unchanged — "
+ "its content is still in context."
+)
+
+
+def _use(name, **inp):
+ return (name, json.dumps(inp))
+
+
+# ---------------------------------------------------------------------------
+# Parsing
+# ---------------------------------------------------------------------------
+
+
+def test_parses_each_surface_with_its_target():
+ for text, surface, category, target in (
+ (NUDGE, "read", "skeleton_nudge", "pkg/core/thing.py"),
+ (TRIAGE, "search", "triage", "pkg/core/loader.py"),
+ (RESCUE, "search", "rescue", "pkg/core/loader.py"),
+ (FIXES, "fix_history", "edit_notice", "pkg/core/loader.py"),
+ (REREAD, "read", "reread", "pkg/core/thing.py"),
+ ):
+ (firing,) = parse_emission(text)
+ assert (firing.surface, firing.category) == (surface, category)
+ assert firing.targets[0] == target
+
+
+def test_one_emission_can_carry_several_firings():
+ """A file that is both governed and much-fixed emits two notices at once."""
+ firings = parse_emission(f"{FIXES}\n{NUDGE}")
+ assert [(f.surface, f.category) for f in firings] == [
+ ("fix_history", "edit_notice"),
+ ("read", "skeleton_nudge"),
+ ]
+
+
+def test_triage_keeps_its_ranked_file_list_in_order():
+ (firing,) = parse_emission(TRIAGE)
+ assert firing.targets == ["pkg/core/loader.py", "pkg/core/schema.py"]
+ assert firing.pattern == "parse_yaml"
+
+
+def test_digest_paths_are_normalized_from_windows_spelling():
+ """The digest groups whatever spelling grep emitted, backslashes included."""
+ text = (
+ "[repowise] Search flood — compact digest (files ordered by match count):\n"
+ " pkg\\core\\loader.py (10 matches)\n"
+ " pkg\\core\\schema.py (3 matches)"
+ )
+ (firing,) = parse_emission(text)
+ assert firing.targets == ["pkg/core/loader.py", "pkg/core/schema.py"]
+
+
+def test_unrelated_repowise_text_is_not_a_firing():
+ assert parse_emission("[repowise] Index is behind HEAD: indexed abc, now def.") == []
+
+
+# ---------------------------------------------------------------------------
+# Classification
+# ---------------------------------------------------------------------------
+
+
+def test_skeleton_nudge_acted_only_on_a_structure_call():
+ (firing,) = parse_emission(NUDGE)
+ classify(firing, [_use("mcp__repowise__get_context", targets=["pkg/core/thing.py"],
+ include=["skeleton"])])
+ assert firing.acted is True
+ assert firing.evidence == "skeleton_call"
+ assert firing.distance == 0
+
+
+def test_ranged_read_after_a_nudge_is_ambiguous_not_acted():
+ """Reading a range of a file you just read in full is ordinary edit prep.
+
+ Crediting it is what turns the nudge's real 0.2% into a flattering 19.7%.
+ """
+ (firing,) = parse_emission(NUDGE)
+ classify(firing, [_use("Read", file_path="pkg/core/thing.py", offset=40, limit=20)])
+ assert firing.acted is False
+ assert firing.evidence == AMBIGUOUS
+
+
+def test_a_real_action_after_an_ambiguous_one_still_counts():
+ (firing,) = parse_emission(NUDGE)
+ classify(
+ firing,
+ [
+ _use("Read", file_path="pkg/core/thing.py", offset=40, limit=20),
+ _use("mcp__repowise__get_symbol", id="pkg/core/thing.py::Widget"),
+ ],
+ )
+ assert firing.acted is True
+ assert firing.distance == 1
+
+
+def test_triage_records_which_rank_the_agent_took():
+ (firing,) = parse_emission(TRIAGE)
+ classify(firing, [_use("Read", file_path="/repo/pkg/core/schema.py")])
+ assert (firing.acted, firing.evidence) == (True, "touched_rank1")
+
+
+def test_rescue_counts_the_offered_symbol_as_well_as_the_file():
+ (firing,) = parse_emission(RESCUE)
+ classify(firing, [_use("Grep", pattern="parse_yaml")])
+ assert firing.acted is True
+
+
+def test_fix_history_acted_on_a_test_run_or_a_history_look():
+ for call, evidence in (
+ (_use("Bash", command="pytest tests/unit/core"), "ran_test"),
+ (_use("Bash", command="git log -p pkg/core/loader.py"), "read_history"),
+ ):
+ (firing,) = parse_emission(FIXES)
+ classify(firing, [call])
+ assert (firing.acted, firing.evidence) == (True, evidence)
+
+
+def test_fix_history_ignores_unrelated_shell_work():
+ (firing,) = parse_emission(FIXES)
+ classify(firing, [_use("Bash", command="ls -la")])
+ assert firing.acted is False
+
+
+def test_reread_is_respected_unless_the_file_is_read_in_full_again():
+ """Its success looks like silence, so adoption scoring would report 0%."""
+ (firing,) = parse_emission(REREAD)
+ classify(firing, [_use("Grep", pattern="anything")])
+ assert (firing.acted, firing.evidence) == (True, "respected")
+
+ (offender,) = parse_emission(REREAD)
+ classify(offender, [_use("Read", file_path="pkg/core/thing.py")])
+ assert (offender.acted, offender.evidence) == (False, "reread_again")
+
+ (ranged,) = parse_emission(REREAD)
+ classify(ranged, [_use("Read", file_path="pkg/core/thing.py", offset=5, limit=10)])
+ assert ranged.acted is True
+
+
+def test_stale_read_has_no_action_to_take():
+ text = "[repowise] a/b.py changed (Edit/Write) after your previous read of it — stale."
+ (firing,) = parse_emission(text)
+ classify(firing, [_use("Read", file_path="a/b.py")])
+ assert firing.acted is None
+
+
+def test_action_beyond_the_window_is_not_credited():
+ (firing,) = parse_emission(TRIAGE)
+ following = [_use("Bash", command="ls")] * ACTION_WINDOW
+ classify(firing, [*following, _use("Read", file_path="pkg/core/loader.py")])
+ assert firing.acted is False
+
+
+# ---------------------------------------------------------------------------
+# Transcript replay and ledger ingest
+# ---------------------------------------------------------------------------
+
+
+def _transcript(path, emissions_then_uses):
+ """Write a minimal Claude Code transcript: hook attachments + tool_use lines."""
+ lines = []
+ for kind, value in emissions_then_uses:
+ if kind == "hook":
+ lines.append(
+ json.dumps(
+ {
+ "type": "attachment",
+ "sessionId": "sess-1",
+ "timestamp": "2026-08-03T10:00:00.000Z",
+ "attachment": {
+ "type": "hook_success",
+ "hookName": "PostToolUse:Read",
+ "durationMs": 1200,
+ "stdout": json.dumps(
+ {
+ "hookSpecificOutput": {
+ "hookEventName": "PostToolUse",
+ "additionalContext": value,
+ }
+ }
+ ),
+ },
+ }
+ )
+ )
+ elif kind == "echo":
+ # The paired record the harness also writes for the same firing.
+ lines.append(
+ json.dumps(
+ {"type": "attachment", "attachment": {
+ "type": "hook_additional_context", "content": [value]}}
+ )
+ )
+ else:
+ name, inp = value
+ lines.append(
+ json.dumps(
+ {
+ "type": "assistant",
+ "message": {
+ "content": [
+ {"type": "tool_use", "id": "t1", "name": name,
+ "input": json.loads(inp)}
+ ]
+ },
+ }
+ )
+ )
+ path.write_text("\n".join(lines) + "\n", encoding="utf-8")
+
+
+def test_replay_counts_each_firing_once_despite_the_paired_record(tmp_path):
+ """The harness logs one emission twice; counting both doubles every figure."""
+ t = tmp_path / "sess-1.jsonl"
+ _transcript(t, [("hook", TRIAGE), ("echo", TRIAGE),
+ ("use", _use("Read", file_path="pkg/core/loader.py"))])
+
+ firings = list(iter_transcript_firings(t))
+ assert len(firings) == 1
+ assert firings[0].acted is True
+ assert firings[0].duration_ms == 1200
+
+
+def test_replay_ignores_the_agent_quoting_the_hook_text(tmp_path):
+ """Reading the hook's own source must not register as a firing."""
+ t = tmp_path / "sess-1.jsonl"
+ _transcript(t, [("use", _use("Write", file_path="x.py", content=NUDGE))])
+ assert list(iter_transcript_firings(t)) == []
+
+
+def test_ingest_is_idempotent_and_settles_the_live_row(tmp_path):
+ """Replaying twice must not double-count, and must upgrade the hook's row."""
+ repo = tmp_path / "repo"
+ (repo / ".repowise" / "sessions").mkdir(parents=True)
+ projects = tmp_path / "projects"
+ from repowise.core.sessions.adapters.claude_code import transcript_dir_for
+
+ d = transcript_dir_for(repo.resolve(), projects)
+ d.mkdir(parents=True)
+ _transcript(d / "sess-1.jsonl", [("hook", TRIAGE),
+ ("use", _use("Read", file_path="pkg/core/loader.py"))])
+
+ # The live hook wrote its own row first, keyed on the same text.
+ store = SessionStagingStore.open_default(repo)
+ store.record_firing(session_id="sess-1", key=ledger_key("search", "triage", TRIAGE),
+ surface="search", category="triage", chars=len(TRIAGE),
+ shown_at=1.0, duration_ms=50, acted=None)
+ store.commit()
+ store.close()
+
+ for _ in range(2):
+ ingest_transcript_efficacy(repo, projects_root=projects)
+
+ store = SessionStagingStore.open_default(repo)
+ try:
+ rows = [r for r in store.efficacy_rows() if r["surface"] == "search"]
+ finally:
+ store.close()
+ assert len(rows) == 1
+ assert rows[0]["firings"] == 1, "replay inserted a second row for one firing"
+ assert rows[0]["acted"] == 1
+ assert rows[0]["duration_ms_total"] == 1200, "harness timing should win over the hook's"
+
+
[email protected]("surface,category,text", [
+ ("read", "skeleton_nudge", NUDGE),
+ ("search", "triage", TRIAGE),
+ ("fix_history", "edit_notice", FIXES),
+])
+def test_ledger_key_mirrors_the_hook_side_helper(surface, category, text):
+ """Drift here silently doubles every count — the hook cannot import core."""
+ from repowise.cli.commands.augment_cmd._shared import _ledger_key
+
+ assert ledger_key(surface, category, text) == _ledger_key(surface, category, text)