Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.
Distill: index-aware output distillation
Coding agents spend much of their context window on output they never needed:
300 lines of passing tests around 4 failures, a full git log to learn what
changed recently, a 60k-token diff to review one hunk. Distill compresses that
output before the agent reads it. Errors come first, structure is kept, and
everything dropped can be restored.
Distill is a capability, not an intelligence layer of its own. It reuses what the index already knows (symbol bounds, graph centrality, hotspots) to decide what to keep. It needs no LLM key, and its filters work on repos with no index.
repowise distill pytest -x # run pytest, print a compact errors-first rendering
repowise expand a1b2c3d4e5f6 # restore anything that was omitted
repowise saved # tokens and dollars saved so far
Guarantees (enforced by the engine and asserted by tests):
- Errors survive. Every line classified as an error or failure in the raw output appears in the distilled rendering.
- Reversible. Raw output is stored before any marker is printed;
repowise expand <ref>returns it byte-for-byte. - Fallback to raw. A filter error, a storage failure or a rendering that is not smaller prints the original output unchanged.
- Net-positive only. Output is distilled only when it gets smaller, marker included. Small outputs pass through untouched.
- Exit codes preserved.
repowise distill <cmd>is a drop-in wrapper for scripts and agent tool calls.
The surfaces
1. repowise distill <cmd>: the executor
Runs the command (shell semantics preserved), captures stdout and stderr, picks a filter by command shape and then by content, and prints the compact rendering. These filters ship:
| Filter | Commands | What it keeps |
|---|---|---|
test_output |
pytest, jest, vitest, cargo test, go test | failures, assertion details and the summary; collapses runs of passes |
build_output |
npm/tsc/cargo/go builds | errors and warnings grouped; strips progress and boilerplate |
lint_output |
eslint/biome, ruff/flake8/mypy, clippy, golangci-lint | errors verbatim; warnings grouped by rule id with counts and file:line anchors; fixable totals |
install_output |
pip, uv, poetry, npm/pnpm/yarn install, cargo install, brew, bundle, composer | what changed and the final summary; drops resolver and download progress. An all-satisfied run collapses to install: ok (no changes) |
infra_plan |
terraform/tofu plan, helm diff/upgrade |
the plan summary and the resources that change. A no-op collapses to plan: no changes |
git_status |
git status |
porcelain-style compact status |
git_log |
git log |
recent subjects and counts |
git_diff |
git diff/show, gh pr diff |
stat plus the most relevant hunks (--stat output has its own git_diff_stat filter) |
search_results |
grep / rg floods | digest grouped by file, with per-file counts and anchors |
file_listing |
ls / tree / find | grouped tree rendering |
logs |
anything log-shaped | template collapse with counts (timestamps and ids normalized) |
Dropped content goes to the omission store and is referenced inline:
[repowise#a1b2c3d4e5f6: 230 lines omitted (~6.1k tokens); restore: repowise expand a1b2c3d4e5f6]
2. repowise expand <ref>: the reversal
repowise expand a1b2c3d4e5f6 # full original output
repowise expand a1b2c3d4e5f6 -q "FAILED" # only the matching lines
repowise expand "[repowise#a1b2...]" # a pasted whole marker works too
Looks in the current repo's store first, then the user-level fallback store.
MCP clients without a shell resolve the same refs with
get_symbol("repowise#<ref>") (see MCP).
3. The command-rewrite hook (Claude Code + Codex)
A PreToolUse hook rewrites noisy agent commands: pytest -x becomes
repowise distill pytest -x. Rewrites run without a prompt by default
(permission: allow). That is a bounded substitution, never a new command (see
Safety model). Set permission: ask in
.repowise/config.yaml to approve each rewrite. repowise init offers to
install the hook (default: yes), or install it yourself:
repowise hook rewrite install # or opt in during `repowise init`
repowise hook rewrite status
repowise hook rewrite uninstall # removes only the repowise entries
The hook covers both Claude Code shell tools: Bash and, on Windows, PowerShell.
Codex. When ~/.codex exists, install also covers the Codex CLI, with two
limits its hook protocol imposes:
- Codex applies a PreToolUse rewrite only from version 0.137. On older
builds the hook entry is skipped.
repowise hook rewrite statusreports what your build can do. - Codex cannot show a rewritten command for approval. Under Codex, rewrites fire
only for families resolving to
permission: allow(the default); a family set toaskpasses through unchanged.
The Codex entry lands in ~/.codex/hooks.json, so one install covers every
repo. Codex asks you to review new hooks: run /hooks inside Codex to trust it.
Separately, install maintains a marker-managed "Output Distillation" section
in the repo's AGENTS.md that teaches the agent to run repowise distill <cmd>
itself and to repowise expand markers instead of re-running commands. This
works on every Codex version. uninstall removes the section and restores your
AGENTS.md byte-for-byte.
Cursor. When ~/.cursor exists, install also adds a preToolUse entry
(matcher Shell) to ~/.cursor/hooks.json, beside any hooks you already have.
Cursor's preToolUse honours allow and deny only, so as under Codex, rewrites
fire only for families resolving to permission: allow. repowise agents add --target=cursor --scope=user writes the same entry.
The hook is conservative. It never rewrites:
- redirections and compound commands (
>,&&,;, backticks,$()), and almost all pipes. Two shapes are allowed: a trailing2>&1(distill merges stderr anyway), and on macOS/Linux a single pipe intohead,tail,grep,egrep,fgreporrg, which runs unchanged inside distill's own shell (pytest -q | head -50becomesrepowise distill "pytest -q | head -50"). Thegrep/rgpattern-file forms (-f,--file) are excluded. - watch and follow modes (
--watch,tail -f) - trivial commands on the ignore list, and commands already prefixed
- PowerShell-native constructs (
Verb-Nouncmdlets,& "path"invocations, backtick continuations) and, from PowerShell, alias tokens such asls,catandfindthat do not mean what their unix namesakes mean - commands in repos that have not opted into repowise (no
.repowise/upward)
On macOS/Linux these decisions come from tokenizing the command, so a shell
character inside quotes is text: pytest -k "a|b" is a test run and is
rewritten. On Windows any |, &, ;, <, > or backtick anywhere in the
command passes it through untouched, because cmd.exe re-parses metacharacters
that argv quoting does not cover. An unrecognized shape is always left alone.
Allow rules under ask. A rewrite changes the command string. If you set
permission: ask, an allow rule you already had, such as Bash(git diff:*), no
longer matches repowise distill git diff ..., and the prompt comes back. One
extra rule covers the distill prefix:
// ~/.claude/settings.json
"permissions": {
"allow": [
"Bash(repowise distill:*)",
"PowerShell(repowise distill:*)"
]
}
repowise hook rewrite install --allow-rule adds these for you. Under the
default allow posture they are not needed. repowise distill runs the
wrapped command unchanged and never widens what it can do.
Per-repo behavior lives under distill.commands in .repowise/config.yaml (see
Configuration). Declining the repowise init prompt writes
distill.commands.enabled: false, so a hook installed globally from another
repo stays inert in this one. A multi-repo workspace init asks once and
records the answer in every selected repo; repowise hook rewrite install -w
re-enables them all later.
repowise init also adds a short "Output Distillation" section to the managed
CLAUDE.md, so any agent that runs shell commands can use distill voluntarily,
hook or no hook.
4. Read intelligence: skeletons and stale-read notices
The index knows every symbol's line bounds, so repowise can render a file skeleton: every signature, the imports, and the bodies of only the most central symbols, without parsing anything at query time:
get_context(["src/big_module.py"], include=["skeleton"])
Body selection is ranked by importance (symbol PageRank, hotspot bit, query match), which is where the index beats blind truncation.
The PostToolUse hook adds passive read surfaces (details in HOOKS.md):
- Skeleton replacement (opt-in,
hooks.read_skeleton): an unboundedReadof a large indexed file is served as its skeleton, once per file per session. - Re-read collapse (opt-in,
hooks.read_reread): aReadof a range this session already read, with no edit between and identical bytes, is served as a short notice. Never twice in a row, so one more Read always returns content. - Stale-read notice: after an
Edit/Write, a laterReadof the same file warns that earlier excerpts predate the edit. - Search digest: grep floods (50 or more lines) get a digest grouped by file and ordered by graph centrality.
5. MCP response budget: _meta.omitted
MCP tool responses are token-budgeted, and every drop goes through the same omission store:
"_meta": {
"omitted": {
"refs": ["a1b2c3d4e5f6"],
"tokens": 5840,
"restore": "repowise expand <ref> (CLI) or get_symbol(\"repowise#<ref>\", query?) (MCP)"
}
}
get_symbol resolves omission refs as well as symbol ids. The
repowise#<12-hex> shape cannot be confused with path/to/file.py::Name, and
the optional query parameter searches within the stored content. See
MCP_TOOLS.md.
MCP calls also record a counterfactual saving (the raw file exploration the
answer replaced) as mcp:<tool> rows in the same ledger. Truncation is folded
into that figure, so it is never counted twice. The saving is recorded, not
served: _meta.tokens_saved and _meta.replaced_tokens appear on a response
only with REPOWISE_MCP_DEBUG_META=1 (and on get_overview).
The omission store
.repowise/omissions/omissions.db is a SQLite sidecar, kept separate from
wiki.db so hook-time writes never contend with indexing. Outside a repowise
repo it falls back to ~/.repowise/omissions/.
- Content is keyed by a 12-hex truncated SHA-256: the same ref that appears in markers, so one store serves the CLI, the hook and MCP.
- It is durable across sessions: an agent resuming tomorrow can still expand yesterday's markers.
- Pruning is by age and size (7 days and 50 MB by default, configurable), applied on write. The most recent row is never evicted, so a marker just printed cannot dangle.
repowise saved: the savings report
repowise saved # per-operation rollup, totals, estimated dollars
repowise saved --by surface # distill vs hooks vs MCP
repowise saved --by agent # which agent the savings went to
repowise saved --by day # daily rollup
repowise saved --since 2026-06-01
repowise saved --missed # savings raw commands left on the table
repowise saved --missed --missed-days 30
One ledger covers the distill command and hook path, the hooks that replace a
tool result, and MCP calls (each answer counted against the raw exploration it
replaced). --by surface separates them.
Savings are priced at each event's own rate, captured when it was recorded, so
a later price change never rewrites a past saving. Events recorded without a
rate are reported as unpriced. Saved tokens are input the agent never read, so
the input rate applies, from the same pricing table repowise costs uses. Token
counts are chars/4 estimates.
The total keeps two kinds of evidence apart. Measured savings compare a known before and after. Inferred savings estimate the exploration an answer replaced. The command prints the split and calls the total "estimated" whenever any of it is inferred.
Missed savings: repowise saved --missed
How many tokens did raw commands waste that a filter would have caught?
--missed scans your local Claude Code transcripts for Bash/PowerShell calls in
this repo that did not go through repowise distill, classifies each with the
engine's own router, and estimates the foregone savings from each filter's
conservative fixture floor (the per-filter minimums asserted in CI, not the
medians). The default window is 7 days (--missed-days N for more). Plain
repowise saved appends a one-line summary when there is something to report.
The scan is read-only and local. Commands and outputs are read from your own
transcript directory (~/.claude/projects/...); nothing is uploaded. Codex
transcripts are not scanned.
The local dashboard's Costs page leads with a savings card that combines the distill ledger with MCP tool savings. The dollar figure uses the input rate of the model your coding agent actually ran, detected from local Claude Code and Codex transcripts, with a default when nothing is detectable.
repowise corrections: recurring command fumbles
The same transcript reader, pointed at commands the agent got wrong and then fixed. It finds consecutive runs of one base command where the first failed (the transcript records a real exit code) and a later variant succeeded, classifies the fumble, and aggregates recurring rules:
repowise corrections # report only (default window: 30 days)
repowise corrections --days 60
repowise corrections --write # maintain the managed guidance block
| Kind | Example rule |
|---|---|
| wrong tool | use .venv\Scripts\python.exe instead of bare python |
| wrong path | pytest: use tests/unit/cli/, not ../../tests/unit/cli/ |
| unknown flag | pytest does not support --looponfail |
| missing arg | tool needs --required-thing |
Classification favors precision. Apart from the structural wrong-tool case, a rule forms only when the error text names the dropped flag or path, so a red-green loop re-running tests with different selections never becomes a "correction". Wrong-path rules consult the symbol index when available.
--write (opt-in) maintains a "Known command corrections" block between
REPOWISE_CORRECTIONS markers in the repo's .claude/CLAUDE.md (and
AGENTS.md when one exists): the most frequent rules first, at least 2
occurrences each, at most 10. Re-running refreshes it; when no rule clears the
threshold the block is removed. Content outside the markers is never touched.
The same privacy contract as the missed scan applies.
Measured savings
On a public OSS repository (microdot), one run per command, tokens estimated as chars/4 (the ledger's estimator):
| Command | Raw tokens | Distilled | Saved |
|---|---|---|---|
pytest -q (11 failures) |
3,374 | 1,317 | 61%, all 11 FAILED lines kept |
git log -50 |
3,064 | 331 | 89% |
git diff (30 commits of history) |
62,833 | 8,635 | 86% |
git log --oneline -30 |
321 | 321 | 0%, already compact, passed through |
git status (clean tree) |
83 | 83 | 0%, too small to distill, passed through |
The 0% rows are the net-positive guard: distill never bloats small output. In an agent spot-check on the same repo (a seeded 11-failure bug), the agent found the root-cause line and fix from the distilled test output, the same conclusion as the raw-output run.
Fixture-suite medians across the core filters: at least 60% reduction on test, build and lint output with zero error-line loss (asserted in CI).
Install and infra-plan logs compress harder because they are mostly repeated boilerplate. Measured on real captures from the repowise repository, one run each:
| Command | Saved |
|---|---|
npm ci (1,344 packages) |
99.5% |
pip install (everything already satisfied) |
collapses to install: ok (no changes) |
terraform plan |
80.2% |
Each still round-trips through repowise expand, and the error line in a
failing install survives verbatim.
Configuration
The distill: block in .repowise/config.yaml:
distill:
enabled: true # master switch for this repo
commands:
enabled: true # the command path (CLI + hook rewrites)
permission: allow # ask | allow | off: hook posture (default allow)
families: # per-filter overrides: ask | allow | off | deny
test_output: allow # auto-allow rewrites for test runs
git_diff: deny # never rewrite git diff here
disabled_filters: [] # filters to skip entirely, e.g. [logs]
omission_store:
ttl_days: 7 # prune stored omissions after this
max_mb: 50 # size cap, oldest pruned first
With no block present, these defaults apply. repowise doctor validates the
block (unknown keys, bad permission values, unknown filter names, non-positive
store sizes) and reports the store size against its cap and whether the rewrite
hook is installed. Full key reference: CONFIG.md.
Safety model, in one place
| Risk | Mitigation |
|---|---|
| A filter eats a critical line | errors-first invariant, fixture tests, expand recovery, fallback to raw |
| Silent permission escalation | a rewrite is always repowise distill <one recognized command> from a closed family set, never an arbitrary command behind the wrapper. Set permission: ask to approve each one. Codex has no ask primitive, so only families resolving to allow rewrite there |
| Marker with nothing behind it | content is stored before the marker prints; a store failure prints raw output |
| Compound-command semantics | compound commands, substitution and redirects are never rewritten. The one pipe shape that is (macOS/Linux only) is passed as one quoted token and runs verbatim in distill's shell; anything that could break out of that quoting bails |
| Unindexed or stale repo | filters work without an index; the index only improves ranking |
| Store growth | age and size caps, pruned on write; repowise doctor reports size |
See also
- CLI_REFERENCE.md:
distill,expand,saved,hook rewrite - MCP_TOOLS.md:
_meta.omitted, the skeleton include,get_symbolrefs - CONFIG.md: the
distill:block - HOOKS.md: every agent hook, including the read surfaces