Skip to content

Experience-Driven Black-Box Cost Control for Coding Agents

teaser fixed

Aaronec8ff5ec7ed5
diff --git a/CLAUDE.md b/CLAUDE.md
index 51e95a9..23ae32b 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -38,13 +38,13 @@ agent behavior, and `docs/ARCHITECTURE.md` for how the package fits together.
`plugin/config.py`); write new code against the current names.
- No run outputs in the repository: transcripts, rows, streams, figures rendered from runs and
tables derived from them stay out of git (`.gitignore`). The one exception is the paper's
- headline results in `README.md` (the line under the badges, the Results section and
- `docs/assets/results_overview.*`, and the teaser's closing results, which read
- `results_overview.py`'s `ROWS`): quoted from the paper's Table 1 and attributed to it.
- Change them only with the paper, and redraw both figures with
+ headline results in `README.md` (the line under the badges, the Results section, and the
+ teaser's closing results, which read `docs/assets/results_overview.py`'s `ROWS`; that script
+ also draws `results_overview.*`, which the README no longer shows): quoted from the paper's
+ Table 1 and attributed to it. Change them only with the paper, and redraw both figures with
`python docs/assets/results_overview.py` and `python docs/assets/teaser.py --plex FONTS`
(its docstring says where to get IBM Plex). The teaser's middle act is a schematic with
- made-up tasks and no numbers.
+ made-up sessions and no numbers.
- The plugin's own text (`plugin/`, its manifests, slash commands and skills, and what the
hooks print) makes no claims of effect.
- Tests: `python -m pytest tests experiments/tests`. A change to what the plugin learns or
diff --git a/README.md b/README.md
index 5a0025b..eb6a552 100644
--- a/README.md
+++ b/README.md
@@ -34,84 +34,85 @@
<p align="center">
<picture>
<source media="(prefers-reduced-motion: reduce)" srcset="docs/assets/teaser.png">
- <img src="docs/assets/teaser.gif" width="100%" alt="An animation that opens on a title card: TokenEconomizer, experience-driven black-box cost control for coding agents, with its three mechanisms (micro-skills, tool pruning, failure-loop intervention), the agents it works with (Claude Code, Codex, GitHub Copilot CLI, Pi) and the UC Berkeley and Microsoft logos. Then a schematic of six tasks in one repository, each a bar of what it cost: the work itself plus three kinds of waste, namely re-learning the repo, tools never used, and getting stuck in loops. Each of the first three tasks teaches TokenEconomizer one thing, in order: 1 micro-skills remember how the repo works (run tests with python -m pytest), 2 tool pruning skips tools never used here (Read and Edit kept, Notebook and Browser left out), and 3 failure-loop intervention spots loops and steps in (the same error three times meets a nudge to try another way). In the task after each lesson that waste shrinks, so the bars step down below a dashed line marking the cost without TokenEconomizer. It ends on results measured on SWE-Bench tasks: 18.4–30.9% lower mean cost across five agent–model combinations, cheaper than default in 30 of 30 repository × combination cells, and Pass@1 up in 4 of 5; per combination, cost 23.6% lower and Pass@1 +1.3 for Claude Code with Sonnet 5, 30.9% and −1.0 with Opus 4.8, 24.7% and +0.9 for Codex with GPT-5.6 Terra, 26.1% and +1.6 with GPT-5.6 Sol, and 18.4% and +2.6 for Copilot CLI with Sonnet 5.">
+ <img src="docs/assets/teaser.gif" width="100%" alt="An animation that opens on a title card: TokenEconomizer, experience-driven black-box cost control for coding agents, with its three mechanisms (micro-skills, tool pruning, failure-loop intervention), the agents it works with (Claude Code, Codex, GitHub Copilot CLI, Pi) and the UC Berkeley and Microsoft logos. Then a schematic of six sessions in one repository, each a bar of what it cost: the work itself plus three kinds of waste, namely re-learning the repo, tools never used, and getting stuck in loops. Each of the first three sessions teaches TokenEconomizer one thing, in order: 1 micro-skills remember how the repo works (run tests with python -m pytest), 2 tool pruning skips tools never used here (Read and Edit kept, Notebook and Browser left out), and 3 failure-loop intervention spots loops and steps in (the same error three times meets a nudge to try another way). In the session after each lesson that waste shrinks, so the bars step down below a dashed line marking the cost without TokenEconomizer. It ends on results measured on SWE-Bench tasks: 18.4–30.9% lower mean cost across five agent–model combinations, cheaper than default in 30 of 30 repository × combination cells, and Pass@1 up in 4 of 5; per combination, cost 23.6% lower and Pass@1 +1.3 for Claude Code with Sonnet 5, 30.9% and −1.0 with Opus 4.8, 24.7% and +0.9 for Codex with GPT-5.6 Terra, 26.1% and +1.6 with GPT-5.6 Sol, and 18.4% and +2.6 for Copilot CLI with Sonnet 5.">
</picture>
</p>
-TokenEconomizer is a plugin for **Claude Code**, **Codex**, the **GitHub Copilot CLI** and
-**Pi**. It leaves the agent as it is, reads each finished session in a repository, and changes
-what the next sessions in that repository pay for:
-
-- **Micro-skills.** Repository knowledge your sessions keep needing, in any language -- that a
- build step must run before anything else works, which spelling of a tool works here (`npx tsc`
- where `tsc` is missing, `python -m pytest` where `pytest` is), an environment variable a
- command needs, where generated files live -- is learned from finished sessions and stated at
- the start of the next one. A fact is kept only while later sessions confirm it. Every *k*
- sessions (*k* = 3 by default) a model, called through your own agent, proposes facts, and that
- cost is counted with your sessions'. How much it helps depends on the repository's state and
- the tasks you run in it.
-- **Tool pruning.** A tool that stays loaded but goes unused over *p* prompts across at least *m*
- sessions, or over *s* sessions however short (by default *p* = 10, *m* = 2, *s* = 5), is left
- out of the next sessions through the agent's own settings. By default this happens without
- asking you: the next session start says which tools were left out, and one command gives them
- back. Set `toolprune=propose` to approve each deny list yourself instead.
-- **Failure-loop intervention.** A turn that keeps hitting the same failures is put to a judge,
- a model called through your own agent, and gets a redirect only if the judge finds it stuck
- with confidence *c* or more, at most *r* times per turn (by default *c* = 0.7, *r* = 1). If it
- is still stuck after that, the plugin tells you how to stop it; it never stops a turn itself.
-
-Each mechanism can be switched on or off on its own, and each letter in italics is a setting you
-can change ([Settings](#settings) names them all). The plugin's actions are announced in one
-line, and every hook fails open. Its only model calls are micro-skills' proposer and the loop
-guard's judge, made through your own agent. What it learns stays on your machine, under
-`~/.tokeneconomizer`.
+TokenEconomizer is a plugin that makes **Claude Code**, **Codex**, the **GitHub Copilot CLI**
+and **Pi** cheaper to run in the repositories you work in often. It does not change the agent;
+it learns from your sessions. Its unit is the *session*: one conversation with your agent, from
+start to exit, however many tasks you do in it. After each session in a repository ends, the
+plugin reads its log, and it changes what the next sessions there start with. It has three
+mechanisms:
+
+- **Micro-skills: remember how the repository works.** Sessions keep rediscovering the same
+ facts: a build step that must run first, the spelling of a command that works here
+ (`python -m pytest` where `pytest` is missing, `npx tsc` where `tsc` is), an environment
+ variable a command needs, where generated files live. Micro-skills learns such facts, in any
+ language, from two sources: failures that repeat across sessions, and a model that reviews the
+ recent sessions every *k* sessions (*k* = 3) and proposes facts. A fact is stated at the start
+ of each session once later sessions confirm it, and dropped when one contradicts it.
+- **Tool pruning: stop loading tools that go unused.** Every loaded tool is sent with every
+ request. A tool that goes unused over *p* prompts spread across at least *m* sessions, or over
+ *s* sessions of any length (*p* = 10, *m* = 2, *s* = 5), is left out of the next sessions
+ through the agent's own settings. One command brings it back.
+- **Failure-loop intervention (the loop guard): step in when a turn is stuck.** When a turn (the
+ agent's work on one prompt) keeps hitting the same failures, a judge model checks whether it is
+ stuck. If the judge is at least *c* sure (*c* = 0.7), the agent gets a short redirect, at most
+ *r* times per turn (*r* = 1). If the turn is still stuck after that, the plugin tells you how to
+ stop it; it never stops a turn itself.
+
+Each italic letter is a setting, shown with its default ([Settings](#settings)), and each
+mechanism can be turned on or off. The plugin announces each action in one line, and if any part
+of it fails, your session carries on as if it were not installed. Its only model calls (the fact
+proposer and the judge) run through your own agent, on the same plan or login, and their cost is
+included in its reports. What it learns stays on your machine, in `~/.tokeneconomizer`.
## Results
-<p align="center"><img src="docs/assets/results_overview.png" width="100%" alt="SWE-Bench results from the paper. Mean cost per task, lower than default: Claude Code with Sonnet 5, 23.6%; Claude Code with Opus 4.8, 30.9%; Codex with GPT-5.6 Terra, 24.7%; Codex with GPT-5.6 Sol, 26.1%; Copilot CLI with Sonnet 5, 18.4%. Pass@1 changes by +1.3, −1.0, +0.9, +1.6 and +2.6 points."></p>
-
-The paper runs five agent–model combinations on the same 277 SWE-Bench tasks from six
-repositories, each repository's tasks in order and starting from no traces; costs include every
-call TokenEconomizer made. It was cheaper than default in each of the 30 repository ×
-combination cells, by 5–40%, and the savings carried over to Web-Bench (16–21% lower) and to two
-security-auditing benchmarks where every task is a new repository (6–20% lower). The
-[project page](https://tokeneconomizer.github.io) has interactive charts, the per-repository
-numbers, and nine annotated runs; [`experiments/`](experiments/README.md) reruns the SWE-Bench and
+The paper ran five agent–model combinations on the same 277 SWE-Bench tasks from six repositories.
+Each task ran as its own session, and each repository's tasks ran in order, starting with nothing
+learned. Costs include every model call TokenEconomizer made. Within each repository and
+combination, cost fell by 5–40%. The savings held on Web-Bench (16–21% lower) and on two
+security-auditing benchmarks where every task is in a new repository (6–20% lower). The
+[project page](https://tokeneconomizer.github.io) has interactive charts, per-repository numbers
+and nine annotated runs, and [`experiments/`](experiments/README.md) reruns the SWE-Bench and
Web-Bench experiments.
## How it works
<p align="center"><img src="docs/assets/framework_overview.png" width="100%" alt="TokenEconomizer: the plugin keeps what it knows about a repository and applies micro-skills, tool pruning and failure-loop intervention to your unchanged coding agent; each finished session's log updates all three"></p>
-Coding agents are built for task success, not for what a task costs, and every turn re-bills
-the agent's instructions, every loaded tool, and the conversation so far. Developers use the same
-agent on the same codebase over many tasks, so much of that cost recurs: the same environment
-facts rediscovered, the same unused tools re-sent, the same kind of failing loop. TokenEconomizer
-holds the agent fixed -- no change to its model, weights or code -- and treats a repository's
-stream of sessions as the unit to optimize: what finished sessions show about the repository
-shapes what later sessions are given, and anything the plugin spends is amortized over the stream.
-Four principles shape the mechanisms:
-
-- **Act through what the agent already reads.** Facts go into the context at session start and
- tools leave through the agent's own settings, so nothing depends on the agent adopting a new
- tool.
+Coding agents are built to finish tasks, not to keep them cheap. Every turn pays again for the
+agent's instructions, every loaded tool, and the conversation so far. Developers use the same
+agent on the same repository again and again, so much of that cost repeats: the same facts
+rediscovered, the same unused tools sent, the same kinds of failing loops. TokenEconomizer leaves
+the agent's model, weights and code alone and learns across a repository's sessions instead of
+treating each one on its own, so whatever the plugin spends is paid back over the sessions that
+follow. Four principles guide the mechanisms:
+
+- **Work through what the agent already reads.** Facts enter the context at session start, and
+ tools are removed through the agent's own settings, so nothing depends on the agent learning a
+ new tool.
- **Count cost as it is billed.** Most tokens in a session are cached re-reads, so cost is
- measured on a cache-aware basis, including any model calls the plugin makes.
-- **Keep task success.** Facts form only from recurring evidence, tools are pruned only after
- going unused, and the loop guard spares a turn that is still making progress.
-- **Recover from mistakes.** A fact a later session contradicts is retired, an applied deny list
- can be restored, and the guard only redirects and tells you: stopping a turn is your call.
+ computed with cache pricing, and it includes the plugin's own model calls.
+- **Protect task success.** Facts form only from repeated evidence, tools are removed only after
+ going unused, and the loop guard leaves alone a turn that is still making progress.
+- **Keep every action reversible.** A fact that a later session contradicts is dropped, removed
+ tools can be given back, and the loop guard only redirects and tells you: stopping a turn is
+ your call.
## Quick start
You need one of the four agents, signed in any way it supports, and Python 3.10 or newer on
-`PATH`. The plugin runs inside your agent's own sessions and never handles credentials, so any
-plan or login works, and it needs no API key of its own.
+`PATH`. The plugin needs no API key of its own and never handles credentials, so any plan or
+login works.
### Install
-Run the shell commands where the clone should stay: Claude Code, Codex and the Copilot CLI run
-the plugin from it. Then start the agent inside one of your repositories.
+Run these commands in the folder where you want to keep the clone. Claude Code, Codex and the
+Copilot CLI run the plugin from the clone, so leave it in place. Then start the agent in one of
+your repositories.
**Claude Code**
@@ -157,9 +158,10 @@ pi install git:github.com/microsoft/TokenEconomizer
Start `pi` and run `/te status`.
-The status command prints a short status block. `refresh-catalog` records which tools a session
-loads, with no model call, so that tool pruning sees all of them. The marketplace path must be
-absolute: the Copilot CLI reads `./TokenEconomizer` as the name of a GitHub repository.
+If the install worked, the status command prints a short status block. `refresh-catalog`
+records every tool a session loads (without a model call), so that tool pruning can consider all
+of them. Keep the marketplace path absolute, as above: the Copilot CLI reads a relative path such
+as `./TokenEconomizer` as the name of a GitHub repository.
To update:
@@ -169,8 +171,8 @@ To update:
| Codex | `git pull` in the clone, then the `codex plugin add` line again |
| Pi | `pi update --extensions` |
-Uninstalling, Copilot with your own model key, and troubleshooting are in
-**[`plugin/README.md`](plugin/README.md)**, which walks through each agent.
+**[`plugin/README.md`](plugin/README.md)** walks through each agent in more detail, including
+uninstalling, using Copilot with your own model key, and troubleshooting.
### Supported versions
@@ -181,8 +183,8 @@ Uninstalling, Copilot with your own model key, and troubleshooting are in
| Copilot CLI | 1.0.83 to 1.0.102 | not supported |
| Pi | 1.2.0 to 1.3.0 | 0.99.x: load the extension by path (below) |
-The commands above were checked on both ends of each range; later versions have not been
-checked.
+The commands above were tested on the oldest and newest version in each range; newer versions
+have not been tested yet.
- **Codex before 0.141.0** does not run plugin hooks at all.
- **Pi 0.99.x**: instead of `pi install`, clone the repository and start each session with
@@ -191,8 +193,8 @@ checked.
### Everyday commands
-Work as usual; the plugin needs nothing from you. Its commands have the same names on every
-agent; only how you type them differs:
+Work as usual; the plugin needs nothing from you. To see or change what it does, use its
+commands. They have the same names on every agent; only the way you type them differs:
| Agent | Type a command as |
|---|---|
@@ -210,52 +212,54 @@ agent; only how you type them differs:
| `config` | every setting in effect |
| `config set microskills=on` | turn a mechanism on (or `=off`) here; add `--global` for every repository |
-Inside an agent each command is one model turn; from a terminal it costs nothing. Every other
-command (`repos`, `doctor`, `config unset`, ...) and how each agent asks before running one:
+Inside an agent, a command costs one model turn; from a terminal it is free. The other commands
+(`repos`, `doctor`, `config unset`, ...) and each agent's permission prompts are listed under
[Commands](plugin/README.md#commands).
## Settings
-Tool pruning and the loop guard are on from the start. Micro-skills is off until you turn it on
-(`config set microskills=on`): it adds the proposer's model calls, and what it gains depends on
-the repository and the tasks you run in it. Each mechanism takes `on` or `off`, and the letters
-used above are settings you change with the same `config set` command (in Claude Code's form
-here):
+Tool pruning and the loop guard are on by default; with these defaults, the plugin's only model
+call is the judge, asked only about a turn that shows signs of looping. Micro-skills is off by
+default, because its proposer adds model calls and how much it saves depends on the repository
+and the work you do there. Each mechanism takes `on` or `off`, and the italic letters above are
+these settings:
| | Setting | Default | What it sets |
|---|---|---|---|
| *k* | `microskills_propose_every` | 3 | sessions between runs of micro-skills' fact proposer |
-| *p* | `toolprune_evidence` | 10 | prompts a never-used tool must go unused before it is left out |
+| *p* | `toolprune_evidence` | 10 | prompts a tool must go unused before it is left out |
| *m* | `toolprune_min_sessions` | 2 | sessions those prompts must span |
-| *s* | `toolprune_sessions` | 5 | or this many sessions, however short |
+| *s* | `toolprune_sessions` | 5 | or this many sessions, of any length |
| *q* | `toolprune_quiet_minutes` | 60 | minutes your sessions here must be idle before a deny list is written |
| *e* | `loopguard.envelope_min_samples` | 5 | turns here that ended with passing checks before the loop guard acts |
| *c* | `loopguard.judge_confidence` | 0.7 | how sure the judge must be that a turn is stuck |
| *r* | `loopguard_nudges` | 1 | redirects per turn before the guard only tells you |
-| *b* | `loopguard.backstop_mult` | 2.5 | how many times the usual turn length a churning turn may run before it is redirected without the judge |
+| *b* | `loopguard.backstop_mult` | 2.5 | how many times its usual length a churning turn may run before it is redirected without the judge |
-Prompts let long sessions count for more; the *m*-session floor keeps one session's work from
-deciding alone. The deny list is written once your sessions in the repository have been idle
-for *q* minutes: a conversation resumed after its tools change writes its whole history to the
-prompt cache again instead of reading it, and after an idle hour that cache has expired anyway.
-With `toolprune=propose` the plugin only proposes the list, and nothing changes until you run
-`toolprune apply`.
+Change them with `config set`, shown here in Claude Code's form (other agents type it as in
+[Everyday commands](#everyday-commands)):
```text
-/tokeneconomizer:config set microskills=on # learn and state repository facts
+/tokeneconomizer:config set microskills=on # turn micro-skills on
/tokeneconomizer:config set microskills_propose_every=5 # k = 5: propose facts less often
/tokeneconomizer:config set toolprune_evidence=20 # p = 20: wait for twenty unused prompts
/tokeneconomizer:config set loopguard_nudges=2 # r = 2: redirect twice before telling you
```
-For one session only, set an environment variable when you start the agent:
-`TE_DISABLE=1 claude` turns the whole plugin off, and `TE_LOOPGUARD=off codex` (likewise
-`TE_MICROSKILLS`, `TE_TOOLPRUNE`) one mechanism. The plugin's model calls run through your own
-agent's CLI, on the same plan or login as your sessions. With the defaults the only one is the
-loop guard's judge, asked only about a turn that shows signs of looping; turning micro-skills on
-adds its proposer (`microskills=envfacts` keeps only the facts found without a model call), and
-`loopguard=off` turns the judge off with the guard. Every setting, its default and its
-environment variable: [Settings](plugin/README.md#settings).
+Tool pruning counts prompts so that long sessions weigh more, and requires *m* sessions so that a
+single session never decides alone. It writes the deny list only once your sessions in the
+repository have been idle for *q* minutes: a conversation resumed after its tools change must
+write its whole history to the prompt cache again instead of reading it, and after an idle hour
+that cache has expired anyway.
+
+Besides `on` and `off`, two modes are useful: `toolprune=propose` only proposes the deny list, and
+nothing changes until you run `toolprune apply`; `microskills=envfacts` learns facts only from
+repeated failures, with no model calls.
+
+To change a single session, set an environment variable when you start the agent:
+`TE_DISABLE=1 claude` turns the whole plugin off, and `TE_LOOPGUARD=off codex` turns off one
+mechanism (likewise `TE_MICROSKILLS` and `TE_TOOLPRUNE`). Every setting, its default and its
+environment variable are listed under [Settings](plugin/README.md#settings).
## What it does on each agent
@@ -269,33 +273,31 @@ agent's own settings, so what it can leave out differs:
| Copilot CLI | whole MCP servers | `.github/copilot/settings.local.json` |
| Pi | nothing: Pi loads four tools | — |
-Tool pruning writes only the entries it adds and never touches deny rules you wrote; a settings
-file it creates is kept out of git (`.git/info/exclude`). The next session start says what it
-left out. On Claude Code that line shows only in the detailed transcript (**Ctrl+O**), so
-`/tokeneconomizer:toolprune` is the place to check. `toolprune restore` removes the plugin's
-entries and no others, and what you give back stays back. If the plugin cannot edit a file
-safely (a `.codex/config.toml` your repository commits, a settings file that is not valid JSON,
-a path through a symlink), the list stays a proposal and the session start says why;
-`toolprune apply` tries again. How each agent's hooks and settings were verified to behave is in
-**[`docs/PLUGIN.md`](docs/PLUGIN.md)**, and how the package fits together in
-**[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)**.
+Tool pruning adds only its own entries and never touches deny rules you wrote; a settings file
+it creates is kept out of git (`.git/info/exclude`). On Claude Code, the session-start line that
+says what it left out shows only in the detailed transcript (**Ctrl+O**), so check with
+`/tokeneconomizer:toolprune`. `toolprune restore` removes the plugin's entries and nothing else,
+and tools you give back are not removed again. If the plugin cannot edit a file safely (a
+`.codex/config.toml` your repository commits, a settings file that is not valid JSON, a path
+through a symlink), the list stays a proposal and the session start says why; `toolprune apply`
+tries again. **[`docs/PLUGIN.md`](docs/PLUGIN.md)** records how each agent's hooks and settings
+were verified to behave, and **[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)** how the package
+fits together.
### When a turn gets stuck
-Once *e* turns in a repository have ended with passing checks, the loop guard knows how long a
-turn there usually runs. From then on, when a turn shows signs of looping -- the same failures
-repeating, or errors piling up -- the guard asks its judge, and only if the judge finds the turn
-stuck (confidence *c* or more) does it add a redirect to the model's context, at most *r* per
-turn. (One exception: a turn past *b* times this repository's usual length that also keeps
-churning the same edits through the same failures is redirected even if the judge disagrees.)
-If the turn is still stuck after that, it shows you one line, ending with how to stop the turn:
+The loop guard first learns how long a turn in this repository usually runs: it waits until *e*
+turns here have ended with passing checks. After that, a turn that shows signs of looping (the
+same failures repeating, or errors piling up) goes to the judge. One case is redirected without
+the judge: a turn that runs past *b* times the usual length while churning the same edits through
+the same failures. If a turn is still stuck after its redirects, you see one line that ends with
+how to stop it:
> TokenEconomizer: this turn still looks stuck after a redirect (envelope, 12 tool calls). Press
> Esc to interrupt it.
-It never stops a turn itself; whether to stop is up to you. Where the line shows, and how you
-stop, in the agent's terminal UI and in a headless run (`claude -p`, `codex exec`, `copilot -p`,
-`pi -p`):
+Where that line appears, and how to stop the turn, in the terminal UI and in a headless run
+(`claude -p`, `codex exec`, `copilot -p`, `pi -p`):
| Agent | Interactive: shown | Stop with | Headless: shown | Stop with |
|---|---|---|---|---|
@@ -304,15 +306,15 @@ stop, in the agent's terminal UI and in a headless run (`claude -p`, `codex exec
| Copilot CLI | in the conversation | Esc twice | on stdout | Ctrl+C |
| Pi | as a notification | Esc | on stderr | Ctrl+C |
-- **Claude Code** keeps a plugin's own lines to its detailed transcript (Ctrl+O), so the plugin
- asks Claude to repeat the line; a model may word it differently or leave it out.
+- **Claude Code** shows a plugin's own lines only in its detailed transcript (Ctrl+O), so the
+ plugin asks Claude to repeat this one in its reply; the model may word it differently or leave
+ it out.
- **Codex**: Esc interrupts the turn, but a command it started keeps running until `/stop`.
`codex exec` prints no hook messages.
- **Pi**: Ctrl+C ends a headless run, but a command it started runs on until it finishes.
-In a headless run nobody may be watching, so a flagged turn keeps running until it ends by
-itself or reaches the agent's own limits; `te status` counts the redirects and "still stuck"
-notices afterwards.
+In a headless run nobody may be watching, so a flagged turn runs until it ends or reaches the
+agent's own limits; afterwards, `te status` counts the redirects and "still stuck" notices.
## Repository
@@ -337,7 +339,7 @@ The package is standard-library only. Run the tests with `python -m pytest tests
The code behind the paper's results -- a harness that runs Claude Code, Codex and the Copilot CLI
on SWE-Bench and Web-Bench, the launchers, and the evaluated task sets -- is in
**[`experiments/`](experiments/README.md)**, whose README gives the setup and one command per
-result. It is not packaged with the plugin and shares only `tokeneconomizer/`.
+result. It is not part of the installed plugin; the two share only `tokeneconomizer/`.
## Citation
diff --git a/docs/assets/results_overview.py b/docs/assets/results_overview.py
index d99574e..144a18a 100644
--- a/docs/assets/results_overview.py
+++ b/docs/assets/results_overview.py
@@ -1,5 +1,6 @@
#!/usr/bin/env python3
-"""The README's results figure: the paper's SWE-Bench headline, per agent-model combination.
+"""The results figure: the paper's SWE-Bench headline, per agent-model combination. Its ROWS
+are also the README teaser's closing numbers (teaser.py); the README no longer shows the figure.
python docs/assets/results_overview.py # needs matplotlib; writes the PNG and SVG here
diff --git a/docs/assets/teaser.gif b/docs/assets/teaser.gif
index a97e862..054fb28 100644
Binary files a/docs/assets/teaser.gif and b/docs/assets/teaser.gif differ
diff --git a/docs/assets/teaser.png b/docs/assets/teaser.png
index b77029f..fadfb7b 100644
Binary files a/docs/assets/teaser.png and b/docs/assets/teaser.png differ
diff --git a/docs/assets/teaser.py b/docs/assets/teaser.py
index 3ce4f44..8e3bf94 100644
--- a/docs/assets/teaser.py
+++ b/docs/assets/teaser.py
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
-"""The teaser: TokenEconomizer's title card, how it trims the waste of a repository's tasks, and
+"""The teaser: TokenEconomizer's title card, how it trims the waste of a repository's sessions, and
the paper's headline results.
python docs/assets/teaser.py --plex FONTS # the README's: teaser.gif and teaser.png
@@ -15,13 +15,15 @@ Three acts, crossfaded:
Title the page hero's two-tone name, the paper's subtitle, the three mechanisms in the paper's
order, the agents it works with, and the authors' institutions' logos (logo_*.png: the
project page's SVGs, rasterized with a transparent background, as Pillow draws no SVG).
-Method a sketch, not a measurement: six made-up tasks, alike, each a bar of the work itself and
- three kinds of waste, named beside the first bar, under a dashed line at what a task costs
- without TokenEconomizer. On the right, what TokenEconomizer has learned: tasks 1-3 each
- fly into it and teach it one mechanism, with a plain example; in the next task that waste
- shrinks while its card lights up, and the last tasks start lean. No number in it comes
- from a run (runs stay out of the repository); the waste is sized so the last tasks land
- inside the paper's range of savings, and nothing is labelled with a number.
+Method a sketch, not a measurement: six made-up sessions, alike, each a bar of the work itself
+ and three kinds of waste, named beside the first bar, under a dashed line at what a
+ session costs without TokenEconomizer. Sessions, not tasks: the plugin learns from each
+ finished session, however many tasks it held. On the right, what TokenEconomizer has
+ learned: sessions 1-3 each fly into it and teach it one mechanism, with a plain example;
+ in the next session that waste shrinks while its card lights up, and the last sessions
+ start lean. No number in it comes from a run (runs stay out of the repository); the
+ waste is sized so the last sessions land inside the paper's range of savings, and
+ nothing is labelled with a number.
Results the paper's SWE-Bench headline, as the README's line under the badges and the project
page's tiles state it, and per agent-model combination the cost cut and the change in
Pass@1. The numbers are results_overview.py's ROWS (the paper's Table 1), not copied.
@@ -107,21 +109,21 @@ def page_look(fonts):
L = None # the look being drawn; main() sets it, with the fonts it is given
-# ---- the tasks ------------------------------------------------------------------------------
-# Every task does the same work, and wastes the same three things until TokenEconomizer has
+# ---- the sessions ---------------------------------------------------------------------------
+# Every session does the same work, and wastes the same three things until TokenEconomizer has
# learned how not to: cost units, the full amount and what is left once it has.
N = 6
WORK = 160
MECHS = ("facts", "prune", "guard") # in the paper's order, top to bottom on each bar
FULL = {"facts": 40, "prune": 44, "guard": 40}
LEFT = {"facts": 8, "prune": 22, "guard": 16}
-LEARNED_AFTER = {"facts": 1, "prune": 2, "guard": 3} # the task that teaches it
+LEARNED_AFTER = {"facts": 1, "prune": 2, "guard": 3} # the session that teaches it
NAME = {"facts": "Micro-skills", "prune": "Tool pruning", "guard": "Failure-loop intervention"}
WASTE = {"facts": "re-learning the repo", "prune": "tools never used",
"guard": "stuck in loops"}
DOES = {"facts": "Remembers how the repo works", "prune": "Skips tools never used here",
"guard": "Spots loops and steps in"}
-CAPTION = {"facts": "told to the agent as each task starts",
+CAPTION = {"facts": "told to the agent as each session starts",
"prune": "unused tools are no longer loaded",
"guard": "a nudge when the same error repeats"}
@@ -143,8 +145,8 @@ PITCH = (CX1 - BX0) / N
BAR_W = 46
BASE_Y = 528 # the bars' baseline
UNIT = 1.52 # pixels per cost unit
-TASK_Y = 550
-FULL_Y = BASE_Y - (WORK + sum(FULL.values())) * UNIT # a task without TokenEconomizer
+SESSION_Y = 550
+FULL_Y = BASE_Y - (WORK + sum(FULL.values())) * UNIT # a session without TokenEconomizer
LEAN_Y = BASE_Y - (WORK + sum(LEFT.values())) * UNIT
PX0, PX1, PY0, PY1 = 704, 1088, 24, 576 # TokenEconomizer's panel
@@ -227,8 +229,8 @@ def rgb(color):
def amount(m, i, f):
- """What task i spends on waste m at frame f: all of it until TokenEconomizer has learned
- from an earlier task, shrinking in the first task after, and what is left from then on."""
+ """What session i spends on waste m at frame f: all of it until TokenEconomizer has learned
+ from an earlier session, shrinking in the first session after, and what is left from then on."""
after = LEARNED_AFTER[m]
if after >= i:
return FULL[m]
@@ -238,7 +240,7 @@ def amount(m, i, f):
def trim_glow(m, f):
- """How lit m's card is while the task after it learned shrinks m's waste: 0 to 1 and back."""
+ """How lit m's card is while the session after it learned shrinks m's waste: 0 to 1 and back."""
i = LEARNED_AFTER[m] + 1
if i > N or ("trim", i) not in AT:
return 0.0
@@ -368,27 +370,27 @@ class Painter:
def draw_static(p):
"""What never moves in the method: the title, the baseline, the panel and its header."""
- title = "Each finished task makes the next ones waste less"
+ title = "Each finished session makes the next ones waste less"
p.text((CX0, TITLE_Y), title, 22, L.ink, "bold", anchor="ls")
assert CX0 + p.width(title, 22, "bold") <= CX1, "the title outgrows the chart"
p.line([(BX0 - 8, BASE_Y), (CX1, BASE_Y)], L.faint, 1.4)
p.rect((PX0, PY0, PX1, PY1), L.panel, L.line, 1.6, radius=14)
p.text((CARD_X0 + 4, 56), "TokenEconomizer", 20, L.ink, "bold")
- p.text((CARD_X0 + 4, 82), "learns from each finished task", 15, L.muted)
+ p.text((CARD_X0 + 4, 82), "learns from each finished session", 15, L.muted)
p.line([(CARD_X0, 100), (CARD_X1, 100)], L.line, 1.2)
def stack(i, f):
- """Task i's bar, bottom to top, as (units, color, waste or None)."""
+ """Session i's bar, bottom to top, as (units, color, waste or None)."""
return [(WORK, L.work, None)] + [(amount(m, i, f), color(m), m) for m in reversed(MECHS)]
def draw_bar(p, i, f):
start = AT["grow", i]
grow = GROW_FIRST if i == 1 else GROW
- p.text((col_x(i), TASK_Y), f"task {i}", 14, mix(L.faint, L.ink, clamp((f - start) / 4)),
+ p.text((col_x(i), SESSION_Y), f"session {i}", 14, mix(L.faint, L.ink, clamp((f - start) / 4)),
anchor="mm")
- assert p.width(f"task {i}", 14) <= PITCH - 8, "task labels meet"
+ assert p.width(f"session {i}", 14) <= PITCH - 8, "session labels meet"
if f < start:
return
segs = stack(i, f)
@@ -431,7 +433,7 @@ def draw_names(p, segs, shown):
def draw_without(p, f):
- """The dashed line at a task's full cost, drawn when TokenEconomizer first trims one."""
+ """The dashed line at a session's full cost, drawn when TokenEconomizer first trims one."""
if f < AT["trim", 2]:
return
x0 = col_x(1) + BAR_W / 2 + 6
diff --git a/plugin/README.md b/plugin/README.md
index a78ae99..6658e37 100644
--- a/plugin/README.md
+++ b/plugin/README.md
@@ -59,38 +59,35 @@ This records which tools a Claude Code session loads, with no model call. Do it
machine, and again after Claude Code updates. **Don't skip it:** without it the deny list sees
only the few tools Claude Code defers, not the ones every session loads.
-**4. Work normally.** See [What to expect](#what-to-expect). Once a tool has gone unused long
-enough (by default ten prompts across two or more sessions, or five sessions however short), the
-plugin leaves it out of the next sessions without asking you (it writes the list once your
-sessions here have been idle, an hour by default; each number is a [setting](#settings)), and
-the session start says so once:
+**4. Work normally.** When the plugin leaves tools out ([What to expect](#what-to-expect) says
+when), the next session start says so once. To see or undo it:
```text
/tokeneconomizer:toolprune # the tools it left out, and the evidence
/tokeneconomizer:toolprune restore # give them back, from your next session; they stay back
```
-The rules go into this repository's `.claude/settings.local.json`, which the plugin keeps out of
-git, and it never touches deny rules you wrote. To approve each deny list yourself instead:
+The deny rules go into this repository's `.claude/settings.local.json`, which the plugin keeps
+out of git; deny rules you wrote are never touched. To approve each deny list yourself instead:
`/tokeneconomizer:config set toolprune=propose`, then `/tokeneconomizer:toolprune apply`.
-**Where its messages appear.** Claude Code shows the plugin's one-line messages (facts
-added, tools left out) only in its detailed transcript: press **Ctrl+O**, where they read
-`<event> says: TokenEconomizer: …`. A turn that still looks stuck is the exception:
-the plugin also asks Claude to repeat that line word for word, so it shows in Claude's reply,
-and under `claude -p` in its final reply. That is a request to the model, which may word it
-differently or leave it out; the detailed transcript always has it. To stop a turn, press
-**Esc**; under `claude -p`, Ctrl+C ends the run. With `--output-format stream-json --verbose`
-every message also arrives as a `hook_response` event.
+**Where its messages appear.** Claude Code shows the plugin's one-line messages (facts added,
+tools left out) only in its detailed transcript: press **Ctrl+O**, where they read
+`<event> says: TokenEconomizer: …`. The exception is a turn that still looks stuck: the plugin
+also asks Claude to repeat that line word for word, so it shows in Claude's reply (under
+`claude -p`, in the final reply). Claude may word it differently or leave it out; the detailed
+transcript always has it. Press **Esc** to stop the turn; under `claude -p`, Ctrl+C ends the
+run. With `--output-format stream-json --verbose`, every message also arrives as a
+`hook_response` event.
-Each slash command is one model turn. The same commands are free from a terminal, as
+Each slash command costs one model turn; the same commands are free from a terminal, as
`~/.tokeneconomizer/bin/te --host claude <command>` run inside the repository
-([Commands](#commands)). For one session without installing anything:
+([Commands](#commands)). To try the plugin for one session without installing it:
`claude --plugin-dir /path/to/TokenEconomizer/plugin`.
-**Update.** `git pull` in the clone, then start a new session (or `/reload-plugins`). Claude Code
-runs a plugin added from a local folder from that folder. If `/tokeneconomizer:status` still
-shows the old behaviour, run `claude plugin update tokeneconomizer@tokeneconomizer`.
+**Update.** `git pull` in the clone, then start a new session (or `/reload-plugins`): Claude Code
+runs the plugin straight from the clone. If `/tokeneconomizer:status` still shows the old
+behaviour, run `claude plugin update tokeneconomizer@tokeneconomizer`.
**Uninstall.**
@@ -114,37 +111,38 @@ codex plugin add tokeneconomizer@tokeneconomizer
Hooks run from the next session on (the session in which you approve gets no session-start
injection), so quit with `/exit` and start `codex` again. To approve later: `/hooks`.
-**3. Check it.** Type `$tokeneconomizer:status`. Codex has no plugin slash commands; the plugin's
-commands are Codex skills, named like the slash commands on the other agents (type
-`$tokeneconomizer` to pick one): `$tokeneconomizer:status`, `:microskills`, `:toolprune`,
-`:config` and `:repos`, each followed by its arguments (`$tokeneconomizer:config set
-loopguard=off`). Name the skill: a question in plain words (*"what has TokenEconomizer learned
-here?"*) may be answered from the plugin's files without running its command. A command that
-changes something (`config set` or `unset`, `toolprune apply` or `restore`, `repos migrate` or
-`merge`) writes the plugin's state in `~/.tokeneconomizer`, which Codex's sandbox keeps
-read-only, so Codex asks first: choose **Yes, proceed**. From a terminal, inside the repository:
+**3. Check it.** Type `$tokeneconomizer:status`. Codex has no plugin slash commands, so the
+plugin's commands are Codex skills with the same names as on the other agents:
+`$tokeneconomizer:status`, `:microskills`, `:toolprune`, `:config` and `:repos`, each followed by
+its arguments (`$tokeneconomizer:config set loopguard=off`); type `$tokeneconomizer` to pick one.
+Name the skill rather than asking in plain words: a question such as *"what has TokenEconomizer
+learned here?"* may be answered from the plugin's files without running its command. Commands
+that change something (`config set` or `unset`, `toolprune apply` or `restore`, `repos migrate`
+or `merge`) write to `~/.tokeneconomizer`, which Codex's sandbox keeps read-only, so Codex asks
+first: choose **Yes, proceed**. From a terminal, inside the repository:
`~/.tokeneconomizer/bin/te --host codex <command>` (the launcher appears after the first
session).
-**4. Work normally.** See [What to expect](#what-to-expect). Codex's deny list turns off the
-sub-agent (`collaboration`), `goals` and image (`image_media`) packages, each only once it has
-been loaded and unused long enough ([What to expect](#what-to-expect)); a package you used even
-once is kept. It is
-written to the project's `.codex/config.toml`, which Codex reads **only in a trusted project**
-(the session start that announces it says so otherwise), and never into one your repository
-commits: that stays a proposal, and the session start says why. The web, asking you questions
-and the bundled skills are never turned off. `$tokeneconomizer:toolprune restore` gives the
-packages back, and `toolprune measure` shows the difference with no model call.
-
-In `codex exec`, Codex does not print hook messages, so the plugin's notices are not shown
-there; a loop guard redirect still reaches the model. In the terminal UI a turn that is still
-stuck after the redirect shows "↳ Hook · TokenEconomizer: … Press Esc to interrupt it, then /stop
+**4. Work normally.** See [What to expect](#what-to-expect). On Codex the deny list can turn off
+the sub-agent (`collaboration`), `goals` and image (`image_media`) packages, each only after it
+has been loaded and gone unused long enough; a package you used even once is kept. Web access,
+asking you questions and the bundled skills are never turned off. The list is written to the
+project's `.codex/config.toml`, which Codex reads **only in a trusted project** (otherwise the
+session start that announces it says so), and never into a `config.toml` your repository
+commits: that stays a proposal, and the session start says why.
+`$tokeneconomizer:toolprune restore` gives the packages back, and `toolprune measure` shows the
+difference with no model call.
+
+**Messages.** Under `codex exec`, Codex prints no hook messages, so the plugin's notices are not
+shown, though a loop guard redirect still reaches the model. In the terminal UI, a turn still
+stuck after its redirect shows "↳ Hook · TokenEconomizer: … Press Esc to interrupt it, then /stop
to end a command it left running": Esc interrupts the turn, but a command it started keeps
running in the background until `/stop`. Under `codex exec`, Ctrl+C ends the run and its command.
-Quitting Codex's interactive mode (`/exit`) keeps the
-session open to `codex resume` and sends no session end, so the plugin learns from it at a later
-session start, once it has been quiet for ten minutes; `codex exec` sessions are learned from
-as they end.
+
+**When a session is learned from.** Quitting interactive Codex (`/exit`) keeps the session open
+for `codex resume` and sends no session end, so the plugin learns from it at a later session
+start, once it has been quiet for ten minutes. `codex exec` sessions are learned from as they
+end.
**Update.** `git pull` in the clone, then `codex plugin add tokeneconomizer@tokeneconomizer`
again to refresh Codex's installed copy (your hook approval is kept).
@@ -192,14 +190,13 @@ commands work only in interactive mode; the plugin's notices also appear under `
With no model call. Do it once on this machine, and again after Copilot updates.
-**4. Work normally.** See [What to expect](#what-to-expect). Copilot has no setting that removes a
-single built-in tool, only one that turns off a whole MCP server, so the deny list turns off a
-server only when **none** of its tools was used long enough ([What to expect](#what-to-expect)):
-one of yours from
+**4. Work normally.** See [What to expect](#what-to-expect). Copilot cannot remove a single
+built-in tool, only turn off a whole MCP server, so the deny list turns off a server only when
+**all** of its tools have gone unused long enough: one of yours from
`~/.copilot/mcp-config.json`, or the built-in `github-mcp-server`, which loads only when you are
signed in to GitHub. It is written to `.github/copilot/settings.local.json`, and the next session
-start says so; `/tokeneconomizer:toolprune restore` turns the server back on. `toolprune
-measure` shows the difference with no model call.
+start says so. `/tokeneconomizer:toolprune restore` turns the server back on, and
+`toolprune measure` shows the difference with no model call.
From a terminal, inside the repository: `~/.tokeneconomizer/bin/te --host copilot <command>` (the
launcher appears after the first session). A turn that is still stuck after the loop guard's
@@ -252,13 +249,13 @@ To check the whole plugin on your own setup:
~/.tokeneconomizer/bin/te --host copilot probe # 7 short real sessions, in a temp repo
```
-It builds a throwaway repository that is set up for pytest but has no pytest installed, and
-whose tests pass under `python -m unittest`, and runs the same prompt in real Copilot sessions
-with the plugin, under a state directory of its own. It reports, per session, whether the fact
-the plugin learned was in the context and which test command the model ran first. Then it
-applies the deny list it proposed, and shows the tool definition tokens of one more session
-before restoring. It makes real model calls on your key or quota, and leaves those sessions in
-Copilot's history.
+It builds a throwaway repository that is set up for pytest, without pytest installed, and whose
+tests pass under `python -m unittest`. It runs the same prompt in real Copilot sessions with the
+plugin, under a state directory of its own, and reports for each session whether the fact the
+plugin learned was in the context and which test command the model ran first. Then it applies the
+deny list it proposed, shows the tool-definition tokens of one more session, and restores the
+list. It makes real model calls on your key or quota, and leaves those sessions in Copilot's
+history.
## Pi
@@ -289,44 +286,47 @@ from); `pi config` turns it off without removing it. Optional: `rm -rf ~/.tokene
## What to expect
+The plugin works session by session. A session is one conversation with the agent, from start to
+exit, however many tasks you do in it; resuming it continues the same session. The plugin learns
+from a session once it ends, and acts at the start of the next one; only the loop guard acts
+during a turn.
+
- **Facts** (once you turn micro-skills on; it is off by default). Two sources propose them, in
- any language or toolchain: the same environment failure coming up in a few sessions (a tool,
- interpreter, build or test command that fails as typed and works spelled another way), with
- no model call; and, every `microskills_propose_every` sessions (3 by default), a model reading digests of the recent ones (the
- commands run, the build, test and lint commands among them, the files edited, and each
- failing command with its error line, paths relative to the repository) and writing facts
- about the repository, each with the patterns that test it against later sessions. A fact is
- injected at every session start once later sessions have confirmed it (three confirming ones,
- among at least four where it applied), and retired when one contradicts it. How much this helps
- depends on the repository's state and the tasks you run in it. A session start that adds facts
- says so once, in one line. A resumed session keeps the facts it started with, word for word, so
- its cached prompt still matches. `microskills` shows the facts injected and those forming.
-- **Deny list.** Applied without asking you once a tool, package or server never used here has
- gone unused over `toolprune_evidence` prompts (10 by default) across at least
- `toolprune_min_sessions` sessions (2), or over `toolprune_sessions` sessions however short (5)
- -- three times as long for a few higher-risk tools -- from the next session on; that session's
- start says so once. Prompts let long sessions count for more, and the session floor keeps one
- session's work from deciding alone. A session that ran no prompt counts for nothing. It is
- written once the repository's sessions have been idle for `toolprune_quiet_minutes` (60 by
- default): a conversation resumed
- after its tools change writes its whole history to the prompt cache again instead of reading
- it, and after an idle hour that cache has expired anyway. `toolprune` shows a list still
- waiting. `toolprune restore` gives back what the plugin removed, and only that, and what you
+ any language or toolchain. The first needs no model call: the same environment failure in a
+ few sessions (a tool, interpreter, build or test command that fails as typed and works spelled
+ another way). The second runs every `microskills_propose_every` sessions (3 by default): a
+ model reads digests of the recent sessions (the commands run, including build, test and lint
+ commands, the files edited, and each failing command with its error line, with paths relative
+ to the repository) and writes facts about the repository, each with patterns that test it
+ against later sessions. A fact is injected at every session start once later sessions confirm
+ it (three confirming sessions among at least four where it applied), and retired when one
+ contradicts it. How much this helps depends on the repository's state and the tasks you run in
+ it. A session start that adds facts says so once, in one line. A resumed session keeps the
+ facts it started with, word for word, so its cached prompt still matches. `microskills` shows
+ the facts injected and those still forming.
+- **Deny list.** A tool, package or server that is loaded but never used here is left out without
+ asking you, once it has gone unused over `toolprune_evidence` prompts (10 by default) across at
+ least `toolprune_min_sessions` sessions (2), or over `toolprune_sessions` sessions of any length
+ (5); a few higher-risk tools need three times as long. A session that ran no prompt does not
+ count. The list is written once the repository's sessions have been idle for
+ `toolprune_quiet_minutes` (60 by default), so that no conversation you resume loses its prompt
+ cache, and it applies from the next session, whose start says so once. `toolprune` shows a list
+ still waiting. `toolprune restore` gives back what the plugin removed, and only that; what you
give back is never removed again (`toolprune apply NAME` removes one again). With
- `toolprune=propose`, it is only proposed, the session start says so, and nothing changes
- until you run `toolprune apply`. A file the plugin creates is added to `.git/info/exclude`.
- Each checkout keeps its own settings file: a repository's other clones share what it learned,
- and each gets the list when one of its own sessions ends.
-- **Loop guard.** Quiet on normal turns, and it does nothing until `loopguard.envelope_min_samples`
- turns in a repository (5 by default) have ended with passing checks. A turn that keeps hitting
- the same failures, or piles up errors, after `loopguard.min_turns` tool calls is put to a judge
- model; only a "stuck" verdict with confidence `loopguard.judge_confidence` (0.7) or more
+ `toolprune=propose`, the list is only proposed, the session start says so, and nothing changes
+ until you run `toolprune apply`. A settings file the plugin creates is added to
+ `.git/info/exclude`. Each checkout keeps its own settings file: a repository's clones share
+ what was learned, and each gets the list when one of its own sessions ends.
+- **Loop guard.** Quiet on normal turns, and idle until `loopguard.envelope_min_samples` turns in
+ a repository (5 by default) have ended with passing checks. After that, a turn that keeps
+ hitting the same failures, or piles up errors, after `loopguard.min_turns` tool calls is put to
+ a judge model. Only a "stuck" verdict with confidence `loopguard.judge_confidence` (0.7) or more
attaches a redirect (written by that model) to a tool result, at most `loopguard_nudges` (1)
- per turn, and if the turn is still stuck after that, a one-line notice tells you how to stop
- it. A long turn with no such sign is never judged. A turn past `loopguard.backstop_mult` (2.5)
+ per turn; if the turn is still stuck after that, a one-line notice tells you how to stop it. A
+ long turn with no sign of looping is never judged. A turn past `loopguard.backstop_mult` (2.5)
times the repository's usual length that also keeps churning edits through the same failures
- is redirected even if the judge disagrees. The plugin never stops a turn itself.
- Where the notice appears, and how to stop, on each agent and in each mode:
+ is redirected even if the judge disagrees. The plugin never stops a turn itself. Where the
+ notice appears, and how to stop the turn, on each agent and in each mode:
[When a turn gets stuck](../README.md#when-a-turn-gets-stuck).
- **Cost.** `status` shows each session's tokens, priced after it ends, and the plugin's own
overhead: the loop guard's judge, and micro-skills' proposer when it is on.
@@ -396,7 +396,7 @@ and are shared by every agent working in it. The examples use Claude Code's form
| Mechanism | Key | `on` means | Other values |
|---|---|---|---|
| Micro-skills (off by default) | `microskills` | `envfacts+distiller`: facts found without model calls, plus a model proposing facts every `microskills_propose_every` sessions | `off`; `envfacts`: only the facts found without model calls; `distiller`: only the proposer |
-| Tool pruning | `toolprune` | `auto`: applied once it forms and your sessions have been idle an hour, from the next session, and announced | `off`; `propose`: only proposed, applied when you run `toolprune apply` |
+| Tool pruning | `toolprune` | `auto`: applied without asking once your sessions have been idle for `toolprune_quiet_minutes`, from the next session, which announces it | `off`; `propose`: only proposed, applied when you run `toolprune apply` |
| Loop guard | `loopguard` | `nudge`: a judge model decides whether a looping turn is stuck; if so, one mid-turn redirect, then a notice to you if the turn is still stuck | `off` (no judge either); `stop` is read as `nudge`, with a warning |
**Tune a mechanism:**
@@ -517,21 +517,23 @@ not rewritten, so the commands that fact recommends keep working.
## Model calls and API keys
-**By default the plugin's only model calls are the loop guard's**: its judge, asked about a turn
-that shows signs of looping, and, when the judge finds it stuck, the redirect it writes. Tool
-pruning never calls a model. Turning micro-skills on adds its fact proposer, once every
-`microskills_propose_every` sessions (`microskills=envfacts` keeps only the facts found without
-a model call); `loopguard=off` stops the judge with the guard. These calls run through **your
-own agent CLI** (`claude -p`; `codex exec` on Codex; `copilot -p`, with every tool removed, on
-Copilot; `pi -p`, with nothing loaded but the model, on Pi), so they use whatever your sessions
-already use: a Claude subscription or Anthropic API key, your Codex login or OpenAI key,
-Copilot's subscription, or the provider and login Pi is set up with. (Copilot with BYOK is the
-exception; see [Copilot CLI](#copilot-cli).) Their cost is billed there too, and shown in
-`status`. A call that gets no reply leaves a line in `~/.tokeneconomizer/errors.log`; one whose
-CLI answered that it is not signed in also pauses that agent's calls (the loop guard then
-redirects only at its backstop), the next session start says so once, and
-`config set microskills=on` (or `loopguard=on`) tries again. `toolprune measure` on Claude Code
-makes its two calls the same way, and only when you run it.
+**With the defaults, the plugin's only model calls are the loop guard's**: the judge, asked about
+a turn that shows signs of looping, and, when the judge finds the turn stuck, the redirect it
+writes. Tool pruning never calls a model. Turning micro-skills on adds its fact proposer, once
+every `microskills_propose_every` sessions (`microskills=envfacts` keeps only the facts found
+without a model call); `loopguard=off` turns the judge off along with the guard.
+
+These calls run through **your own agent's CLI**: `claude -p`; `codex exec`; `copilot -p`, with
+every tool removed; and `pi -p`, with nothing loaded but the model. So they use whatever your
+sessions already use (a Claude subscription or Anthropic API key, your Codex login or OpenAI key,
+Copilot's subscription, or the provider and login Pi is set up with), are billed there, and are
+shown in `status`. Copilot with BYOK is the exception; see [Copilot CLI](#copilot-cli).
+
+A call that gets no reply leaves a line in `~/.tokeneconomizer/errors.log`. If the CLI answers
+that it is not signed in, that agent's calls are also paused (the loop guard then redirects only
+at its backstop), the next session start says so once, and `config set microskills=on` (or
+`loopguard=on`) tries again. `toolprune measure` on Claude Code makes its two calls the same way,
+and only when you run it.
## Troubleshooting