Experience-Driven Black-Box Cost Control for Coding Agents
Python agentic-aiagentic-codingai-codingai-coding-assistantanthropicartificial-intelligenceclaude-codecodexcoding-agentcoding-agentscost-optimizationcost-reductiondeveloper-toolsgithub-copilotllmopen-sourceopenaiproductivitytoken-optimization
teaser fixed
Aaronec8ff5ec7ed5
diff --git a/CLAUDE.md b/CLAUDE.mdindex 51e95a9..23ae32b 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -38,13 +38,13 @@ agent behavior, and `docs/ARCHITECTURE.md` for how the package fits together.`plugin/config.py`); write new code against the current names.- No run outputs in the repository: transcripts, rows, streams, figures rendered from runs andtables derived from them stay out of git (`.gitignore`). The one exception is the paper's- headline results in `README.md` (the line under the badges, the Results section and- `docs/assets/results_overview.*`, and the teaser's closing results, which read- `results_overview.py`'s `ROWS`): quoted from the paper's Table 1 and attributed to it.- Change them only with the paper, and redraw both figures with+ headline results in `README.md` (the line under the badges, the Results section, and the+ teaser's closing results, which read `docs/assets/results_overview.py`'s `ROWS`; that script+ also draws `results_overview.*`, which the README no longer shows): quoted from the paper's+ Table 1 and attributed to it. Change them only with the paper, and redraw both figures with`python docs/assets/results_overview.py` and `python docs/assets/teaser.py --plex FONTS`(its docstring says where to get IBM Plex). The teaser's middle act is a schematic with- made-up tasks and no numbers.+ made-up sessions and no numbers.- The plugin's own text (`plugin/`, its manifests, slash commands and skills, and what thehooks print) makes no claims of effect.- Tests: `python -m pytest tests experiments/tests`. A change to what the plugin learns ordiff --git a/README.md b/README.mdindex 5a0025b..eb6a552 100644--- a/README.md+++ b/README.md@@ -34,84 +34,85 @@<p align="center"><picture><source media="(prefers-reduced-motion: reduce)" srcset="docs/assets/teaser.png">- <img src="docs/assets/teaser.gif" width="100%" alt="An animation that opens on a title card: TokenEconomizer, experience-driven black-box cost control for coding agents, with its three mechanisms (micro-skills, tool pruning, failure-loop intervention), the agents it works with (Claude Code, Codex, GitHub Copilot CLI, Pi) and the UC Berkeley and Microsoft logos. Then a schematic of six tasks in one repository, each a bar of what it cost: the work itself plus three kinds of waste, namely re-learning the repo, tools never used, and getting stuck in loops. Each of the first three tasks teaches TokenEconomizer one thing, in order: 1 micro-skills remember how the repo works (run tests with python -m pytest), 2 tool pruning skips tools never used here (Read and Edit kept, Notebook and Browser left out), and 3 failure-loop intervention spots loops and steps in (the same error three times meets a nudge to try another way). In the task after each lesson that waste shrinks, so the bars step down below a dashed line marking the cost without TokenEconomizer. It ends on results measured on SWE-Bench tasks: 18.4–30.9% lower mean cost across five agent–model combinations, cheaper than default in 30 of 30 repository × combination cells, and Pass@1 up in 4 of 5; per combination, cost 23.6% lower and Pass@1 +1.3 for Claude Code with Sonnet 5, 30.9% and −1.0 with Opus 4.8, 24.7% and +0.9 for Codex with GPT-5.6 Terra, 26.1% and +1.6 with GPT-5.6 Sol, and 18.4% and +2.6 for Copilot CLI with Sonnet 5.">+ <img src="docs/assets/teaser.gif" width="100%" alt="An animation that opens on a title card: TokenEconomizer, experience-driven black-box cost control for coding agents, with its three mechanisms (micro-skills, tool pruning, failure-loop intervention), the agents it works with (Claude Code, Codex, GitHub Copilot CLI, Pi) and the UC Berkeley and Microsoft logos. Then a schematic of six sessions in one repository, each a bar of what it cost: the work itself plus three kinds of waste, namely re-learning the repo, tools never used, and getting stuck in loops. Each of the first three sessions teaches TokenEconomizer one thing, in order: 1 micro-skills remember how the repo works (run tests with python -m pytest), 2 tool pruning skips tools never used here (Read and Edit kept, Notebook and Browser left out), and 3 failure-loop intervention spots loops and steps in (the same error three times meets a nudge to try another way). In the session after each lesson that waste shrinks, so the bars step down below a dashed line marking the cost without TokenEconomizer. It ends on results measured on SWE-Bench tasks: 18.4–30.9% lower mean cost across five agent–model combinations, cheaper than default in 30 of 30 repository × combination cells, and Pass@1 up in 4 of 5; per combination, cost 23.6% lower and Pass@1 +1.3 for Claude Code with Sonnet 5, 30.9% and −1.0 with Opus 4.8, 24.7% and +0.9 for Codex with GPT-5.6 Terra, 26.1% and +1.6 with GPT-5.6 Sol, and 18.4% and +2.6 for Copilot CLI with Sonnet 5."></picture></p>-TokenEconomizer is a plugin for **Claude Code**, **Codex**, the **GitHub Copilot CLI** and-**Pi**. It leaves the agent as it is, reads each finished session in a repository, and changes-what the next sessions in that repository pay for:--- **Micro-skills.** Repository knowledge your sessions keep needing, in any language -- that a- build step must run before anything else works, which spelling of a tool works here (`npx tsc`- where `tsc` is missing, `python -m pytest` where `pytest` is), an environment variable a- command needs, where generated files live -- is learned from finished sessions and stated at- the start of the next one. A fact is kept only while later sessions confirm it. Every *k*- sessions (*k* = 3 by default) a model, called through your own agent, proposes facts, and that- cost is counted with your sessions'. How much it helps depends on the repository's state and- the tasks you run in it.-- **Tool pruning.** A tool that stays loaded but goes unused over *p* prompts across at least *m*- sessions, or over *s* sessions however short (by default *p* = 10, *m* = 2, *s* = 5), is left- out of the next sessions through the agent's own settings. By default this happens without- asking you: the next session start says which tools were left out, and one command gives them- back. Set `toolprune=propose` to approve each deny list yourself instead.-- **Failure-loop intervention.** A turn that keeps hitting the same failures is put to a judge,- a model called through your own agent, and gets a redirect only if the judge finds it stuck- with confidence *c* or more, at most *r* times per turn (by default *c* = 0.7, *r* = 1). If it- is still stuck after that, the plugin tells you how to stop it; it never stops a turn itself.--Each mechanism can be switched on or off on its own, and each letter in italics is a setting you-can change ([Settings](#settings) names them all). The plugin's actions are announced in one-line, and every hook fails open. Its only model calls are micro-skills' proposer and the loop-guard's judge, made through your own agent. What it learns stays on your machine, under-`~/.tokeneconomizer`.+TokenEconomizer is a plugin that makes **Claude Code**, **Codex**, the **GitHub Copilot CLI**+and **Pi** cheaper to run in the repositories you work in often. It does not change the agent;+it learns from your sessions. Its unit is the *session*: one conversation with your agent, from+start to exit, however many tasks you do in it. After each session in a repository ends, the+plugin reads its log, and it changes what the next sessions there start with. It has three+mechanisms:++- **Micro-skills: remember how the repository works.** Sessions keep rediscovering the same+ facts: a build step that must run first, the spelling of a command that works here+ (`python -m pytest` where `pytest` is missing, `npx tsc` where `tsc` is), an environment+ variable a command needs, where generated files live. Micro-skills learns such facts, in any+ language, from two sources: failures that repeat across sessions, and a model that reviews the+ recent sessions every *k* sessions (*k* = 3) and proposes facts. A fact is stated at the start+ of each session once later sessions confirm it, and dropped when one contradicts it.+- **Tool pruning: stop loading tools that go unused.** Every loaded tool is sent with every+ request. A tool that goes unused over *p* prompts spread across at least *m* sessions, or over+ *s* sessions of any length (*p* = 10, *m* = 2, *s* = 5), is left out of the next sessions+ through the agent's own settings. One command brings it back.+- **Failure-loop intervention (the loop guard): step in when a turn is stuck.** When a turn (the+ agent's work on one prompt) keeps hitting the same failures, a judge model checks whether it is+ stuck. If the judge is at least *c* sure (*c* = 0.7), the agent gets a short redirect, at most+ *r* times per turn (*r* = 1). If the turn is still stuck after that, the plugin tells you how to+ stop it; it never stops a turn itself.++Each italic letter is a setting, shown with its default ([Settings](#settings)), and each+mechanism can be turned on or off. The plugin announces each action in one line, and if any part+of it fails, your session carries on as if it were not installed. Its only model calls (the fact+proposer and the judge) run through your own agent, on the same plan or login, and their cost is+included in its reports. What it learns stays on your machine, in `~/.tokeneconomizer`.## Results-<p align="center"><img src="docs/assets/results_overview.png" width="100%" alt="SWE-Bench results from the paper. Mean cost per task, lower than default: Claude Code with Sonnet 5, 23.6%; Claude Code with Opus 4.8, 30.9%; Codex with GPT-5.6 Terra, 24.7%; Codex with GPT-5.6 Sol, 26.1%; Copilot CLI with Sonnet 5, 18.4%. Pass@1 changes by +1.3, −1.0, +0.9, +1.6 and +2.6 points."></p>--The paper runs five agent–model combinations on the same 277 SWE-Bench tasks from six-repositories, each repository's tasks in order and starting from no traces; costs include every-call TokenEconomizer made. It was cheaper than default in each of the 30 repository ×-combination cells, by 5–40%, and the savings carried over to Web-Bench (16–21% lower) and to two-security-auditing benchmarks where every task is a new repository (6–20% lower). The-[project page](https://tokeneconomizer.github.io) has interactive charts, the per-repository-numbers, and nine annotated runs; [`experiments/`](experiments/README.md) reruns the SWE-Bench and+The paper ran five agent–model combinations on the same 277 SWE-Bench tasks from six repositories.+Each task ran as its own session, and each repository's tasks ran in order, starting with nothing+learned. Costs include every model call TokenEconomizer made. Within each repository and+combination, cost fell by 5–40%. The savings held on Web-Bench (16–21% lower) and on two+security-auditing benchmarks where every task is in a new repository (6–20% lower). The+[project page](https://tokeneconomizer.github.io) has interactive charts, per-repository numbers+and nine annotated runs, and [`experiments/`](experiments/README.md) reruns the SWE-Bench andWeb-Bench experiments.## How it works<p align="center"><img src="docs/assets/framework_overview.png" width="100%" alt="TokenEconomizer: the plugin keeps what it knows about a repository and applies micro-skills, tool pruning and failure-loop intervention to your unchanged coding agent; each finished session's log updates all three"></p>-Coding agents are built for task success, not for what a task costs, and every turn re-bills-the agent's instructions, every loaded tool, and the conversation so far. Developers use the same-agent on the same codebase over many tasks, so much of that cost recurs: the same environment-facts rediscovered, the same unused tools re-sent, the same kind of failing loop. TokenEconomizer-holds the agent fixed -- no change to its model, weights or code -- and treats a repository's-stream of sessions as the unit to optimize: what finished sessions show about the repository-shapes what later sessions are given, and anything the plugin spends is amortized over the stream.-Four principles shape the mechanisms:--- **Act through what the agent already reads.** Facts go into the context at session start and- tools leave through the agent's own settings, so nothing depends on the agent adopting a new- tool.+Coding agents are built to finish tasks, not to keep them cheap. Every turn pays again for the+agent's instructions, every loaded tool, and the conversation so far. Developers use the same+agent on the same repository again and again, so much of that cost repeats: the same facts+rediscovered, the same unused tools sent, the same kinds of failing loops. TokenEconomizer leaves+the agent's model, weights and code alone and learns across a repository's sessions instead of+treating each one on its own, so whatever the plugin spends is paid back over the sessions that+follow. Four principles guide the mechanisms:++- **Work through what the agent already reads.** Facts enter the context at session start, and+ tools are removed through the agent's own settings, so nothing depends on the agent learning a+ new tool.- **Count cost as it is billed.** Most tokens in a session are cached re-reads, so cost is- measured on a cache-aware basis, including any model calls the plugin makes.-- **Keep task success.** Facts form only from recurring evidence, tools are pruned only after- going unused, and the loop guard spares a turn that is still making progress.-- **Recover from mistakes.** A fact a later session contradicts is retired, an applied deny list- can be restored, and the guard only redirects and tells you: stopping a turn is your call.+ computed with cache pricing, and it includes the plugin's own model calls.+- **Protect task success.** Facts form only from repeated evidence, tools are removed only after+ going unused, and the loop guard leaves alone a turn that is still making progress.+- **Keep every action reversible.** A fact that a later session contradicts is dropped, removed+ tools can be given back, and the loop guard only redirects and tells you: stopping a turn is+ your call.## Quick startYou need one of the four agents, signed in any way it supports, and Python 3.10 or newer on-`PATH`. The plugin runs inside your agent's own sessions and never handles credentials, so any-plan or login works, and it needs no API key of its own.+`PATH`. The plugin needs no API key of its own and never handles credentials, so any plan or+login works.### Install-Run the shell commands where the clone should stay: Claude Code, Codex and the Copilot CLI run-the plugin from it. Then start the agent inside one of your repositories.+Run these commands in the folder where you want to keep the clone. Claude Code, Codex and the+Copilot CLI run the plugin from the clone, so leave it in place. Then start the agent in one of+your repositories.**Claude Code**@@ -157,9 +158,10 @@ pi install git:github.com/microsoft/TokenEconomizerStart `pi` and run `/te status`.-The status command prints a short status block. `refresh-catalog` records which tools a session-loads, with no model call, so that tool pruning sees all of them. The marketplace path must be-absolute: the Copilot CLI reads `./TokenEconomizer` as the name of a GitHub repository.+If the install worked, the status command prints a short status block. `refresh-catalog`+records every tool a session loads (without a model call), so that tool pruning can consider all+of them. Keep the marketplace path absolute, as above: the Copilot CLI reads a relative path such+as `./TokenEconomizer` as the name of a GitHub repository.To update:@@ -169,8 +171,8 @@ To update:| Codex | `git pull` in the clone, then the `codex plugin add` line again || Pi | `pi update --extensions` |-Uninstalling, Copilot with your own model key, and troubleshooting are in-**[`plugin/README.md`](plugin/README.md)**, which walks through each agent.+**[`plugin/README.md`](plugin/README.md)** walks through each agent in more detail, including+uninstalling, using Copilot with your own model key, and troubleshooting.### Supported versions@@ -181,8 +183,8 @@ Uninstalling, Copilot with your own model key, and troubleshooting are in| Copilot CLI | 1.0.83 to 1.0.102 | not supported || Pi | 1.2.0 to 1.3.0 | 0.99.x: load the extension by path (below) |-The commands above were checked on both ends of each range; later versions have not been-checked.+The commands above were tested on the oldest and newest version in each range; newer versions+have not been tested yet.- **Codex before 0.141.0** does not run plugin hooks at all.- **Pi 0.99.x**: instead of `pi install`, clone the repository and start each session with@@ -191,8 +193,8 @@ checked.### Everyday commands-Work as usual; the plugin needs nothing from you. Its commands have the same names on every-agent; only how you type them differs:+Work as usual; the plugin needs nothing from you. To see or change what it does, use its+commands. They have the same names on every agent; only the way you type them differs:| Agent | Type a command as ||---|---|@@ -210,52 +212,54 @@ agent; only how you type them differs:| `config` | every setting in effect || `config set microskills=on` | turn a mechanism on (or `=off`) here; add `--global` for every repository |-Inside an agent each command is one model turn; from a terminal it costs nothing. Every other-command (`repos`, `doctor`, `config unset`, ...) and how each agent asks before running one:+Inside an agent, a command costs one model turn; from a terminal it is free. The other commands+(`repos`, `doctor`, `config unset`, ...) and each agent's permission prompts are listed under[Commands](plugin/README.md#commands).## Settings-Tool pruning and the loop guard are on from the start. Micro-skills is off until you turn it on-(`config set microskills=on`): it adds the proposer's model calls, and what it gains depends on-the repository and the tasks you run in it. Each mechanism takes `on` or `off`, and the letters-used above are settings you change with the same `config set` command (in Claude Code's form-here):+Tool pruning and the loop guard are on by default; with these defaults, the plugin's only model+call is the judge, asked only about a turn that shows signs of looping. Micro-skills is off by+default, because its proposer adds model calls and how much it saves depends on the repository+and the work you do there. Each mechanism takes `on` or `off`, and the italic letters above are+these settings:| | Setting | Default | What it sets ||---|---|---|---|| *k* | `microskills_propose_every` | 3 | sessions between runs of micro-skills' fact proposer |-| *p* | `toolprune_evidence` | 10 | prompts a never-used tool must go unused before it is left out |+| *p* | `toolprune_evidence` | 10 | prompts a tool must go unused before it is left out || *m* | `toolprune_min_sessions` | 2 | sessions those prompts must span |-| *s* | `toolprune_sessions` | 5 | or this many sessions, however short |+| *s* | `toolprune_sessions` | 5 | or this many sessions, of any length || *q* | `toolprune_quiet_minutes` | 60 | minutes your sessions here must be idle before a deny list is written || *e* | `loopguard.envelope_min_samples` | 5 | turns here that ended with passing checks before the loop guard acts || *c* | `loopguard.judge_confidence` | 0.7 | how sure the judge must be that a turn is stuck || *r* | `loopguard_nudges` | 1 | redirects per turn before the guard only tells you |-| *b* | `loopguard.backstop_mult` | 2.5 | how many times the usual turn length a churning turn may run before it is redirected without the judge |+| *b* | `loopguard.backstop_mult` | 2.5 | how many times its usual length a churning turn may run before it is redirected without the judge |-Prompts let long sessions count for more; the *m*-session floor keeps one session's work from-deciding alone. The deny list is written once your sessions in the repository have been idle-for *q* minutes: a conversation resumed after its tools change writes its whole history to the-prompt cache again instead of reading it, and after an idle hour that cache has expired anyway.-With `toolprune=propose` the plugin only proposes the list, and nothing changes until you run-`toolprune apply`.+Change them with `config set`, shown here in Claude Code's form (other agents type it as in+[Everyday commands](#everyday-commands)):```text-/tokeneconomizer:config set microskills=on # learn and state repository facts+/tokeneconomizer:config set microskills=on # turn micro-skills on/tokeneconomizer:config set microskills_propose_every=5 # k = 5: propose facts less often/tokeneconomizer:config set toolprune_evidence=20 # p = 20: wait for twenty unused prompts/tokeneconomizer:config set loopguard_nudges=2 # r = 2: redirect twice before telling you```-For one session only, set an environment variable when you start the agent:-`TE_DISABLE=1 claude` turns the whole plugin off, and `TE_LOOPGUARD=off codex` (likewise-`TE_MICROSKILLS`, `TE_TOOLPRUNE`) one mechanism. The plugin's model calls run through your own-agent's CLI, on the same plan or login as your sessions. With the defaults the only one is the-loop guard's judge, asked only about a turn that shows signs of looping; turning micro-skills on-adds its proposer (`microskills=envfacts` keeps only the facts found without a model call), and-`loopguard=off` turns the judge off with the guard. Every setting, its default and its-environment variable: [Settings](plugin/README.md#settings).+Tool pruning counts prompts so that long sessions weigh more, and requires *m* sessions so that a+single session never decides alone. It writes the deny list only once your sessions in the+repository have been idle for *q* minutes: a conversation resumed after its tools change must+write its whole history to the prompt cache again instead of reading it, and after an idle hour+that cache has expired anyway.++Besides `on` and `off`, two modes are useful: `toolprune=propose` only proposes the deny list, and+nothing changes until you run `toolprune apply`; `microskills=envfacts` learns facts only from+repeated failures, with no model calls.++To change a single session, set an environment variable when you start the agent:+`TE_DISABLE=1 claude` turns the whole plugin off, and `TE_LOOPGUARD=off codex` turns off one+mechanism (likewise `TE_MICROSKILLS` and `TE_TOOLPRUNE`). Every setting, its default and its+environment variable are listed under [Settings](plugin/README.md#settings).## What it does on each agent@@ -269,33 +273,31 @@ agent's own settings, so what it can leave out differs:| Copilot CLI | whole MCP servers | `.github/copilot/settings.local.json` || Pi | nothing: Pi loads four tools | — |-Tool pruning writes only the entries it adds and never touches deny rules you wrote; a settings-file it creates is kept out of git (`.git/info/exclude`). The next session start says what it-left out. On Claude Code that line shows only in the detailed transcript (**Ctrl+O**), so-`/tokeneconomizer:toolprune` is the place to check. `toolprune restore` removes the plugin's-entries and no others, and what you give back stays back. If the plugin cannot edit a file-safely (a `.codex/config.toml` your repository commits, a settings file that is not valid JSON,-a path through a symlink), the list stays a proposal and the session start says why;-`toolprune apply` tries again. How each agent's hooks and settings were verified to behave is in-**[`docs/PLUGIN.md`](docs/PLUGIN.md)**, and how the package fits together in-**[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)**.+Tool pruning adds only its own entries and never touches deny rules you wrote; a settings file+it creates is kept out of git (`.git/info/exclude`). On Claude Code, the session-start line that+says what it left out shows only in the detailed transcript (**Ctrl+O**), so check with+`/tokeneconomizer:toolprune`. `toolprune restore` removes the plugin's entries and nothing else,+and tools you give back are not removed again. If the plugin cannot edit a file safely (a+`.codex/config.toml` your repository commits, a settings file that is not valid JSON, a path+through a symlink), the list stays a proposal and the session start says why; `toolprune apply`+tries again. **[`docs/PLUGIN.md`](docs/PLUGIN.md)** records how each agent's hooks and settings+were verified to behave, and **[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)** how the package+fits together.### When a turn gets stuck-Once *e* turns in a repository have ended with passing checks, the loop guard knows how long a-turn there usually runs. From then on, when a turn shows signs of looping -- the same failures-repeating, or errors piling up -- the guard asks its judge, and only if the judge finds the turn-stuck (confidence *c* or more) does it add a redirect to the model's context, at most *r* per-turn. (One exception: a turn past *b* times this repository's usual length that also keeps-churning the same edits through the same failures is redirected even if the judge disagrees.)-If the turn is still stuck after that, it shows you one line, ending with how to stop the turn:+The loop guard first learns how long a turn in this repository usually runs: it waits until *e*+turns here have ended with passing checks. After that, a turn that shows signs of looping (the+same failures repeating, or errors piling up) goes to the judge. One case is redirected without+the judge: a turn that runs past *b* times the usual length while churning the same edits through+the same failures. If a turn is still stuck after its redirects, you see one line that ends with+how to stop it:> TokenEconomizer: this turn still looks stuck after a redirect (envelope, 12 tool calls). Press> Esc to interrupt it.-It never stops a turn itself; whether to stop is up to you. Where the line shows, and how you-stop, in the agent's terminal UI and in a headless run (`claude -p`, `codex exec`, `copilot -p`,-`pi -p`):+Where that line appears, and how to stop the turn, in the terminal UI and in a headless run+(`claude -p`, `codex exec`, `copilot -p`, `pi -p`):| Agent | Interactive: shown | Stop with | Headless: shown | Stop with ||---|---|---|---|---|@@ -304,15 +306,15 @@ stop, in the agent's terminal UI and in a headless run (`claude -p`, `codex exec| Copilot CLI | in the conversation | Esc twice | on stdout | Ctrl+C || Pi | as a notification | Esc | on stderr | Ctrl+C |-- **Claude Code** keeps a plugin's own lines to its detailed transcript (Ctrl+O), so the plugin- asks Claude to repeat the line; a model may word it differently or leave it out.+- **Claude Code** shows a plugin's own lines only in its detailed transcript (Ctrl+O), so the+ plugin asks Claude to repeat this one in its reply; the model may word it differently or leave+ it out.- **Codex**: Esc interrupts the turn, but a command it started keeps running until `/stop`.`codex exec` prints no hook messages.- **Pi**: Ctrl+C ends a headless run, but a command it started runs on until it finishes.-In a headless run nobody may be watching, so a flagged turn keeps running until it ends by-itself or reaches the agent's own limits; `te status` counts the redirects and "still stuck"-notices afterwards.+In a headless run nobody may be watching, so a flagged turn runs until it ends or reaches the+agent's own limits; afterwards, `te status` counts the redirects and "still stuck" notices.## Repository@@ -337,7 +339,7 @@ The package is standard-library only. Run the tests with `python -m pytest testsThe code behind the paper's results -- a harness that runs Claude Code, Codex and the Copilot CLIon SWE-Bench and Web-Bench, the launchers, and the evaluated task sets -- is in**[`experiments/`](experiments/README.md)**, whose README gives the setup and one command per-result. It is not packaged with the plugin and shares only `tokeneconomizer/`.+result. It is not part of the installed plugin; the two share only `tokeneconomizer/`.## Citationdiff --git a/docs/assets/results_overview.py b/docs/assets/results_overview.pyindex d99574e..144a18a 100644--- a/docs/assets/results_overview.py+++ b/docs/assets/results_overview.py@@ -1,5 +1,6 @@#!/usr/bin/env python3-"""The README's results figure: the paper's SWE-Bench headline, per agent-model combination.+"""The results figure: the paper's SWE-Bench headline, per agent-model combination. Its ROWS+are also the README teaser's closing numbers (teaser.py); the README no longer shows the figure.python docs/assets/results_overview.py # needs matplotlib; writes the PNG and SVG herediff --git a/docs/assets/teaser.gif b/docs/assets/teaser.gifindex a97e862..054fb28 100644Binary files a/docs/assets/teaser.gif and b/docs/assets/teaser.gif differdiff --git a/docs/assets/teaser.png b/docs/assets/teaser.pngindex b77029f..fadfb7b 100644Binary files a/docs/assets/teaser.png and b/docs/assets/teaser.png differdiff --git a/docs/assets/teaser.py b/docs/assets/teaser.pyindex 3ce4f44..8e3bf94 100644--- a/docs/assets/teaser.py+++ b/docs/assets/teaser.py@@ -1,5 +1,5 @@#!/usr/bin/env python3-"""The teaser: TokenEconomizer's title card, how it trims the waste of a repository's tasks, and+"""The teaser: TokenEconomizer's title card, how it trims the waste of a repository's sessions, andthe paper's headline results.python docs/assets/teaser.py --plex FONTS # the README's: teaser.gif and teaser.png@@ -15,13 +15,15 @@ Three acts, crossfaded:Title the page hero's two-tone name, the paper's subtitle, the three mechanisms in the paper'sorder, the agents it works with, and the authors' institutions' logos (logo_*.png: theproject page's SVGs, rasterized with a transparent background, as Pillow draws no SVG).-Method a sketch, not a measurement: six made-up tasks, alike, each a bar of the work itself and- three kinds of waste, named beside the first bar, under a dashed line at what a task costs- without TokenEconomizer. On the right, what TokenEconomizer has learned: tasks 1-3 each- fly into it and teach it one mechanism, with a plain example; in the next task that waste- shrinks while its card lights up, and the last tasks start lean. No number in it comes- from a run (runs stay out of the repository); the waste is sized so the last tasks land- inside the paper's range of savings, and nothing is labelled with a number.+Method a sketch, not a measurement: six made-up sessions, alike, each a bar of the work itself+ and three kinds of waste, named beside the first bar, under a dashed line at what a+ session costs without TokenEconomizer. Sessions, not tasks: the plugin learns from each+ finished session, however many tasks it held. On the right, what TokenEconomizer has+ learned: sessions 1-3 each fly into it and teach it one mechanism, with a plain example;+ in the next session that waste shrinks while its card lights up, and the last sessions+ start lean. No number in it comes from a run (runs stay out of the repository); the+ waste is sized so the last sessions land inside the paper's range of savings, and+ nothing is labelled with a number.Results the paper's SWE-Bench headline, as the README's line under the badges and the projectpage's tiles state it, and per agent-model combination the cost cut and the change inPass@1. The numbers are results_overview.py's ROWS (the paper's Table 1), not copied.@@ -107,21 +109,21 @@ def page_look(fonts):L = None # the look being drawn; main() sets it, with the fonts it is given-# ---- the tasks -------------------------------------------------------------------------------# Every task does the same work, and wastes the same three things until TokenEconomizer has+# ---- the sessions ---------------------------------------------------------------------------+# Every session does the same work, and wastes the same three things until TokenEconomizer has# learned how not to: cost units, the full amount and what is left once it has.N = 6WORK = 160MECHS = ("facts", "prune", "guard") # in the paper's order, top to bottom on each barFULL = {"facts": 40, "prune": 44, "guard": 40}LEFT = {"facts": 8, "prune": 22, "guard": 16}-LEARNED_AFTER = {"facts": 1, "prune": 2, "guard": 3} # the task that teaches it+LEARNED_AFTER = {"facts": 1, "prune": 2, "guard": 3} # the session that teaches itNAME = {"facts": "Micro-skills", "prune": "Tool pruning", "guard": "Failure-loop intervention"}WASTE = {"facts": "re-learning the repo", "prune": "tools never used","guard": "stuck in loops"}DOES = {"facts": "Remembers how the repo works", "prune": "Skips tools never used here","guard": "Spots loops and steps in"}-CAPTION = {"facts": "told to the agent as each task starts",+CAPTION = {"facts": "told to the agent as each session starts","prune": "unused tools are no longer loaded","guard": "a nudge when the same error repeats"}@@ -143,8 +145,8 @@ PITCH = (CX1 - BX0) / NBAR_W = 46BASE_Y = 528 # the bars' baselineUNIT = 1.52 # pixels per cost unit-TASK_Y = 550-FULL_Y = BASE_Y - (WORK + sum(FULL.values())) * UNIT # a task without TokenEconomizer+SESSION_Y = 550+FULL_Y = BASE_Y - (WORK + sum(FULL.values())) * UNIT # a session without TokenEconomizerLEAN_Y = BASE_Y - (WORK + sum(LEFT.values())) * UNITPX0, PX1, PY0, PY1 = 704, 1088, 24, 576 # TokenEconomizer's panel@@ -227,8 +229,8 @@ def rgb(color):def amount(m, i, f):- """What task i spends on waste m at frame f: all of it until TokenEconomizer has learned- from an earlier task, shrinking in the first task after, and what is left from then on."""+ """What session i spends on waste m at frame f: all of it until TokenEconomizer has learned+ from an earlier session, shrinking in the first session after, and what is left from then on."""after = LEARNED_AFTER[m]if after >= i:return FULL[m]@@ -238,7 +240,7 @@ def amount(m, i, f):def trim_glow(m, f):- """How lit m's card is while the task after it learned shrinks m's waste: 0 to 1 and back."""+ """How lit m's card is while the session after it learned shrinks m's waste: 0 to 1 and back."""i = LEARNED_AFTER[m] + 1if i > N or ("trim", i) not in AT:return 0.0@@ -368,27 +370,27 @@ class Painter:def draw_static(p):"""What never moves in the method: the title, the baseline, the panel and its header."""- title = "Each finished task makes the next ones waste less"+ title = "Each finished session makes the next ones waste less"p.text((CX0, TITLE_Y), title, 22, L.ink, "bold", anchor="ls")assert CX0 + p.width(title, 22, "bold") <= CX1, "the title outgrows the chart"p.line([(BX0 - 8, BASE_Y), (CX1, BASE_Y)], L.faint, 1.4)p.rect((PX0, PY0, PX1, PY1), L.panel, L.line, 1.6, radius=14)p.text((CARD_X0 + 4, 56), "TokenEconomizer", 20, L.ink, "bold")- p.text((CARD_X0 + 4, 82), "learns from each finished task", 15, L.muted)+ p.text((CARD_X0 + 4, 82), "learns from each finished session", 15, L.muted)p.line([(CARD_X0, 100), (CARD_X1, 100)], L.line, 1.2)def stack(i, f):- """Task i's bar, bottom to top, as (units, color, waste or None)."""+ """Session i's bar, bottom to top, as (units, color, waste or None)."""return [(WORK, L.work, None)] + [(amount(m, i, f), color(m), m) for m in reversed(MECHS)]def draw_bar(p, i, f):start = AT["grow", i]grow = GROW_FIRST if i == 1 else GROW- p.text((col_x(i), TASK_Y), f"task {i}", 14, mix(L.faint, L.ink, clamp((f - start) / 4)),+ p.text((col_x(i), SESSION_Y), f"session {i}", 14, mix(L.faint, L.ink, clamp((f - start) / 4)),anchor="mm")- assert p.width(f"task {i}", 14) <= PITCH - 8, "task labels meet"+ assert p.width(f"session {i}", 14) <= PITCH - 8, "session labels meet"if f < start:returnsegs = stack(i, f)@@ -431,7 +433,7 @@ def draw_names(p, segs, shown):def draw_without(p, f):- """The dashed line at a task's full cost, drawn when TokenEconomizer first trims one."""+ """The dashed line at a session's full cost, drawn when TokenEconomizer first trims one."""if f < AT["trim", 2]:returnx0 = col_x(1) + BAR_W / 2 + 6diff --git a/plugin/README.md b/plugin/README.mdindex a78ae99..6658e37 100644--- a/plugin/README.md+++ b/plugin/README.md@@ -59,38 +59,35 @@ This records which tools a Claude Code session loads, with no model call. Do itmachine, and again after Claude Code updates. **Don't skip it:** without it the deny list seesonly the few tools Claude Code defers, not the ones every session loads.-**4. Work normally.** See [What to expect](#what-to-expect). Once a tool has gone unused long-enough (by default ten prompts across two or more sessions, or five sessions however short), the-plugin leaves it out of the next sessions without asking you (it writes the list once your-sessions here have been idle, an hour by default; each number is a [setting](#settings)), and-the session start says so once:+**4. Work normally.** When the plugin leaves tools out ([What to expect](#what-to-expect) says+when), the next session start says so once. To see or undo it:```text/tokeneconomizer:toolprune # the tools it left out, and the evidence/tokeneconomizer:toolprune restore # give them back, from your next session; they stay back```-The rules go into this repository's `.claude/settings.local.json`, which the plugin keeps out of-git, and it never touches deny rules you wrote. To approve each deny list yourself instead:+The deny rules go into this repository's `.claude/settings.local.json`, which the plugin keeps+out of git; deny rules you wrote are never touched. To approve each deny list yourself instead:`/tokeneconomizer:config set toolprune=propose`, then `/tokeneconomizer:toolprune apply`.-**Where its messages appear.** Claude Code shows the plugin's one-line messages (facts-added, tools left out) only in its detailed transcript: press **Ctrl+O**, where they read-`<event> says: TokenEconomizer: …`. A turn that still looks stuck is the exception:-the plugin also asks Claude to repeat that line word for word, so it shows in Claude's reply,-and under `claude -p` in its final reply. That is a request to the model, which may word it-differently or leave it out; the detailed transcript always has it. To stop a turn, press-**Esc**; under `claude -p`, Ctrl+C ends the run. With `--output-format stream-json --verbose`-every message also arrives as a `hook_response` event.+**Where its messages appear.** Claude Code shows the plugin's one-line messages (facts added,+tools left out) only in its detailed transcript: press **Ctrl+O**, where they read+`<event> says: TokenEconomizer: …`. The exception is a turn that still looks stuck: the plugin+also asks Claude to repeat that line word for word, so it shows in Claude's reply (under+`claude -p`, in the final reply). Claude may word it differently or leave it out; the detailed+transcript always has it. Press **Esc** to stop the turn; under `claude -p`, Ctrl+C ends the+run. With `--output-format stream-json --verbose`, every message also arrives as a+`hook_response` event.-Each slash command is one model turn. The same commands are free from a terminal, as+Each slash command costs one model turn; the same commands are free from a terminal, as`~/.tokeneconomizer/bin/te --host claude <command>` run inside the repository-([Commands](#commands)). For one session without installing anything:+([Commands](#commands)). To try the plugin for one session without installing it:`claude --plugin-dir /path/to/TokenEconomizer/plugin`.-**Update.** `git pull` in the clone, then start a new session (or `/reload-plugins`). Claude Code-runs a plugin added from a local folder from that folder. If `/tokeneconomizer:status` still-shows the old behaviour, run `claude plugin update tokeneconomizer@tokeneconomizer`.+**Update.** `git pull` in the clone, then start a new session (or `/reload-plugins`): Claude Code+runs the plugin straight from the clone. If `/tokeneconomizer:status` still shows the old+behaviour, run `claude plugin update tokeneconomizer@tokeneconomizer`.**Uninstall.**@@ -114,37 +111,38 @@ codex plugin add tokeneconomizer@tokeneconomizerHooks run from the next session on (the session in which you approve gets no session-startinjection), so quit with `/exit` and start `codex` again. To approve later: `/hooks`.-**3. Check it.** Type `$tokeneconomizer:status`. Codex has no plugin slash commands; the plugin's-commands are Codex skills, named like the slash commands on the other agents (type-`$tokeneconomizer` to pick one): `$tokeneconomizer:status`, `:microskills`, `:toolprune`,-`:config` and `:repos`, each followed by its arguments (`$tokeneconomizer:config set-loopguard=off`). Name the skill: a question in plain words (*"what has TokenEconomizer learned-here?"*) may be answered from the plugin's files without running its command. A command that-changes something (`config set` or `unset`, `toolprune apply` or `restore`, `repos migrate` or-`merge`) writes the plugin's state in `~/.tokeneconomizer`, which Codex's sandbox keeps-read-only, so Codex asks first: choose **Yes, proceed**. From a terminal, inside the repository:+**3. Check it.** Type `$tokeneconomizer:status`. Codex has no plugin slash commands, so the+plugin's commands are Codex skills with the same names as on the other agents:+`$tokeneconomizer:status`, `:microskills`, `:toolprune`, `:config` and `:repos`, each followed by+its arguments (`$tokeneconomizer:config set loopguard=off`); type `$tokeneconomizer` to pick one.+Name the skill rather than asking in plain words: a question such as *"what has TokenEconomizer+learned here?"* may be answered from the plugin's files without running its command. Commands+that change something (`config set` or `unset`, `toolprune apply` or `restore`, `repos migrate`+or `merge`) write to `~/.tokeneconomizer`, which Codex's sandbox keeps read-only, so Codex asks+first: choose **Yes, proceed**. From a terminal, inside the repository:`~/.tokeneconomizer/bin/te --host codex <command>` (the launcher appears after the firstsession).-**4. Work normally.** See [What to expect](#what-to-expect). Codex's deny list turns off the-sub-agent (`collaboration`), `goals` and image (`image_media`) packages, each only once it has-been loaded and unused long enough ([What to expect](#what-to-expect)); a package you used even-once is kept. It is-written to the project's `.codex/config.toml`, which Codex reads **only in a trusted project**-(the session start that announces it says so otherwise), and never into one your repository-commits: that stays a proposal, and the session start says why. The web, asking you questions-and the bundled skills are never turned off. `$tokeneconomizer:toolprune restore` gives the-packages back, and `toolprune measure` shows the difference with no model call.--In `codex exec`, Codex does not print hook messages, so the plugin's notices are not shown-there; a loop guard redirect still reaches the model. In the terminal UI a turn that is still-stuck after the redirect shows "↳ Hook · TokenEconomizer: … Press Esc to interrupt it, then /stop+**4. Work normally.** See [What to expect](#what-to-expect). On Codex the deny list can turn off+the sub-agent (`collaboration`), `goals` and image (`image_media`) packages, each only after it+has been loaded and gone unused long enough; a package you used even once is kept. Web access,+asking you questions and the bundled skills are never turned off. The list is written to the+project's `.codex/config.toml`, which Codex reads **only in a trusted project** (otherwise the+session start that announces it says so), and never into a `config.toml` your repository+commits: that stays a proposal, and the session start says why.+`$tokeneconomizer:toolprune restore` gives the packages back, and `toolprune measure` shows the+difference with no model call.++**Messages.** Under `codex exec`, Codex prints no hook messages, so the plugin's notices are not+shown, though a loop guard redirect still reaches the model. In the terminal UI, a turn still+stuck after its redirect shows "↳ Hook · TokenEconomizer: … Press Esc to interrupt it, then /stopto end a command it left running": Esc interrupts the turn, but a command it started keepsrunning in the background until `/stop`. Under `codex exec`, Ctrl+C ends the run and its command.-Quitting Codex's interactive mode (`/exit`) keeps the-session open to `codex resume` and sends no session end, so the plugin learns from it at a later-session start, once it has been quiet for ten minutes; `codex exec` sessions are learned from-as they end.++**When a session is learned from.** Quitting interactive Codex (`/exit`) keeps the session open+for `codex resume` and sends no session end, so the plugin learns from it at a later session+start, once it has been quiet for ten minutes. `codex exec` sessions are learned from as they+end.**Update.** `git pull` in the clone, then `codex plugin add tokeneconomizer@tokeneconomizer`again to refresh Codex's installed copy (your hook approval is kept).@@ -192,14 +190,13 @@ commands work only in interactive mode; the plugin's notices also appear under `With no model call. Do it once on this machine, and again after Copilot updates.-**4. Work normally.** See [What to expect](#what-to-expect). Copilot has no setting that removes a-single built-in tool, only one that turns off a whole MCP server, so the deny list turns off a-server only when **none** of its tools was used long enough ([What to expect](#what-to-expect)):-one of yours from+**4. Work normally.** See [What to expect](#what-to-expect). Copilot cannot remove a single+built-in tool, only turn off a whole MCP server, so the deny list turns off a server only when+**all** of its tools have gone unused long enough: one of yours from`~/.copilot/mcp-config.json`, or the built-in `github-mcp-server`, which loads only when you aresigned in to GitHub. It is written to `.github/copilot/settings.local.json`, and the next session-start says so; `/tokeneconomizer:toolprune restore` turns the server back on. `toolprune-measure` shows the difference with no model call.+start says so. `/tokeneconomizer:toolprune restore` turns the server back on, and+`toolprune measure` shows the difference with no model call.From a terminal, inside the repository: `~/.tokeneconomizer/bin/te --host copilot <command>` (thelauncher appears after the first session). A turn that is still stuck after the loop guard's@@ -252,13 +249,13 @@ To check the whole plugin on your own setup:~/.tokeneconomizer/bin/te --host copilot probe # 7 short real sessions, in a temp repo```-It builds a throwaway repository that is set up for pytest but has no pytest installed, and-whose tests pass under `python -m unittest`, and runs the same prompt in real Copilot sessions-with the plugin, under a state directory of its own. It reports, per session, whether the fact-the plugin learned was in the context and which test command the model ran first. Then it-applies the deny list it proposed, and shows the tool definition tokens of one more session-before restoring. It makes real model calls on your key or quota, and leaves those sessions in-Copilot's history.+It builds a throwaway repository that is set up for pytest, without pytest installed, and whose+tests pass under `python -m unittest`. It runs the same prompt in real Copilot sessions with the+plugin, under a state directory of its own, and reports for each session whether the fact the+plugin learned was in the context and which test command the model ran first. Then it applies the+deny list it proposed, shows the tool-definition tokens of one more session, and restores the+list. It makes real model calls on your key or quota, and leaves those sessions in Copilot's+history.## Pi@@ -289,44 +286,47 @@ from); `pi config` turns it off without removing it. Optional: `rm -rf ~/.tokene## What to expect+The plugin works session by session. A session is one conversation with the agent, from start to+exit, however many tasks you do in it; resuming it continues the same session. The plugin learns+from a session once it ends, and acts at the start of the next one; only the loop guard acts+during a turn.+- **Facts** (once you turn micro-skills on; it is off by default). Two sources propose them, in- any language or toolchain: the same environment failure coming up in a few sessions (a tool,- interpreter, build or test command that fails as typed and works spelled another way), with- no model call; and, every `microskills_propose_every` sessions (3 by default), a model reading digests of the recent ones (the- commands run, the build, test and lint commands among them, the files edited, and each- failing command with its error line, paths relative to the repository) and writing facts- about the repository, each with the patterns that test it against later sessions. A fact is- injected at every session start once later sessions have confirmed it (three confirming ones,- among at least four where it applied), and retired when one contradicts it. How much this helps- depends on the repository's state and the tasks you run in it. A session start that adds facts- says so once, in one line. A resumed session keeps the facts it started with, word for word, so- its cached prompt still matches. `microskills` shows the facts injected and those forming.-- **Deny list.** Applied without asking you once a tool, package or server never used here has- gone unused over `toolprune_evidence` prompts (10 by default) across at least- `toolprune_min_sessions` sessions (2), or over `toolprune_sessions` sessions however short (5)- -- three times as long for a few higher-risk tools -- from the next session on; that session's- start says so once. Prompts let long sessions count for more, and the session floor keeps one- session's work from deciding alone. A session that ran no prompt counts for nothing. It is- written once the repository's sessions have been idle for `toolprune_quiet_minutes` (60 by- default): a conversation resumed- after its tools change writes its whole history to the prompt cache again instead of reading- it, and after an idle hour that cache has expired anyway. `toolprune` shows a list still- waiting. `toolprune restore` gives back what the plugin removed, and only that, and what you+ any language or toolchain. The first needs no model call: the same environment failure in a+ few sessions (a tool, interpreter, build or test command that fails as typed and works spelled+ another way). The second runs every `microskills_propose_every` sessions (3 by default): a+ model reads digests of the recent sessions (the commands run, including build, test and lint+ commands, the files edited, and each failing command with its error line, with paths relative+ to the repository) and writes facts about the repository, each with patterns that test it+ against later sessions. A fact is injected at every session start once later sessions confirm+ it (three confirming sessions among at least four where it applied), and retired when one+ contradicts it. How much this helps depends on the repository's state and the tasks you run in+ it. A session start that adds facts says so once, in one line. A resumed session keeps the+ facts it started with, word for word, so its cached prompt still matches. `microskills` shows+ the facts injected and those still forming.+- **Deny list.** A tool, package or server that is loaded but never used here is left out without+ asking you, once it has gone unused over `toolprune_evidence` prompts (10 by default) across at+ least `toolprune_min_sessions` sessions (2), or over `toolprune_sessions` sessions of any length+ (5); a few higher-risk tools need three times as long. A session that ran no prompt does not+ count. The list is written once the repository's sessions have been idle for+ `toolprune_quiet_minutes` (60 by default), so that no conversation you resume loses its prompt+ cache, and it applies from the next session, whose start says so once. `toolprune` shows a list+ still waiting. `toolprune restore` gives back what the plugin removed, and only that; what yougive back is never removed again (`toolprune apply NAME` removes one again). With- `toolprune=propose`, it is only proposed, the session start says so, and nothing changes- until you run `toolprune apply`. A file the plugin creates is added to `.git/info/exclude`.- Each checkout keeps its own settings file: a repository's other clones share what it learned,- and each gets the list when one of its own sessions ends.-- **Loop guard.** Quiet on normal turns, and it does nothing until `loopguard.envelope_min_samples`- turns in a repository (5 by default) have ended with passing checks. A turn that keeps hitting- the same failures, or piles up errors, after `loopguard.min_turns` tool calls is put to a judge- model; only a "stuck" verdict with confidence `loopguard.judge_confidence` (0.7) or more+ `toolprune=propose`, the list is only proposed, the session start says so, and nothing changes+ until you run `toolprune apply`. A settings file the plugin creates is added to+ `.git/info/exclude`. Each checkout keeps its own settings file: a repository's clones share+ what was learned, and each gets the list when one of its own sessions ends.+- **Loop guard.** Quiet on normal turns, and idle until `loopguard.envelope_min_samples` turns in+ a repository (5 by default) have ended with passing checks. After that, a turn that keeps+ hitting the same failures, or piles up errors, after `loopguard.min_turns` tool calls is put to+ a judge model. Only a "stuck" verdict with confidence `loopguard.judge_confidence` (0.7) or moreattaches a redirect (written by that model) to a tool result, at most `loopguard_nudges` (1)- per turn, and if the turn is still stuck after that, a one-line notice tells you how to stop- it. A long turn with no such sign is never judged. A turn past `loopguard.backstop_mult` (2.5)+ per turn; if the turn is still stuck after that, a one-line notice tells you how to stop it. A+ long turn with no sign of looping is never judged. A turn past `loopguard.backstop_mult` (2.5)times the repository's usual length that also keeps churning edits through the same failures- is redirected even if the judge disagrees. The plugin never stops a turn itself.- Where the notice appears, and how to stop, on each agent and in each mode:+ is redirected even if the judge disagrees. The plugin never stops a turn itself. Where the+ notice appears, and how to stop the turn, on each agent and in each mode:[When a turn gets stuck](../README.md#when-a-turn-gets-stuck).- **Cost.** `status` shows each session's tokens, priced after it ends, and the plugin's ownoverhead: the loop guard's judge, and micro-skills' proposer when it is on.@@ -396,7 +396,7 @@ and are shared by every agent working in it. The examples use Claude Code's form| Mechanism | Key | `on` means | Other values ||---|---|---|---|| Micro-skills (off by default) | `microskills` | `envfacts+distiller`: facts found without model calls, plus a model proposing facts every `microskills_propose_every` sessions | `off`; `envfacts`: only the facts found without model calls; `distiller`: only the proposer |-| Tool pruning | `toolprune` | `auto`: applied once it forms and your sessions have been idle an hour, from the next session, and announced | `off`; `propose`: only proposed, applied when you run `toolprune apply` |+| Tool pruning | `toolprune` | `auto`: applied without asking once your sessions have been idle for `toolprune_quiet_minutes`, from the next session, which announces it | `off`; `propose`: only proposed, applied when you run `toolprune apply` || Loop guard | `loopguard` | `nudge`: a judge model decides whether a looping turn is stuck; if so, one mid-turn redirect, then a notice to you if the turn is still stuck | `off` (no judge either); `stop` is read as `nudge`, with a warning |**Tune a mechanism:**@@ -517,21 +517,23 @@ not rewritten, so the commands that fact recommends keep working.## Model calls and API keys-**By default the plugin's only model calls are the loop guard's**: its judge, asked about a turn-that shows signs of looping, and, when the judge finds it stuck, the redirect it writes. Tool-pruning never calls a model. Turning micro-skills on adds its fact proposer, once every-`microskills_propose_every` sessions (`microskills=envfacts` keeps only the facts found without-a model call); `loopguard=off` stops the judge with the guard. These calls run through **your-own agent CLI** (`claude -p`; `codex exec` on Codex; `copilot -p`, with every tool removed, on-Copilot; `pi -p`, with nothing loaded but the model, on Pi), so they use whatever your sessions-already use: a Claude subscription or Anthropic API key, your Codex login or OpenAI key,-Copilot's subscription, or the provider and login Pi is set up with. (Copilot with BYOK is the-exception; see [Copilot CLI](#copilot-cli).) Their cost is billed there too, and shown in-`status`. A call that gets no reply leaves a line in `~/.tokeneconomizer/errors.log`; one whose-CLI answered that it is not signed in also pauses that agent's calls (the loop guard then-redirects only at its backstop), the next session start says so once, and-`config set microskills=on` (or `loopguard=on`) tries again. `toolprune measure` on Claude Code-makes its two calls the same way, and only when you run it.+**With the defaults, the plugin's only model calls are the loop guard's**: the judge, asked about+a turn that shows signs of looping, and, when the judge finds the turn stuck, the redirect it+writes. Tool pruning never calls a model. Turning micro-skills on adds its fact proposer, once+every `microskills_propose_every` sessions (`microskills=envfacts` keeps only the facts found+without a model call); `loopguard=off` turns the judge off along with the guard.++These calls run through **your own agent's CLI**: `claude -p`; `codex exec`; `copilot -p`, with+every tool removed; and `pi -p`, with nothing loaded but the model. So they use whatever your+sessions already use (a Claude subscription or Anthropic API key, your Codex login or OpenAI key,+Copilot's subscription, or the provider and login Pi is set up with), are billed there, and are+shown in `status`. Copilot with BYOK is the exception; see [Copilot CLI](#copilot-cli).++A call that gets no reply leaves a line in `~/.tokeneconomizer/errors.log`. If the CLI answers+that it is not signed in, that agent's calls are also paused (the loop guard then redirects only+at its backstop), the next session start says so once, and `config set microskills=on` (or+`loopguard=on`) tries again. `toolprune measure` on Claude Code makes its two calls the same way,+and only when you run it.## Troubleshooting