Experience-Driven Black-Box Cost Control for Coding Agents
Python agentic-aiagentic-codingai-codingai-coding-assistantanthropicartificial-intelligenceclaude-codecodexcoding-agentcoding-agentscost-optimizationcost-reductiondeveloper-toolsgithub-copilotllmopen-sourceopenaiproductivitytoken-optimization
Verify Cursor hooks and OpenCode 2.1.4
Shadowde9e2fc7f24b
diff --git a/README.md b/README.mdindex a5d8865..8660598 100644--- a/README.md+++ b/README.md@@ -211,7 +211,7 @@ uninstalling, using Copilot with your own model key, and troubleshooting.| Copilot CLI | 1.0.83 to 1.0.102 | not supported || Pi | 1.2.0 to 1.3.0 | 0.99.x: load the extension by path (below) || Cursor Agent (experimental) | current command-hook schema | no version range verified |-| opencode (experimental) | 1.19.5 and 2.2.0 | not tested |+| opencode (experimental) | 1.19.5, 2.1.4, and 2.2.0 | not tested |The commands above were tested on the oldest and newest version in each range; newer versionshave not been tested yet.@@ -220,7 +220,7 @@ have not been tested yet.- **Pi 0.99.x**: instead of `pi install`, clone the repository and start each session with`pi --extension /path/to/TokenEconomizer/plugin/pi/tokeneconomizer.ts`, using your clone'sfull path; `git pull` in the clone updates it.-- **opencode** support is experimental: it was tested on 1.19.5 and 2.2.0 only, and not in the+- **opencode** support is experimental: it was tested on 1.19.5, 2.1.4, and 2.2.0, and not in the2.x terminal UI.- **Cursor Agent** support is experimental; the hook event mapping has not yet been verified inthe live Cursor app.@@ -254,11 +254,10 @@ Inside an agent, a command costs one model turn; from a terminal it is free. The## Settings-The loop guard is on by default. Tool pruning is off by default and never removes tools or MCP-servers unless you explicitly enable it. With these defaults, the plugin calls a model only when a-turn shows signs of looping: the judge, and the redirect it writes if it finds the turn stuck.-Micro-skills is off by default, because its proposer adds model calls and how much it saves depends-on the repository and the work you do there. Each mechanism has its own switch+Tool pruning and the loop guard are on by default; with these defaults, the plugin calls a model+only when a turn shows signs of looping: the judge, and the redirect it writes if it finds the+turn stuck. Micro-skills is off by default, because its proposer adds model calls and how much it+saves depends on the repository and the work you do there. Each mechanism has its own switch(`microskills`, `toolprune`, `loopguard`) that takes `on` or `off`, and the italic letters aboveare these settings:diff --git a/docs/PLUGIN.md b/docs/PLUGIN.mdindex 033831c..f4cfc6e 100644--- a/docs/PLUGIN.md+++ b/docs/PLUGIN.md@@ -54,7 +54,7 @@ Every hook fails open: any error becomes an empty output (no change) and a trace| Unit of work | one benchmark task | a **session** for micro-skills and tool pruning; a **prompt-turn** for the loop guard | In interactive use a person already splits work into short turns. A whole session folded as one trajectory would look stuck merely by being long || Learning signal | the official grader's `resolved` | the envelope learns from turns that **ended green**: the agent stopped on its own, was not stopped by the guard, and no verification after its last edit failed | There is no grader. Micro-skills and evidence pruning never needed one || Injection channel | a prefix on the task prompt | `SessionStart` `additionalContext` | Verified to reach the model (§3) |-| Tool removal | `--disallowed-tools` / Codex config at launch | off by default; when explicitly enabled, a **deny list** is applied after evidence forms and repository sessions have been idle longer than a prompt cache lives (§3), and announced at the next session start (`toolprune=propose` waits for approval instead), written to the repository's own settings file — tools in `.claude/settings.local.json`, Codex capability packages in `.codex/config.toml`, Copilot MCP servers in `.github/copilot/settings.local.json` — effective from the **next** session | The plugin launches nothing. A session keeps the tools it started with, but a conversation resumed after a change sends the new ones, re-writing its cached prefix, hence the idle wait. Each agent's file can express a different unit (§3) |+| Tool removal | `--disallowed-tools` / Codex config at launch | a **deny list** is applied by default after evidence forms and repository sessions have been idle longer than a prompt cache lives (§3), and announced at the next session start (`toolprune=propose` waits for approval instead), written to the repository's own settings file — tools in `.claude/settings.local.json`, Codex capability packages in `.codex/config.toml`, Copilot MCP servers in `.github/copilot/settings.local.json` — effective from the **next** session | The plugin launches nothing. A session keeps the tools it started with, but a conversation resumed after a change sends the new ones, re-writing its cached prefix, hence the idle wait. Each agent's file can express a different unit (§3) || Pruning candidates | `toolprune.tools.INAPPLICABLE_TOOLS` — orchestration, Q&A, plan mode, web — "inapplicable by construction" for a sealed headless task | nothing is inapplicable by construction; only **evidence** (never used, and unused over `toolprune_evidence` prompts (10) across at least `toolprune_min_sessions` sessions (2), or over `toolprune_sessions` sessions (5)) proposes a tool, and a person's tools — AskUserQuestion, plan mode, sub-agents, the web — are protected | A person asks questions, plans and browses in a real session. A deny rule the plugin wrote for a tool that has since become protected is given back at the next fold || Network | every arm, baseline included, carries an answer-key guard — a `PreToolUse` hook and an `/etc/hosts` block refuse the benchmarks' upstream sources — and drops the web tools | **no guard**: no `PreToolUse` hook (on Pi, no `tool_call` handler), no blocked command or host; the web tools stay | A benchmark's answer is a fetch away. A real session has no answer key, and the web is part of the work || Who decides a turn is stuck | in the guard's arms, the trace judge, woken by the rule gate or by a turn past the envelope (the agents ran under a turn cap); without a judge, the strict rule | **always the judge** when the guard is on, woken only by a sign of trouble (a repeated failure, or dense errors) after `min_turns` calls, and never before the repository has an envelope; a `productive` or ambiguous verdict spares the turn (at the `aggressive` preset an ambiguous one past the envelope gets the redirect). The hard backstop (far past the envelope AND the strict stall rule) still acts without it. While the agent's model calls are paused, the judge gives no verdict and only the backstop acts | The rule alone redirected turns that had found their cause and were working around it (§5); interactive turns outrun a small envelope all the time, so the envelope alone wakes nothing |@@ -525,7 +525,7 @@ Which repository a session belongs to (`plugin/paths.py`, `plugin/repos.py`):agent files its own session log. `te status` run inside a session says so when the two differ.* **Deny lists stay per checkout.** The clones of a repository share its evidence and proposal,but each has its own settings file, so `untool_applied.json` records per file what was added,- and what the person gave back with `restore`: `toolprune: off`, the default, never applies that+ and what the person gave back with `restore`: `toolprune: auto`, the default, never applies thatagain (an explicit `toolprune apply NAME` does), as the evidence would keep proposing it.* **State from earlier keys is carried over.** Up to 0.1.0 a checkout's folder name was part ofthe key, any `.git` entry made a checkout, and a session was recorded wherever the agent was.diff --git a/plugin/README.md b/plugin/README.mdindex fcbb429..3af995f 100644--- a/plugin/README.md+++ b/plugin/README.md@@ -324,7 +324,7 @@ uses them to learn repository facts and watch for loops, and reports Cursor sessunavailable. The loop guard's model calls use `cursor-agent` in read-only Ask mode; their tokenusage is also unavailable in the current CLI output.-Tool pruning is off by default and is not applied to Cursor. Cursor commands run from a terminal:+Tool pruning follows the global setting and is not applied to Cursor. Cursor commands run from a terminal:`~/.tokeneconomizer/bin/te --host cursor <command>`.## opencode@@ -494,7 +494,7 @@ and are shared by every agent working in it. The examples use Claude Code's form| Mechanism | Key | `on` means | Other values ||---|---|---|---|| Micro-skills (off by default) | `microskills` | `envfacts+distiller`: facts found without model calls, plus a model proposing facts every `microskills_propose_every` sessions | `off`; `envfacts`: only the facts found without model calls; `distiller`: only the proposer |-| Tool pruning (off by default) | `toolprune` | `auto`: applied without asking once your sessions have been idle for `toolprune_quiet_minutes` (or a day after it formed), from the next session, which announces it | `off`: never remove tools or MCP servers; `propose`: only proposed, applied when you run `toolprune apply` |+| Tool pruning (on by default) | `toolprune` | `auto`: applied without asking once your sessions have been idle for `toolprune_quiet_minutes` (or a day after it formed), from the next session, which announces it | `off`: never remove tools or MCP servers; `propose`: only proposed, applied when you run `toolprune apply` || Loop guard | `loopguard` | `nudge`: a judge model decides whether a looping turn is stuck; if so, up to `loopguard_nudges` mid-turn redirects, then a notice to you if the turn is still stuck | `off` (no judge either); `stop` is read as `nudge`, with a warning |**Tune a mechanism:**diff --git a/tests/test_cursor.py b/tests/test_cursor.pynew file mode 100644index 0000000..93da637--- /dev/null+++ b/tests/test_cursor.py@@ -0,0 +1,120 @@+"""Cursor hook adapter: published payloads, private event capture and trace reconstruction."""+import json+import os+from pathlib import Path++from tokeneconomizer import trace+from tokeneconomizer.plugin.config import LiveConfig+from tokeneconomizer.plugin.hosts import Cursor+from tokeneconomizer.plugin.hook import run+from tokeneconomizer.plugin.paths import state_root+++def test_tool_pruning_can_be_disabled_by_a_user_config():+ config = LiveConfig()+ assert config.toolprune == "auto"+ assert config.set("toolprune", "off") and config.toolprune == "off"+++def test_cursor_translates_prompt_and_tool_hook_schemas(tmp_path):+ host = Cursor()+ env = {"TOKENECONOMIZER_HOME": str(tmp_path / "state"),+ "CURSOR_PROJECT_DIR": str(tmp_path / "repo")}++ prompt = host.prepare({+ "hook_event_name": "beforeSubmitPrompt", "conversation_id": "conv-1",+ "generation_id": "gen-1", "prompt": "Fix the bug",+ "workspace_roots": [str(tmp_path / "repo")],+ }, env)+ assert prompt["hook_event_name"] == "UserPromptSubmit"+ assert (prompt["session_id"], prompt["prompt_id"], prompt["cwd"]) == (+ "conv-1", "gen-1", str(tmp_path / "repo"))+ assert prompt["transcript_path"] == str(+ tmp_path / "state" / "cursor-sessions" / "conv-1.jsonl")++ success = host.prepare({+ "hook_event_name": "postToolUse", "conversation_id": "conv-1",+ "generation_id": "gen-1", "tool_name": "Shell",+ "tool_input": {"command": "pytest"},+ "tool_output": '{"exitCode":1,"stdout":"failed","stderr":"details"}',+ }, env)+ assert success["hook_event_name"] == "PostToolUse"+ assert success["tool_use_id"] == "gen-1"+ assert success["tool_response"] == {+ "content": "failed\ndetails", "exit_code": 1, "isError": False}++ failure = host.prepare({+ "hook_event_name": "postToolUseFailure", "conversation_id": "conv-1",+ "generation_id": "gen-1", "tool_name": "Shell",+ "tool_input": {"command": "rm -rf /"}, "error_message": "Denied",+ "failure_type": "permission_denied",+ }, env)+ assert failure["hook_event_name"] == "PostToolUseFailure"+ assert failure["error"] == "Permission denied: Denied"+++def test_cursor_plugin_declares_all_supported_hook_events():+ root = Path(__file__).resolve().parents[1]+ manifest = json.loads((root / "plugin/.cursor-plugin/plugin.json").read_text())+ hooks = json.loads((root / "plugin/hooks/cursor-hooks.json").read_text())["hooks"]+ assert manifest["name"] == "tokeneconomizer"+ assert manifest["hooks"] == "./hooks/cursor-hooks.json"+ assert set(hooks) == {+ "sessionStart", "beforeSubmitPrompt", "postToolUse", "postToolUseFailure",+ "stop", "sessionEnd",+ }+ assert all("${CURSOR_PLUGIN_ROOT}/scripts/te-hook" in hook["command"]+ and "--host cursor" in hook["command"]+ for entries in hooks.values() for hook in entries)+++def test_cursor_records_only_allowlisted_repository_events(tmp_path):+ allowed = tmp_path / "allowed"+ (allowed / ".git" / "info").mkdir(parents=True)+ (allowed / ".git" / "HEAD").write_text("ref: refs/heads/main\n")+ (allowed / ".git" / "config").write_text(+ '[remote "origin"]\n\turl = https://github.com/acme/widget.git\n')+ excluded = tmp_path / "excluded"+ (excluded / ".git" / "info").mkdir(parents=True)+ (excluded / ".git" / "HEAD").write_text("ref: refs/heads/main\n")+ (excluded / ".git" / "config").write_text(+ '[remote "origin"]\n\turl = https://github.com/acme/other.git\n')+ state = tmp_path / "state"+ os.makedirs(state)+ (state / "config.json").write_text(json.dumps({"repositories": [str(allowed)]}))+ env = {"TOKENECONOMIZER_HOME": str(state), "TE_LOOPGUARD": "off",+ "GIT_CEILING_DIRECTORIES": str(tmp_path), "PATH": os.environ.get("PATH", "")}++ event = {"hook_event_name": "beforeSubmitPrompt", "conversation_id": "conversation-a",+ "generation_id": "generation-a", "prompt": "Fix the issue",+ "workspace_roots": [str(excluded)]}+ assert run(event, env, host_hint="cursor") == {}+ assert not os.path.exists(state_root(env) + "/cursor-sessions")++ event["workspace_roots"] = [str(allowed)]+ assert run(event, env, host_hint="cursor") == {}+ spool = state / "cursor-sessions" / "conversation-a.jsonl"+ rows = [json.loads(line) for line in spool.read_text().splitlines()]+ assert len(rows) == 1 and rows[0]["prompt"] == "Fix the issue"+++def test_cursor_events_reconstruct_tool_results_and_count_prompts(tmp_path):+ source, target = tmp_path / "events.jsonl", tmp_path / "trace.jsonl"+ events = [+ {"hook_event_name": "UserPromptSubmit", "prompt": "Fix the bug"},+ {"hook_event_name": "PostToolUse", "tool_use_id": "t1", "tool_name": "Shell",+ "tool_input": {"command": "pytest"},+ "tool_response": {"content": "3 passed", "exit_code": 0}},+ {"hook_event_name": "PostToolUseFailure", "tool_use_id": "t2",+ "tool_name": "Read", "tool_input": {"file_path": "missing"},+ "error": "not found"},+ ]+ source.write_text("\n".join(json.dumps(event) for event in events) + "\n")++ assert trace.cursor_prompt_count(str(source)) == 1+ assert trace.cursor_events_to_claude(str(source), str(target)) == 5+ calls = trace.load_trajectory(str(target))+ assert [(name, args, ok) for name, args, _result, ok in calls] == [+ ("Bash", {"command": "pytest"}, True),+ ("Read", {"file_path": "missing"}, False),+ ]diff --git a/tokeneconomizer/plugin/config.py b/tokeneconomizer/plugin/config.pyindex ee22c97..315aa4b 100644--- a/tokeneconomizer/plugin/config.py+++ b/tokeneconomizer/plugin/config.py@@ -8,10 +8,12 @@ The defaults:calls a model through the agent's own CLI (charged to theledger) to write facts beyond them, as in the experiments."envfacts" alone makes no model call;- * toolprune = "off" -- no tools or MCP servers are removed by default. Set it to `auto`- to apply a deny list after the evidence forms and the repository's- sessions have been idle, or to `propose` to wait for `toolprune- apply`;+ * toolprune = "auto" -- a deny list is applied once the evidence forms it and the+ repository's sessions have been idle (toolprune_quiet_minutes),+ from the next session, and announced once at that session's start;+ `toolprune restore` gives it back. "propose" only proposes it,+ and writes nothing to the agent's settings until the person+ runs `toolprune apply`;* loopguard = "nudge" -- a turn that shows signs of looping is put to a judge, a modelcalled through the agent's own CLI, and redirected mid-turn onlywhen the judge finds it stuck (the redirect written by that@@ -149,7 +151,7 @@ class LiveConfig:microskills_max_facts: int = 4microskills_max_tokens: int = 150microskills_propose_every: int = 3- toolprune: str = "off"+ toolprune: str = "auto"toolprune_evidence: int = 10toolprune_min_sessions: int = 2toolprune_sessions: int = 5diff --git a/tokeneconomizer/plugin/hosts.py b/tokeneconomizer/plugin/hosts.pyindex 043e59a..f9a1716 100644--- a/tokeneconomizer/plugin/hosts.py+++ b/tokeneconomizer/plugin/hosts.py@@ -414,14 +414,43 @@ class Cursor(Host):"sessionEnd": "SessionEnd",}+ def session_id(self, p):+ return str(p.get("session_id") or p.get("conversation_id") or "")++ def cwd(self, p, env):+ roots = p.get("workspace_roots")+ root = roots[0] if isinstance(roots, list) and roots else ""+ return p.get("cwd") or env.get("CURSOR_PROJECT_DIR") or root or os.getcwd()++ def turn_key(self, p):+ return str(p.get("generation_id") or p.get("prompt_id") or "")++ def context(self, event, text):+ return {"additional_context": text}++ def notice(self, text):+ return {"additional_context": self.relay(text)}++ def transcript(self, p):+ if p.get("transcript_path"):+ return p["transcript_path"]+ sid = self.session_id(p)+ if not sid:+ return ""+ from .paths import state_root+ safe_id = os.path.basename(sid)+ return os.path.join(state_root(), "cursor-sessions", safe_id + ".jsonl")+def prepare(self, payload, env):p = dict(payload or {})p["session_id"] = p.get("conversation_id") or p.get("session_id") or ""- roots = p.get("workspace_roots")- root = roots[0] if isinstance(roots, list) and roots else ""- p["cwd"] = p.get("cwd") or env.get("CURSOR_PROJECT_DIR") or root or os.getcwd()+ p["cwd"] = (p.get("cwd") or env.get("CURSOR_PROJECT_DIR") or+ ((p.get("workspace_roots") or [""])[0]) or os.getcwd())p["hook_event_name"] = self._EVENTS.get(p.get("hook_event_name"), "")- p["transcript_path"] = self.transcript(p)+ if p.get("session_id"):+ from .paths import state_root+ p["transcript_path"] = os.path.join(+ state_root(env), "cursor-sessions", os.path.basename(p["session_id"]) + ".jsonl")if p["hook_event_name"] == "SessionStart":p["source"] = "startup"if p["hook_event_name"] == "UserPromptSubmit":@@ -449,31 +478,6 @@ class Cursor(Host):p["error"] = "Permission denied: " + str(p["error"])return p- def session_id(self, p):- return str(p.get("session_id") or p.get("conversation_id") or "")-- def cwd(self, p, env):- roots = p.get("workspace_roots")- root = roots[0] if isinstance(roots, list) and roots else ""- return p.get("cwd") or env.get("CURSOR_PROJECT_DIR") or root or os.getcwd()-- def turn_key(self, p):- return str(p.get("generation_id") or p.get("prompt_id") or "")-- def context(self, event, text):- return {"additional_context": text}-- def notice(self, text):- return {"additional_context": self.relay(text)}-- def transcript(self, p):- sid = self.session_id(p)- if not sid:- return ""- from .paths import state_root- safe_id = os.path.basename(sid)- return os.path.join(state_root(), "cursor-sessions", safe_id + ".jsonl")-def record_event(self, payload):event = payload.get("hook_event_name")if event not in ("UserPromptSubmit", "PostToolUse", "PostToolUseFailure"):