Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
fix(init/codex): merge into hooks.json instead of overwriting it (#2173)
Abhay Singh84f66da36f0b
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 85c35f733..85671853c 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -115,6 +115,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Bug Fixes* **proxy:** run a cold-start fast pass before background-compression deferral so byte-identical freeze doesn't lock sessions to the uncompressed transcript. Since #1850, a session's provider-cached prefix is frozen in whatever form its cold start forwarded; deferring the WHOLE pipeline (`HEADROOM_BACKGROUND_COMPRESSION=1`, frozen=0, ≥50k tokens) therefore cached the raw transcript and forfeited the session's compression savings for its lifetime — including sub-second lossless wins like `read_lifecycle` stale-read drops, observed in the field as sessions permanently stuck at 0 savings. The deferral branch now runs the pipeline synchronously with the new `skip_kompress=True` kwarg (everything except the Kompress ML stage — the only stage that can blow the request budget per #1171) under a bounded fast-pass budget (`HEADROOM_COLD_START_FAST_PASS_TIMEOUT_SECONDS`, default 10s), forwards the pruned form, and defers only Kompress to the background job (tagged `deferred:kompress_background`). Fail-open: on fast-pass timeout/error the request forwards uncompressed exactly as before. Units routed to Kompress under `skip_kompress` take the same fallback as when the model isn't ready.+* **init/codex:** don't overwrite the user's `hooks.json`. `_ensure_codex_hooks` wrote a fresh payload containing only Headroom's two hooks, wholesale-replacing `~/.codex/hooks.json` — so any user-managed Codex hooks (and other top-level keys) were silently destroyed on `headroom init codex`. It now read-merges: existing entries are preserved, Headroom's are deduped on the `headroom-init-codex` marker and appended, matching `_ensure_claude_hooks` / `_ensure_copilot_hooks`.* **ccr:** detect `read_lifecycle` stale/superseded markers in the retrieve-tool injector so they stay redeemable. Those markers (`[Read content stale: … Retrieve original: hash=<hash>]`) store the original bytes in the CCR store under a valid hash, but none of `CCRToolInjector`'s patterns matched them — every pattern required the word "compressed" or the `<<ccr:` form. So on a frozen-prefix turn (both `read_lifecycle` and prefix freezing are on by default) the injector reported no compressed content, the `headroom_retrieve` tool was not injected, and the model was handed a marker advertising `Retrieve original: hash=X` with no tool to redeem it — silent data loss for stale reads, where retrieval is the only way to recover the original-at-read-time content (the exact case the #1006 guard exists to prevent). Added a pattern matching the load-bearing `Retrieve original: hash=` phrase, aligning the injector with the sibling `read_maturation` marker that was already (incidentally) detected.* **init/codex:** stop `headroom init codex` from deleting per-profile provider settings. `_ensure_codex_provider` removed the root-level `model_provider`/`openai_base_url` (which init owns) with a multiline regex that matched those keys in **every** table, so a user's `[profiles.*]` overrides (e.g. `[profiles.work] model_provider = "azure"`) were silently stripped and those profiles fell through to the injected `"headroom"` default — config corruption. The strip is now scoped to the document root (everything before the first table header), so per-profile overrides are preserved while init still replaces a root-level assignment.* **proxy:** strip output-only content blocks from request messages before forwarding. Anthropic's server-side refusal-fallback feature (`server-side-fallback-2026-06-01`) emits a `{"type":"fallback","from":{...},"to":{...}}` block inside the assistant response to signal that a refused request was re-served by the fallback model. That block is valid on the *response* path but rejected on the *request* path, so when a client replays the assistant turn the next request 400s (`invalid_request_error: messages.N.content.0: Input tag 'fallback' ...`) and the conversation gets permanently stuck through the proxy. `read_request_json_with_bytes` (Anthropic/OpenAI/Bedrock) and `_read_request_json` (Gemini) now drop such blocks — re-encoding the raw bytes so byte-faithful passthrough cannot leak the pre-strip body, backfilling a benign text block if a turn is emptied, and leaving requests without such blocks byte-identical (no cache churn).diff --git a/headroom/cli/init.py b/headroom/cli/init.pyindex b37ef7ced..259d4fc8f 100644--- a/headroom/cli/init.py+++ b/headroom/cli/init.py@@ -476,24 +476,43 @@ def _ensure_codex_feature_flag(path: Path) -> None:def _ensure_codex_hooks(path: Path, profile: str) -> None:logger.debug("ensure codex hooks: %s (profile=%s)", path, profile)command = f"{_hook_command('--profile', profile)} --marker {_CODEX_HOOK_MARKER}"- payload = {- "hooks": {- "SessionStart": [- {- "matcher": "startup|resume",- "hooks": [{"type": "command", "command": command, "timeout": 15}],- }- ],- "PreToolUse": [- {- "matcher": "Bash",- "hooks": [{"type": "command", "command": command, "timeout": 15}],- }- ],- }- }- path.parent.mkdir(parents=True, exist_ok=True)- path.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")+ # Read-merge-write rather than overwrite: the previous version wrote a fresh+ # payload wholesale, destroying any user-managed hooks (and other top-level+ # keys) in codex hooks.json. Merge per event and dedup on the Headroom+ # marker, matching _ensure_claude_hooks / _ensure_copilot_hooks.+ payload = _json_file(path)+ hooks = dict(payload.get("hooks") or {}) if isinstance(payload.get("hooks"), dict) else {}+ for event, matcher in (+ ("SessionStart", "startup|resume"),+ ("PreToolUse", "Bash"),+ ):+ entries = list(hooks.get(event) or []) if isinstance(hooks.get(event), list) else []+ retained: list[dict[str, Any]] = []+ for entry in entries:+ if not isinstance(entry, dict):+ retained.append(entry)+ continue+ hook_items = entry.get("hooks")+ if not isinstance(hook_items, list):+ retained.append(entry)+ continue+ has_headroom = any(+ isinstance(item, dict)+ and item.get("command")+ and _CODEX_HOOK_MARKER in str(item.get("command"))+ for item in hook_items+ )+ if not has_headroom:+ retained.append(entry)+ retained.append(+ {+ "matcher": matcher,+ "hooks": [{"type": "command", "command": command, "timeout": 15}],+ }+ )+ hooks[event] = retained+ payload["hooks"] = hooks+ _write_json(path, payload)def _manifest_changed(diff --git a/tests/test_cli/test_init_cli.py b/tests/test_cli/test_init_cli.pyindex f91469f27..20dffb7b7 100644--- a/tests/test_cli/test_init_cli.py+++ b/tests/test_cli/test_init_cli.py@@ -464,6 +464,42 @@ def test_ensure_copilot_hooks_replaces_existing_marker(monkeypatch, tmp_path: Paassert commands == ["echo keep", "headroom init hook ensure --marker headroom-init-copilot"]+def test_ensure_codex_hooks_preserves_user_hooks(monkeypatch, tmp_path: Path) -> None:+ """init codex must merge into hooks.json, not overwrite it — a user's own+ hooks (and unrelated top-level keys) must survive."""+ init_cli, _ = _load_init_module(monkeypatch)+ path = tmp_path / "hooks.json"+ path.write_text(+ json.dumps(+ {+ "notify": True, # unrelated top-level key+ "hooks": {+ "SessionStart": [+ {+ "matcher": "startup",+ "hooks": [{"type": "command", "command": "echo keep"}],+ }+ ]+ },+ }+ ),+ encoding="utf-8",+ )+ monkeypatch.setattr(init_cli, "_hook_command", lambda *parts: "headroom init hook ensure")++ init_cli._ensure_codex_hooks(path, "init-user")++ payload = json.loads(path.read_text(encoding="utf-8"))+ # The unrelated top-level key survives.+ assert payload["notify"] is True+ # The user's own hook is retained and Headroom's is appended exactly once.+ ss_commands = [+ item["command"] for entry in payload["hooks"]["SessionStart"] for item in entry["hooks"]+ ]+ assert "echo keep" in ss_commands+ assert sum(1 for c in ss_commands if "headroom-init-codex" in c) == 1++def test_replace_marker_block_replaces_existing_block(monkeypatch) -> None:init_cli, _ = _load_init_module(monkeypatch)content = "before\n# start\nold\n# end\nafter\n"