Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

fix(cli/init): fail clearly on a target settings file with invalid JSON (#2227)

Abhay Singhdaca1dd75619
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 89d0b4198..92f330f66 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -127,6 +127,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* **savings:** record the pre-compression original as the ledger `before`, not the forwarded count. In `PrometheusMetrics.record_request` the durable savings ledger was written with `tokens_before=input_tokens`, but `input_tokens` there is the optimized (post-compression) count that was forwarded (`emit_request_outcome` passes `outcome.optimized_tokens`). `headroom savings` derives the reported reduction percent as saved / before, so understating `before` by `tokens_saved` inflated it — a real 40% reduction (1000 → 600) was reported as ~67% (400 / 600). The event now records `tokens_before=input_tokens + tokens_saved` (the reconstructed original) and `tokens_after=input_tokens` (the forwarded count); the `saved` and cost figures are unchanged.
* **proxy/gemini:** forward a non-JSON upstream error body with its real status instead of a synthetic 502. In `handle_gemini_generate_content` the token-extraction `except (KeyError, TypeError, AttributeError)` guarding `response.json()` omitted `json.JSONDecodeError` / `ValueError`, so a non-JSON body (an HTML/empty error page from an overloaded Google/Vertex/Copilot frontend, common on 5xx/429) escaped to the outer `except Exception` and was returned as a generic 502 — discarding the real upstream `status_code` and body and defeating the client's retry/backoff. The except now catches the JSON/ValueError family, matching the all-non-text sibling branch, so the true status and body are forwarded verbatim.
* **init/codex:** don't overwrite the user's `hooks.json`. `_ensure_codex_hooks` wrote a fresh payload containing only Headroom's two hooks, wholesale-replacing `~/.codex/hooks.json` — so any user-managed Codex hooks (and other top-level keys) were silently destroyed on `headroom init codex`. It now read-merges: existing entries are preserved, Headroom's are deduped on the `headroom-init-codex` marker and appended, matching `_ensure_claude_hooks` / `_ensure_copilot_hooks`.
+* **cli/init:** fail with an actionable error when a target's settings file contains invalid JSON. `_json_file` (used to read-merge-write Claude's `settings.json`, Codex's `hooks.json`, etc.) called `json.loads` unguarded, so a hand-edit typo (e.g. a trailing comma) crashed `headroom init` with a raw `JSONDecodeError` traceback. It now raises a `ClickException` naming the file and the parse error and telling the user to fix it or move it aside, without touching the file (returning `{}` would have made the follow-up write overwrite the user's settings).
* **ccr:** detect `read_lifecycle` stale/superseded markers in the retrieve-tool injector so they stay redeemable. Those markers (`[Read content stale: … Retrieve original: hash=<hash>]`) store the original bytes in the CCR store under a valid hash, but none of `CCRToolInjector`'s patterns matched them — every pattern required the word "compressed" or the `<<ccr:` form. So on a frozen-prefix turn (both `read_lifecycle` and prefix freezing are on by default) the injector reported no compressed content, the `headroom_retrieve` tool was not injected, and the model was handed a marker advertising `Retrieve original: hash=X` with no tool to redeem it — silent data loss for stale reads, where retrieval is the only way to recover the original-at-read-time content (the exact case the #1006 guard exists to prevent). Added a pattern matching the load-bearing `Retrieve original: hash=` phrase, aligning the injector with the sibling `read_maturation` marker that was already (incidentally) detected.
* **install:** don't crash `headroom install status` when the health payload's `config` is a non-dict. The command did `payload.get('config', {}).get('backend', manifest.backend)`, but `dict.get(..., {})` only defaults on a *missing* key — a present-but-non-dict `config` (`null`, a string, a list, e.g. when a different or older service is answering on the port) reached the chained `.get('backend', ...)` and raised `AttributeError`, crashing the command with a raw traceback. The value is now guarded with `isinstance(config, dict)` before the lookup, mirroring `wrap.py`'s `_proxy_health_config`, so it falls back to the manifest's backend.
* **telemetry:** only advance the usage-report baseline after a confirmed 200. `UsageReporter._report_usage` sends usage as a delta against the last snapshot, but it called `_snapshot_metrics()` (and advanced `_last_report_time`) unconditionally after the POST — including when the send returned non-200 or raised. So a report that failed to reach the cloud (which the module is explicitly designed to tolerate) still rebased the baseline, permanently dropping that window's requests/tokens from usage-based billing/quota; the next report started from the advanced baseline and never re-included them. The baseline now advances only on a 200, so a failed send leaves the window intact for the next report to retry.
diff --git a/headroom/cli/init.py b/headroom/cli/init.py
index 259d4fc8f..8a153ee22 100644
--- a/headroom/cli/init.py
+++ b/headroom/cli/init.py
@@ -150,7 +150,18 @@ def _json_file(path: Path) -> dict[str, Any]:
content = path.read_text(encoding="utf-8").strip()
if not content:
return {}
- payload = json.loads(content)
+ try:
+ payload = json.loads(content)
+ except json.JSONDecodeError as e:
+ # This is a user-owned file (e.g. ~/.claude/settings.json or Codex's
+ # hooks.json) that the callers read-merge-write. Returning {} would make
+ # the following _write_json overwrite it, silently discarding the user's
+ # settings; letting the raw JSONDecodeError propagate crashes `headroom
+ # init` with a traceback. Abort with an actionable message so the user
+ # can fix the JSON (or move it aside) without losing it.
+ raise click.ClickException(
+ f"{path} contains invalid JSON ({e}); fix it and re-run, or move it aside."
+ ) from e
return payload if isinstance(payload, dict) else {}
diff --git a/tests/test_cli/test_init_cli.py b/tests/test_cli/test_init_cli.py
index 20dffb7b7..bfcf1aa77 100644
--- a/tests/test_cli/test_init_cli.py
+++ b/tests/test_cli/test_init_cli.py
@@ -389,6 +389,22 @@ def test_json_file_handles_missing_empty_and_non_mapping(monkeypatch, tmp_path:
assert init_cli._json_file(array_payload) == {}
+def test_json_file_rejects_malformed_json(monkeypatch, tmp_path: Path) -> None:
+ # A user-owned settings file with a hand-edit typo (trailing comma) must
+ # fail with an actionable error rather than crashing init with a raw
+ # JSONDecodeError or silently overwriting the file (which the caller's
+ # subsequent _write_json would do if this returned {}).
+ init_cli, _ = _load_init_module(monkeypatch)
+ malformed = tmp_path / "settings.json"
+ malformed.write_text('{"env": {"A": "B",}}\n', encoding="utf-8")
+
+ with pytest.raises(click.ClickException, match="invalid JSON"):
+ init_cli._json_file(malformed)
+
+ # The file is left untouched (not overwritten).
+ assert malformed.read_text(encoding="utf-8") == '{"env": {"A": "B",}}\n'
+
+
def test_ensure_claude_hooks_rewrites_existing_entries(monkeypatch, tmp_path: Path) -> None:
init_cli, _ = _load_init_module(monkeypatch)
settings_path = tmp_path / "settings.json"