Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
fix(mcp/opencode): don't clobber an unparseable opencode.json on register (#1661)
Abhay Singhd079614b1fe8
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex c1541f99a..b6a7591f0 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -47,6 +47,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0* **cli/proxy:** honor `HEADROOM_MIN_TOKENS=0` / `HEADROOM_MAX_ITEMS=0`. The Click `proxy` command built these with `_get_env_int_optional(name) or 500`/`or 50`, so an explicit `0` — a legitimate value (`min_tokens_to_crush=0` means "crush every item") — was treated as falsy and silently replaced with the default. The `headroom proxy` argparse path already preserved `0` via `_get_env_int`, so the two entry points disagreed. The Click path now uses the same None-checking helper.* **proxy:** strip the inbound `Content-Encoding`/`Transfer-Encoding` request headers on the Anthropic `/v1/messages` and OpenAI `/v1/chat/completions` paths before forwarding upstream. `read_request_json_with_bytes` already decompresses the inbound body (zstd/gzip/deflate/br), so the bytes forwarded upstream are plain JSON — but these two handlers left the original `content-encoding` header in place, so a client (or an edge proxy like a Cloudflare Worker) that sent a compressed body got its request rejected with upstream HTTP 400 because the provider tried to decompress already-decoded JSON. The `/v1/responses` handler already carried this fix (#1542); it is now applied to the messages and chat paths too.* **models:** fix the model registry's prefix fallback silently returning the wrong context window. `ModelRegistry.get` accepted any registered name as a `str.startswith` prefix and returned the *first* match, so `gpt-4-32k-0613` resolved to `gpt-4` (8192) instead of `gpt-4-32k` (32768), and unregistered ids like `gpt-4.1`/`gpt-4.5` inherited `gpt-4`'s 8192-token window — making the proxy think a nearly-empty context was almost full and compress far too aggressively. The fallback now requires the registered name to end at a version boundary in the query (so `gpt-4.1` no longer matches `gpt-4`) and picks the longest qualifying name (so `gpt-4-32k-0613` → `gpt-4-32k`).+* **mcp/opencode:** stop the OpenCode MCP registrar from destroying an existing but unparseable `opencode.json`. `_write_entry` read the config via a helper that returns `{}` on `JSONDecodeError`, then rewrote the whole file with only `{"mcp": {...}}` — wiping the user's `theme`/`model`/`provider` and any other MCP servers (OpenCode configs are commonly JSONC / hand-edited). The write path now refuses to overwrite a present-but-invalid config and returns a `FAILED` result; absent/empty files still register fresh and valid files still merge with all other keys preserved. (Same class of fix as the Claude registrar.)* **proxy:** include the system prompt, tools, and the response-shaping request fields in the SemanticCache key. `_compute_key` hashed only `{model, messages}`, so two non-streaming requests with identical messages but a different top-level `system` prompt, tool set, sampling config, or output-shaping field collided on one key and the second caller was served the first's cached response — generated under different request semantics, in the default config (`cache_enabled` defaults on). The key now folds the request fields that shape generation — `temperature`/`top_p`/`top_k`/`max_tokens`/`stop`, plus OpenAI `tool_choice`/`response_format`/`parallel_tool_calls`/`seed`/`presence_penalty`/`frequency_penalty`/`logit_bias`/`n`/`logprobs`/`top_logprobs`/`reasoning_effort`/`verbosity`/`modalities` and Anthropic `thinking`/`tool_choice`/`output_config` — canonicalizing `system`/`tools` so a moved `cache_control` breakpoint does not fragment it, and the handlers snapshot the fields once at the cache read and reuse them at write so a body mutated by the pipeline cannot diverge the key. Non-streaming path only.* **learn (verbosity):** `--verbosity --apply --all` now aggregates the savings baseline across every project instead of overwriting it per project (last-project-wins), which previously left the output shaper with a tiny, unrepresentative baseline. The applied verbosity level comes from the project with the most samples ([#1288](https://github.com/headroomlabs-ai/headroom/pull/1288)).* **proxy/anthropic:** restore token-mode compression on continued Claude Code turns with a frozen prefix and deferred CCR tool injection. Token mode now runs request-side compression even when the client did not pre-register `headroom_retrieve`, relying on the existing marker-triggered injection override to keep emitted CCR markers redeemable ([#1487](https://github.com/headroomlabs-ai/headroom/issues/1487)).diff --git a/headroom/mcp_registry/opencode.py b/headroom/mcp_registry/opencode.pyindex 6ab6cca91..7a71a1c98 100644--- a/headroom/mcp_registry/opencode.py+++ b/headroom/mcp_registry/opencode.py@@ -35,7 +35,13 @@ def _opencode_config_path() -> Path:def _read_json(path: Path) -> dict[str, Any]:- """Read a JSON file, returning empty dict if absent or unparseable."""+ """Read a JSON file, returning empty dict if absent or unparseable.++ Safe for READ-ONLY callers only. Do NOT use before a full-file rewrite:+ an unparseable existing file returns ``{}`` here, and writing that back+ would destroy the user's other config. Use :func:`_read_json_for_write`+ on the write path instead.+ """if not path.exists():return {}try:@@ -48,6 +54,37 @@ def _read_json(path: Path) -> dict[str, Any]:return data+class _MalformedConfigError(Exception):+ """The target config file exists but is not a parseable JSON object.++ Raised on the write path so we refuse to clobber a config we can't safely+ merge into, rather than silently overwriting the user's other settings.+ """+++def _read_json_for_write(path: Path) -> dict[str, Any]:+ """Read a JSON object for a subsequent full-file rewrite.++ Returns ``{}`` only when the file is absent or empty (safe to start fresh).+ If the file exists with content but does not parse as a JSON object, raise+ :class:`_MalformedConfigError` so the caller aborts instead of overwriting+ unrelated user config — ``opencode.json`` holds ``theme``/``model``/+ ``provider``/other MCP servers alongside the ``mcp`` block.+ """+ if not path.exists():+ return {}+ raw = path.read_text(encoding="utf-8") # OSError propagates to the caller+ if not raw.strip():+ return {}+ try:+ data = json.loads(raw)+ except json.JSONDecodeError as exc:+ raise _MalformedConfigError(str(exc)) from exc+ if not isinstance(data, dict):+ raise _MalformedConfigError("top-level JSON is not an object")+ return data++def _write_json(path: Path, data: dict[str, Any]) -> None:path.parent.mkdir(parents=True, exist_ok=True)with open(path, "w", encoding="utf-8") as f:@@ -164,13 +201,21 @@ class OpencodeRegistrar(MCPRegistrar):def _write_entry(self, spec: ServerSpec) -> RegisterResult:try:self._config_path.parent.mkdir(parents=True, exist_ok=True)- data = _read_json(self._config_path)+ data = _read_json_for_write(self._config_path)mcp = data.setdefault("mcp", {})if not isinstance(mcp, dict):mcp = {}data["mcp"] = mcpmcp[spec.name] = _spec_to_entry(spec)_write_json(self._config_path, data)+ except _MalformedConfigError as exc:+ # Refuse to overwrite: opencode.json holds theme/model/provider and+ # other MCP servers that a blind rewrite would wipe.+ return RegisterResult(+ RegisterStatus.FAILED,+ f"{self._config_path} exists but is not valid JSON ({exc}); "+ "refusing to overwrite. Fix or remove the file, then re-run.",+ )except OSError as exc:return RegisterResult(RegisterStatus.FAILED, f"could not write {self._config_path}: {exc}"diff --git a/tests/test_mcp_registry_opencode.py b/tests/test_mcp_registry_opencode.pyindex 99f4d869d..2340ad649 100644--- a/tests/test_mcp_registry_opencode.py+++ b/tests/test_mcp_registry_opencode.py@@ -96,6 +96,48 @@ def test_register_server_creates_config_when_missing(tmp_path: Path) -> None:}[email protected]("contents", ["not json", "{", '{"theme": }', "[]"])+def test_register_server_preserves_malformed_config(tmp_path: Path, contents: str) -> None:+ """Registering must NOT clobber an existing but unparseable opencode.json.++ The file holds theme/model/provider and other MCP servers; before the fix a+ malformed file was read as {} and rewritten with only {"mcp": ...}."""+ from headroom.mcp_registry.base import ServerSpec++ config_path = tmp_path / "opencode.json"+ config_path.write_text(contents, encoding="utf-8")+ registrar = _registrar(tmp_path)++ spec = ServerSpec(name="headroom", command="headroom", args=("mcp", "serve"))+ result = registrar.register_server(spec)++ assert result.status == RegisterStatus.FAILED+ assert "not valid JSON" in result.detail+ assert config_path.read_text(encoding="utf-8") == contents+++def test_register_server_preserves_other_keys(tmp_path: Path) -> None:+ """The happy path merges: theme/model and an existing MCP server survive."""+ from headroom.mcp_registry.base import ServerSpec++ config_path = tmp_path / "opencode.json"+ _write_json(+ config_path,+ {"theme": "dark", "model": "anthropic/claude", "mcp": {"other": {"type": "local"}}},+ )+ registrar = _registrar(tmp_path)++ spec = ServerSpec(name="headroom", command="headroom", args=("mcp", "serve"))+ result = registrar.register_server(spec)++ assert result.status == RegisterStatus.REGISTERED+ data = json.loads(config_path.read_text(encoding="utf-8"))+ assert data["theme"] == "dark"+ assert data["model"] == "anthropic/claude"+ assert data["mcp"]["other"] == {"type": "local"}+ assert "headroom" in data["mcp"]++def test_register_server_idempotent(tmp_path: Path) -> None:"""register_server is a no-op when the same spec is already present."""registrar = _registrar(tmp_path)@@ -288,17 +330,22 @@ def test_register_then_re_register_with_different_env_returns_mismatch(tmp_path:def test_register_server_on_malformed_config_file(tmp_path: Path) -> None:- """register_server overwrites a malformed config file, preserving nothing."""+ """register_server refuses to overwrite a malformed config, preserving it.++ Updated for the clobber guard: opencode.json holds theme/model/provider and+ other MCP servers, so a present-but-unparseable file is left untouched and+ registration fails rather than silently wiping the user's config.+ """registrar = _registrar(tmp_path)(tmp_path / "opencode.json").write_text("not valid json at all")from headroom.mcp_registry.base import ServerSpecspec = ServerSpec(name="headroom", command="headroom", args=("mcp", "serve"))result = registrar.register_server(spec)- assert result.status == RegisterStatus.REGISTERED+ assert result.status == RegisterStatus.FAILED- data = json.loads((tmp_path / "opencode.json").read_text())- assert "headroom" in data["mcp"]+ # The malformed file is left byte-for-byte untouched rather than clobbered.+ assert (tmp_path / "opencode.json").read_text() == "not valid json at all"def test_entry_to_spec_command_as_string() -> None: