Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

fix(mcp/opencode): don't clobber an unparseable opencode.json on register (#1661)

Abhay Singhd079614b1fe8
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c1541f99a..b6a7591f0 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -47,6 +47,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* **cli/proxy:** honor `HEADROOM_MIN_TOKENS=0` / `HEADROOM_MAX_ITEMS=0`. The Click `proxy` command built these with `_get_env_int_optional(name) or 500`/`or 50`, so an explicit `0` — a legitimate value (`min_tokens_to_crush=0` means "crush every item") — was treated as falsy and silently replaced with the default. The `headroom proxy` argparse path already preserved `0` via `_get_env_int`, so the two entry points disagreed. The Click path now uses the same None-checking helper.
* **proxy:** strip the inbound `Content-Encoding`/`Transfer-Encoding` request headers on the Anthropic `/v1/messages` and OpenAI `/v1/chat/completions` paths before forwarding upstream. `read_request_json_with_bytes` already decompresses the inbound body (zstd/gzip/deflate/br), so the bytes forwarded upstream are plain JSON — but these two handlers left the original `content-encoding` header in place, so a client (or an edge proxy like a Cloudflare Worker) that sent a compressed body got its request rejected with upstream HTTP 400 because the provider tried to decompress already-decoded JSON. The `/v1/responses` handler already carried this fix (#1542); it is now applied to the messages and chat paths too.
* **models:** fix the model registry's prefix fallback silently returning the wrong context window. `ModelRegistry.get` accepted any registered name as a `str.startswith` prefix and returned the *first* match, so `gpt-4-32k-0613` resolved to `gpt-4` (8192) instead of `gpt-4-32k` (32768), and unregistered ids like `gpt-4.1`/`gpt-4.5` inherited `gpt-4`'s 8192-token window — making the proxy think a nearly-empty context was almost full and compress far too aggressively. The fallback now requires the registered name to end at a version boundary in the query (so `gpt-4.1` no longer matches `gpt-4`) and picks the longest qualifying name (so `gpt-4-32k-0613` → `gpt-4-32k`).
+* **mcp/opencode:** stop the OpenCode MCP registrar from destroying an existing but unparseable `opencode.json`. `_write_entry` read the config via a helper that returns `{}` on `JSONDecodeError`, then rewrote the whole file with only `{"mcp": {...}}` — wiping the user's `theme`/`model`/`provider` and any other MCP servers (OpenCode configs are commonly JSONC / hand-edited). The write path now refuses to overwrite a present-but-invalid config and returns a `FAILED` result; absent/empty files still register fresh and valid files still merge with all other keys preserved. (Same class of fix as the Claude registrar.)
* **proxy:** include the system prompt, tools, and the response-shaping request fields in the SemanticCache key. `_compute_key` hashed only `{model, messages}`, so two non-streaming requests with identical messages but a different top-level `system` prompt, tool set, sampling config, or output-shaping field collided on one key and the second caller was served the first's cached response — generated under different request semantics, in the default config (`cache_enabled` defaults on). The key now folds the request fields that shape generation — `temperature`/`top_p`/`top_k`/`max_tokens`/`stop`, plus OpenAI `tool_choice`/`response_format`/`parallel_tool_calls`/`seed`/`presence_penalty`/`frequency_penalty`/`logit_bias`/`n`/`logprobs`/`top_logprobs`/`reasoning_effort`/`verbosity`/`modalities` and Anthropic `thinking`/`tool_choice`/`output_config` — canonicalizing `system`/`tools` so a moved `cache_control` breakpoint does not fragment it, and the handlers snapshot the fields once at the cache read and reuse them at write so a body mutated by the pipeline cannot diverge the key. Non-streaming path only.
* **learn (verbosity):** `--verbosity --apply --all` now aggregates the savings baseline across every project instead of overwriting it per project (last-project-wins), which previously left the output shaper with a tiny, unrepresentative baseline. The applied verbosity level comes from the project with the most samples ([#1288](https://github.com/headroomlabs-ai/headroom/pull/1288)).
* **proxy/anthropic:** restore token-mode compression on continued Claude Code turns with a frozen prefix and deferred CCR tool injection. Token mode now runs request-side compression even when the client did not pre-register `headroom_retrieve`, relying on the existing marker-triggered injection override to keep emitted CCR markers redeemable ([#1487](https://github.com/headroomlabs-ai/headroom/issues/1487)).
diff --git a/headroom/mcp_registry/opencode.py b/headroom/mcp_registry/opencode.py
index 6ab6cca91..7a71a1c98 100644
--- a/headroom/mcp_registry/opencode.py
+++ b/headroom/mcp_registry/opencode.py
@@ -35,7 +35,13 @@ def _opencode_config_path() -> Path:
def _read_json(path: Path) -> dict[str, Any]:
- """Read a JSON file, returning empty dict if absent or unparseable."""
+ """Read a JSON file, returning empty dict if absent or unparseable.
+
+ Safe for READ-ONLY callers only. Do NOT use before a full-file rewrite:
+ an unparseable existing file returns ``{}`` here, and writing that back
+ would destroy the user's other config. Use :func:`_read_json_for_write`
+ on the write path instead.
+ """
if not path.exists():
return {}
try:
@@ -48,6 +54,37 @@ def _read_json(path: Path) -> dict[str, Any]:
return data
+class _MalformedConfigError(Exception):
+ """The target config file exists but is not a parseable JSON object.
+
+ Raised on the write path so we refuse to clobber a config we can't safely
+ merge into, rather than silently overwriting the user's other settings.
+ """
+
+
+def _read_json_for_write(path: Path) -> dict[str, Any]:
+ """Read a JSON object for a subsequent full-file rewrite.
+
+ Returns ``{}`` only when the file is absent or empty (safe to start fresh).
+ If the file exists with content but does not parse as a JSON object, raise
+ :class:`_MalformedConfigError` so the caller aborts instead of overwriting
+ unrelated user config — ``opencode.json`` holds ``theme``/``model``/
+ ``provider``/other MCP servers alongside the ``mcp`` block.
+ """
+ if not path.exists():
+ return {}
+ raw = path.read_text(encoding="utf-8") # OSError propagates to the caller
+ if not raw.strip():
+ return {}
+ try:
+ data = json.loads(raw)
+ except json.JSONDecodeError as exc:
+ raise _MalformedConfigError(str(exc)) from exc
+ if not isinstance(data, dict):
+ raise _MalformedConfigError("top-level JSON is not an object")
+ return data
+
+
def _write_json(path: Path, data: dict[str, Any]) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
@@ -164,13 +201,21 @@ class OpencodeRegistrar(MCPRegistrar):
def _write_entry(self, spec: ServerSpec) -> RegisterResult:
try:
self._config_path.parent.mkdir(parents=True, exist_ok=True)
- data = _read_json(self._config_path)
+ data = _read_json_for_write(self._config_path)
mcp = data.setdefault("mcp", {})
if not isinstance(mcp, dict):
mcp = {}
data["mcp"] = mcp
mcp[spec.name] = _spec_to_entry(spec)
_write_json(self._config_path, data)
+ except _MalformedConfigError as exc:
+ # Refuse to overwrite: opencode.json holds theme/model/provider and
+ # other MCP servers that a blind rewrite would wipe.
+ return RegisterResult(
+ RegisterStatus.FAILED,
+ f"{self._config_path} exists but is not valid JSON ({exc}); "
+ "refusing to overwrite. Fix or remove the file, then re-run.",
+ )
except OSError as exc:
return RegisterResult(
RegisterStatus.FAILED, f"could not write {self._config_path}: {exc}"
diff --git a/tests/test_mcp_registry_opencode.py b/tests/test_mcp_registry_opencode.py
index 99f4d869d..2340ad649 100644
--- a/tests/test_mcp_registry_opencode.py
+++ b/tests/test_mcp_registry_opencode.py
@@ -96,6 +96,48 @@ def test_register_server_creates_config_when_missing(tmp_path: Path) -> None:
}
[email protected]("contents", ["not json", "{", '{"theme": }', "[]"])
+def test_register_server_preserves_malformed_config(tmp_path: Path, contents: str) -> None:
+ """Registering must NOT clobber an existing but unparseable opencode.json.
+
+ The file holds theme/model/provider and other MCP servers; before the fix a
+ malformed file was read as {} and rewritten with only {"mcp": ...}."""
+ from headroom.mcp_registry.base import ServerSpec
+
+ config_path = tmp_path / "opencode.json"
+ config_path.write_text(contents, encoding="utf-8")
+ registrar = _registrar(tmp_path)
+
+ spec = ServerSpec(name="headroom", command="headroom", args=("mcp", "serve"))
+ result = registrar.register_server(spec)
+
+ assert result.status == RegisterStatus.FAILED
+ assert "not valid JSON" in result.detail
+ assert config_path.read_text(encoding="utf-8") == contents
+
+
+def test_register_server_preserves_other_keys(tmp_path: Path) -> None:
+ """The happy path merges: theme/model and an existing MCP server survive."""
+ from headroom.mcp_registry.base import ServerSpec
+
+ config_path = tmp_path / "opencode.json"
+ _write_json(
+ config_path,
+ {"theme": "dark", "model": "anthropic/claude", "mcp": {"other": {"type": "local"}}},
+ )
+ registrar = _registrar(tmp_path)
+
+ spec = ServerSpec(name="headroom", command="headroom", args=("mcp", "serve"))
+ result = registrar.register_server(spec)
+
+ assert result.status == RegisterStatus.REGISTERED
+ data = json.loads(config_path.read_text(encoding="utf-8"))
+ assert data["theme"] == "dark"
+ assert data["model"] == "anthropic/claude"
+ assert data["mcp"]["other"] == {"type": "local"}
+ assert "headroom" in data["mcp"]
+
+
def test_register_server_idempotent(tmp_path: Path) -> None:
"""register_server is a no-op when the same spec is already present."""
registrar = _registrar(tmp_path)
@@ -288,17 +330,22 @@ def test_register_then_re_register_with_different_env_returns_mismatch(tmp_path:
def test_register_server_on_malformed_config_file(tmp_path: Path) -> None:
- """register_server overwrites a malformed config file, preserving nothing."""
+ """register_server refuses to overwrite a malformed config, preserving it.
+
+ Updated for the clobber guard: opencode.json holds theme/model/provider and
+ other MCP servers, so a present-but-unparseable file is left untouched and
+ registration fails rather than silently wiping the user's config.
+ """
registrar = _registrar(tmp_path)
(tmp_path / "opencode.json").write_text("not valid json at all")
from headroom.mcp_registry.base import ServerSpec
spec = ServerSpec(name="headroom", command="headroom", args=("mcp", "serve"))
result = registrar.register_server(spec)
- assert result.status == RegisterStatus.REGISTERED
+ assert result.status == RegisterStatus.FAILED
- data = json.loads((tmp_path / "opencode.json").read_text())
- assert "headroom" in data["mcp"]
+ # The malformed file is left byte-for-byte untouched rather than clobbered.
+ assert (tmp_path / "opencode.json").read_text() == "not valid json at all"
def test_entry_to_spec_command_as_string() -> None: