Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

install: couple Codex routing to persistent runtime readiness (#2043)

Rod Boev3a39cb99ad4c
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 28485abe8..ac9657062 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -144,6 +144,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* **mcp/codex:** don't corrupt an unparseable or non-table `config.toml` on register. `CodexRegistrar.register_server` only guarded against clobbering a user-managed entry when `get_server` returned one, but `get_server` returns `None` both for an unparseable TOML file and for an `mcp_servers`/`mcp_servers.<name>` that is present but not a table. In those cases `register_server` fell through to `_write_block`, which blindly appended a `[mcp_servers.<name>]` table — appending into an unparseable file, or creating a duplicate `[mcp_servers.headroom]` key alongside a non-table entry (e.g. `headroom = "..."`), which `tomllib`/codex then reject, destroying a previously-valid config. It now refuses (`FAILED`) and leaves the file untouched, mirroring the claude (#1660) and opencode (#1661) guards.
* **proxy/vertex:** route Vertex `publisher=google` (Gemini) requests to the region matching the request path. `vertex_generate_content`, `vertex_stream_generate_content`, and `vertex_count_tokens` discarded the path's `location` and forwarded to the single fixed host from `_api_target(proxy, "vertex")` (default `us-central1`), instead of the region-aware `_vertex_target_for_location` the sibling Anthropic `rawPredict` route already uses. So a request to `.../locations/europe-west1/publishers/google/...` was sent to a `us-central1` host, which Vertex rejects on the region/host mismatch. The three google routes now derive the host from the request's `location` (operator-pinned upstreams are still honored).
* **proxy/anthropic:** give each Anthropic conversation its own session id. `SessionTrackerStore.compute_session_id` derived its fallback id from `model` + system text harvested only from `role:"system"` entries inside `messages` — but Anthropic carries the system prompt as a top-level `body["system"]` field, so genuine Anthropic requests (which never carry `x-headroom-session-id`) collapsed to `md5(model:[])` and every conversation on the same model shared one `PrefixCacheTracker`. That let session-sticky state cross-contaminate: conversation A's sticky `headroom_retrieve`/memory tools and `anthropic-beta` headers were injected into conversation B, and frozen-prefix/compression-cache state mixed across conversations. The Anthropic handler now folds the top-level `system` into the session-id inputs (prepending a synthetic `role:"system"` message used only to derive the id), giving distinct conversations distinct ids.
+* **install/codex:** persistent provider routing is now lifecycle-coupled. `headroom install apply` waits until the runtime is ready before writing managed Codex routing, and stop/remove/recovery paths revert that routing before tearing the proxy down, so Codex Desktop is not left pointed at a dead `127.0.0.1:8787` provider after a failed or stopped deployment ([#2038](https://github.com/headroomlabs-ai/headroom/issues/2038)).
* **cache/semantic:** key entries by the full-context hash, not the trailing query text. `SemanticCache.put` stored each response under `sha256(query)[:16]` where `query` is only the last user message, and the exact-match branch of `get` returned the slot without checking the stored entry's `messages_hash`. Two requests that share a trailing message ("continue", "yes", "run the tests") but differ in earlier context therefore collided on one slot — the second overwrote the first, and the first's hash then resolved to the second's cached response (wrong data served). Entries are now keyed by `messages_hash` when present, and `get` verifies `entry.messages_hash` before returning.
* **wrap/opencode:** `headroom unwrap opencode` now removes the Headroom rtk instruction block from the project and global `AGENTS.md`. `wrap opencode` injects the marker-fenced "prefix shell commands with `rtk`" guidance into both `./AGENTS.md` and `<opencode-home>/AGENTS.md`, but unwrap only restored the config and MCP state — so a plain `opencode` launch kept following the rtk guidance and failed once the managed rtk binary was off PATH. Unwrap now strips the marker-fenced block from both files, mirroring `unwrap codex` (#1421) and `unwrap copilot`.
* **proxy/savings:** stop billing the $3/M fallback rate for genuinely free models. `_estimate_compression_savings_usd` and `_estimate_input_cost_usd` read `input_cost_per_token` from litellm and used `if not input_cost_per_token: raise`, which treats a legitimate `0.0` (a free / local / vendored-at-0 model that litellm does carry) as "price unavailable" and falls back to `DEFAULT_FALLBACK_INPUT_COST_PER_TOKEN` — fabricating dollar savings/cost for a model that costs nothing. Both now use an explicit `is None` check so a present `0.0` flows through as `$0` while a missing key still falls back.
diff --git a/headroom/cli/install.py b/headroom/cli/install.py
index 3a16594df..8ba43ca52 100644
--- a/headroom/cli/install.py
+++ b/headroom/cli/install.py
@@ -107,17 +107,33 @@ def _stop_deployment(manifest: DeploymentManifest) -> None:
stop_runtime(manifest)
+def _deactivate_deployment_mutations(
+ manifest: DeploymentManifest, *, persist_manifest: bool = True
+) -> None:
+ if not manifest.mutations:
+ return
+ revert_mutations(manifest)
+ manifest.mutations = []
+ if persist_manifest:
+ save_manifest(manifest)
+
+
+def _activate_deployment_mutations(manifest: DeploymentManifest) -> None:
+ manifest.mutations = apply_mutations(manifest)
+ save_manifest(manifest)
+
+
def _remove_deployment(manifest: DeploymentManifest) -> None:
try:
- _stop_deployment(manifest)
+ _deactivate_deployment_mutations(manifest, persist_manifest=False)
except Exception:
pass
try:
- remove_supervisor(manifest)
+ _stop_deployment(manifest)
except Exception:
pass
try:
- revert_mutations(manifest)
+ remove_supervisor(manifest)
except Exception:
pass
delete_manifest(manifest.profile)
@@ -125,10 +141,10 @@ def _remove_deployment(manifest: DeploymentManifest) -> None:
def _restore_deployment(manifest: DeploymentManifest) -> None:
restored = deepcopy(manifest)
- restored.mutations = apply_mutations(restored)
restored.artifacts = install_supervisor(restored)
save_manifest(restored)
_start_deployment(restored)
+ _activate_deployment_mutations(restored)
def _reject_task_lifecycle(manifest: DeploymentManifest, action: str) -> None:
@@ -280,10 +296,10 @@ def install_apply(
_remove_deployment(existing)
try:
- manifest.mutations = apply_mutations(manifest)
manifest.artifacts = install_supervisor(manifest)
save_manifest(manifest)
_start_deployment(manifest)
+ _activate_deployment_mutations(manifest)
except Exception as exc:
_remove_deployment(manifest)
if existing is not None:
@@ -331,7 +347,11 @@ def install_start(profile: str) -> None:
manifest = _require_manifest(profile)
_reject_task_lifecycle(manifest, "start")
+ if not probe_ready(manifest.health_url):
+ _deactivate_deployment_mutations(manifest)
_start_deployment(manifest)
+ if probe_ready(manifest.health_url) and not manifest.mutations:
+ _activate_deployment_mutations(manifest)
click.echo(f"Started deployment '{profile}'.")
@@ -342,6 +362,7 @@ def install_stop(profile: str) -> None:
manifest = _require_manifest(profile)
_reject_task_lifecycle(manifest, "stop")
+ _deactivate_deployment_mutations(manifest)
_stop_deployment(manifest)
click.echo(f"Stopped deployment '{profile}'.")
@@ -353,8 +374,10 @@ def install_restart(profile: str) -> None:
manifest = _require_manifest(profile)
_reject_task_lifecycle(manifest, "restart")
+ _deactivate_deployment_mutations(manifest)
_stop_deployment(manifest)
_start_deployment(manifest)
+ _activate_deployment_mutations(manifest)
click.echo(f"Restarted deployment '{profile}'.")
@@ -364,6 +387,7 @@ def install_remove(profile: str) -> None:
"""Remove a persistent deployment and undo managed config."""
manifest = _require_manifest(profile)
+ _deactivate_deployment_mutations(manifest, persist_manifest=False)
try:
if manifest.supervisor_kind == SupervisorKind.SERVICE.value:
stop_supervisor(manifest)
@@ -377,7 +401,6 @@ def install_remove(profile: str) -> None:
remove_supervisor(manifest)
except Exception:
pass
- revert_mutations(manifest)
delete_manifest(profile)
click.echo(f"Removed deployment '{profile}'.")
@@ -423,6 +446,10 @@ def install_agent_ensure(profile: str) -> None:
if wait_ready(manifest, timeout_seconds=_STARTUP_READY_TIMEOUT_SECONDS):
click.echo(f"Deployment '{profile}' is healthy.")
return
+ _deactivate_deployment_mutations(manifest)
stop_runtime(manifest)
+ else:
+ _deactivate_deployment_mutations(manifest)
_start_deployment(manifest, assume_start_lock=True)
+ _activate_deployment_mutations(manifest)
click.echo(f"Deployment '{profile}' is healthy.")
diff --git a/tests/test_cli/test_install_cli.py b/tests/test_cli/test_install_cli.py
index 6cccaeb92..f52cc0b85 100644
--- a/tests/test_cli/test_install_cli.py
+++ b/tests/test_cli/test_install_cli.py
@@ -17,15 +17,20 @@ def test_install_apply_starts_service_supervisor(monkeypatch) -> None:
supervisor_kind = "service"
scope = "user"
health_url = "http://127.0.0.1:8787/readyz"
- targets = ["claude", "codex"]
+ mutations = [object()]
+ mutations = [object()]
mutations = []
+ targets = ["claude", "codex"]
artifacts = []
manifest = Manifest()
monkeypatch.setattr("headroom.cli.install.build_manifest", lambda **_: manifest)
monkeypatch.setattr("headroom.cli.install.load_manifest", lambda profile: None)
- monkeypatch.setattr("headroom.cli.install.apply_mutations", lambda deployment: [])
+ monkeypatch.setattr(
+ "headroom.cli.install.apply_mutations",
+ lambda deployment: calls.append("apply") or [],
+ )
monkeypatch.setattr("headroom.cli.install.install_supervisor", lambda deployment: [])
monkeypatch.setattr(
"headroom.cli.install.save_manifest", lambda deployment: calls.append("save")
@@ -51,7 +56,7 @@ def test_install_apply_starts_service_supervisor(monkeypatch) -> None:
assert result.exit_code == 0, result.output
assert "Installed persistent deployment 'default'" in result.output
assert "Targets: claude, codex" in result.output
- assert calls == ["save", "start_service"]
+ assert calls == ["save", "start_service", "apply", "save"]
def test_install_apply_forwards_no_http2_to_build_manifest(monkeypatch) -> None:
@@ -65,6 +70,8 @@ def test_install_apply_forwards_no_http2_to_build_manifest(monkeypatch) -> None:
supervisor_kind = "service"
scope = "user"
health_url = "http://127.0.0.1:8787/readyz"
+ mutations = [object()]
+ mutations = [object()]
targets = ["claude"]
mutations = []
artifacts = []
@@ -141,8 +148,12 @@ def test_install_restart_uses_internal_helpers(monkeypatch) -> None:
supervisor_kind = "service"
scope = "user"
health_url = "http://127.0.0.1:8787/readyz"
+ mutations = [object()]
monkeypatch.setattr("headroom.cli.install.load_manifest", lambda profile: Manifest())
+ monkeypatch.setattr(
+ "headroom.cli.install.revert_mutations", lambda manifest: calls.append("revert")
+ )
monkeypatch.setattr(
"headroom.cli.install.stop_supervisor", lambda manifest: calls.append("stop_supervisor")
)
@@ -155,6 +166,10 @@ def test_install_restart_uses_internal_helpers(monkeypatch) -> None:
monkeypatch.setattr(
"headroom.cli.install.wait_ready", lambda manifest, timeout_seconds=45: True
)
+ monkeypatch.setattr(
+ "headroom.cli.install.apply_mutations", lambda manifest: calls.append("apply") or []
+ )
+ monkeypatch.setattr("headroom.cli.install.save_manifest", lambda manifest: calls.append("save"))
monkeypatch.setattr("headroom.cli.install.probe_ready", lambda url: False)
monkeypatch.setattr("headroom.cli.install.runtime_status", lambda manifest: "stopped")
@@ -162,7 +177,15 @@ def test_install_restart_uses_internal_helpers(monkeypatch) -> None:
assert result.exit_code == 0, result.output
assert "Restarted deployment 'default'." in result.output
- assert calls == ["stop_supervisor", "stop_runtime", "start_supervisor"]
+ assert calls == [
+ "revert",
+ "save",
+ "stop_supervisor",
+ "stop_runtime",
+ "start_supervisor",
+ "apply",
+ "save",
+ ]
def test_install_start_noops_when_already_healthy(monkeypatch) -> None:
@@ -176,6 +199,7 @@ def test_install_start_noops_when_already_healthy(monkeypatch) -> None:
supervisor_kind = "service"
scope = "user"
health_url = "http://127.0.0.1:8787/readyz"
+ mutations = [object()]
monkeypatch.setattr("headroom.cli.install.load_manifest", lambda profile: Manifest())
monkeypatch.setattr("headroom.cli.install.probe_ready", lambda url: True)
@@ -200,6 +224,7 @@ def test_install_start_noops_for_healthy_docker_without_docker_on_path(monkeypat
supervisor_kind = "none"
scope = "user"
health_url = "http://127.0.0.1:8787/readyz"
+ mutations = [object()]
monkeypatch.setattr("headroom.cli.install.load_manifest", lambda profile: Manifest())
monkeypatch.setattr("headroom.cli.install.probe_ready", lambda url: True)
@@ -222,8 +247,10 @@ def test_install_start_does_not_spawn_when_start_lock_is_contended(monkeypatch)
supervisor_kind = "service"
scope = "user"
health_url = "http://127.0.0.1:8787/readyz"
+ mutations = []
monkeypatch.setattr("headroom.cli.install.load_manifest", lambda profile: Manifest())
+ monkeypatch.setattr("headroom.cli.install.probe_ready", lambda url: False)
import contextlib
@@ -254,14 +281,28 @@ def test_install_start_restarts_wedged_runtime_under_single_lock(monkeypatch) ->
supervisor_kind = "service"
scope = "user"
health_url = "http://127.0.0.1:8787/readyz"
+ mutations = [object()]
monkeypatch.setattr("headroom.cli.install.load_manifest", lambda profile: Manifest())
- monkeypatch.setattr("headroom.cli.install.probe_ready", lambda url: False)
+ probe_calls = {"count": 0}
+
+ def fake_probe_ready(url: str) -> bool:
+ probe_calls["count"] += 1
+ return probe_calls["count"] > 2
+
+ monkeypatch.setattr("headroom.cli.install.probe_ready", fake_probe_ready)
monkeypatch.setattr("headroom.cli.install.runtime_status", lambda manifest: "running")
wait_results = iter([False, True])
monkeypatch.setattr(
"headroom.cli.install.wait_ready", lambda manifest, timeout_seconds: next(wait_results)
)
+ monkeypatch.setattr(
+ "headroom.cli.install.revert_mutations", lambda manifest: calls.append("revert")
+ )
+ monkeypatch.setattr(
+ "headroom.cli.install.apply_mutations", lambda manifest: calls.append("apply") or []
+ )
+ monkeypatch.setattr("headroom.cli.install.save_manifest", lambda manifest: calls.append("save"))
monkeypatch.setattr("headroom.cli.install.stop_runtime", lambda manifest: calls.append("stop"))
monkeypatch.setattr(
"headroom.cli.install.start_supervisor", lambda manifest: calls.append("start_supervisor")
@@ -270,7 +311,7 @@ def test_install_start_restarts_wedged_runtime_under_single_lock(monkeypatch) ->
result = runner.invoke(main, ["install", "start"])
assert result.exit_code == 0, result.output
- assert calls == ["stop", "start_supervisor"]
+ assert calls == ["revert", "save", "stop", "start_supervisor", "apply", "save"]
def test_install_apply_rejects_invalid_profile() -> None:
@@ -363,6 +404,7 @@ def test_install_apply_restores_previous_deployment_after_failed_update(monkeypa
new_manifest = Manifest("default", ["claude"])
existing_manifest = Manifest("default", ["codex"])
+ existing_manifest.mutations = [object()]
monkeypatch.setattr("headroom.cli.install.build_manifest", lambda **_: new_manifest)
monkeypatch.setattr("headroom.cli.install.load_manifest", lambda profile: existing_manifest)
@@ -411,24 +453,23 @@ def test_install_apply_restores_previous_deployment_after_failed_update(monkeypa
assert result.exit_code != 0
assert "Restoring previous deployment 'default'" in result.output
assert calls == [
+ "revert:codex",
"stop-supervisor:codex",
"stop-runtime:codex",
"remove-supervisor:codex",
- "revert:codex",
"delete:default",
- "apply:claude",
"supervisor:claude",
"save:claude",
"start:claude",
"stop-supervisor:claude",
"stop-runtime:claude",
"remove-supervisor:claude",
- "revert:claude",
"delete:default",
- "apply:codex",
"supervisor:codex",
"save:codex",
"start:codex",
+ "apply:codex",
+ "save:codex",
]
@@ -508,8 +549,12 @@ def test_install_remove_continues_when_runtime_teardown_errors(monkeypatch) -> N
supervisor_kind = "service"
scope = "user"
health_url = "http://127.0.0.1:8787/readyz"
+ mutations = [object()]
monkeypatch.setattr("headroom.cli.install.load_manifest", lambda profile: Manifest())
+ monkeypatch.setattr(
+ "headroom.cli.install.revert_mutations", lambda manifest: calls.append("revert")
+ )
monkeypatch.setattr(
"headroom.cli.install.stop_supervisor",
lambda manifest: (_ for _ in ()).throw(RuntimeError("boom")),
@@ -521,9 +566,6 @@ def test_install_remove_continues_when_runtime_teardown_errors(monkeypatch) -> N
monkeypatch.setattr(
"headroom.cli.install.remove_supervisor", lambda manifest: calls.append("remove_supervisor")
)
- monkeypatch.setattr(
- "headroom.cli.install.revert_mutations", lambda manifest: calls.append("revert")
- )
monkeypatch.setattr(
"headroom.cli.install.delete_manifest", lambda profile: calls.append("delete")
)
@@ -531,7 +573,7 @@ def test_install_remove_continues_when_runtime_teardown_errors(monkeypatch) -> N
result = runner.invoke(main, ["install", "remove"])
assert result.exit_code == 0, result.output
- assert calls == ["remove_supervisor", "revert", "delete"]
+ assert calls == ["revert", "remove_supervisor", "delete"]
def test_install_agent_ensure_reports_already_healthy(monkeypatch) -> None:
@@ -609,11 +651,26 @@ def test_install_agent_ensure_stops_wedged_runtime_before_restart(monkeypatch) -
health_url = "http://127.0.0.1:8787/readyz"
preset = "persistent-task"
supervisor_kind = "none"
+ scope = "user"
+ mutations = []
+ scope = "user"
+ mutations = []
+ scope = "user"
+ mutations = []
+ scope = "user"
+ mutations = [object()]
monkeypatch.setattr("headroom.cli.install.load_manifest", lambda profile: Manifest())
monkeypatch.setattr("headroom.cli.install.probe_ready", lambda url: False)
monkeypatch.setattr("headroom.cli.install.runtime_status", lambda manifest: "running")
monkeypatch.setattr("headroom.cli.install.wait_ready", lambda manifest, timeout_seconds: False)
+ monkeypatch.setattr(
+ "headroom.cli.install.revert_mutations", lambda manifest: calls.append("revert")
+ )
+ monkeypatch.setattr(
+ "headroom.cli.install.apply_mutations", lambda manifest: calls.append("apply") or []
+ )
+ monkeypatch.setattr("headroom.cli.install.save_manifest", lambda manifest: calls.append("save"))
monkeypatch.setattr("headroom.cli.install.stop_runtime", lambda manifest: calls.append("stop"))
monkeypatch.setattr(
"headroom.cli.install.start_detached_agent",
@@ -639,7 +696,9 @@ def test_install_agent_ensure_stops_wedged_runtime_before_restart(monkeypatch) -
result = runner.invoke(main, ["install", "agent", "ensure"])
assert result.exit_code == 0, result.output
# stop must come before start_deployment — that's the bug guard.
+ assert calls.index("revert") < calls.index("stop")
assert calls.index("stop") < calls.index("start_deployment")
+ assert calls.index("start_deployment") < calls.index("apply")
assert "start_agent" not in calls
assert "start_docker" not in calls
@@ -654,10 +713,16 @@ def test_install_agent_ensure_starts_when_stopped_and_lock_acquired(monkeypatch)
health_url = "http://127.0.0.1:8787/readyz"
preset = "persistent-task"
supervisor_kind = "none"
+ scope = "user"
+ mutations = []
monkeypatch.setattr("headroom.cli.install.load_manifest", lambda profile: Manifest())
monkeypatch.setattr("headroom.cli.install.probe_ready", lambda url: False)
monkeypatch.setattr("headroom.cli.install.runtime_status", lambda manifest: "stopped")
+ monkeypatch.setattr(
+ "headroom.cli.install.apply_mutations", lambda manifest: calls.append("apply") or []
+ )
+ monkeypatch.setattr("headroom.cli.install.save_manifest", lambda manifest: calls.append("save"))
monkeypatch.setattr(
"headroom.cli.install.start_detached_agent",
lambda profile: calls.append("start_agent"),
@@ -678,7 +743,7 @@ def test_install_agent_ensure_starts_when_stopped_and_lock_acquired(monkeypatch)
result = runner.invoke(main, ["install", "agent", "ensure"])
assert result.exit_code == 0, result.output
- assert calls == ["start_agent"]
+ assert calls == ["start_agent", "apply", "save"]
def test_install_agent_ensure_no_duplicate_spawn_after_lock_recheck(monkeypatch) -> None:
@@ -729,6 +794,8 @@ def test_install_agent_ensure_propagates_start_deployment_failure(monkeypatch) -
health_url = "http://127.0.0.1:8787/readyz"
preset = "persistent-task"
supervisor_kind = "none"
+ scope = "user"
+ mutations = []
monkeypatch.setattr("headroom.cli.install.load_manifest", lambda profile: Manifest())
monkeypatch.setattr("headroom.cli.install.probe_ready", lambda url: False)