Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
docs: document HEADROOM_CONFIG_DIR / HEADROOM_WORKSPACE_DIR filesystem contract
JerrettDavis1192f657eb7c
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 62db943fb..33b09f290 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -5,6 +5,20 @@ All notable changes to Headroom will be documented in this file.The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).+## [Unreleased]++### Added+- **Canonical filesystem contract** (issue #175) — new `HEADROOM_CONFIG_DIR`+ (default `~/.headroom/config`, read-mostly) and `HEADROOM_WORKSPACE_DIR`+ (default `~/.headroom`, read-write state) env vars recognized by the Python+ proxy/CLI and the npm SDK. Additive; all existing per-resource env vars+ (`HEADROOM_SAVINGS_PATH`, `HEADROOM_TOIN_PATH`,+ `HEADROOM_SUBSCRIPTION_STATE_PATH`, `HEADROOM_MODEL_LIMITS`) continue to+ work with identical semantics. Docker install scripts and+ `docker-compose.native.yml` forward the new vars into containers so+ savings, logs, and telemetry resolve to the bind-mounted `.headroom` path.+ See [`wiki/filesystem-contract.md`](wiki/filesystem-contract.md).+## [0.5.22] - 2026-04-11### Addeddiff --git a/README.md b/README.mdindex 6545e7462..02f5eb6af 100644--- a/README.md+++ b/README.md@@ -535,6 +535,7 @@ Python 3.10+| [Learn](docs/content/docs/failure-learning.mdx) | Plugin-based failure learning (Claude, Codex, Gemini, extensible) || [Installation](docs/content/docs/installation.mdx) | pip, npm, Docker install methods || [Configuration](docs/content/docs/configuration.mdx) | All options |+| [Filesystem Contract](wiki/filesystem-contract.md) | `HEADROOM_CONFIG_DIR` / `HEADROOM_WORKSPACE_DIR` and per-resource path overrides |---diff --git a/docs/content/docs/configuration.mdx b/docs/content/docs/configuration.mdxindex aa080cb4a..5b572bbb5 100644--- a/docs/content/docs/configuration.mdx+++ b/docs/content/docs/configuration.mdx@@ -249,9 +249,26 @@ headroom proxy --llmlingua --llmlingua-device cuda --llmlingua-rate 0.4| `HEADROOM_MODEL_LIMITS` | Custom model config (JSON string or file path) | -- || `HEADROOM_BASE_URL` | Base URL of the Headroom proxy (TypeScript SDK) | `http://localhost:8787` || `HEADROOM_API_KEY` | API key for Headroom Cloud authentication | -- |-| `HEADROOM_SAVINGS_PATH` | Override persistent savings file location | `~/.headroom/proxy_savings.json` |+| `HEADROOM_CONFIG_DIR` | Canonical config (read-mostly) root. Derives `models.json` and per-plugin config paths when set. | `~/.headroom/config` |+| `HEADROOM_WORKSPACE_DIR` | Canonical workspace (read-write state) root. Derives savings, memory DB, logs, TOIN, subscription state, and more when set. | `~/.headroom` |+| `HEADROOM_SAVINGS_PATH` | Override persistent savings file location. Always wins when set. | derived from `${HEADROOM_WORKSPACE_DIR}` |+| `HEADROOM_TOIN_PATH` | Override TOIN telemetry file location. Always wins when set. | derived from `${HEADROOM_WORKSPACE_DIR}` |+| `HEADROOM_SUBSCRIPTION_STATE_PATH` | Override subscription tracker state file. Always wins when set. | derived from `${HEADROOM_WORKSPACE_DIR}` || `HEADROOM_TELEMETRY` | Set to `off` to disable anonymous telemetry | `on` |+### Filesystem Contract++Headroom resolves every on-disk resource through a two-root model+(`HEADROOM_CONFIG_DIR` + `HEADROOM_WORKSPACE_DIR`) with additive+precedence rules: explicit argument > per-resource env var > derived+from canonical root > default. Every legacy env var continues to work+unchanged.++See the **[Filesystem Contract](https://github.com/chopratejas/headroom/blob/main/wiki/filesystem-contract.md)**+page for the full bucket table, plugin-author guidance, and the Docker+naming overlap note (`HEADROOM_WORKSPACE` is *not* the same as+`HEADROOM_WORKSPACE_DIR`).+## Custom Model ConfigurationConfigure context limits and pricing for new or custom models:@@ -280,12 +297,17 @@ Configure context limits and pricing for new or custom models:}```-Save as `~/.headroom/models.json`, or set `HEADROOM_MODEL_LIMITS` to a JSON string or file path.+Save as `${HEADROOM_CONFIG_DIR}/models.json` (defaults to+`~/.headroom/config/models.json`), or set `HEADROOM_MODEL_LIMITS` to a+JSON string or file path. Installs that still have+`~/.headroom/models.json` (the legacy location) continue to work.Settings are resolved in this order (later overrides earlier):1. Built-in defaults-2. `~/.headroom/models.json` config file+2. `${HEADROOM_CONFIG_DIR}/models.json` (new canonical location); falls+ back to `~/.headroom/models.json` (legacy) when the canonical file+ is absent3. `HEADROOM_MODEL_LIMITS` environment variable4. SDK constructor argumentsdiff --git a/wiki/cli.md b/wiki/cli.mdindex b162de9bb..9414b713b 100644--- a/wiki/cli.md+++ b/wiki/cli.md@@ -318,7 +318,9 @@ headroom perf --raw| `--hours` | `168.0` | Time window in hours || `--raw` | off | Print raw PERF records instead of the summarized report |-The command reads `~/.headroom/logs/proxy.log`.+The command reads `${HEADROOM_WORKSPACE_DIR}/logs/proxy.log` (defaults+to `~/.headroom/logs/proxy.log` — see the+[Filesystem Contract](filesystem-contract.md)).## `headroom evals`@@ -628,7 +630,10 @@ headroom install apply --preset persistent-docker --scope user| `--no-telemetry` | off | Disable anonymous telemetry || `--image` | `ghcr.io/chopratejas/headroom:latest` | Docker image for Docker-backed installs |-`apply` stores a manifest under `~/.headroom/deploy/<profile>/manifest.json`, applies managed tool configuration, starts the chosen runtime, and waits for `readyz`.+`apply` stores a manifest under+`${HEADROOM_WORKSPACE_DIR}/deploy/<profile>/manifest.json` (default+`~/.headroom/deploy/<profile>/manifest.json`), applies managed tool+configuration, starts the chosen runtime, and waits for `readyz`.Docker-native host wrappers expose a narrower `headroom install` subset for `persistent-docker` only: `apply`, `status`, `start`, `stop`, `restart`, and `remove`. Those wrapper flows preserve the same port and manifest behavior, but they intentionally reject `persistent-service`, `persistent-task`, and provider mutation flags like `--scope`, `--providers`, and `--target`.diff --git a/wiki/configuration.md b/wiki/configuration.mdindex fcd7c7281..5e0787383 100644--- a/wiki/configuration.md+++ b/wiki/configuration.md@@ -271,6 +271,27 @@ Some settings can be configured via environment variables:| `HEADROOM_STORE_URL` | Database URL | temp directory || `HEADROOM_DEFAULT_MODE` | Default mode | `optimize` || `HEADROOM_MODEL_LIMITS` | Custom model config (JSON string or file path) | - |+| `HEADROOM_CONFIG_DIR` | Canonical config (read-mostly) root. Derives `models.json` and per-plugin config paths when set. | `~/.headroom/config` |+| `HEADROOM_WORKSPACE_DIR` | Canonical workspace (read-write state) root. Derives savings ledger, memory DB, logs, TOIN, subscription state, and more when set. | `~/.headroom` |+| `HEADROOM_SAVINGS_PATH` | Full path to the proxy savings JSON ledger. Always wins when set. | derived from `${HEADROOM_WORKSPACE_DIR}` |+| `HEADROOM_TOIN_PATH` | Full path to the TOIN telemetry JSON file. Always wins when set. | derived from `${HEADROOM_WORKSPACE_DIR}` |+| `HEADROOM_SUBSCRIPTION_STATE_PATH` | Full path to the subscription tracker state. Always wins when set. | derived from `${HEADROOM_WORKSPACE_DIR}` |++## Filesystem Contract++Headroom resolves every on-disk resource through a two-root model:++- `HEADROOM_CONFIG_DIR` (default `~/.headroom/config`) — read-mostly+ configuration+- `HEADROOM_WORKSPACE_DIR` (default `~/.headroom`) — read-write state++Precedence for each resource is: explicit argument > per-resource env+var > derived from canonical root > default. Every legacy env var+continues to work unchanged.++See **[Filesystem Contract](filesystem-contract.md)** for the full+bucket table, plugin-author guidance, and the Docker naming overlap+note (`HEADROOM_WORKSPACE` is *not* the same as `HEADROOM_WORKSPACE_DIR`).---@@ -285,7 +306,9 @@ Configure context limits and pricing for new or custom models. Useful when:Settings are resolved in this order (later overrides earlier):1. Built-in defaults-2. `~/.headroom/models.json` config file+2. `${HEADROOM_CONFIG_DIR}/models.json` (defaults to+ `~/.headroom/config/models.json`); falls back to the legacy location+ `~/.headroom/models.json` when the canonical file is absent3. `HEADROOM_MODEL_LIMITS` environment variable4. SDK constructor argumentsdiff --git a/wiki/docker-install.md b/wiki/docker-install.mdindex 7aa3fedd2..9093c90c5 100644--- a/wiki/docker-install.md+++ b/wiki/docker-install.md@@ -132,6 +132,25 @@ docker compose -f docker/docker-compose.native.yml up -d proxyThis remains a supported persistent-Docker path when you want the proxy managed explicitly through Compose instead of the installed wrapper.+#### `HEADROOM_WORKSPACE` vs `HEADROOM_WORKSPACE_DIR`++These are two different variables — both are set by the compose file,+and both are retained for backward compatibility:++- **`HEADROOM_WORKSPACE`** (host-side) is the directory the compose file+ bind-mounts into the container as `/workspace`. It behaves like CWD+ in a native (non-Docker) run.+- **`HEADROOM_WORKSPACE_DIR`** (inside-the-container) is the canonical+ Headroom state root — part of the [filesystem contract][fs]+ introduced in issue #175. The compose file sets it to+ `/tmp/headroom-home/.headroom` so the proxy resolves savings, logs,+ TOIN, and memory under the bind-mounted `${HOME}/.headroom`.++You do not need to set `HEADROOM_WORKSPACE_DIR` manually when using the+shipped compose file — it is already in the `environment:` block.++[fs]: filesystem-contract.md+### macOS / Linux```bashdiff --git a/wiki/filesystem-contract.md b/wiki/filesystem-contract.mdnew file mode 100644index 000000000..57eb0a4ce--- /dev/null+++ b/wiki/filesystem-contract.md@@ -0,0 +1,168 @@+# Filesystem Contract++Headroom writes configuration, runtime state, logs, and caches to a small+set of well-known paths under the user's home directory. This page is the+source of truth for where those paths live, how to override them, and how+they behave inside Docker containers.++## Two-root model++| Variable | Default | Purpose | Typical access |+|---|---|---|---|+| `HEADROOM_CONFIG_DIR` | `~/.headroom/config` | User/admin-authored configuration (model catalogs, plugin settings, etc.) | Read-mostly |+| `HEADROOM_WORKSPACE_DIR` | `~/.headroom` | Runtime state written by the proxy and CLI (savings, logs, memory DB, telemetry, caches) | Read-write |++Both variables are recognized by the Python proxy / CLI and the npm SDK.+They are **additive** — every pre-existing per-resource env var+(`HEADROOM_SAVINGS_PATH`, `HEADROOM_TOIN_PATH`,+`HEADROOM_SUBSCRIPTION_STATE_PATH`, `HEADROOM_MODEL_LIMITS`, ...)+continues to work with identical semantics.++## Precedence++For every per-resource helper, resolution follows this order:++```+explicit argument+ │ falls through when None/""+ ▼+per-resource env var (e.g. HEADROOM_SAVINGS_PATH)+ │ falls through when unset/blank+ ▼+derived from canonical root+ │ e.g. ${HEADROOM_WORKSPACE_DIR}/proxy_savings.json+ ▼+default (e.g. ~/.headroom/proxy_savings.json)+```++Examples:++- `HEADROOM_WORKSPACE_DIR=/mnt/state` → savings land at+ `/mnt/state/proxy_savings.json` unless `HEADROOM_SAVINGS_PATH` overrides.+- `HEADROOM_SAVINGS_PATH=/custom/savings.json` always wins, even when+ `HEADROOM_WORKSPACE_DIR` is set.+- Unset both and the default is `~/.headroom/proxy_savings.json`.++## Bucket assignments++### Workspace bucket (`HEADROOM_WORKSPACE_DIR`)++| Resource | Default path | Legacy env var |+|---|---|---|+| Proxy savings ledger | `${WORKSPACE_DIR}/proxy_savings.json` | `HEADROOM_SAVINGS_PATH` |+| TOIN telemetry JSON | `${WORKSPACE_DIR}/toin.json` | `HEADROOM_TOIN_PATH` |+| Subscription tracker state | `${WORKSPACE_DIR}/subscription_state.json` | `HEADROOM_SUBSCRIPTION_STATE_PATH` |+| Memory SQLite | `${WORKSPACE_DIR}/memory.db` | CLI `--memory-db-path` |+| Native memory directory | `${WORKSPACE_DIR}/memories/` | `MemoryConfig.native_memory_dir` |+| License cache | `${WORKSPACE_DIR}/license_cache.json` | — |+| Session stats JSONL | `${WORKSPACE_DIR}/session_stats.jsonl` | — |+| Memory sync state | `${WORKSPACE_DIR}/sync_state.json` | — |+| Memory bridge state | `${WORKSPACE_DIR}/bridge_state.json` | — |+| Proxy log directory | `${WORKSPACE_DIR}/logs/` | — |+| HTTP 400 debug dumps | `${WORKSPACE_DIR}/logs/debug_400/` | — |+| Vendored `rtk` binary | `${WORKSPACE_DIR}/bin/rtk[.exe]` | — |+| Deployment profiles | `${WORKSPACE_DIR}/deploy/` | — |+| Beacon lock file | `${WORKSPACE_DIR}/.beacon_lock_<port>` | — |++### Config bucket (`HEADROOM_CONFIG_DIR`)++| Resource | Default path | Legacy env var |+|---|---|---|+| Models catalog | `${CONFIG_DIR}/models.json` | `HEADROOM_MODEL_LIMITS` (content override) |+| Plugin settings | `${CONFIG_DIR}/plugins/<name>/...` | — |++### Backward compatibility — models.json++`models.json` historically lived at `~/.headroom/models.json` (i.e. in the+workspace root, not in `config/`). For a seamless migration the Python+providers check **both** locations in this order:++1. `${HEADROOM_CONFIG_DIR}/models.json` (new canonical location)+2. `${HEADROOM_WORKSPACE_DIR}/models.json` (legacy fallback)++Existing installs continue to work unchanged. New installs are encouraged+to put `models.json` in the config bucket.++## Plugin authors++Two helpers give plugins isolated, per-plugin directories under both+roots:++### Python++```python+from headroom import paths++cfg_dir = paths.plugin_config_dir("my-plugin")+# → ~/.headroom/config/plugins/my-plugin++state_dir = paths.plugin_workspace_dir("my-plugin")+# → ~/.headroom/plugins/my-plugin++cfg_dir.mkdir(parents=True, exist_ok=True)+(cfg_dir / "settings.json").write_text("{}")+```++### npm SDK++```typescript+import { pluginConfigDir, pluginWorkspaceDir } from "@headroom/sdk";++const cfgDir = pluginConfigDir("my-plugin");+const stateDir = pluginWorkspaceDir("my-plugin");+```++Plugin-author helpers reject names containing `/` or `\` to keep the+namespace flat.++## Docker naming overlap: `HEADROOM_WORKSPACE` vs `HEADROOM_WORKSPACE_DIR`++These are **two different variables** with different semantics, both+retained for backward compatibility:++| Variable | Scope | Meaning |+|---|---|---|+| `HEADROOM_WORKSPACE` | Host-side (Docker) | Directory to bind-mount into the container as `/workspace` (equivalent to CWD in native runs). Used by `docker-compose.native.yml`. |+| `HEADROOM_WORKSPACE_DIR` | Inside-the-container | Canonical Headroom state root. Resolves to `/tmp/headroom-home/.headroom` inside the official container image, which in turn bind-mounts to `${HOME}/.headroom` on the host. |++The official Docker bootstrap (compose file, `scripts/install.sh`, and the+Python `install` command) sets `HEADROOM_WORKSPACE_DIR` and+`HEADROOM_CONFIG_DIR` inside the container so the proxy resolves state to+the bind-mounted path without any user action.++## Project-scoped `.headroom/` directories++A few code paths deliberately use **project-local** `.headroom/` paths+resolved relative to the current working directory rather than the+canonical workspace root:++- `headroom/proxy/server.py` — project-scoped memory DB default+- `headroom/memory/mcp_server.py` — project-scoped memory DB default+- `headroom/cli/wrap.py` — project-scoped memory and hook artifacts++These **do not obey** `HEADROOM_WORKSPACE_DIR`. This is intentional: it+preserves the "project memory lives in the project directory" invariant+documented in [memory.md](memory.md). Users who want a single centrally+located memory store can pass `--memory-db-path <path>` explicitly or set+the path via the plugin API.++## Legacy per-resource env vars++Every legacy env var continues to work with its original semantics (raw+string in, raw string out — no tilde expansion, no path-separator+normalization), ensuring byte-for-byte backward compatibility.++Full list:++- `HEADROOM_SAVINGS_PATH`+- `HEADROOM_TOIN_PATH`+- `HEADROOM_SUBSCRIPTION_STATE_PATH`+- `HEADROOM_MODEL_LIMITS` (content override — JSON string or file path)++## See also++- [configuration.md](configuration.md) — general configuration reference+- [docker-install.md](docker-install.md) — Docker install details+- [persistent-installs.md](persistent-installs.md) — persistent+ deployment profiles+- [memory.md](memory.md) — memory-system paths and project scopingdiff --git a/wiki/index.md b/wiki/index.mdindex 516509d6c..b83f0d782 100644--- a/wiki/index.md+++ b/wiki/index.md@@ -413,6 +413,7 @@ Requires Python 3.10+.- **[Architecture](ARCHITECTURE.md)** — How the pipeline works under the hood- **[Benchmarks](benchmarks.md)** — Accuracy and latency data- **[Limitations](LIMITATIONS.md)** — When compression helps and when it doesn't+- **[Filesystem Contract](filesystem-contract.md)** — Canonical config/workspace env vars and paths---diff --git a/wiki/mcp.md b/wiki/mcp.mdindex e89816279..fea4c7102 100644--- a/wiki/mcp.md+++ b/wiki/mcp.md@@ -102,7 +102,12 @@ Returns:- proxy (request count, cache hits, cost saved — if proxy is running)```-Sub-agent stats are aggregated via a shared stats file (`~/.headroom/session_stats.jsonl`). Each MCP server instance (main session and sub-agents) writes events there, and `headroom_stats` reads across all of them.+Sub-agent stats are aggregated via a shared stats file at+`${HEADROOM_WORKSPACE_DIR}/session_stats.jsonl` (default+`~/.headroom/session_stats.jsonl` — see the+[Filesystem Contract](filesystem-contract.md)). Each MCP server instance+(main session and sub-agents) writes events there, and `headroom_stats`+reads across all of them.## Architecture@@ -222,4 +227,4 @@ headroom proxy # In another terminal### Sub-agent stats not showing-Sub-agent stats appear in `headroom_stats` only after sub-agents have run compressions. The shared stats file is at `~/.headroom/session_stats.jsonl`.+Sub-agent stats appear in `headroom_stats` only after sub-agents have run compressions. The shared stats file is at `${HEADROOM_WORKSPACE_DIR}/session_stats.jsonl` (defaults to `~/.headroom/session_stats.jsonl`).diff --git a/wiki/memory.md b/wiki/memory.mdindex ce440a726..cd0c6eadb 100644--- a/wiki/memory.md+++ b/wiki/memory.md@@ -74,7 +74,16 @@ Claude Code Codex CLI Gemini CLI### Project-Scoped Database-Memory is stored per-project at `{cwd}/.headroom/memory.db`. Each project has its own memory — no cross-project contamination. Override with `--memory-db-path` for a custom location.+Memory is stored per-project at `{cwd}/.headroom/memory.db`. Each+project has its own memory — no cross-project contamination. Override+with `--memory-db-path` for a custom location.++> **Filesystem contract note.** Project-scoped memory paths resolve+> relative to the current working directory and **do not** obey the+> canonical `HEADROOM_WORKSPACE_DIR` env var. This preserves the+> project-memory isolation invariant. Users who want a single central+> memory store should pass `--memory-db-path` explicitly. See the+> [Filesystem Contract](filesystem-contract.md) for the rationale.### User Identitydiff --git a/wiki/metrics.md b/wiki/metrics.mdindex 7a5bf3c5b..608ee9036 100644--- a/wiki/metrics.md+++ b/wiki/metrics.md@@ -50,8 +50,12 @@ curl http://localhost:8787/stats`/stats` keeps the existing live/session fields, including `savings_history`,for backward compatibility. The new `persistent_savings` block is durable local-proxy compression history stored by default at `~/.headroom/proxy_savings.json`.-Use `HEADROOM_SAVINGS_PATH` to override the file location.+proxy compression history stored by default at+`${HEADROOM_WORKSPACE_DIR}/proxy_savings.json` (i.e.+`~/.headroom/proxy_savings.json` when `HEADROOM_WORKSPACE_DIR` is unset).+Use `HEADROOM_SAVINGS_PATH` to override the file location directly, or+set `HEADROOM_WORKSPACE_DIR` to relocate the entire state root. See the+[Filesystem Contract](filesystem-contract.md) for details.For Anthropic-style providers that return cache-write TTL buckets, `/stats`also surfaces observed cache TTL usage under `prefix_cache`:diff --git a/wiki/persistent-installs.md b/wiki/persistent-installs.mdindex 98e424fdd..3c2184d37 100644--- a/wiki/persistent-installs.md+++ b/wiki/persistent-installs.md@@ -146,6 +146,13 @@ docker compose -f docker/docker-compose.native.yml up -d proxyThat keeps `localhost:8787` stable and restarts the proxy automatically.+> **Note:** `HEADROOM_WORKSPACE` (the host-side bind-mount source used+> by the compose file) is **not** the same variable as+> `HEADROOM_WORKSPACE_DIR` (the canonical Headroom state root inside+> the container). Both are retained; the compose file sets the latter+> automatically. See [Filesystem Contract](filesystem-contract.md) for+> the full bucket model.+## Related guides- [CLI Reference](cli.md)diff --git a/wiki/proxy.md b/wiki/proxy.mdindex 58c6b7dac..3f6464d36 100644--- a/wiki/proxy.md+++ b/wiki/proxy.md@@ -245,8 +245,12 @@ other Headroom frontends. It returns:- derived hourly, daily, weekly, and monthly rollups for charts- UTC timestamps throughout-By default the proxy stores this history at `~/.headroom/proxy_savings.json`.-Set `HEADROOM_SAVINGS_PATH` to override the location.+By default the proxy stores this history at+`${HEADROOM_WORKSPACE_DIR}/proxy_savings.json` (i.e.+`~/.headroom/proxy_savings.json` when `HEADROOM_WORKSPACE_DIR` is unset).+Set `HEADROOM_SAVINGS_PATH` to override the location directly, or set+`HEADROOM_WORKSPACE_DIR` to relocate the full state root. See the+[Filesystem Contract](filesystem-contract.md).`/dashboard` uses this endpoint directly for its historical view, including thedaily/weekly/monthly rollups and built-in JSON / CSV export buttons.