Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

refactor: extract provider logic into slices

JerrettDavisb17c6d81cc39
diff --git a/README.md b/README.md
index 7090a50c8..09d9d3bae 100644
--- a/README.md
+++ b/README.md
@@ -1,286 +1,285 @@
-<div align="center">
-
-# Headroom
-
-**Compress everything your AI agent reads. Same answers, fraction of the tokens.**
-
-[![CI](https://github.com/chopratejas/headroom/actions/workflows/ci.yml/badge.svg)](https://github.com/chopratejas/headroom/actions/workflows/ci.yml)
-[![codecov](https://codecov.io/gh/chopratejas/headroom/graph/badge.svg)](https://app.codecov.io/gh/chopratejas/headroom)
-[![PyPI](https://img.shields.io/pypi/v/headroom-ai.svg)](https://pypi.org/project/headroom-ai/)
-[![npm](https://img.shields.io/npm/v/headroom-ai.svg)](https://www.npmjs.com/package/headroom-ai)
-[![Model: Kompress-base](https://img.shields.io/badge/model-Kompress--base-yellow.svg)](https://huggingface.co/chopratejas/kompress-base)
-[![Tokens saved: 60B+](https://img.shields.io/badge/tokens%20saved-60B%2B-2ea44f)](https://headroomlabs.ai/dashboard)
-[![License: Apache 2.0](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
-[![Docs](https://img.shields.io/badge/docs-online-blue.svg)](https://headroom-docs.vercel.app/docs)
-
-<img src="HeadroomDemo-Fast.gif" alt="Headroom in action" width="820">
-
-</div>
-
----
-
-Every tool call, log line, DB read, RAG chunk, and file your agent injects into a prompt is mostly boilerplate. Headroom strips the noise and keeps the signal — **losslessly, locally, and without touching accuracy.**
-
-> **100 logs. One FATAL error buried at position 67. Both runs found it.**
-> Baseline **10,144 tokens** → Headroom **1,260 tokens** — **87% fewer, identical answer.**
-> `python examples/needle_in_haystack_test.py`
-
----
-
-## Quick start
-
-Works with Anthropic, OpenAI, Google, Bedrock, Vertex, Azure, OpenRouter, and 100+ models via LiteLLM.
-
-**Wrap your coding agent — one command:**
-
-```bash
-pip install "headroom-ai[all]"
-
-headroom wrap claude # Claude Code
-headroom wrap codex # Codex
-headroom wrap cursor # Cursor
-headroom wrap aider # Aider
-headroom wrap copilot # GitHub Copilot CLI
-```
-
-**Prefer a one-time durable install instead of wrapping every launch:**
-
-```bash
-headroom init -g # Detect installed user-scoped agents and wire them to Headroom
-headroom init claude # Install repo-local Claude hooks for just this project
-headroom init copilot -g # Install user-scoped Copilot hooks and provider routing
-```
-
-**Drop it into your own code — Python or TypeScript:**
-
-```python
-from headroom import compress
-
-result = compress(messages, model="claude-sonnet-4-5")
-response = client.messages.create(model="claude-sonnet-4-5", messages=result.messages)
-print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})")
-```
-
-```typescript
-import { compress } from 'headroom-ai';
-const result = await compress(messages, { model: 'gpt-4o' });
-```
-
-**Or run it as a proxy — zero code changes, any language:**
-
-```bash
-headroom proxy --port 8787
-ANTHROPIC_BASE_URL=http://localhost:8787 your-app
-OPENAI_BASE_URL=http://localhost:8787/v1 your-app
-```
-
----
-
-## Why Headroom
-
-- **Accuracy-preserving.** GSM8K **0.870 → 0.870** (±0.000). TruthfulQA **+0.030**. SQuAD v2 and BFCL both **97%** accuracy after compression. Validated on public OSS benchmarks you can rerun yourself.
-- **Runs on your machine.** No cloud API, no data egress. Compression latency is milliseconds — faster end-to-end for Sonnet / Opus / GPT-4 class models than a hosted service round-trip.
-- **[Kompress-base](https://huggingface.co/chopratejas/kompress-base) on HuggingFace.** Our open-source text compressor, fine-tuned on real agentic traces — tool outputs, logs, RAG chunks, code. Install with `pip install "headroom-ai[ml]"`.
-- **Cross-agent memory and learning.** Claude Code saves a fact, Codex reads it back. `headroom learn` mines failed sessions and writes corrections straight to `CLAUDE.md` / `AGENTS.md` / `GEMINI.md` — reliability compounds over time.
-- **Reversible (CCR).** Compression is not deletion. The model can always call `headroom_retrieve` to pull the original bytes. Nothing is thrown away.
-
-Bundles the [RTK](https://github.com/rtk-ai/rtk) binary for shell-output rewriting — full [attribution below](#compared-to).
-
----
-
-## How it fits
-
-```
- Your agent / app
- (Claude Code, Cursor, Codex, LangChain, Agno, Strands, your own code…)
- │ prompts · tool outputs · logs · RAG results · files
- ▼
- ┌────────────────────────────────────────────────────┐
- │ Headroom (runs locally — your data stays here) │
- │ ─────────────────────────────────────────────── │
- │ CacheAligner → ContentRouter → CCR │
- │ ├─ SmartCrusher (JSON) │
- │ ├─ CodeCompressor (AST) │
- │ └─ Kompress-base (text, HF) │
- │ │
- │ Cross-agent memory · headroom learn · MCP │
- └────────────────────────────────────────────────────┘
- │ compressed prompt + retrieval tool
- ▼
- LLM provider (Anthropic · OpenAI · Bedrock · …)
-```
-
-→ [Architecture](https://headroom-docs.vercel.app/docs/architecture) · [CCR reversible compression](https://headroom-docs.vercel.app/docs/ccr) · [Kompress-base model card](https://huggingface.co/chopratejas/kompress-base)
-
-### Canonical pipeline lifecycle
-
-Headroom now exposes one stable request lifecycle across `compress()`, the SDK, and the proxy:
-
-`Setup` → `Pre-Start` → `Post-Start` → `Input Received` → `Input Cached` → `Input Routed` → `Input Compressed` → `Input Remembered` → `Pre-Send` → `Post-Send` → `Response Received`
-
-- **Transforms** still do the work: CacheAligner, ContentRouter, SmartCrusher, CodeCompressor, Kompress-base, IntelligentContext / RollingWindow.
-- **Pipeline extensions** observe or customize those lifecycle stages via `on_pipeline_event(...)`.
-- **Compression hooks** still work and now sit alongside the canonical lifecycle instead of being the only extension seam.
-- **Proxy extensions** remain the server/app integration seam for ASGI middleware, routes, and startup policy.
-
----
-
-## Proof
-
-**Savings on real agent workloads:**
-
-| Workload | Before | After | Savings |
-|-------------------------------|-------:|-------:|--------:|
-| Code search (100 results) | 17,765 | 1,408 | **92%** |
-| SRE incident debugging | 65,694 | 5,118 | **92%** |
-| GitHub issue triage | 54,174 | 14,761 | **73%** |
-| Codebase exploration | 78,502 | 41,254 | **47%** |
-
-**Accuracy preserved on standard benchmarks:**
-
-| Benchmark | Category | N | Baseline | Headroom | Delta |
-|------------|----------|----:|---------:|---------:|----------:|
-| GSM8K | Math | 100 | 0.870 | 0.870 | **±0.000**|
-| TruthfulQA | Factual | 100 | 0.530 | 0.560 | **+0.030**|
-| SQuAD v2 | QA | 100 | — | **97%** | 19% compression |
-| BFCL | Tools | 100 | — | **97%** | 32% compression |
-
-Reproduce:
-
-```bash
-python -m headroom.evals suite --tier 1
-```
-
-**Community, live:**
-
-<div align="center">
- <a href="https://headroomlabs.ai/dashboard">
- <img src="headroom-savings.png" alt="60B+ tokens saved — community leaderboard" width="820">
- </a>
- <p><b><a href="https://headroomlabs.ai/dashboard">60B+ tokens saved by the community in the last 20 days — live leaderboard →</a></b></p>
-</div>
-
-→ [Full benchmarks & methodology](https://headroom-docs.vercel.app/docs/benchmarks)
-
----
-
-## Built for coding agents
-
-| Agent | Durable init / one-shot wrap | Notes |
-|--------------------|------------------------------------|------------------------------------------------------------------|
-| **Claude Code** | `headroom init claude -g` / `headroom wrap claude` | `init` installs user or repo-local hooks; `wrap` is still useful for ad hoc sessions |
-| **Codex** | `headroom init codex -g` / `headroom wrap codex --memory` | `init` installs provider config plus lifecycle hooks where supported |
-| **Cursor** | `headroom wrap cursor` | Prints Cursor config — durable init not available yet |
-| **Aider** | `headroom wrap aider` | Starts proxy, launches Aider |
-| **Copilot CLI** | `headroom init copilot -g` / `headroom wrap copilot` | `init` installs hooks and BYOK provider routing for the current user |
-| **OpenClaw** | `headroom init openclaw -g` / `headroom wrap openclaw` | Installs Headroom as ContextEngine plugin |
-
-MCP-native too — `headroom mcp install` exposes `headroom_compress`, `headroom_retrieve`, and `headroom_stats` to any MCP client.
-
-<div align="center">
- <img src="headroom_learn.gif" alt="headroom learn in action" width="720">
-</div>
-
----
-
-## Integrations
-
-<details>
-<summary><b>Drop Headroom into any stack</b></summary>
-
-| Your setup | Hook in with |
-|-------------------------|------------------------------------------------------------------|
-| Any Python app | `compress(messages, model=…)` |
-| Any TypeScript app | `await compress(messages, { model })` |
-| Anthropic / OpenAI SDK | `withHeadroom(new Anthropic())` · `withHeadroom(new OpenAI())` |
-| Vercel AI SDK | `wrapLanguageModel({ model, middleware: headroomMiddleware() })` |
-| LiteLLM | `litellm.callbacks = [HeadroomCallback()]` |
-| LangChain | `HeadroomChatModel(your_llm)` |
-| Agno | `HeadroomAgnoModel(your_model)` |
-| Strands | [Strands guide](https://headroom-docs.vercel.app/docs/strands) |
-| ASGI apps | `app.add_middleware(CompressionMiddleware)` |
-| Multi-agent | `SharedContext().put / .get` |
-| MCP clients | `headroom mcp install` |
-
-</details>
-
-<details>
-<summary><b>What's inside</b></summary>
-
-- **SmartCrusher** — universal JSON: arrays of dicts, nested objects, mixed types.
-- **CodeCompressor** — AST-aware for Python, JS, Go, Rust, Java, C++.
-- **Kompress-base** — our HuggingFace model, trained on agentic traces.
-- **Image compression** — 40–90% reduction via trained ML router.
-- **CacheAligner** — stabilizes prefixes so Anthropic/OpenAI KV caches actually hit.
-- **IntelligentContext** — score-based context fitting with learned importance.
-- **CCR** — reversible compression; LLM retrieves originals on demand.
-- **Cross-agent memory** — shared store, agent provenance, auto-dedup.
-- **SharedContext** — compressed context passing across multi-agent workflows.
-- **`headroom learn`** — plugin-based failure mining for Claude, Codex, Gemini.
-
-</details>
-
----
-
-## Install
-
-```bash
-pip install "headroom-ai[all]" # Python, everything
-npm install headroom-ai # TypeScript / Node
-docker pull ghcr.io/chopratejas/headroom:latest
-```
-
-Granular extras: `[proxy]`, `[mcp]`, `[ml]` (Kompress-base), `[agno]`, `[langchain]`, `[evals]`. Requires **Python 3.10+**.
-
-→ [Installation guide](https://headroom-docs.vercel.app/docs/installation) — Docker tags, persistent service, PowerShell, devcontainers.
-
----
-
-## Documentation
-
-| Start here | Go deeper |
-|-------------------------------------------------------------------------|------------------------------------------------------------------------|
-| [Quickstart](https://headroom-docs.vercel.app/docs/quickstart) | [Architecture](https://headroom-docs.vercel.app/docs/architecture) |
-| [Proxy](https://headroom-docs.vercel.app/docs/proxy) | [How compression works](https://headroom-docs.vercel.app/docs/how-compression-works) |
-| [MCP tools](https://headroom-docs.vercel.app/docs/mcp) | [CCR — reversible compression](https://headroom-docs.vercel.app/docs/ccr) |
-| [Memory](https://headroom-docs.vercel.app/docs/memory) | [Cache optimization](https://headroom-docs.vercel.app/docs/cache-optimization) |
-| [Failure learning](https://headroom-docs.vercel.app/docs/failure-learning) | [Benchmarks](https://headroom-docs.vercel.app/docs/benchmarks) |
-| [Configuration](https://headroom-docs.vercel.app/docs/configuration) | [Limitations](https://headroom-docs.vercel.app/docs/limitations) |
-
----
-
-## Compared to
-
-Headroom runs **locally**, covers **every** content type (not just CLI or text), works with every major framework, and is **reversible**.
-
-| | Scope | Deploy | Local | Reversible |
-|----------------------------------|-------------------------------------------------|-------------------------------------|:-----:|:----------:|
-| **Headroom** | All context — tools, RAG, logs, files, history | Proxy · library · middleware · MCP | Yes | Yes |
-| [RTK](https://github.com/rtk-ai/rtk) | CLI command outputs | CLI wrapper | Yes | No |
-| [Compresr](https://compresr.ai), [Token Co.](https://thetokencompany.ai) | Text sent to their API | Hosted API call | No | No |
-| OpenAI Compaction | Conversation history | Provider-native | No | No |
-
-> **Attribution.** Headroom ships with the excellent [RTK](https://github.com/rtk-ai/rtk) binary for shell-output rewriting — `git show` → `git show --short`, noisy `ls` → scoped, chatty installers → summarized. Huge thanks to the RTK team; their tool is a first-class part of our stack, and Headroom compresses everything downstream of it.
-
----
-
-## Contributing
-
-```bash
-git clone https://github.com/chopratejas/headroom.git && cd headroom
-pip install -e ".[dev]" && pytest
-```
-
-Devcontainers in `.devcontainer/` (default + `memory-stack` with Qdrant & Neo4j). See [CONTRIBUTING.md](CONTRIBUTING.md).
-
----
-
-## Community
-
-- **[Live leaderboard](https://headroomlabs.ai/dashboard)** — 60B+ tokens saved and counting.
-- **[Discord](https://discord.gg/yRmaUNpsPJ)** — questions, feedback, war stories.
-- **[Kompress-base on HuggingFace](https://huggingface.co/chopratejas/kompress-base)** — the model behind our text compression.
-
-## License
-
-Apache 2.0 — see [LICENSE](LICENSE).
+<div align="center">
+
+# Headroom
+
+**Compress everything your AI agent reads. Same answers, fraction of the tokens.**
+
+[![CI](https://github.com/chopratejas/headroom/actions/workflows/ci.yml/badge.svg)](https://github.com/chopratejas/headroom/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/headroom-ai.svg)](https://pypi.org/project/headroom-ai/)
+[![npm](https://img.shields.io/npm/v/headroom-ai.svg)](https://www.npmjs.com/package/headroom-ai)
+[![Model: Kompress-base](https://img.shields.io/badge/model-Kompress--base-yellow.svg)](https://huggingface.co/chopratejas/kompress-base)
+[![Tokens saved: 60B+](https://img.shields.io/badge/tokens%20saved-60B%2B-2ea44f)](https://headroomlabs.ai/dashboard)
+[![License: Apache 2.0](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
+[![Docs](https://img.shields.io/badge/docs-online-blue.svg)](https://headroom-docs.vercel.app/docs)
+
+<img src="HeadroomDemo-Fast.gif" alt="Headroom in action" width="820">
+
+</div>
+
+---
+
+Every tool call, log line, DB read, RAG chunk, and file your agent injects into a prompt is mostly boilerplate. Headroom strips the noise and keeps the signal — **losslessly, locally, and without touching accuracy.**
+
+> **100 logs. One FATAL error buried at position 67. Both runs found it.**
+> Baseline **10,144 tokens** → Headroom **1,260 tokens** — **87% fewer, identical answer.**
+> `python examples/needle_in_haystack_test.py`
+
+---
+
+## Quick start
+
+Works with Anthropic, OpenAI, Google, Bedrock, Vertex, Azure, OpenRouter, and 100+ models via LiteLLM.
+
+**Wrap your coding agent — one command:**
+
+```bash
+pip install "headroom-ai[all]"
+
+headroom wrap claude # Claude Code
+headroom wrap codex # Codex
+headroom wrap cursor # Cursor
+headroom wrap aider # Aider
+headroom wrap copilot # GitHub Copilot CLI
+```
+
+**Drop it into your own code — Python or TypeScript:**
+
+```python
+from headroom import compress
+
+result = compress(messages, model="claude-sonnet-4-5")
+response = client.messages.create(model="claude-sonnet-4-5", messages=result.messages)
+print(f"Saved {result.tokens_saved} tokens ({result.compression_ratio:.0%})")
+```
+
+```typescript
+import { compress } from 'headroom-ai';
+const result = await compress(messages, { model: 'gpt-4o' });
+```
+
+**Or run it as a proxy — zero code changes, any language:**
+
+```bash
+headroom proxy --port 8787
+ANTHROPIC_BASE_URL=http://localhost:8787 your-app
+OPENAI_BASE_URL=http://localhost:8787/v1 your-app
+```
+
+---
+
+## Why Headroom
+
+- **Accuracy-preserving.** GSM8K **0.870 → 0.870** (±0.000). TruthfulQA **+0.030**. SQuAD v2 and BFCL both **97%** accuracy after compression. Validated on public OSS benchmarks you can rerun yourself.
+- **Runs on your machine.** No cloud API, no data egress. Compression latency is milliseconds — faster end-to-end for Sonnet / Opus / GPT-4 class models than a hosted service round-trip.
+- **[Kompress-base](https://huggingface.co/chopratejas/kompress-base) on HuggingFace.** Our open-source text compressor, fine-tuned on real agentic traces — tool outputs, logs, RAG chunks, code. Install with `pip install "headroom-ai[ml]"`.
+- **Cross-agent memory and learning.** Claude Code saves a fact, Codex reads it back. `headroom learn` mines failed sessions and writes corrections straight to `CLAUDE.md` / `AGENTS.md` / `GEMINI.md` — reliability compounds over time.
+- **Reversible (CCR).** Compression is not deletion. The model can always call `headroom_retrieve` to pull the original bytes. Nothing is thrown away.
+
+Bundles the [RTK](https://github.com/rtk-ai/rtk) binary for shell-output rewriting — full [attribution below](#compared-to).
+
+---
+
+## How it fits
+
+```
+ Your agent / app
+ (Claude Code, Cursor, Codex, LangChain, Agno, Strands, your own code…)
+ │ prompts · tool outputs · logs · RAG results · files
+ ▼
+ ┌────────────────────────────────────────────────────┐
+ │ Headroom (runs locally — your data stays here) │
+ │ ─────────────────────────────────────────────── │
+ │ CacheAligner → ContentRouter → CCR │
+ │ ├─ SmartCrusher (JSON) │
+ │ ├─ CodeCompressor (AST) │
+ │ └─ Kompress-base (text, HF) │
+ │ │
+ │ Cross-agent memory · headroom learn · MCP │
+ └────────────────────────────────────────────────────┘
+ │ compressed prompt + retrieval tool
+ ▼
+ LLM provider (Anthropic · OpenAI · Bedrock · …)
+```
+
+→ [Architecture](https://headroom-docs.vercel.app/docs/architecture) · [CCR reversible compression](https://headroom-docs.vercel.app/docs/ccr) · [Kompress-base model card](https://huggingface.co/chopratejas/kompress-base)
+
+### Canonical pipeline lifecycle
+
+Headroom now exposes one stable request lifecycle across `compress()`, the SDK, and the proxy:
+
+`Setup` → `Pre-Start` → `Post-Start` → `Input Received` → `Input Cached` → `Input Routed` → `Input Compressed` → `Input Remembered` → `Pre-Send` → `Post-Send` → `Response Received`
+
+- **Transforms** still do the work: CacheAligner, ContentRouter, SmartCrusher, CodeCompressor, Kompress-base, IntelligentContext / RollingWindow.
+- **Pipeline extensions** observe or customize those lifecycle stages via `on_pipeline_event(...)`.
+- **Compression hooks** still work and now sit alongside the canonical lifecycle instead of being the only extension seam.
+- **Proxy extensions** remain the server/app integration seam for ASGI middleware, routes, and startup policy.
+
+### Provider slices
+
+Provider and tool-specific behavior is being moved behind dedicated modules under `headroom/providers/` so core orchestration stays focused on lifecycle, sequencing, and policy.
+
+- **CLI/tool slices**: `headroom/providers/claude`, `copilot`, `codex`, `openclaw`
+- **Provider runtime slices**: `headroom/providers/claude`, `gemini`, plus shared backend/runtime dispatch in `headroom/providers/registry.py`
+- **Core files stay orchestration-first**: `wrap.py`, `client.py`, `cli/proxy.py`, and `proxy/server.py` now delegate provider-specific env shaping, API target normalization, backend selection, and transport dispatch instead of inlining those rules.
+
+---
+
+## Proof
+
+**Savings on real agent workloads:**
+
+| Workload | Before | After | Savings |
+|-------------------------------|-------:|-------:|--------:|
+| Code search (100 results) | 17,765 | 1,408 | **92%** |
+| SRE incident debugging | 65,694 | 5,118 | **92%** |
+| GitHub issue triage | 54,174 | 14,761 | **73%** |
+| Codebase exploration | 78,502 | 41,254 | **47%** |
+
+**Accuracy preserved on standard benchmarks:**
+
+| Benchmark | Category | N | Baseline | Headroom | Delta |
+|------------|----------|----:|---------:|---------:|----------:|
+| GSM8K | Math | 100 | 0.870 | 0.870 | **±0.000**|
+| TruthfulQA | Factual | 100 | 0.530 | 0.560 | **+0.030**|
+| SQuAD v2 | QA | 100 | — | **97%** | 19% compression |
+| BFCL | Tools | 100 | — | **97%** | 32% compression |
+
+Reproduce:
+
+```bash
+python -m headroom.evals suite --tier 1
+```
+
+**Community, live:**
+
+<div align="center">
+ <a href="https://headroomlabs.ai/dashboard">
+ <img src="headroom-savings.png" alt="60B+ tokens saved — community leaderboard" width="820">
+ </a>
+ <p><b><a href="https://headroomlabs.ai/dashboard">60B+ tokens saved by the community in the last 20 days — live leaderboard →</a></b></p>
+</div>
+
+→ [Full benchmarks & methodology](https://headroom-docs.vercel.app/docs/benchmarks)
+
+---
+
+## Built for coding agents
+
+| Agent | One-command wrap | Notes |
+|--------------------|------------------------------------|------------------------------------------------------------------|
+| **Claude Code** | `headroom wrap claude` | `--memory` for cross-agent memory, `--code-graph` for codebase intel |
+| **Codex** | `headroom wrap codex --memory` | Shares the same memory store as Claude |
+| **Cursor** | `headroom wrap cursor` | Prints Cursor config — paste once, done |
+| **Aider** | `headroom wrap aider` | Starts proxy, launches Aider |
+| **Copilot CLI** | `headroom wrap copilot` | Starts proxy, launches Copilot |
+| **OpenClaw** | `headroom wrap openclaw` | Installs Headroom as ContextEngine plugin |
+
+MCP-native too — `headroom mcp install` exposes `headroom_compress`, `headroom_retrieve`, and `headroom_stats` to any MCP client.
+
+<div align="center">
+ <img src="headroom_learn.gif" alt="headroom learn in action" width="720">
+</div>
+
+---
+
+## Integrations
+
+<details>
+<summary><b>Drop Headroom into any stack</b></summary>
+
+| Your setup | Hook in with |
+|-------------------------|------------------------------------------------------------------|
+| Any Python app | `compress(messages, model=…)` |
+| Any TypeScript app | `await compress(messages, { model })` |
+| Anthropic / OpenAI SDK | `withHeadroom(new Anthropic())` · `withHeadroom(new OpenAI())` |
+| Vercel AI SDK | `wrapLanguageModel({ model, middleware: headroomMiddleware() })` |
+| LiteLLM | `litellm.callbacks = [HeadroomCallback()]` |
+| LangChain | `HeadroomChatModel(your_llm)` |
+| Agno | `HeadroomAgnoModel(your_model)` |
+| Strands | [Strands guide](https://headroom-docs.vercel.app/docs/strands) |
+| ASGI apps | `app.add_middleware(CompressionMiddleware)` |
+| Multi-agent | `SharedContext().put / .get` |
+| MCP clients | `headroom mcp install` |
+
+</details>
+
+<details>
+<summary><b>What's inside</b></summary>
+
+- **SmartCrusher** — universal JSON: arrays of dicts, nested objects, mixed types.
+- **CodeCompressor** — AST-aware for Python, JS, Go, Rust, Java, C++.
+- **Kompress-base** — our HuggingFace model, trained on agentic traces.
+- **Image compression** — 40–90% reduction via trained ML router.
+- **CacheAligner** — stabilizes prefixes so Anthropic/OpenAI KV caches actually hit.
+- **IntelligentContext** — score-based context fitting with learned importance.
+- **CCR** — reversible compression; LLM retrieves originals on demand.
+- **Cross-agent memory** — shared store, agent provenance, auto-dedup.
+- **SharedContext** — compressed context passing across multi-agent workflows.
+- **`headroom learn`** — plugin-based failure mining for Claude, Codex, Gemini.
+
+</details>
+
+---
+
+## Install
+
+```bash
+pip install "headroom-ai[all]" # Python, everything
+npm install headroom-ai # TypeScript / Node
+docker pull ghcr.io/chopratejas/headroom:latest
+```
+
+Granular extras: `[proxy]`, `[mcp]`, `[ml]` (Kompress-base), `[agno]`, `[langchain]`, `[evals]`. Requires **Python 3.10+**.
+
+→ [Installation guide](https://headroom-docs.vercel.app/docs/installation) — Docker tags, persistent service, PowerShell, devcontainers.
+
+---
+
+## Documentation
+
+| Start here | Go deeper |
+|-------------------------------------------------------------------------|------------------------------------------------------------------------|
+| [Quickstart](https://headroom-docs.vercel.app/docs/quickstart) | [Architecture](https://headroom-docs.vercel.app/docs/architecture) |
+| [Proxy](https://headroom-docs.vercel.app/docs/proxy) | [How compression works](https://headroom-docs.vercel.app/docs/how-compression-works) |
+| [MCP tools](https://headroom-docs.vercel.app/docs/mcp) | [CCR — reversible compression](https://headroom-docs.vercel.app/docs/ccr) |
+| [Memory](https://headroom-docs.vercel.app/docs/memory) | [Cache optimization](https://headroom-docs.vercel.app/docs/cache-optimization) |
+| [Failure learning](https://headroom-docs.vercel.app/docs/failure-learning) | [Benchmarks](https://headroom-docs.vercel.app/docs/benchmarks) |
+| [Configuration](https://headroom-docs.vercel.app/docs/configuration) | [Limitations](https://headroom-docs.vercel.app/docs/limitations) |
+
+---
+
+## Compared to
+
+Headroom runs **locally**, covers **every** content type (not just CLI or text), works with every major framework, and is **reversible**.
+
+| | Scope | Deploy | Local | Reversible |
+|----------------------------------|-------------------------------------------------|-------------------------------------|:-----:|:----------:|
+| **Headroom** | All context — tools, RAG, logs, files, history | Proxy · library · middleware · MCP | Yes | Yes |
+| [RTK](https://github.com/rtk-ai/rtk) | CLI command outputs | CLI wrapper | Yes | No |
+| [Compresr](https://compresr.ai), [Token Co.](https://thetokencompany.ai) | Text sent to their API | Hosted API call | No | No |
+| OpenAI Compaction | Conversation history | Provider-native | No | No |
+
+> **Attribution.** Headroom ships with the excellent [RTK](https://github.com/rtk-ai/rtk) binary for shell-output rewriting — `git show` → `git show --short`, noisy `ls` → scoped, chatty installers → summarized. Huge thanks to the RTK team; their tool is a first-class part of our stack, and Headroom compresses everything downstream of it.
+
+---
+
+## Contributing
+
+```bash
+git clone https://github.com/chopratejas/headroom.git && cd headroom
+pip install -e ".[dev]" && pytest
+```
+
+Devcontainers in `.devcontainer/` (default + `memory-stack` with Qdrant & Neo4j). See [CONTRIBUTING.md](CONTRIBUTING.md).
+
+---
+
+## Community
+
+- **[Live leaderboard](https://headroomlabs.ai/dashboard)** — 60B+ tokens saved and counting.
+- **[Discord](https://discord.gg/yRmaUNpsPJ)** — questions, feedback, war stories.
+- **[Kompress-base on HuggingFace](https://huggingface.co/chopratejas/kompress-base)** — the model behind our text compression.
+
+## License
+
+Apache 2.0 — see [LICENSE](LICENSE).
diff --git a/headroom/cli/proxy.py b/headroom/cli/proxy.py
index 5564ff639..7204406ae 100644
--- a/headroom/cli/proxy.py
+++ b/headroom/cli/proxy.py
@@ -1,550 +1,554 @@
-"""Proxy server CLI commands."""
-
-import os
-import sys
-
-import click
-
-from headroom.proxy.modes import PROXY_MODE_TOKEN, normalize_proxy_mode
-
-from .main import main
-
-
- "--host",
- default="127.0.0.1",
- envvar="HEADROOM_HOST",
- help="Host to bind to (default: 127.0.0.1, env: HEADROOM_HOST)",
-)
- "--port",
- "-p",
- default=8787,
- type=int,
- envvar="HEADROOM_PORT",
- help="Port to bind to (default: 8787, env: HEADROOM_PORT)",
-)
- "--mode",
- default=None,
- type=click.Choice(
- [
- "token",
- "cache",
- "token_mode",
- "cache_mode",
- "token_savings",
- "cost_savings",
- "token_headroom",
- ]
- ),
- help=(
- "Optimization mode: token (prioritize compression) or cache "
- "(freeze prior turns for prefix-cache stability). "
- "Legacy aliases are accepted. Default: token. Env: HEADROOM_MODE"
- ),
-)
- "--intercept-tool-results",
- is_flag=True,
- help=(
- "Opt in to tool_result interceptors (ast-grep Read outliner, etc.). "
- "Off by default while this feature ships."
- ),
-)
[email protected]("--no-optimize", is_flag=True, help="Disable optimization (passthrough mode)")
[email protected]("--no-cache", is_flag=True, help="Disable semantic caching")
[email protected]("--no-rate-limit", is_flag=True, help="Disable rate limiting")
- "--retry-max-attempts",
- type=int,
- default=None,
- help="Maximum upstream retry attempts for connect/read/5xx failures (default: 3)",
-)
- "--connect-timeout-seconds",
- type=int,
- default=None,
- help="Upstream connection timeout in seconds (default: 10)",
-)
- "--anthropic-pre-upstream-concurrency",
- type=int,
- default=None,
- envvar="HEADROOM_ANTHROPIC_PRE_UPSTREAM_CONCURRENCY",
- help=(
- "Cap the number of Anthropic HTTP requests that may run pre-upstream work "
- "(request parse / deep-copy / first compression stage / memory context / upstream connect) "
- "concurrently. Prevents cold-start replay storms from starving /livez and new Codex WS opens. "
- "Default: max(2, min(8, os.cpu_count() or 4)). "
- "Set to 0 or negative to disable (unbounded). "
- "Env: HEADROOM_ANTHROPIC_PRE_UPSTREAM_CONCURRENCY."
- ),
-)
- "--anthropic-pre-upstream-acquire-timeout-seconds",
- type=float,
- default=None,
- envvar="HEADROOM_ANTHROPIC_PRE_UPSTREAM_ACQUIRE_TIMEOUT_SECONDS",
- help=(
- "Fail-fast timeout for waiting on the Anthropic pre-upstream semaphore "
- "before returning 503 + Retry-After. "
- "Default: 15.0 seconds. "
- "Env: HEADROOM_ANTHROPIC_PRE_UPSTREAM_ACQUIRE_TIMEOUT_SECONDS."
- ),
-)
- "--anthropic-pre-upstream-memory-context-timeout-seconds",
- type=float,
- default=None,
- envvar="HEADROOM_ANTHROPIC_PRE_UPSTREAM_MEMORY_CONTEXT_TIMEOUT_SECONDS",
- help=(
- "Fail-open timeout for Anthropic memory-context lookup while the request "
- "still holds a pre-upstream slot. "
- "Default: 2.0 seconds. "
- "Env: HEADROOM_ANTHROPIC_PRE_UPSTREAM_MEMORY_CONTEXT_TIMEOUT_SECONDS."
- ),
-)
[email protected]("--log-file", default=None, help="Path to JSONL log file")
- "--log-messages",
- is_flag=True,
- help="Enable full message logging (request/response content stored for live feed)",
-)
- "--budget",
- type=float,
- default=None,
- envvar="HEADROOM_BUDGET",
- help="Daily budget limit in USD (env: HEADROOM_BUDGET)",
-)
-# Code graph: indexes project + watches files for live reindex via codebase-memory-mcp
- "--code-graph",
- is_flag=True,
- help="Enable code graph intelligence (indexes project, watches files for live reindex via codebase-memory-mcp)",
-)
-# Read lifecycle (ON by default: compresses stale/superseded Read outputs)
- "--no-read-lifecycle",
- is_flag=True,
- help="Disable Read lifecycle management (stale/superseded Read compression)",
-)
-# Intelligent Context Management (ON by default)
- "--no-intelligent-context",
- is_flag=True,
- help="Disable IntelligentContextManager (fall back to RollingWindow)",
-)
- "--no-intelligent-scoring",
- is_flag=True,
- help="Disable multi-factor importance scoring (use position-based)",
-)
- "--no-compress-first",
- is_flag=True,
- help="Disable trying deeper compression before dropping messages",
-)
-# Memory System (Multi-Provider Support)
- "--memory",
- is_flag=True,
- help="Enable persistent user memory. Auto-detects provider and uses appropriate tool format. "
- "Set x-headroom-user-id header for per-user memory (defaults to 'default').",
-)
- "--memory-db-path",
- default="",
- help="Path to memory database file (default: {cwd}/.headroom/memory.db)",
-)
[email protected]("--no-memory-tools", is_flag=True, help="Disable automatic memory tool injection")
- "--no-memory-context", is_flag=True, help="Disable automatic memory context injection"
-)
- "--memory-top-k",
- type=int,
- default=10,
- help="Number of memories to inject as context (default: 10)",
-)
-# Traffic Learning (live pattern extraction from proxy traffic)
- "--learn",
- is_flag=True,
- help="Enable live traffic learning: extract error→recovery patterns, environment facts, "
- "and user preferences from proxy traffic. Implies --memory. "
- "Learned patterns are saved to agent-native memory files (MEMORY.md, .cursor/rules, AGENTS.md).",
-)
- "--no-learn",
- is_flag=True,
- help="Explicitly disable traffic learning even when --memory is set.",
-)
-# Backend configuration
- "--backend",
- default="anthropic",
- help=(
- "API backend: 'anthropic' (direct), 'bedrock' (AWS), 'openrouter' (OpenRouter), "
- "'anyllm' (any-llm), or 'litellm-<provider>' (e.g., litellm-vertex)"
- ),
-)
- "--anyllm-provider",
- default="openai",
- help="Provider for any-llm backend: openai, mistral, groq, ollama, etc. (default: openai)",
-)
- "--anthropic-api-url",
- default=None,
- help="Custom Anthropic API URL for passthrough endpoints (env: ANTHROPIC_TARGET_API_URL)",
-)
- "--openai-api-url",
- default=None,
- help="Custom OpenAI API URL for passthrough endpoints (env: OPENAI_TARGET_API_URL)",
-)
- "--gemini-api-url",
- default=None,
- help="Custom Gemini API URL for passthrough endpoints (env: GEMINI_TARGET_API_URL)",
-)
- "--cloudcode-api-url",
- default=None,
- help="Custom Cloud Code Assist API URL for compatibility endpoints (env: CLOUDCODE_TARGET_API_URL)",
-)
- "--region",
- default="us-west-2",
- help="Cloud region for Bedrock/Vertex/etc (default: us-west-2)",
-)
- "--bedrock-region",
- default=None,
- help="(deprecated, use --region) AWS region for Bedrock",
-)
- "--bedrock-profile",
- default=None,
- help="AWS profile name for Bedrock (default: use default credentials)",
-)
- "--no-telemetry",
- is_flag=True,
- help="Disable anonymous usage telemetry (env: HEADROOM_TELEMETRY=off)",
-)
- "--stateless",
- is_flag=True,
- help="Disable all filesystem writes — run purely in-memory. "
- "For containerized / read-only / load-balanced deployments. "
- "(env: HEADROOM_STATELESS=true)",
-)
-def proxy(
- ctx: click.Context,
- mode: str | None,
- host: str,
- port: int,
- intercept_tool_results: bool,
- no_optimize: bool,
- no_cache: bool,
- no_rate_limit: bool,
- retry_max_attempts: int | None,
- connect_timeout_seconds: int | None,
- anthropic_pre_upstream_concurrency: int | None,
- anthropic_pre_upstream_acquire_timeout_seconds: float | None,
- anthropic_pre_upstream_memory_context_timeout_seconds: float | None,
- log_file: str | None,
- log_messages: bool,
- budget: float | None,
- code_graph: bool,
- no_read_lifecycle: bool,
- no_intelligent_context: bool,
- no_intelligent_scoring: bool,
- no_compress_first: bool,
- memory: bool,
- memory_db_path: str,
- no_memory_tools: bool,
- no_memory_context: bool,
- memory_top_k: int,
- learn: bool,
- no_learn: bool,
- backend: str,
- anyllm_provider: str,
- anthropic_api_url: str | None,
- openai_api_url: str | None,
- gemini_api_url: str | None,
- cloudcode_api_url: str | None,
- region: str,
- bedrock_region: str | None,
- bedrock_profile: str | None,
- no_telemetry: bool,
- stateless: bool,
-) -> None:
- """Start the optimization proxy server.
-
- \b
- Examples:
- headroom proxy Start proxy on port 8787
- headroom proxy --port 8080 Start proxy on port 8080
- headroom proxy --no-optimize Passthrough mode (no optimization)
-
- \b
- Usage with Claude Code:
- ANTHROPIC_BASE_URL=http://localhost:8787 claude
-
- \b
- Usage with OpenAI-compatible clients:
- OPENAI_BASE_URL=http://localhost:8787/v1 your-app
- """
- # Import here to avoid slow startup
- try:
- from headroom.proxy.server import ProxyConfig, run_server
- except ImportError as e:
- click.echo("Error: Proxy dependencies not installed. Run: pip install headroom[proxy]")
- click.echo(f"Details: {e}")
- raise SystemExit(1) from None
-
- # Opt-in: turn on tool_result interceptors (ast-grep Read outline, etc.).
- # Only fetch the bundled CLI tool binaries when the feature is enabled —
- # otherwise we'd pay a network round-trip and risk a readonly-FS failure
- # for capabilities the user hasn't asked for. The TransformPipeline reads
- # this env var at construction time.
- if intercept_tool_results:
- from headroom.binaries import ensure_tools
-
- resolved_tools = ensure_tools()
- critical_tools = ["ast-grep"]
- missing = [t for t in critical_tools if not resolved_tools.get(t)]
- if missing:
- # User explicitly opted in — fail fast rather than silently starting
- # with non-functional interceptors. They can retry with the tool
- # installed, or drop the flag if they want pass-through behavior.
- click.secho(
- f"error: --intercept-tool-results requires tool(s) that could not "
- f"be installed: {missing}. Run `headroom tools doctor` to diagnose, "
- "or omit the flag to start the proxy without interceptors.",
- fg="red",
- err=True,
- )
- sys.exit(1)
- os.environ["HEADROOM_INTERCEPT_ENABLED"] = "1"
-
- # Resolve API URL overrides: CLI flag > env var > None
- effective_anthropic_api_url = anthropic_api_url or os.environ.get("ANTHROPIC_TARGET_API_URL")
- effective_openai_api_url = openai_api_url or os.environ.get("OPENAI_TARGET_API_URL")
- effective_gemini_api_url = gemini_api_url or os.environ.get("GEMINI_TARGET_API_URL")
- effective_cloudcode_api_url = cloudcode_api_url or os.environ.get("CLOUDCODE_TARGET_API_URL")
-
- # Resolve anyllm provider: env var takes precedence over CLI default (matches argparse path)
- effective_anyllm_provider = os.environ.get("HEADROOM_ANYLLM_PROVIDER") or anyllm_provider
-
- # Resolve mode: CLI flag > env var > default
- effective_mode: str = normalize_proxy_mode(
- mode or os.environ.get("HEADROOM_MODE") or PROXY_MODE_TOKEN
- )
-
- # Stateless mode: CLI flag or env var
- is_stateless = stateless or os.environ.get("HEADROOM_STATELESS", "").lower() in (
- "true",
- "1",
- "yes",
- "on",
- )
-
- # Telemetry opt-out: --no-telemetry flag sets the env var
- if no_telemetry:
- os.environ["HEADROOM_TELEMETRY"] = "off"
-
- # Stateless mode: suppress TOIN filesystem persistence
- if is_stateless:
- os.environ["HEADROOM_TOIN_BACKEND"] = "none"
-
- # License key for managed/enterprise deployments (optional)
- license_key = os.environ.get("HEADROOM_LICENSE_KEY")
-
- config = ProxyConfig(
- host=host,
- port=port,
- anthropic_api_url=effective_anthropic_api_url,
- openai_api_url=effective_openai_api_url,
- gemini_api_url=effective_gemini_api_url,
- cloudcode_api_url=effective_cloudcode_api_url,
- mode=effective_mode,
- optimize=not no_optimize,
- cache_enabled=not no_cache,
- rate_limit_enabled=not no_rate_limit,
- retry_max_attempts=retry_max_attempts if retry_max_attempts is not None else 3,
- connect_timeout_seconds=connect_timeout_seconds
- if connect_timeout_seconds is not None
- else 10,
- log_file=None if is_stateless else log_file,
- log_full_messages=log_messages
- or os.environ.get("HEADROOM_LOG_MESSAGES", "").lower() in ("true", "1", "yes", "on"),
- budget_limit_usd=budget,
- # Code graph: live file watcher for incremental reindexing
- code_graph_watcher=code_graph,
- # Read lifecycle: ON by default (use --no-read-lifecycle to disable)
- read_lifecycle=not no_read_lifecycle,
- # Intelligent Context: ON by default (use --no-intelligent-context to disable)
- intelligent_context=not no_intelligent_context,
- intelligent_context_scoring=not no_intelligent_scoring,
- intelligent_context_compress_first=not no_compress_first,
- # Memory System (Multi-Provider with auto-detection)
- # --learn implies --memory (need backend for storing patterns)
- # Stateless mode disables memory (requires SQLite on disk)
- memory_enabled=False if is_stateless else (memory or (learn and not no_learn)),
- memory_db_path=memory_db_path,
- memory_inject_tools=not no_memory_tools,
- memory_inject_context=not no_memory_context,
- memory_top_k=memory_top_k,
- # Traffic Learning: only with --learn, never with --no-learn
- # Stateless mode disables learning (requires filesystem)
- traffic_learning_enabled=False if is_stateless else (learn and not no_learn),
- traffic_learning_agent_type=os.environ.get("HEADROOM_AGENT_TYPE", "unknown"),
- # Backend (Anthropic direct, Bedrock, LiteLLM, or any-llm)
- backend=backend,
- bedrock_region=bedrock_region or region,
- bedrock_profile=bedrock_profile,
- anyllm_provider=effective_anyllm_provider,
- # License / Usage Reporting (managed/enterprise)
- license_key=license_key,
- # Stateless mode: disable all filesystem writes
- stateless=is_stateless,
- # Unit 4: bounded pre-upstream concurrency on the Anthropic HTTP
- # path. ``None`` -> HeadroomProxy computes ``max(2, min(8,
- # os.cpu_count() or 4))``; ``<= 0`` -> disabled (unbounded).
- # Precedence: CLI > env > auto-compute (click's ``envvar``
- # handles the env-var fallback).
- anthropic_pre_upstream_concurrency=anthropic_pre_upstream_concurrency,
- anthropic_pre_upstream_acquire_timeout_seconds=(
- anthropic_pre_upstream_acquire_timeout_seconds
- if anthropic_pre_upstream_acquire_timeout_seconds is not None
- else 15.0
- ),
- anthropic_pre_upstream_memory_context_timeout_seconds=(
- anthropic_pre_upstream_memory_context_timeout_seconds
- if anthropic_pre_upstream_memory_context_timeout_seconds is not None
- else 2.0
- ),
- )
-
- memory_status = "DISABLED"
- if config.memory_enabled:
- memory_status = "ENABLED (multi-provider)"
-
- license_status = "OSS (no license key)"
- if license_key:
- license_status = f"MANAGED (key={license_key[:8]}...)"
-
- anthropic_url = config.anthropic_api_url or "https://api.anthropic.com"
- openai_url = config.openai_api_url or "https://api.openai.com"
- cloudcode_url = config.cloudcode_api_url or "https://cloudcode-pa.googleapis.com"
- backend_section = ""
-
- if config.backend == "anyllm" or config.backend.startswith("anyllm-"):
- # any-llm backend
- backend_section = """
- Set credentials for your provider (e.g., OPENAI_API_KEY, MISTRAL_API_KEY)
- Providers: https://mozilla-ai.github.io/any-llm/providers/
-"""
- elif config.backend != "anthropic":
- # LiteLLM backend
- from headroom.backends.litellm import get_provider_config
-
- provider = config.backend.replace("litellm-", "")
- provider_config = get_provider_config(provider)
-
- # Build usage instructions from provider config
- env_vars_str = (
- ", ".join(provider_config.env_vars) if provider_config.env_vars else "See docs"
- )
- backend_section = f"""
-IMPORTANT for {provider_config.display_name} users:
- 1. Set credentials: {env_vars_str}
- 2. Set a dummy Anthropic key: ANTHROPIC_API_KEY="sk-ant-dummy"
- (Headroom ignores this - it uses your {provider_config.display_name} credentials)
- 3. Set base URL: ANTHROPIC_BASE_URL=http://{config.host}:{config.port}"""
- if provider_config.model_format_hint:
- backend_section += f"\n 4. Use model names: {provider_config.model_format_hint}"
- backend_section += "\n"
-
- # Build memory section if enabled
- memory_section = ""
- if config.memory_enabled:
- memory_section = f"""
-Memory (Multi-Provider):
- - Auto-detects provider from request (Anthropic, OpenAI, Gemini, etc.)
- - Anthropic: Uses native memory tool (memory_20250818) - subscription safe
- - OpenAI/Gemini/Others: Uses function calling format
- - All providers share the same semantic vector store backend
- - Set x-headroom-user-id header for per-user memory (defaults to 'default')
- - Tools: {"ENABLED" if config.memory_inject_tools else "DISABLED"}
- - Context injection: {"ENABLED" if config.memory_inject_context else "DISABLED"}
- - Database: {config.memory_db_path}
-"""
-
- # Stateless mode warning
- stateless_line = ""
- if is_stateless:
- stateless_line = (
- " Stateless: YES (no filesystem writes — memory, logs, TOIN disabled)\n"
- )
-
- from headroom.telemetry.beacon import is_telemetry_enabled
-
- # Build telemetry section for the startup banner
- if is_telemetry_enabled():
- telemetry_line = (
- " Telemetry: ENABLED (anonymous aggregate stats)\n"
- " Disable: HEADROOM_TELEMETRY=off or headroom proxy --no-telemetry"
- )
- else:
- telemetry_line = " Telemetry: DISABLED"
-
- click.echo(f"""
-╔═══════════════════════════════════════════════════════════════════════╗
-║ HEADROOM PROXY ║
-║ The Context Optimization Layer for LLM Applications ║
-╚═══════════════════════════════════════════════════════════════════════╝
-
-Starting proxy server...
-
- URL: http://{config.host}:{config.port}
- Mode: {config.mode}
- Optimization: {"ENABLED" if config.optimize else "DISABLED"}
- Caching: {"ENABLED" if config.cache_enabled else "DISABLED"}
- Rate Limit: {"ENABLED" if config.rate_limit_enabled else "DISABLED"}
- Memory: {memory_status}
- License: {license_status}
-{stateless_line}{telemetry_line}
-{backend_section}
-Routing:
- /v1/messages → {anthropic_url}
- /v1/chat/completions → {openai_url}
- /v1/responses → {openai_url} (HTTP + WebSocket)
- /v1internal:streamGenerateContent → {cloudcode_url}
-
-Usage:
- Claude Code: ANTHROPIC_BASE_URL=http://{config.host}:{config.port} claude
- Codex / OpenAI: OPENAI_BASE_URL=http://{config.host}:{config.port}/v1 your-app
-{memory_section}
-Endpoints:
- GET /livez Process liveness
- GET /readyz Traffic readiness
- GET /health Aggregate health
- GET /stats Detailed statistics
- GET /stats-history Durable compression history + display session
- GET /metrics Prometheus metrics
-
-Press Ctrl+C to stop.
-""")
-
- try:
- run_server(config)
- except KeyboardInterrupt:
- click.echo("\nShutting down...")
+"""Proxy server CLI commands."""
+
+import os
+import sys
+
+import click
+
+from headroom.providers.registry import resolve_api_overrides, resolve_api_targets
+from headroom.proxy.modes import PROXY_MODE_TOKEN, normalize_proxy_mode
+
+from .main import main
+
+
+ "--host",
+ default="127.0.0.1",
+ envvar="HEADROOM_HOST",
+ help="Host to bind to (default: 127.0.0.1, env: HEADROOM_HOST)",
+)
+ "--port",
+ "-p",
+ default=8787,
+ type=int,
+ envvar="HEADROOM_PORT",
+ help="Port to bind to (default: 8787, env: HEADROOM_PORT)",
+)
+ "--mode",
+ default=None,
+ type=click.Choice(
+ [
+ "token",
+ "cache",
+ "token_mode",
+ "cache_mode",
+ "token_savings",
+ "cost_savings",
+ "token_headroom",
+ ]
+ ),
+ help=(
+ "Optimization mode: token (prioritize compression) or cache "
+ "(freeze prior turns for prefix-cache stability). "
+ "Legacy aliases are accepted. Default: token. Env: HEADROOM_MODE"
+ ),
+)
+ "--intercept-tool-results",
+ is_flag=True,
+ help=(
+ "Opt in to tool_result interceptors (ast-grep Read outliner, etc.). "
+ "Off by default while this feature ships."
+ ),
+)
[email protected]("--no-optimize", is_flag=True, help="Disable optimization (passthrough mode)")
[email protected]("--no-cache", is_flag=True, help="Disable semantic caching")
[email protected]("--no-rate-limit", is_flag=True, help="Disable rate limiting")
+ "--retry-max-attempts",
+ type=int,
+ default=None,
+ help="Maximum upstream retry attempts for connect/read/5xx failures (default: 3)",
+)
+ "--connect-timeout-seconds",
+ type=int,
+ default=None,
+ help="Upstream connection timeout in seconds (default: 10)",
+)
+ "--anthropic-pre-upstream-concurrency",
+ type=int,
+ default=None,
+ envvar="HEADROOM_ANTHROPIC_PRE_UPSTREAM_CONCURRENCY",
+ help=(
+ "Cap the number of Anthropic HTTP requests that may run pre-upstream work "
+ "(request parse / deep-copy / first compression stage / memory context / upstream connect) "
+ "concurrently. Prevents cold-start replay storms from starving /livez and new Codex WS opens. "
+ "Default: max(2, min(8, os.cpu_count() or 4)). "
+ "Set to 0 or negative to disable (unbounded). "
+ "Env: HEADROOM_ANTHROPIC_PRE_UPSTREAM_CONCURRENCY."
+ ),
+)
+ "--anthropic-pre-upstream-acquire-timeout-seconds",
+ type=float,
+ default=None,
+ envvar="HEADROOM_ANTHROPIC_PRE_UPSTREAM_ACQUIRE_TIMEOUT_SECONDS",
+ help=(
+ "Fail-fast timeout for waiting on the Anthropic pre-upstream semaphore "
+ "before returning 503 + Retry-After. "
+ "Default: 15.0 seconds. "
+ "Env: HEADROOM_ANTHROPIC_PRE_UPSTREAM_ACQUIRE_TIMEOUT_SECONDS."
+ ),
+)
+ "--anthropic-pre-upstream-memory-context-timeout-seconds",
+ type=float,
+ default=None,
+ envvar="HEADROOM_ANTHROPIC_PRE_UPSTREAM_MEMORY_CONTEXT_TIMEOUT_SECONDS",
+ help=(
+ "Fail-open timeout for Anthropic memory-context lookup while the request "
+ "still holds a pre-upstream slot. "
+ "Default: 2.0 seconds. "
+ "Env: HEADROOM_ANTHROPIC_PRE_UPSTREAM_MEMORY_CONTEXT_TIMEOUT_SECONDS."
+ ),
+)
[email protected]("--log-file", default=None, help="Path to JSONL log file")
+ "--log-messages",
+ is_flag=True,
+ help="Enable full message logging (request/response content stored for live feed)",
+)
+ "--budget",
+ type=float,
+ default=None,
+ envvar="HEADROOM_BUDGET",
+ help="Daily budget limit in USD (env: HEADROOM_BUDGET)",
+)
+# Code graph: indexes project + watches files for live reindex via codebase-memory-mcp
+ "--code-graph",
+ is_flag=True,
+ help="Enable code graph intelligence (indexes project, watches files for live reindex via codebase-memory-mcp)",
+)
+# Read lifecycle (ON by default: compresses stale/superseded Read outputs)
+ "--no-read-lifecycle",
+ is_flag=True,
+ help="Disable Read lifecycle management (stale/superseded Read compression)",
+)
+# Intelligent Context Management (ON by default)
+ "--no-intelligent-context",
+ is_flag=True,
+ help="Disable IntelligentContextManager (fall back to RollingWindow)",
+)
+ "--no-intelligent-scoring",
+ is_flag=True,
+ help="Disable multi-factor importance scoring (use position-based)",
+)
+ "--no-compress-first",
+ is_flag=True,
+ help="Disable trying deeper compression before dropping messages",
+)
+# Memory System (Multi-Provider Support)
+ "--memory",
+ is_flag=True,
+ help="Enable persistent user memory. Auto-detects provider and uses appropriate tool format. "
+ "Set x-headroom-user-id header for per-user memory (defaults to 'default').",
+)
+ "--memory-db-path",
+ default="",
+ help="Path to memory database file (default: {cwd}/.headroom/memory.db)",
+)
[email protected]("--no-memory-tools", is_flag=True, help="Disable automatic memory tool injection")
+ "--no-memory-context", is_flag=True, help="Disable automatic memory context injection"
+)
+ "--memory-top-k",
+ type=int,
+ default=10,
+ help="Number of memories to inject as context (default: 10)",
+)
+# Traffic Learning (live pattern extraction from proxy traffic)
+ "--learn",
+ is_flag=True,
+ help="Enable live traffic learning: extract error→recovery patterns, environment facts, "
+ "and user preferences from proxy traffic. Implies --memory. "
+ "Learned patterns are saved to agent-native memory files (MEMORY.md, .cursor/rules, AGENTS.md).",
+)
+ "--no-learn",
+ is_flag=True,
+ help="Explicitly disable traffic learning even when --memory is set.",
+)
+# Backend configuration
+ "--backend",
+ default="anthropic",
+ help=(
+ "API backend: 'anthropic' (direct), 'bedrock' (AWS), 'openrouter' (OpenRouter), "
+ "'anyllm' (any-llm), or 'litellm-<provider>' (e.g., litellm-vertex)"
+ ),
+)
+ "--anyllm-provider",
+ default="openai",
+ help="Provider for any-llm backend: openai, mistral, groq, ollama, etc. (default: openai)",
+)
+ "--anthropic-api-url",
+ default=None,
+ help="Custom Anthropic API URL for passthrough endpoints (env: ANTHROPIC_TARGET_API_URL)",
+)
+ "--openai-api-url",
+ default=None,
+ help="Custom OpenAI API URL for passthrough endpoints (env: OPENAI_TARGET_API_URL)",
+)
+ "--gemini-api-url",
+ default=None,
+ help="Custom Gemini API URL for passthrough endpoints (env: GEMINI_TARGET_API_URL)",
+)
+ "--cloudcode-api-url",
+ default=None,
+ help="Custom Cloud Code Assist API URL for compatibility endpoints (env: CLOUDCODE_TARGET_API_URL)",
+)
+ "--region",
+ default="us-west-2",
+ help="Cloud region for Bedrock/Vertex/etc (default: us-west-2)",
+)
+ "--bedrock-region",
+ default=None,
+ help="(deprecated, use --region) AWS region for Bedrock",
+)
+ "--bedrock-profile",
+ default=None,
+ help="AWS profile name for Bedrock (default: use default credentials)",
+)
+ "--no-telemetry",
+ is_flag=True,
+ help="Disable anonymous usage telemetry (env: HEADROOM_TELEMETRY=off)",
+)
+ "--stateless",
+ is_flag=True,
+ help="Disable all filesystem writes — run purely in-memory. "
+ "For containerized / read-only / load-balanced deployments. "
+ "(env: HEADROOM_STATELESS=true)",
+)
+def proxy(
+ ctx: click.Context,
+ mode: str | None,
+ host: str,
+ port: int,
+ intercept_tool_results: bool,
+ no_optimize: bool,
+ no_cache: bool,
+ no_rate_limit: bool,
+ retry_max_attempts: int | None,
+ connect_timeout_seconds: int | None,
+ anthropic_pre_upstream_concurrency: int | None,
+ anthropic_pre_upstream_acquire_timeout_seconds: float | None,
+ anthropic_pre_upstream_memory_context_timeout_seconds: float | None,
+ log_file: str | None,
+ log_messages: bool,
+ budget: float | None,
+ code_graph: bool,
+ no_read_lifecycle: bool,
+ no_intelligent_context: bool,
+ no_intelligent_scoring: bool,
+ no_compress_first: bool,
+ memory: bool,
+ memory_db_path: str,
+ no_memory_tools: bool,
+ no_memory_context: bool,
+ memory_top_k: int,
+ learn: bool,
+ no_learn: bool,
+ backend: str,
+ anyllm_provider: str,
+ anthropic_api_url: str | None,
+ openai_api_url: str | None,
+ gemini_api_url: str | None,
+ cloudcode_api_url: str | None,
+ region: str,
+ bedrock_region: str | None,
+ bedrock_profile: str | None,
+ no_telemetry: bool,
+ stateless: bool,
+) -> None:
+ """Start the optimization proxy server.
+
+ \b
+ Examples:
+ headroom proxy Start proxy on port 8787
+ headroom proxy --port 8080 Start proxy on port 8080
+ headroom proxy --no-optimize Passthrough mode (no optimization)
+
+ \b
+ Usage with Claude Code:
+ ANTHROPIC_BASE_URL=http://localhost:8787 claude
+
+ \b
+ Usage with OpenAI-compatible clients:
+ OPENAI_BASE_URL=http://localhost:8787/v1 your-app
+ """
+ # Import here to avoid slow startup
+ try:
+ from headroom.proxy.server import ProxyConfig, run_server
+ except ImportError as e:
+ click.echo("Error: Proxy dependencies not installed. Run: pip install headroom[proxy]")
+ click.echo(f"Details: {e}")
+ raise SystemExit(1) from None
+
+ # Opt-in: turn on tool_result interceptors (ast-grep Read outline, etc.).
+ # Only fetch the bundled CLI tool binaries when the feature is enabled —
+ # otherwise we'd pay a network round-trip and risk a readonly-FS failure
+ # for capabilities the user hasn't asked for. The TransformPipeline reads
+ # this env var at construction time.
+ if intercept_tool_results:
+ from headroom.binaries import ensure_tools
+
+ resolved_tools = ensure_tools()
+ critical_tools = ["ast-grep"]
+ missing = [t for t in critical_tools if not resolved_tools.get(t)]
+ if missing:
+ # User explicitly opted in — fail fast rather than silently starting
+ # with non-functional interceptors. They can retry with the tool
+ # installed, or drop the flag if they want pass-through behavior.
+ click.secho(
+ f"error: --intercept-tool-results requires tool(s) that could not "
+ f"be installed: {missing}. Run `headroom tools doctor` to diagnose, "
+ "or omit the flag to start the proxy without interceptors.",
+ fg="red",
+ err=True,
+ )
+ sys.exit(1)
+ os.environ["HEADROOM_INTERCEPT_ENABLED"] = "1"
+
+ provider_api_overrides = resolve_api_overrides(
+ anthropic_api_url=anthropic_api_url,
+ openai_api_url=openai_api_url,
+ gemini_api_url=gemini_api_url,
+ cloudcode_api_url=cloudcode_api_url,
+ environ=os.environ,
+ )
+
+ # Resolve anyllm provider: env var takes precedence over CLI default (matches argparse path)
+ effective_anyllm_provider = os.environ.get("HEADROOM_ANYLLM_PROVIDER") or anyllm_provider
+
+ # Resolve mode: CLI flag > env var > default
+ effective_mode: str = normalize_proxy_mode(
+ mode or os.environ.get("HEADROOM_MODE") or PROXY_MODE_TOKEN
+ )
+
+ # Stateless mode: CLI flag or env var
+ is_stateless = stateless or os.environ.get("HEADROOM_STATELESS", "").lower() in (
+ "true",
+ "1",
+ "yes",
+ "on",
+ )
+
+ # Telemetry opt-out: --no-telemetry flag sets the env var
+ if no_telemetry:
+ os.environ["HEADROOM_TELEMETRY"] = "off"
+
+ # Stateless mode: suppress TOIN filesystem persistence
+ if is_stateless:
+ os.environ["HEADROOM_TOIN_BACKEND"] = "none"
+
+ # License key for managed/enterprise deployments (optional)
+ license_key = os.environ.get("HEADROOM_LICENSE_KEY")
+
+ config = ProxyConfig(
+ host=host,
+ port=port,
+ anthropic_api_url=provider_api_overrides.anthropic,
+ openai_api_url=provider_api_overrides.openai,
+ gemini_api_url=provider_api_overrides.gemini,
+ cloudcode_api_url=provider_api_overrides.cloudcode,
+ mode=effective_mode,
+ optimize=not no_optimize,
+ cache_enabled=not no_cache,
+ rate_limit_enabled=not no_rate_limit,
+ retry_max_attempts=retry_max_attempts if retry_max_attempts is not None else 3,
+ connect_timeout_seconds=connect_timeout_seconds
+ if connect_timeout_seconds is not None
+ else 10,
+ log_file=None if is_stateless else log_file,
+ log_full_messages=log_messages
+ or os.environ.get("HEADROOM_LOG_MESSAGES", "").lower() in ("true", "1", "yes", "on"),
+ budget_limit_usd=budget,
+ # Code graph: live file watcher for incremental reindexing
+ code_graph_watcher=code_graph,
+ # Read lifecycle: ON by default (use --no-read-lifecycle to disable)
+ read_lifecycle=not no_read_lifecycle,
+ # Intelligent Context: ON by default (use --no-intelligent-context to disable)
+ intelligent_context=not no_intelligent_context,
+ intelligent_context_scoring=not no_intelligent_scoring,
+ intelligent_context_compress_first=not no_compress_first,
+ # Memory System (Multi-Provider with auto-detection)
+ # --learn implies --memory (need backend for storing patterns)
+ # Stateless mode disables memory (requires SQLite on disk)
+ memory_enabled=False if is_stateless else (memory or (learn and not no_learn)),
+ memory_db_path=memory_db_path,
+ memory_inject_tools=not no_memory_tools,
+ memory_inject_context=not no_memory_context,
+ memory_top_k=memory_top_k,
+ # Traffic Learning: only with --learn, never with --no-learn
+ # Stateless mode disables learning (requires filesystem)
+ traffic_learning_enabled=False if is_stateless else (learn and not no_learn),
+ traffic_learning_agent_type=os.environ.get("HEADROOM_AGENT_TYPE", "unknown"),
+ # Backend (Anthropic direct, Bedrock, LiteLLM, or any-llm)
+ backend=backend,
+ bedrock_region=bedrock_region or region,
+ bedrock_profile=bedrock_profile,
+ anyllm_provider=effective_anyllm_provider,
+ # License / Usage Reporting (managed/enterprise)
+ license_key=license_key,
+ # Stateless mode: disable all filesystem writes
+ stateless=is_stateless,
+ # Unit 4: bounded pre-upstream concurrency on the Anthropic HTTP
+ # path. ``None`` -> HeadroomProxy computes ``max(2, min(8,
+ # os.cpu_count() or 4))``; ``<= 0`` -> disabled (unbounded).
+ # Precedence: CLI > env > auto-compute (click's ``envvar``
+ # handles the env-var fallback).
+ anthropic_pre_upstream_concurrency=anthropic_pre_upstream_concurrency,
+ anthropic_pre_upstream_acquire_timeout_seconds=(
+ anthropic_pre_upstream_acquire_timeout_seconds
+ if anthropic_pre_upstream_acquire_timeout_seconds is not None
+ else 15.0
+ ),
+ anthropic_pre_upstream_memory_context_timeout_seconds=(
+ anthropic_pre_upstream_memory_context_timeout_seconds
+ if anthropic_pre_upstream_memory_context_timeout_seconds is not None
+ else 2.0
+ ),
+ )
+
+ memory_status = "DISABLED"
+ if config.memory_enabled:
+ memory_status = "ENABLED (multi-provider)"
+
+ license_status = "OSS (no license key)"
+ if license_key:
+ license_status = f"MANAGED (key={license_key[:8]}...)"
+
+ provider_api_targets = resolve_api_targets(config.provider_api_overrides)
+ anthropic_url = provider_api_targets.anthropic
+ openai_url = provider_api_targets.openai
+ cloudcode_url = provider_api_targets.cloudcode
+ backend_section = ""
+
+ if config.backend == "anyllm" or config.backend.startswith("anyllm-"):
+ # any-llm backend
+ backend_section = """
+ Set credentials for your provider (e.g., OPENAI_API_KEY, MISTRAL_API_KEY)
+ Providers: https://mozilla-ai.github.io/any-llm/providers/
+"""
+ elif config.backend != "anthropic":
+ # LiteLLM backend
+ from headroom.backends.litellm import get_provider_config
+
+ provider = config.backend.replace("litellm-", "")
+ provider_config = get_provider_config(provider)
+
+ # Build usage instructions from provider config
+ env_vars_str = (
+ ", ".join(provider_config.env_vars) if provider_config.env_vars else "See docs"
+ )
+ backend_section = f"""
+IMPORTANT for {provider_config.display_name} users:
+ 1. Set credentials: {env_vars_str}
+ 2. Set a dummy Anthropic key: ANTHROPIC_API_KEY="sk-ant-dummy"
+ (Headroom ignores this - it uses your {provider_config.display_name} credentials)
+ 3. Set base URL: ANTHROPIC_BASE_URL=http://{config.host}:{config.port}"""
+ if provider_config.model_format_hint:
+ backend_section += f"\n 4. Use model names: {provider_config.model_format_hint}"
+ backend_section += "\n"
+
+ # Build memory section if enabled
+ memory_section = ""
+ if config.memory_enabled:
+ memory_section = f"""
+Memory (Multi-Provider):
+ - Auto-detects provider from request (Anthropic, OpenAI, Gemini, etc.)
+ - Anthropic: Uses native memory tool (memory_20250818) - subscription safe
+ - OpenAI/Gemini/Others: Uses function calling format
+ - All providers share the same semantic vector store backend
+ - Set x-headroom-user-id header for per-user memory (defaults to 'default')
+ - Tools: {"ENABLED" if config.memory_inject_tools else "DISABLED"}
+ - Context injection: {"ENABLED" if config.memory_inject_context else "DISABLED"}
+ - Database: {config.memory_db_path}
+"""
+
+ # Stateless mode warning
+ stateless_line = ""
+ if is_stateless:
+ stateless_line = (
+ " Stateless: YES (no filesystem writes — memory, logs, TOIN disabled)\n"
+ )
+
+ from headroom.telemetry.beacon import is_telemetry_enabled
+
+ # Build telemetry section for the startup banner
+ if is_telemetry_enabled():
+ telemetry_line = (
+ " Telemetry: ENABLED (anonymous aggregate stats)\n"
+ " Disable: HEADROOM_TELEMETRY=off or headroom proxy --no-telemetry"
+ )
+ else:
+ telemetry_line = " Telemetry: DISABLED"
+
+ click.echo(f"""
+╔═══════════════════════════════════════════════════════════════════════╗
+║ HEADROOM PROXY ║
+║ The Context Optimization Layer for LLM Applications ║
+╚═══════════════════════════════════════════════════════════════════════╝
+
+Starting proxy server...
+
+ URL: http://{config.host}:{config.port}
+ Mode: {config.mode}
+ Optimization: {"ENABLED" if config.optimize else "DISABLED"}
+ Caching: {"ENABLED" if config.cache_enabled else "DISABLED"}
+ Rate Limit: {"ENABLED" if config.rate_limit_enabled else "DISABLED"}
+ Memory: {memory_status}
+ License: {license_status}
+{stateless_line}{telemetry_line}
+{backend_section}
+Routing:
+ /v1/messages → {anthropic_url}
+ /v1/chat/completions → {openai_url}
+ /v1/responses → {openai_url} (HTTP + WebSocket)
+ /v1internal:streamGenerateContent → {cloudcode_url}
+
+Usage:
+ Claude Code: ANTHROPIC_BASE_URL=http://{config.host}:{config.port} claude
+ Codex / OpenAI: OPENAI_BASE_URL=http://{config.host}:{config.port}/v1 your-app
+{memory_section}
+Endpoints:
+ GET /livez Process liveness
+ GET /readyz Traffic readiness
+ GET /health Aggregate health
+ GET /stats Detailed statistics
+ GET /stats-history Durable compression history + display session
+ GET /metrics Prometheus metrics
+
+Press Ctrl+C to stop.
+""")
+
+ try:
+ run_server(config)
+ except KeyboardInterrupt:
+ click.echo("\nShutting down...")
diff --git a/headroom/cli/wrap.py b/headroom/cli/wrap.py
index 6fbec6551..ae1731537 100644
--- a/headroom/cli/wrap.py
+++ b/headroom/cli/wrap.py
@@ -1,2211 +1,2069 @@
-"""Wrap CLI commands to run through Headroom proxy.
-
-Usage:
- headroom wrap claude # Start proxy + rtk + claude
- headroom wrap copilot -- --model ... # Start proxy + launch GitHub Copilot CLI
- headroom wrap codex # Start proxy + OpenAI Codex CLI
- headroom wrap aider # Start proxy + aider
- headroom wrap cursor # Start proxy + print Cursor config instructions
- headroom wrap openclaw # Install + configure OpenClaw plugin
- headroom wrap claude --no-rtk # Without rtk hooks
- headroom wrap claude --port 9999 # Custom proxy port
- headroom wrap claude -- --model opus # Pass args to claude
-"""
-
-from __future__ import annotations
-
-import io
-import json
-import os
-import shutil
-import signal
-import socket
-import subprocess
-import sys
-import time
-import urllib.error
-import urllib.request
-from pathlib import Path
-from typing import Any
-
-# Fix Windows cp1252 encoding — box-drawing characters require UTF-8
-if sys.platform == "win32" and hasattr(sys.stdout, "buffer"):
- if sys.stdout.encoding and sys.stdout.encoding.lower().replace("-", "") != "utf8":
- sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8", errors="replace")
- sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding="utf-8", errors="replace")
-
-import click
-
-from headroom.copilot_auth import DEFAULT_API_URL as COPILOT_API_URL
-from headroom.copilot_auth import has_oauth_auth, resolve_client_bearer_token
-
-from .main import main
-
-
-def _print_telemetry_notice() -> None:
- """Print a telemetry notice when anonymous telemetry is enabled.
-
- Respects the HEADROOM_TELEMETRY and HEADROOM_TELEMETRY_WARN feature flags.
- Does nothing when telemetry or warnings are disabled.
- """
- from headroom.telemetry.beacon import format_telemetry_notice
-
- notice = format_telemetry_notice(prefix=" ")
- if notice:
- click.echo(notice)
-
-
-# Proxy health check (reused from evals/suite_runner.py pattern)
-
-
-def _check_proxy(port: int) -> bool:
- """Check if Headroom proxy is running on given port."""
- try:
- with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
- s.settimeout(1)
- s.connect(("127.0.0.1", port))
- return True
- except (TimeoutError, ConnectionRefusedError, OSError):
- return False
-
-
-def _get_log_path() -> Path:
- """Get path for proxy log file."""
- from headroom import paths as _paths
-
- log_dir = _paths.log_dir()
- log_dir.mkdir(parents=True, exist_ok=True)
- return log_dir / "proxy.log"
-
-
-def _start_proxy(
- port: int,
- *,
- learn: bool = False,
- memory: bool = False,
- agent_type: str = "unknown",
- code_graph: bool = False,
- backend: str | None = None,
- anyllm_provider: str | None = None,
- region: str | None = None,
- openai_api_url: str | None = None,
-) -> subprocess.Popen:
- """Start Headroom proxy as a background subprocess.
-
- Logs are written to ~/.headroom/logs/proxy.log to avoid pipe buffer
- deadlocks (macOS pipe buffer is ~64KB — a busy proxy fills it quickly,
- blocking the process).
- """
- cmd = [sys.executable, "-m", "headroom.cli", "proxy", "--port", str(port)]
-
- # Forward HEADROOM_MODE env var so the proxy respects the user's mode choice
- headroom_mode = os.environ.get("HEADROOM_MODE")
- if headroom_mode:
- cmd.extend(["--mode", headroom_mode])
-
- # Forward --learn flag to proxy subprocess
- if learn:
- cmd.append("--learn")
-
- # Forward --memory flag to proxy subprocess
- if memory:
- cmd.append("--memory")
-
- # Forward --code-graph flag to proxy subprocess (live file watcher)
- if code_graph:
- cmd.append("--code-graph")
-
- # Forward backend configuration to proxy subprocess
- _backend = backend or os.environ.get("HEADROOM_BACKEND")
- if _backend:
- cmd.extend(["--backend", _backend])
-
- _anyllm = anyllm_provider or os.environ.get("HEADROOM_ANYLLM_PROVIDER")
- if _anyllm:
- cmd.extend(["--anyllm-provider", _anyllm])
-
- _region = region or os.environ.get("HEADROOM_REGION")
- if _region:
- cmd.extend(["--region", _region])
-
- if openai_api_url:
- cmd.extend(["--openai-api-url", openai_api_url])
-
- log_path = _get_log_path()
- log_file = open(log_path, "a") # noqa: SIM115
-
- # Ensure proxy subprocess uses UTF-8 (Windows defaults to cp1252)
- proxy_env = os.environ.copy()
- proxy_env["PYTHONIOENCODING"] = "utf-8"
-
- # Tell the proxy which agent is being wrapped (for traffic learning output)
- if agent_type != "unknown":
- proxy_env["HEADROOM_AGENT_TYPE"] = agent_type
- proxy_env.setdefault("HEADROOM_STACK", f"wrap_{agent_type}")
-
- proc = subprocess.Popen(
- cmd,
- stdout=log_file,
- stderr=log_file,
- env=proxy_env,
- )
-
- # Wait for proxy to be ready (up to 45 seconds).
- # ML components (Kompress, Magika, Tree-sitter) load synchronously before
- # uvicorn binds the port. On slower machines this can take 20-30 seconds.
- for _i in range(45):
- time.sleep(1)
- if _check_proxy(port):
- click.echo(f" Logs: {log_path}")
- return proc
- # Check if process died
- if proc.poll() is not None:
- log_file.close()
- # Read last few lines of log for error context
- try:
- tail = log_path.read_text()[-500:]
- except Exception:
- tail = "(no log output)"
- raise RuntimeError(f"Proxy exited with code {proc.returncode}: {tail}")
-
- proc.kill()
- log_file.close()
- raise RuntimeError(f"Proxy failed to start on port {port} within 45 seconds")
-
-
-def _setup_rtk(verbose: bool = False) -> Path | None:
- """Ensure rtk is installed and hooks are registered."""
- from headroom.rtk import get_rtk_path
- from headroom.rtk.installer import ensure_rtk, register_claude_hooks
-
- rtk_path = get_rtk_path()
-
- if rtk_path:
- if verbose:
- click.echo(f" rtk found at {rtk_path}")
- else:
- click.echo(" Downloading rtk (Rust Token Killer)...")
- rtk_path = ensure_rtk()
- if rtk_path:
- click.echo(f" rtk installed at {rtk_path}")
- else:
- click.echo(" rtk download failed — continuing without it")
- return None
-
- # Register hooks (idempotent)
- if register_claude_hooks(rtk_path):
- if verbose:
- click.echo(" rtk hooks registered in Claude Code")
- else:
- click.echo(" rtk hook registration failed — continuing without it")
-
- return rtk_path
-
-
-_CBM_MCP_SERVER_NAME = "codebase-memory-mcp"
-
-
-def _register_cbm_mcp_server(cbm_bin: str) -> None:
- """Register codebase-memory-mcp as an MCP server in Claude Code.
-
- Uses ``claude mcp add`` so the tools appear in ``/mcp`` automatically.
- Idempotent — skips if already registered.
- """
- claude_cli = shutil.which("claude")
- if not claude_cli:
- return
-
- # Check if already registered
- check = subprocess.run(
- [claude_cli, "mcp", "get", _CBM_MCP_SERVER_NAME],
- capture_output=True,
- text=True,
- )
- if check.returncode == 0:
- return # Already registered
-
- result = subprocess.run(
- [claude_cli, "mcp", "add", _CBM_MCP_SERVER_NAME, "-s", "user", "--", cbm_bin],
- capture_output=True,
- text=True,
- )
- if result.returncode == 0:
- click.echo(f" Code graph: registered {_CBM_MCP_SERVER_NAME} MCP server")
- else:
- pass # Non-critical — tools won't appear in /mcp but graph still works
-
-
-def _setup_code_graph(verbose: bool = False) -> bool:
- """Ensure codebase-memory-mcp is installed, registered as MCP server, and project is indexed.
-
- codebase-memory-mcp builds a knowledge graph of the codebase using
- tree-sitter, enabling the LLM to query code structure (call chains,
- function definitions, impact analysis) instead of reading entire files.
-
- Steps:
- 1. Download the binary if not already present.
- 2. Register as an MCP server in Claude Code (``claude mcp add``).
- 3. Index the current project (fast, idempotent).
-
- With Claude Code's MCP Tool Search, the 14 graph tools add ~200 tokens
- overhead per request (not the full ~1,915) — they're lazy-loaded.
-
- Returns True if graph is ready, False if setup failed.
- """
- from headroom.graph.installer import ensure_cbm, get_cbm_path
-
- cbm_path = get_cbm_path()
- if not cbm_path:
- click.echo(" Code graph: downloading codebase-memory-mcp...")
- cbm_path = ensure_cbm()
- if cbm_path:
- click.echo(f" Code graph: installed at {cbm_path}")
- else:
- click.echo(" Code graph: download failed — skipping")
- return False
-
- cbm_bin = str(cbm_path)
-
- # Register as MCP server so tools appear in /mcp
- _register_cbm_mcp_server(cbm_bin)
-
- # Index current project (fast — ~1s for most repos, idempotent)
- project_dir = str(Path.cwd())
- try:
- result = subprocess.run(
- [
- cbm_bin,
- "cli",
- "index_repository",
- json.dumps({"repo_path": project_dir, "mode": "fast"}),
- ],
- capture_output=True,
- text=True,
- timeout=30,
- )
- if result.returncode == 0:
- # Parse node/edge counts from output
- for line in result.stdout.splitlines():
- if '"nodes"' in line:
- try:
- # Parse the JSON response to extract node/edge counts
- import re
-
- m_nodes = re.search(r'"nodes":(\d+)', line)
- m_edges = re.search(r'"edges":(\d+)', line)
- if m_nodes and m_edges:
- nodes = int(m_nodes.group(1))
- edges = int(m_edges.group(1))
- click.echo(
- f" Code graph: indexed ({nodes:,} symbols, "
- f"{edges:,} relationships)"
- )
- else:
- click.echo(" Code graph: indexed")
- except (ValueError, AttributeError):
- click.echo(" Code graph: indexed")
- return True
- click.echo(" Code graph: indexed")
- return True
- else:
- if verbose:
- click.echo(f" Code graph: indexing failed ({result.stderr[:100]})")
- return False
- except subprocess.TimeoutExpired:
- click.echo(" Code graph: indexing timed out (will complete in background)")
- return False
- except Exception as e:
- if verbose:
- click.echo(f" Code graph: setup failed ({e})")
- return False
-
-
-# rtk instructions for tools without hook support (Codex, Cursor, Aider).
-# These get injected into AGENTS.md / .cursorrules so the LLM voluntarily
-# uses rtk-prefixed commands. Kept concise to minimize instruction overhead.
-RTK_INSTRUCTIONS_BLOCK = """\
-<!-- headroom:rtk-instructions -->
-# RTK (Rust Token Killer) - Token-Optimized Commands
-
-When running shell commands, **always prefix with `rtk`**. This reduces context
-usage by 60-90% with zero behavior change. If rtk has no filter for a command,
-it passes through unchanged — so it is always safe to use.
-
-## Key Commands
-```bash
-# Git (59-80% savings)
-rtk git status rtk git diff rtk git log
-
-# Files & Search (60-75% savings)
-rtk ls <path> rtk read <file> rtk grep <pattern>
-rtk find <pattern> rtk diff <file>
-
-# Test (90-99% savings) — shows failures only
-rtk pytest tests/ rtk cargo test rtk test <cmd>
-
-# Build & Lint (80-90% savings) — shows errors only
-rtk tsc rtk lint rtk cargo build
-rtk prettier --check rtk mypy rtk ruff check
-
-# Analysis (70-90% savings)
-rtk err <cmd> rtk log <file> rtk json <file>
-rtk summary <cmd> rtk deps rtk env
-
-# GitHub (26-87% savings)
-rtk gh pr view <n> rtk gh run list rtk gh issue list
-
-# Infrastructure (85% savings)
-rtk docker ps rtk kubectl get rtk docker logs <c>
-
-# Package managers (70-90% savings)
-rtk pip list rtk pnpm install rtk npm run <script>
-```
-
-## Rules
-- In command chains, prefix each segment: `rtk git add . && rtk git commit -m "msg"`
-- For debugging, use raw command without rtk prefix
-- `rtk proxy <cmd>` runs command without filtering but tracks usage
-<!-- /headroom:rtk-instructions -->
-"""
-
-# Marker used to detect if instructions are already injected
-_RTK_MARKER = "<!-- headroom:rtk-instructions -->"
-
-# Memory MCP markers
-_MEMORY_MCP_MARKER = "# --- Headroom memory MCP (auto-injected) ---"
-_MEMORY_MCP_END = "# --- end Headroom memory ---"
-_MEMORY_AGENTS_MARKER = "<!-- headroom:memory-instructions -->"
-
-
-def _ensure_rtk_binary(verbose: bool = False) -> Path | None:
- """Ensure rtk binary is installed (download if needed). No hook registration."""
- from headroom.rtk import get_rtk_path
- from headroom.rtk.installer import ensure_rtk
-
- rtk_path = get_rtk_path()
-
- if rtk_path:
- if verbose:
- click.echo(f" rtk found at {rtk_path}")
- return rtk_path
-
- click.echo(" Downloading rtk (Rust Token Killer)...")
- rtk_path = ensure_rtk()
- if rtk_path:
- click.echo(f" rtk installed at {rtk_path}")
- return rtk_path
-
- click.echo(" rtk download failed — continuing without it")
- return None
-
-
-def _prepare_wrap_rtk(verbose: bool = False, *, label: str | None = None) -> Path | None:
- """Ensure rtk is present for host-bridged wrap flows without host-specific setup."""
- if label:
- click.echo(f" Preparing rtk for {label}...")
- return _ensure_rtk_binary(verbose=verbose)
-
-
-def _inject_codex_provider_config(port: int) -> None:
- """Inject a Headroom model provider into Codex's config.toml.
-
- Codex ignores OPENAI_BASE_URL for WebSocket transport unless a custom
- provider declares ``supports_websockets = true``. This writes a
- ``[model_providers.headroom]`` section that routes both HTTP and WS
- through the proxy, and sets ``model_provider = "headroom"``.
-
- Safe to call multiple times — only writes if the section is missing
- or the port changed.
- """
- config_dir = Path.home() / ".codex"
- config_file = config_dir / "config.toml"
-
- # model_provider must be a top-level TOML key (before any [section]).
- # The [model_providers.headroom] table can go at the end.
- top_level_marker = "# --- Headroom proxy (auto-injected by headroom wrap codex) ---"
- top_level_block = f'{top_level_marker}\nmodel_provider = "headroom"\n'
- provider_section = (
- f"\n[model_providers.headroom]\n"
- f'name = "OpenAI via Headroom proxy"\n'
- f'base_url = "http://127.0.0.1:{port}/v1"\n'
- f'env_key = "OPENAI_API_KEY"\n'
- f"requires_openai_auth = true\n"
- f"supports_websockets = true\n"
- f"# --- end Headroom ---\n"
- )
-
- marker = top_level_marker
- end_marker = "# --- end Headroom ---"
-
- try:
- config_dir.mkdir(parents=True, exist_ok=True)
-
- if config_file.exists():
- content = config_file.read_text()
- if marker in content:
- # Remove existing Headroom blocks entirely
- start = content.index(marker)
- end = content.index(end_marker) + len(end_marker)
- content = content[:start].rstrip("\n") + content[end:].lstrip("\n")
-
- # Strip any stale top-level model_provider left behind
- import re
-
- content = re.sub(r'\nmodel_provider\s*=\s*"headroom"\n', "\n", content)
-
- # Place top-level key at the very beginning, provider table at the end
- content = top_level_block + "\n" + content.strip() + "\n" + provider_section
- else:
- content = top_level_block + "\n" + provider_section
-
- config_file.write_text(content)
- click.echo(f" Codex config: injected Headroom provider (WS + HTTP) into {config_file}")
- except Exception as e:
- click.echo(f" Warning: could not update Codex config: {e}")
-
-
-def _inject_rtk_instructions(file_path: Path, verbose: bool = False) -> bool:
- """Inject rtk instructions into a file (AGENTS.md, .cursorrules, etc.).
-
- Idempotent — skips if marker already present. Appends to existing content.
- Returns True if instructions were written.
- """
- if file_path.exists():
- existing = file_path.read_text()
- if _RTK_MARKER in existing:
- if verbose:
- click.echo(f" rtk instructions already in {file_path.name}")
- return True
- # Append to existing file
- with open(file_path, "a") as f:
- f.write("\n\n" + RTK_INSTRUCTIONS_BLOCK)
- else:
- file_path.parent.mkdir(parents=True, exist_ok=True)
- file_path.write_text(RTK_INSTRUCTIONS_BLOCK)
-
- click.echo(f" rtk instructions injected into {file_path}")
- return True
-
-
-def _inject_memory_mcp_config(db_path: str, user_id: str) -> None:
- """Register headroom memory as an MCP server in Codex's config.toml.
-
- Idempotent — replaces existing section if present.
- """
- import sys
-
- config_dir = Path.home() / ".codex"
- config_file = config_dir / "config.toml"
-
- # Use forward slashes in TOML paths (works on all platforms, avoids
- # backslash escaping issues on Windows)
- python_bin = sys.executable.replace("\\", "/")
- db_path_toml = db_path.replace("\\", "/")
- mcp_section = (
- f"\n{_MEMORY_MCP_MARKER}\n"
- f"[mcp_servers.headroom_memory]\n"
- f'command = "{python_bin}"\n'
- f'args = ["-m", "headroom.memory.mcp_server", "--db", "{db_path_toml}", "--user", "{user_id}"]\n'
- f"startup_timeout_sec = 30\n"
- f"tool_timeout_sec = 30\n"
- f"{_MEMORY_MCP_END}\n"
- )
-
- try:
- config_dir.mkdir(parents=True, exist_ok=True)
-
- if config_file.exists():
- content = config_file.read_text()
- if _MEMORY_MCP_MARKER in content:
- start = content.index(_MEMORY_MCP_MARKER)
- end = content.index(_MEMORY_MCP_END) + len(_MEMORY_MCP_END)
- content = content[:start].rstrip("\n") + mcp_section + content[end:].lstrip("\n")
- else:
- content = content.rstrip() + "\n" + mcp_section
- else:
- content = mcp_section
-
- config_file.write_text(content)
- click.echo(f" Memory MCP: registered in {config_file}")
- except Exception as e:
- click.echo(f" Warning: could not register memory MCP: {e}")
-
-
-def _inject_memory_agents_md(file_path: Path) -> bool:
- """Inject memory usage guidance into AGENTS.md.
-
- Idempotent — skips if marker already present.
- """
- memory_block = (
- f"{_MEMORY_AGENTS_MARKER}\n"
- "## Memory\n\n"
- "Use the `headroom_memory` MCP server for persistent cross-session knowledge.\n\n"
- "**Before** answering questions about prior decisions, conventions, project context,\n"
- "architecture, user preferences, org info, codenames, debugging history, or anything\n"
- "from past sessions — call `memory_search` first.\n\n"
- "**After** making durable decisions, discovering conventions, or learning important\n"
- "facts — call `memory_save` to persist them for future sessions.\n\n"
- "Memory is your first source of truth for anything not visible in the current conversation.\n"
- )
-
- if file_path.exists():
- existing = file_path.read_text()
- if _MEMORY_AGENTS_MARKER in existing:
- return True # Already injected
- with open(file_path, "a") as f:
- f.write("\n\n" + memory_block)
- else:
- file_path.parent.mkdir(parents=True, exist_ok=True)
- file_path.write_text(memory_block)
-
- click.echo(f" Memory guidance injected into {file_path.name}")
- return True
-
-
-def _resolve_copilot_provider_type(backend: str | None, provider_type: str) -> str:
- """Resolve Copilot BYOK provider type for the current proxy backend."""
- if provider_type != "auto":
- return provider_type
-
- effective_backend = backend or os.environ.get("HEADROOM_BACKEND") or "anthropic"
- return "anthropic" if effective_backend == "anthropic" else "openai"
-
-
-def _query_proxy_config(port: int) -> dict[str, Any] | None:
- """Query the running proxy's feature configuration via /health.
-
- Returns a dict with keys like backend, optimize, cache, rate_limit,
- memory, learn, code_graph, pid. Returns None if unreachable or the
- response lacks a config block.
- """
- url = f"http://127.0.0.1:{port}/health"
- try:
- with urllib.request.urlopen(url, timeout=2) as response:
- payload = json.loads(response.read().decode("utf-8"))
- except (OSError, urllib.error.URLError, ValueError, json.JSONDecodeError):
- return None
-
- config = payload.get("config")
- if not isinstance(config, dict):
- return None
- return config
-
-
-def _detect_running_proxy_backend(port: int) -> str | None:
- """Read the backend of an already-running proxy from its health endpoint."""
- config = _query_proxy_config(port)
- if config is None:
- return None
- backend = config.get("backend")
- return backend if isinstance(backend, str) else None
-
-
-def _kill_proxy_by_pid(pid: int, port: int) -> bool:
- """Terminate a proxy process by PID and wait for the port to free up.
-
- Sends SIGTERM first, falls back to SIGKILL after 5 seconds.
- Returns True if the port is free afterwards, False otherwise.
- """
- try:
- os.kill(pid, signal.SIGTERM)
- except ProcessLookupError:
- pass # Already gone
- except PermissionError:
- click.echo(f" Warning: No permission to kill proxy PID {pid}")
- return False
-
- # Wait for port to free (up to 5 seconds)
- for _ in range(50):
- time.sleep(0.1)
- if not _check_proxy(port):
- return True
-
- # SIGTERM didn't work — escalate to SIGKILL (Unix) or terminate (Windows)
- try:
- _kill_signal = getattr(signal, "SIGKILL", signal.SIGTERM)
- os.kill(pid, _kill_signal)
- except (ProcessLookupError, PermissionError):
- pass
-
- for _ in range(20):
- time.sleep(0.1)
- if not _check_proxy(port):
- return True
-
- return False
-
-
-def _find_persistent_manifest(port: int) -> Any:
- """Return a matching persistent deployment manifest for the requested port."""
- from headroom.install.state import list_manifests
-
- manifests = [manifest for manifest in list_manifests() if manifest.port == port]
- manifests.sort(key=lambda manifest: (manifest.profile != "default", manifest.profile))
- return manifests[0] if manifests else None
-
-
-def _recover_persistent_proxy(port: int) -> bool:
- """Start or recover a matching persistent deployment for the requested port."""
- from headroom.install.health import probe_ready
- from headroom.install.models import InstallPreset, SupervisorKind
- from headroom.install.runtime import start_detached_agent, start_persistent_docker, wait_ready
- from headroom.install.supervisors import start_supervisor
-
- manifest = _find_persistent_manifest(port)
- if manifest is None:
- return False
-
- if probe_ready(manifest.health_url):
- click.echo(f" Reusing persistent deployment '{manifest.profile}' on port {port}")
- return True
-
- if manifest.supervisor_kind == SupervisorKind.TASK.value:
- click.echo(
- f" Warning: task-based deployment '{manifest.profile}' cannot be auto-recovered via wrap"
- )
- return False
-
- click.echo(f" Recovering persistent deployment '{manifest.profile}' on port {port}...")
- try:
- if manifest.preset == InstallPreset.PERSISTENT_DOCKER.value:
- start_persistent_docker(manifest)
- elif manifest.supervisor_kind == SupervisorKind.SERVICE.value:
- start_supervisor(manifest)
- else:
- start_detached_agent(manifest.profile)
- except Exception as exc:
- click.echo(
- f" Warning: could not recover persistent deployment '{manifest.profile}': {exc}"
- )
- return False
-
- if wait_ready(manifest, timeout_seconds=45):
- click.echo(f" Recovered persistent deployment '{manifest.profile}' on port {port}")
- return True
-
- click.echo(f" Warning: persistent deployment '{manifest.profile}' did not become ready")
- return False
-
-
-def _copilot_model_configured(copilot_args: tuple[str, ...], env: dict[str, str]) -> bool:
- """Return True when Copilot BYOK model selection is configured."""
- if env.get("COPILOT_MODEL") or env.get("COPILOT_PROVIDER_MODEL_ID"):
- return True
-
- for idx, arg in enumerate(copilot_args):
- if arg == "--model" and idx + 1 < len(copilot_args):
- return True
- if arg.startswith("--model="):
- return True
-
- return False
-
-
-def _should_use_copilot_oauth(
- *,
- backend: str | None,
- provider_type: str,
- env: dict[str, str],
-) -> bool:
- """Prefer existing Copilot auth when the requested routing supports it."""
-
- if env.get("COPILOT_PROVIDER_API_KEY") or env.get("COPILOT_PROVIDER_BEARER_TOKEN"):
- return False
- if provider_type == "anthropic":
- return False
-
- effective_backend = backend or os.environ.get("HEADROOM_BACKEND")
- if effective_backend not in (None, "", "anthropic"):
- return False
-
- return has_oauth_auth()
-
-
-def _ensure_proxy(
- port: int,
- no_proxy: bool,
- *,
- learn: bool = False,
- memory: bool = False,
- agent_type: str = "unknown",
- code_graph: bool = False,
- backend: str | None = None,
- anyllm_provider: str | None = None,
- region: str | None = None,
- openai_api_url: str | None = None,
-) -> subprocess.Popen | None:
- """Start or verify proxy. Returns process handle if we started it."""
- if not no_proxy:
- manifest = _find_persistent_manifest(port)
- if manifest is not None:
- from headroom.install.health import probe_ready
-
- if probe_ready(manifest.health_url):
- click.echo(f" Proxy already running on port {port}")
- return None
- if _recover_persistent_proxy(port):
- return None
- raise click.ClickException(
- f"Persistent deployment '{manifest.profile}' on port {port} is not healthy."
- )
-
- if _check_proxy(port):
- # Proxy is running — check if it has the features we need
- needs_restart = False
- running_config = _query_proxy_config(port)
-
- if running_config is not None:
- missing = []
- if memory and not running_config.get("memory"):
- missing.append("memory")
- if learn and not running_config.get("learn"):
- missing.append("learn")
- if code_graph and not running_config.get("code_graph"):
- missing.append("code_graph")
-
- if missing:
- needs_restart = True
- flags_str = ", ".join(f"--{f.replace('_', '-')}" for f in missing)
- click.echo(f" Proxy on port {port} is missing: {flags_str}")
- click.echo(" Restarting proxy with upgraded configuration...")
-
- # Merge: keep features the running proxy already has
- memory = memory or bool(running_config.get("memory"))
- learn = learn or bool(running_config.get("learn"))
- code_graph = code_graph or bool(running_config.get("code_graph"))
-
- proxy_pid = running_config.get("pid")
- if proxy_pid is not None:
- if not _kill_proxy_by_pid(int(proxy_pid), port):
- raise click.ClickException(
- f"Failed to stop existing proxy (PID {proxy_pid}) on port {port}. "
- "Stop it manually and retry."
- )
- else:
- click.echo(
- " Warning: Running proxy does not expose PID. "
- "Cannot restart automatically."
- )
- click.echo(
- f" Please stop the proxy on port {port} manually "
- f"and rerun with {flags_str}."
- )
- return None
-
- if not needs_restart:
- click.echo(f" Proxy already running on port {port}")
- return None
-
- # Start (or restart) the proxy with the requested flags
- click.echo(f" Starting Headroom proxy on port {port}...")
- try:
- proc = _start_proxy(
- port,
- learn=learn,
- memory=memory,
- agent_type=agent_type,
- code_graph=code_graph,
- backend=backend,
- anyllm_provider=anyllm_provider,
- region=region,
- openai_api_url=openai_api_url,
- )
- click.echo(f" Proxy ready on http://127.0.0.1:{port}")
- return proc
- except RuntimeError as e:
- click.echo(f" Error: {e}")
- raise SystemExit(1) from e
- else:
- if not _check_proxy(port):
- click.echo(f" Warning: No proxy detected on port {port}")
- return None
-
-
-def _make_cleanup(proxy_proc_holder: list, port: int = 8787) -> Any:
- """Create a cleanup function that terminates the proxy on exit.
-
- Only kills the proxy if no other headroom-wrapped clients are using it.
- Checks by looking for other processes with ANTHROPIC_BASE_URL or
- OPENAI_BASE_URL pointing at our port.
- """
-
- def _other_clients_exist() -> bool:
- """Check if other processes are using this proxy."""
- try:
- # Count headroom wrap processes (excluding ourselves)
- result = subprocess.run(
- ["pgrep", "-f", f"127.0.0.1:{port}"],
- capture_output=True,
- text=True,
- )
- pids = [p.strip() for p in result.stdout.strip().split("\n") if p.strip()]
- my_pid = str(os.getpid())
- other_pids = [p for p in pids if p != my_pid]
- return len(other_pids) > 0
- except Exception:
- return False # If we can't check, assume no others
-
- def cleanup(signum: int | None = None, frame: Any = None) -> None:
- proc = proxy_proc_holder[0] if proxy_proc_holder else None
- if proc and proc.poll() is None:
- if _other_clients_exist():
- # Other clients still using the proxy — leave it running
- return
- proc.terminate()
- try:
- proc.wait(timeout=5)
- except subprocess.TimeoutExpired:
- proc.kill()
-
- return cleanup
-
-
-def _launch_tool(
- binary: str,
- args: tuple,
- env: dict[str, str],
- port: int,
- no_proxy: bool,
- tool_label: str,
- env_vars_display: list[str],
- *,
- learn: bool = False,
- memory: bool = False,
- agent_type: str = "unknown",
- code_graph: bool = False,
- backend: str | None = None,
- anyllm_provider: str | None = None,
- region: str | None = None,
- openai_api_url: str | None = None,
-) -> None:
- """Common logic: start proxy, launch tool, clean up."""
- proxy_holder: list[subprocess.Popen | None] = [None]
- cleanup = _make_cleanup(proxy_holder, port)
- signal.signal(signal.SIGINT, cleanup)
- signal.signal(signal.SIGTERM, cleanup)
-
- try:
- click.echo()
- padded = f"HEADROOM WRAP: {tool_label}".center(47)
- click.echo(" ╔═══════════════════════════════════════════════╗")
- click.echo(f" ║{padded}║")
- click.echo(" ╚═══════════════════════════════════════════════╝")
- click.echo()
-
- proxy_holder[0] = _ensure_proxy(
- port,
- no_proxy,
- learn=learn,
- memory=memory,
- agent_type=agent_type,
- code_graph=code_graph,
- backend=backend,
- anyllm_provider=anyllm_provider,
- region=region,
- openai_api_url=openai_api_url,
- )
-
- if code_graph:
- _setup_code_graph(verbose=False)
-
- click.echo()
- click.echo(f" Launching {tool_label} (API routed through Headroom)...")
- for var in env_vars_display:
- click.echo(f" {var}")
- if args:
- click.echo(f" Extra args: {' '.join(args)}")
- _print_telemetry_notice()
- click.echo()
-
- result = subprocess.run([binary, *args], env=env)
- raise SystemExit(result.returncode)
-
- except SystemExit:
- raise
- except Exception as e:
- click.echo(f" Error: {e}")
- raise SystemExit(1) from e
- finally:
- cleanup()
-
-
-def _run_checked(
- cmd: list[str],
- *,
- cwd: Path | None = None,
- action: str,
-) -> subprocess.CompletedProcess[str]:
- """Run subprocess and raise a ClickException with actionable context on failure."""
- try:
- return subprocess.run(
- cmd,
- cwd=str(cwd) if cwd else None,
- check=True,
- capture_output=True,
- text=True,
- encoding="utf-8",
- errors="replace",
- )
- except FileNotFoundError as e:
- raise click.ClickException(f"{action} failed: command not found: {cmd[0]}") from e
- except subprocess.CalledProcessError as e:
- stderr = (e.stderr or "").strip()
- stdout = (e.stdout or "").strip()
- details = stderr or stdout or f"exit code {e.returncode}"
- raise click.ClickException(f"{action} failed: {details}") from e
-
-
-def _resolve_openclaw_extensions_dir(openclaw_bin: str) -> Path:
- """Resolve OpenClaw extension root from active config file path."""
- result = _run_checked([openclaw_bin, "config", "file"], action="openclaw config file")
- lines = result.stdout.strip().splitlines()
- config_path_str = lines[-1].strip() if lines else ""
- if not config_path_str:
- raise click.ClickException(
- "Unable to resolve OpenClaw config path from `openclaw config file`."
- )
- config_path = Path(config_path_str).expanduser()
- return config_path.parent / "extensions"
-
-
-def _normalize_openclaw_gateway_provider_ids(provider_ids: tuple[str, ...] | None) -> list[str]:
- """Normalize configured OpenClaw provider ids, defaulting to openai-codex."""
- values = provider_ids or ()
- seen: set[str] = set()
- normalized: list[str] = []
-
- for entry in values:
- provider_id = entry.strip()
- if not provider_id or provider_id in seen:
- continue
- seen.add(provider_id)
- normalized.append(provider_id)
-
- if normalized:
- return normalized
- return ["openai-codex"]
-
-
-def _read_openclaw_config_value(openclaw_bin: str, path: str) -> Any | None:
- """Read an OpenClaw config value when present, returning None on missing paths."""
- result = subprocess.run(
- [openclaw_bin, "config", "get", path],
- capture_output=True,
- text=True,
- encoding="utf-8",
- errors="replace",
- )
- if result.returncode != 0:
- return None
-
- output = result.stdout.strip()
- if not output:
- return None
-
- try:
- return json.loads(output)
- except json.JSONDecodeError:
- return output
-
-
-def _decode_openclaw_entry_json(raw_value: str | None) -> Any | None:
- """Decode a JSON payload captured from `openclaw config get` when available."""
- if not raw_value:
- return None
-
- try:
- return json.loads(raw_value)
- except json.JSONDecodeError:
- return raw_value
-
-
-def _build_openclaw_plugin_entry(
- *,
- existing_entry: Any,
- proxy_port: int,
- startup_timeout_ms: int,
- python_path: str | None,
- no_auto_start: bool,
- gateway_provider_ids: tuple[str, ...] | None,
- enabled: bool,
-) -> dict[str, object]:
- """Merge managed Headroom plugin settings with any existing entry payload."""
- base_entry = existing_entry if isinstance(existing_entry, dict) else {}
- existing_config = base_entry.get("config")
- next_config = dict(existing_config) if isinstance(existing_config, dict) else {}
-
- next_config["proxyPort"] = proxy_port
- next_config["autoStart"] = not no_auto_start
- next_config["startupTimeoutMs"] = startup_timeout_ms
- next_config["gatewayProviderIds"] = _normalize_openclaw_gateway_provider_ids(
- gateway_provider_ids
- )
-
- if python_path:
- next_config["pythonPath"] = python_path
- else:
- next_config.pop("pythonPath", None)
-
- return {
- **base_entry,
- "enabled": enabled,
- "config": next_config,
- }
-
-
-def _build_openclaw_unwrap_entry(existing_entry: Any) -> dict[str, object]:
- """Disable the managed plugin while preserving unrelated user config."""
- base_entry = existing_entry if isinstance(existing_entry, dict) else {}
- existing_config = {}
- if isinstance(existing_entry, dict) and isinstance(existing_entry.get("config"), dict):
- existing_config = {
- key: value
- for key, value in existing_entry["config"].items()
- if key
- not in {
- "gatewayProviderIds",
- "proxyUrl",
- "proxyPort",
- "autoStart",
- "startupTimeoutMs",
- "pythonPath",
- }
- }
-
- return {**base_entry, "enabled": False, "config": existing_config}
-
-
-def _write_openclaw_plugin_entry(openclaw_bin: str, entry: dict[str, object]) -> None:
- """Persist the Headroom plugin config entry."""
- _run_checked(
- [
- openclaw_bin,
- "config",
- "set",
- "plugins.entries.headroom",
- json.dumps(entry, separators=(",", ":")),
- "--strict-json",
- ],
- action="openclaw config set plugins.entries.headroom",
- )
-
-
-def _set_openclaw_context_engine_slot(openclaw_bin: str, engine_id: str) -> None:
- """Persist the selected OpenClaw context engine slot."""
- _run_checked(
- [
- openclaw_bin,
- "config",
- "set",
- "plugins.slots.contextEngine",
- json.dumps(engine_id),
- "--strict-json",
- ],
- action="openclaw config set plugins.slots.contextEngine",
- )
-
-
-def _restart_or_start_openclaw_gateway(openclaw_bin: str) -> tuple[str, str]:
- """Restart the gateway when running, otherwise start it."""
- restart_result = subprocess.run(
- [openclaw_bin, "gateway", "restart"],
- capture_output=True,
- text=True,
- encoding="utf-8",
- errors="replace",
- )
- if restart_result.returncode == 0:
- output = restart_result.stdout.strip() or restart_result.stderr.strip()
- return "restarted", output
-
- start_result = _run_checked(
- [openclaw_bin, "gateway", "start"],
- action="openclaw gateway start",
- )
- output = start_result.stdout.strip() or start_result.stderr.strip()
- return "started", output
-
-
-def _copy_openclaw_plugin_into_extensions(
- *,
- plugin_dir: Path,
- openclaw_bin: str,
-) -> Path:
- """Fallback install path when `openclaw plugins install` is blocked on linked source."""
- dist_dir = plugin_dir / "dist"
- if not dist_dir.exists():
- raise click.ClickException(
- f"Plugin dist folder missing at {dist_dir}. Build the plugin first."
- )
- hook_shim_dir = plugin_dir / "hook-shim"
- if not hook_shim_dir.exists():
- raise click.ClickException(
- f"Plugin hook-shim folder missing at {hook_shim_dir}. Build the plugin first."
- )
-
- extensions_dir = _resolve_openclaw_extensions_dir(openclaw_bin)
- target_dir = extensions_dir / "headroom"
- target_dist = target_dir / "dist"
- target_hook_shim = target_dir / "hook-shim"
- target_dir.mkdir(parents=True, exist_ok=True)
- if target_dist.exists():
- shutil.rmtree(target_dist)
- if target_hook_shim.exists():
- shutil.rmtree(target_hook_shim)
- shutil.copytree(dist_dir, target_dist)
- shutil.copytree(hook_shim_dir, target_hook_shim)
-
- for filename in ("openclaw.plugin.json", "package.json", "README.md"):
- source = plugin_dir / filename
- if source.exists():
- shutil.copy2(source, target_dir / filename)
-
- return target_dir
-
-
-def wrap() -> None:
- """Wrap CLI tools to run through Headroom.
-
- \b
- Starts a Headroom proxy, configures the environment, and launches
- the target tool so all API calls route through Headroom automatically.
-
- \b
- Supported tools:
- headroom wrap claude # Claude Code (Anthropic)
- headroom wrap copilot -- --model claude-sonnet-4-20250514
- headroom wrap codex # OpenAI Codex CLI
- headroom wrap aider # Aider
- headroom wrap cursor # Cursor (prints config instructions)
- headroom wrap openclaw # OpenClaw plugin bootstrap
- """
-
-
-def unwrap() -> None:
- """Undo durable Headroom wrapping for supported tools."""
-
-
-# =============================================================================
-# Claude Code
-# =============================================================================
-
-
[email protected](context_settings={"ignore_unknown_options": True})
[email protected]("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)")
[email protected]("--no-rtk", is_flag=True, help="Skip rtk installation and hook registration")
- "--code-graph",
- is_flag=True,
- help="Enable code graph indexing via codebase-memory-mcp (optional)",
-)
[email protected]("--no-proxy", is_flag=True, help="Skip proxy startup (use existing proxy)")
- "--learn", is_flag=True, help="Enable live traffic learning (patterns saved to MEMORY.md)"
-)
[email protected]("--memory", is_flag=True, help="Enable persistent cross-session memory")
[email protected]("--verbose", "-v", is_flag=True, help="Verbose output")
[email protected]("--prepare-only", is_flag=True, hidden=True)
[email protected]("claude_args", nargs=-1, type=click.UNPROCESSED)
-def claude(
- port: int,
- no_rtk: bool,
- code_graph: bool,
- no_proxy: bool,
- learn: bool,
- memory: bool,
- verbose: bool,
- prepare_only: bool,
- claude_args: tuple,
-) -> None:
- """Launch Claude Code through Headroom proxy.
-
- \b
- Sets ANTHROPIC_BASE_URL to route all Anthropic API calls through Headroom.
- All unknown flags are passed through to claude (e.g. --resume, --model).
-
- \b
- Examples:
- headroom wrap claude # Start everything
- headroom wrap claude --memory # With persistent memory
- headroom wrap claude --resume <id> # Resume a session
- headroom wrap claude -- -p # Claude in print mode
- headroom wrap claude --code-graph # With code graph intelligence
- headroom wrap claude --no-rtk # Skip rtk (proxy only)
- """
- if prepare_only:
- if not no_rtk:
- _prepare_wrap_rtk(verbose=verbose, label="Claude")
- return
-
- claude_bin = shutil.which("claude")
- if not claude_bin:
- click.echo("Error: 'claude' not found in PATH.")
- click.echo("Install Claude Code: https://docs.anthropic.com/en/docs/claude-code")
- raise SystemExit(1)
-
- # Setup rtk before launching (Claude-specific)
- proxy_holder: list[subprocess.Popen | None] = [None]
- cleanup = _make_cleanup(proxy_holder, port)
- signal.signal(signal.SIGINT, cleanup)
- signal.signal(signal.SIGTERM, cleanup)
-
- # Memory sync BEFORE proxy startup — sync headroom DB ↔ Claude's files
- if memory:
- try:
- import subprocess as _sp
-
- mem_dir = Path.cwd() / ".headroom"
- mem_dir.mkdir(parents=True, exist_ok=True)
- _sync_db = str(mem_dir / "memory.db")
- _sync_user = os.environ.get("USER", os.environ.get("USERNAME", "default"))
-
- click.echo(f" Syncing memory (user={_sync_user})...")
- sync_result = _sp.run(
- [
- sys.executable,
- "-m",
- "headroom.memory.sync",
- "--db",
- _sync_db,
- "--user",
- _sync_user,
- "--agent",
- "claude",
- "--force",
- ],
- capture_output=True,
- text=True,
- timeout=30,
- )
- if sync_result.returncode == 0 and sync_result.stdout.strip():
- import json as _json
-
- stats = _json.loads(sync_result.stdout.strip().split("\n")[-1])
- imp, exp, ms = stats["imported"], stats["exported"], stats["ms"]
- if imp or exp:
- click.echo(f" Memory synced: {imp} imported, {exp} exported ({ms}ms)")
- else:
- click.echo(f" Memory: up to date ({ms}ms)")
- elif sync_result.returncode != 0:
- click.echo(f" Warning: memory sync error: {sync_result.stderr[-200:]}")
- except Exception as e:
- click.echo(f" Warning: memory sync failed: {e}")
-
- try:
- click.echo()
- click.echo(" ╔═══════════════════════════════════════════════╗")
- click.echo(" ║ HEADROOM WRAP: CLAUDE ║")
- click.echo(" ╚═══════════════════════════════════════════════╝")
- click.echo()
-
- proxy_holder[0] = _ensure_proxy(
- port, no_proxy, learn=learn, memory=memory, agent_type="claude", code_graph=code_graph
- )
-
- if not no_rtk:
- click.echo(" Setting up rtk...")
- _setup_rtk(verbose=verbose)
- elif verbose:
- click.echo(" Skipping rtk (--no-rtk)")
-
- if code_graph:
- _setup_code_graph(verbose=verbose)
-
- click.echo()
- click.echo(" Launching Claude Code (API routed through Headroom)...")
- click.echo(f" ANTHROPIC_BASE_URL=http://127.0.0.1:{port}")
- if claude_args:
- click.echo(f" Extra args: {' '.join(claude_args)}")
- _print_telemetry_notice()
- click.echo()
-
- env = os.environ.copy()
- env["ANTHROPIC_BASE_URL"] = f"http://127.0.0.1:{port}"
-
- result = subprocess.run([claude_bin, *claude_args], env=env)
- raise SystemExit(result.returncode)
-
- except SystemExit:
- raise
- except Exception as e:
- click.echo(f" Error: {e}")
- raise SystemExit(1) from e
- finally:
- cleanup()
-
-
-# =============================================================================
-# GitHub Copilot CLI
-# =============================================================================
-
-
[email protected](context_settings={"ignore_unknown_options": True})
[email protected]("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)")
- "--no-rtk",
- is_flag=True,
- help="Skip rtk installation and Copilot instructions injection",
-)
[email protected]("--no-proxy", is_flag=True, help="Skip proxy startup (use existing proxy)")
- "--backend",
- default=None,
- help="API backend for the proxy: 'anthropic', 'anyllm', 'litellm-vertex', etc. (env: HEADROOM_BACKEND)",
-)
- "--anyllm-provider",
- default=None,
- help="Provider for any-llm backend: openai, mistral, groq, etc. (env: HEADROOM_ANYLLM_PROVIDER)",
-)
- "--region", default=None, help="Cloud region for Bedrock/Vertex (env: HEADROOM_REGION)"
-)
- "--provider-type",
- type=click.Choice(["auto", "anthropic", "openai"]),
- default="auto",
- show_default=True,
- help="Copilot BYOK provider mode. 'auto' uses anthropic for the default proxy backend and openai for translated backends.",
-)
- "--wire-api",
- type=click.Choice(["completions", "responses"]),
- default=None,
- help="OpenAI-compatible Copilot wire API. Defaults to 'completions' when provider-type resolves to openai.",
-)
[email protected]("--memory", is_flag=True, help="Enable persistent cross-session memory")
[email protected]("--verbose", "-v", is_flag=True, help="Verbose output")
[email protected]("copilot_args", nargs=-1, type=click.UNPROCESSED)
-def copilot(
- port: int,
- no_rtk: bool,
- no_proxy: bool,
- backend: str | None,
- anyllm_provider: str | None,
- region: str | None,
- provider_type: str,
- wire_api: str | None,
- memory: bool,
- verbose: bool,
- copilot_args: tuple[str, ...],
-) -> None:
- """Launch GitHub Copilot CLI through Headroom proxy.
-
- \b
- Configures Copilot CLI BYOK provider variables so Copilot routes through
- the local Headroom proxy. In auto mode, the wrapper uses Anthropic-style
- routing for the stock proxy backend and OpenAI-compatible routing for
- translated backends such as any-llm and LiteLLM.
-
- \b
- Examples:
- headroom wrap copilot -- --model claude-sonnet-4-20250514
- headroom wrap copilot --backend anyllm --anyllm-provider groq -- --model gpt-4o
- headroom wrap copilot --provider-type openai --wire-api responses -- --model gpt-5.4
- headroom wrap copilot --no-rtk -- --prompt "explain this file"
- """
- copilot_bin = shutil.which("copilot")
- if not copilot_bin:
- click.echo("Error: 'copilot' not found in PATH.")
- click.echo(
- "Install GitHub Copilot CLI: "
- "https://docs.github.com/en/copilot/how-tos/copilot-cli/set-up-copilot-cli/install-copilot-cli"
- )
- raise SystemExit(1)
-
- effective_backend = backend or os.environ.get("HEADROOM_BACKEND")
- if _check_proxy(port):
- running_backend = _detect_running_proxy_backend(port)
- if effective_backend and running_backend and effective_backend != running_backend:
- raise click.ClickException(
- f"Proxy already running on port {port} with backend '{running_backend}'. "
- f"Stop it or rerun with --backend {running_backend}."
- )
- effective_backend = running_backend or effective_backend
-
- effective_provider_type = _resolve_copilot_provider_type(effective_backend, provider_type)
- if effective_provider_type == "anthropic" and wire_api is not None:
- raise click.ClickException(
- "--wire-api is only valid when Copilot is using the openai provider type."
- )
- if wire_api == "responses" and effective_backend not in (None, "anthropic"):
- raise click.ClickException(
- "--wire-api responses is not supported with translated backends; use completions."
- )
-
- if not no_rtk:
- click.echo(" Setting up rtk for Copilot...")
- rtk_path = _ensure_rtk_binary(verbose=verbose)
- if rtk_path:
- copilot_instructions = Path.cwd() / ".github" / "copilot-instructions.md"
- _inject_rtk_instructions(copilot_instructions, verbose=verbose)
-
- env = os.environ.copy()
- env.pop("COPILOT_PROVIDER_WIRE_API", None)
- openai_api_url: str | None = None
-
- if _should_use_copilot_oauth(
- backend=effective_backend,
- provider_type=provider_type,
- env=env,
- ):
- client_bearer = resolve_client_bearer_token()
- if not client_bearer:
- raise click.ClickException(
- "GitHub Copilot auth was detected but no reusable bearer token could be resolved."
- )
-
- env["COPILOT_PROVIDER_TYPE"] = "openai"
- env["COPILOT_PROVIDER_BASE_URL"] = f"http://127.0.0.1:{port}/v1"
- env["COPILOT_PROVIDER_WIRE_API"] = wire_api or "completions"
- env["COPILOT_PROVIDER_BEARER_TOKEN"] = client_bearer
- env.pop("COPILOT_PROVIDER_API_KEY", None)
- env_vars_display = [
- "COPILOT_PROVIDER_TYPE=openai",
- f"COPILOT_PROVIDER_BASE_URL=http://127.0.0.1:{port}/v1",
- f"COPILOT_PROVIDER_WIRE_API={env['COPILOT_PROVIDER_WIRE_API']}",
- "COPILOT_AUTH_MODE=github-oauth",
- ]
- openai_api_url = COPILOT_API_URL
- else:
- env["COPILOT_PROVIDER_TYPE"] = effective_provider_type
-
- # Copilot BYOK requires COPILOT_PROVIDER_API_KEY — propagate from the
- # user's existing provider key so they don't have to set it twice.
- if not env.get("COPILOT_PROVIDER_API_KEY"):
- if effective_provider_type == "anthropic":
- _key = env.get("ANTHROPIC_API_KEY", "")
- else:
- _key = env.get("OPENAI_API_KEY", "")
- if _key:
- env["COPILOT_PROVIDER_API_KEY"] = _key
-
- if effective_provider_type == "anthropic":
- env["COPILOT_PROVIDER_BASE_URL"] = f"http://127.0.0.1:{port}"
- env_vars_display = [
- "COPILOT_PROVIDER_TYPE=anthropic",
- f"COPILOT_PROVIDER_BASE_URL=http://127.0.0.1:{port}",
- ]
- else:
- effective_wire_api = wire_api or "completions"
- env["COPILOT_PROVIDER_BASE_URL"] = f"http://127.0.0.1:{port}/v1"
- env["COPILOT_PROVIDER_WIRE_API"] = effective_wire_api
- env_vars_display = [
- "COPILOT_PROVIDER_TYPE=openai",
- f"COPILOT_PROVIDER_BASE_URL=http://127.0.0.1:{port}/v1",
- f"COPILOT_PROVIDER_WIRE_API={effective_wire_api}",
- ]
-
- if not env.get("COPILOT_PROVIDER_API_KEY"):
- src = (
- "ANTHROPIC_API_KEY" if effective_provider_type == "anthropic" else "OPENAI_API_KEY"
- )
- click.echo(
- f"\n Error: Copilot BYOK mode requires a provider API key.\n"
- f" No reusable GitHub Copilot OAuth session was found, so Headroom fell back to\n"
- f" Copilot's BYOK provider mode. That mode bypasses GitHub's Copilot API and\n"
- f" routes requests directly to the model provider through the Headroom proxy.\n\n"
- f" Set one of:\n"
- f" export {src}=sk-... # recommended\n"
- f" export COPILOT_PROVIDER_API_KEY=sk-... # also works\n"
- )
- raise SystemExit(1)
-
- if not _copilot_model_configured(copilot_args, env):
- click.echo(
- " Note: Copilot BYOK requires a model. Pass `--model <name>` "
- "or set `COPILOT_MODEL` / `COPILOT_PROVIDER_MODEL_ID`."
- )
-
- _launch_tool(
- binary=copilot_bin,
- args=copilot_args,
- env=env,
- port=port,
- no_proxy=no_proxy,
- tool_label="COPILOT",
- env_vars_display=env_vars_display,
- learn=False,
- memory=memory,
- agent_type="copilot",
- backend=backend,
- anyllm_provider=anyllm_provider,
- region=region,
- openai_api_url=openai_api_url,
- )
-
-
-# =============================================================================
-# OpenAI Codex CLI
-# =============================================================================
-
-
[email protected](context_settings={"ignore_unknown_options": True})
[email protected]("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)")
[email protected]("--no-rtk", is_flag=True, help="Skip rtk installation and AGENTS.md injection")
- "--code-graph",
- is_flag=True,
- help="Enable code graph indexing via codebase-memory-mcp (optional)",
-)
[email protected]("--no-proxy", is_flag=True, help="Skip proxy startup (use existing proxy)")
- "--learn", is_flag=True, help="Enable live traffic learning (patterns saved to AGENTS.md)"
-)
- "--backend",
- default=None,
- help="API backend for the proxy: 'anthropic', 'anyllm', 'litellm-vertex', etc. (env: HEADROOM_BACKEND)",
-)
- "--anyllm-provider",
- default=None,
- help="Provider for any-llm backend: openai, mistral, groq, etc. (env: HEADROOM_ANYLLM_PROVIDER)",
-)
- "--region", default=None, help="Cloud region for Bedrock/Vertex (env: HEADROOM_REGION)"
-)
[email protected]("--memory", is_flag=True, help="Enable persistent cross-session memory")
[email protected]("--verbose", "-v", is_flag=True, help="Verbose output")
[email protected]("--prepare-only", is_flag=True, hidden=True)
[email protected]("codex_args", nargs=-1, type=click.UNPROCESSED)
-def codex(
- port: int,
- no_rtk: bool,
- code_graph: bool,
- no_proxy: bool,
- learn: bool,
- memory: bool,
- backend: str | None,
- anyllm_provider: str | None,
- region: str | None,
- verbose: bool,
- prepare_only: bool,
- codex_args: tuple,
-) -> None:
- """Launch OpenAI Codex CLI through Headroom proxy.
-
- \b
- Sets OPENAI_BASE_URL to route all OpenAI API calls through Headroom.
- Installs rtk and injects instructions into AGENTS.md so Codex uses
- token-optimized commands (60-90% savings on shell output).
-
- \b
- Examples:
- headroom wrap codex # Start proxy + rtk + codex
- headroom wrap codex -- "fix the bug" # Pass prompt to codex
- headroom wrap codex --no-rtk # Skip rtk setup
- headroom wrap codex --port 9999 # Custom proxy port
- headroom wrap codex --backend anyllm --anyllm-provider groq
- """
- # Setup rtk for Codex (binary + AGENTS.md instructions, no hooks)
- if not no_rtk:
- click.echo(" Setting up rtk for Codex...")
- rtk_path = _ensure_rtk_binary(verbose=verbose)
- if rtk_path:
- # Inject into project AGENTS.md (Codex reads this automatically)
- agents_md = Path.cwd() / "AGENTS.md"
- _inject_rtk_instructions(agents_md, verbose=verbose)
-
- # Also inject into global ~/.codex/AGENTS.md
- global_agents = Path.home() / ".codex" / "AGENTS.md"
- _inject_rtk_instructions(global_agents, verbose=verbose)
-
- # Setup memory MCP server for Codex (native tool integration)
- if memory:
- click.echo(" Setting up memory for Codex...")
- mem_dir = Path.cwd() / ".headroom"
- mem_dir.mkdir(parents=True, exist_ok=True)
- db_path = str(mem_dir / "memory.db")
- mem_user = os.environ.get("USER", os.environ.get("USERNAME", "default"))
-
- # Register MCP server in Codex config
- _inject_memory_mcp_config(db_path, mem_user)
-
- # Inject memory guidance into project AGENTS.md
- agents_md = Path.cwd() / "AGENTS.md"
- _inject_memory_agents_md(agents_md)
-
- # Sync Claude's memories → DB so MCP search finds them
- try:
- import asyncio
-
- from headroom.memory.backends.local import LocalBackend, LocalBackendConfig
- from headroom.memory.sync import sync_import
- from headroom.memory.sync_adapters.claude_code import (
- ClaudeCodeAdapter,
- get_claude_memory_dir,
- )
-
- claude_memory_dir = get_claude_memory_dir()
-
- async def _import_claude_memories() -> int:
- config = LocalBackendConfig(db_path=db_path)
- backend = LocalBackend(config)
- await backend._ensure_initialized()
- adapter = ClaudeCodeAdapter(claude_memory_dir)
- count = await sync_import(backend, adapter, mem_user)
- await backend.close()
- return count
-
- imported = asyncio.run(_import_claude_memories())
- if imported:
- click.echo(f" Memory: imported {imported} memories from Claude")
- except Exception as e:
- click.echo(f" Warning: Claude memory import failed: {e}")
-
- if prepare_only:
- _inject_codex_provider_config(port)
- return
-
- codex_bin = shutil.which("codex")
- if not codex_bin:
- click.echo("Error: 'codex' not found in PATH.")
- click.echo("Install Codex CLI: npm install -g @openai/codex")
- raise SystemExit(1)
-
- env = os.environ.copy()
- env["OPENAI_BASE_URL"] = f"http://127.0.0.1:{port}/v1"
-
- # Inject Headroom provider into Codex config so WebSocket traffic also
- # routes through the proxy. Codex ignores OPENAI_BASE_URL for its WS
- # transport unless a custom provider declares supports_websockets = true.
- # NOTE: this must run BEFORE _inject_memory_mcp_config because it rewrites
- # the config file. Re-inject MCP config after if memory is enabled.
- _inject_codex_provider_config(port)
- if memory:
- mem_dir = Path.cwd() / ".headroom"
- _inject_memory_mcp_config(
- str(mem_dir / "memory.db"),
- os.environ.get("USER", os.environ.get("USERNAME", "default")),
- )
-
- _launch_tool(
- binary=codex_bin,
- args=codex_args,
- env=env,
- port=port,
- no_proxy=no_proxy,
- tool_label="CODEX",
- env_vars_display=[f"OPENAI_BASE_URL=http://127.0.0.1:{port}/v1"],
- learn=learn,
- memory=memory,
- agent_type="codex",
- code_graph=code_graph,
- backend=backend,
- anyllm_provider=anyllm_provider,
- region=region,
- )
-
-
-# =============================================================================
-# Aider
-# =============================================================================
-
-
[email protected](context_settings={"ignore_unknown_options": True})
[email protected]("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)")
[email protected]("--no-rtk", is_flag=True, help="Skip rtk installation and conventions injection")
- "--code-graph",
- is_flag=True,
- help="Enable code graph indexing via codebase-memory-mcp (optional)",
-)
[email protected]("--no-proxy", is_flag=True, help="Skip proxy startup (use existing proxy)")
[email protected]("--learn", is_flag=True, help="Enable live traffic learning")
[email protected]("--memory", is_flag=True, help="Enable persistent cross-session memory")
- "--backend", default=None, help="API backend: 'anthropic', 'anyllm', 'litellm-vertex', etc."
-)
[email protected]("--anyllm-provider", default=None, help="Provider for any-llm backend")
[email protected]("--region", default=None, help="Cloud region for Bedrock/Vertex")
[email protected]("--verbose", "-v", is_flag=True, help="Verbose output")
[email protected]("--prepare-only", is_flag=True, hidden=True)
[email protected]("aider_args", nargs=-1, type=click.UNPROCESSED)
-def aider(
- port: int,
- no_rtk: bool,
- code_graph: bool,
- no_proxy: bool,
- learn: bool,
- memory: bool,
- backend: str | None,
- anyllm_provider: str | None,
- region: str | None,
- verbose: bool,
- prepare_only: bool,
- aider_args: tuple,
-) -> None:
- """Launch aider through Headroom proxy.
-
- \b
- Sets OPENAI_API_BASE to route all API calls through Headroom.
- Installs rtk and injects instructions into .aider.conf.yml conventions
- so aider uses token-optimized commands.
-
- \b
- Examples:
- headroom wrap aider # Start proxy + rtk + aider
- headroom wrap aider -- --model gpt-4o # Use GPT-4o
- headroom wrap aider -- --model claude-sonnet-4 # Use Claude
- headroom wrap aider --no-rtk # Skip rtk setup
- headroom wrap aider --backend litellm-vertex --region us-central1
- """
- # Setup rtk for aider (binary + CONVENTIONS.md instructions)
- if not no_rtk:
- click.echo(" Setting up rtk for aider...")
- rtk_path = _ensure_rtk_binary(verbose=verbose)
- if rtk_path:
- # aider reads CONVENTIONS.md from project root
- conventions = Path.cwd() / "CONVENTIONS.md"
- _inject_rtk_instructions(conventions, verbose=verbose)
-
- if prepare_only:
- return
-
- aider_bin = shutil.which("aider")
- if not aider_bin:
- click.echo("Error: 'aider' not found in PATH.")
- click.echo("Install aider: pip install aider-chat")
- raise SystemExit(1)
-
- env = os.environ.copy()
- env["OPENAI_API_BASE"] = f"http://127.0.0.1:{port}/v1"
- env["ANTHROPIC_BASE_URL"] = f"http://127.0.0.1:{port}"
-
- _launch_tool(
- binary=aider_bin,
- args=aider_args,
- env=env,
- port=port,
- no_proxy=no_proxy,
- tool_label="AIDER",
- env_vars_display=[
- f"OPENAI_API_BASE=http://127.0.0.1:{port}/v1",
- f"ANTHROPIC_BASE_URL=http://127.0.0.1:{port}",
- ],
- learn=learn,
- memory=memory,
- agent_type="aider",
- code_graph=code_graph,
- backend=backend,
- anyllm_provider=anyllm_provider,
- region=region,
- )
-
-
-# =============================================================================
-# Cursor
-# =============================================================================
-
-
[email protected](context_settings={"ignore_unknown_options": True})
[email protected]("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)")
[email protected]("--no-rtk", is_flag=True, help="Skip rtk installation and .cursorrules injection")
[email protected]("--no-proxy", is_flag=True, help="Skip proxy startup (use existing proxy)")
- "--learn", is_flag=True, help="Enable live traffic learning (patterns saved to .cursor/rules/)"
-)
[email protected]("--memory", is_flag=True, help="Enable persistent cross-session memory")
[email protected]("--verbose", "-v", is_flag=True, help="Verbose output")
[email protected]("--prepare-only", is_flag=True, hidden=True)
-def cursor(
- port: int,
- no_rtk: bool,
- no_proxy: bool,
- learn: bool,
- memory: bool,
- verbose: bool,
- prepare_only: bool,
-) -> None:
- """Start Headroom proxy for use with Cursor.
-
- \b
- Cursor reads its API configuration from its settings UI, not from
- environment variables. This command starts the proxy, installs rtk
- with .cursorrules instructions, and prints the Cursor settings.
-
- \b
- After running this command, open Cursor and configure:
- Settings > Models > OpenAI API Key > Advanced > Override Base URL
-
- \b
- Example:
- headroom wrap cursor # Start proxy + rtk + instructions
- headroom wrap cursor --no-rtk # Proxy only, no rtk
- headroom wrap cursor --port 9999 # Custom proxy port
- """
- if not no_rtk:
- click.echo(" Setting up rtk for Cursor...")
- rtk_path = _ensure_rtk_binary(verbose=verbose)
- if rtk_path:
- cursorrules = Path.cwd() / ".cursorrules"
- _inject_rtk_instructions(cursorrules, verbose=verbose)
-
- if prepare_only:
- return
-
- proxy_holder: list[subprocess.Popen | None] = [None]
- cleanup = _make_cleanup(proxy_holder, port)
- signal.signal(signal.SIGINT, cleanup)
- signal.signal(signal.SIGTERM, cleanup)
-
- try:
- click.echo()
- click.echo(" ╔═══════════════════════════════════════════════╗")
- click.echo(" ║ HEADROOM WRAP: CURSOR ║")
- click.echo(" ╚═══════════════════════════════════════════════╝")
- click.echo()
-
- proxy_holder[0] = _ensure_proxy(
- port, no_proxy, learn=learn, memory=memory, agent_type="cursor"
- )
-
- click.echo()
- click.echo(" Headroom proxy is running. Configure Cursor:")
- click.echo()
- click.echo(" For OpenAI models:")
- click.echo(f" Base URL: http://127.0.0.1:{port}/v1")
- click.echo(" API Key: your-openai-api-key")
- click.echo()
- click.echo(" For Anthropic models:")
- click.echo(f" Base URL: http://127.0.0.1:{port}")
- click.echo(" API Key: your-anthropic-api-key")
- click.echo()
- click.echo(" In Cursor:")
- click.echo(" Settings > Models > OpenAI API Key > Override OpenAI Base URL")
- click.echo(f" Set to: http://127.0.0.1:{port}/v1")
- if not no_rtk:
- click.echo()
- click.echo(" rtk instructions injected into .cursorrules")
- click.echo(" Cursor will use token-optimized commands automatically.")
- click.echo()
- click.echo(" Press Ctrl+C to stop the proxy.")
- click.echo()
-
- # Block until Ctrl+C
- try:
- while True:
- time.sleep(1)
- proc = proxy_holder[0]
- if proc and proc.poll() is not None:
- click.echo(" Proxy process exited unexpectedly.")
- raise SystemExit(1)
- except KeyboardInterrupt:
- click.echo("\n Shutting down...")
-
- except SystemExit:
- raise
- except Exception as e:
- click.echo(f" Error: {e}")
- raise SystemExit(1) from e
- finally:
- cleanup()
-
-
-# =============================================================================
-# OpenClaw
-# =============================================================================
-
-
[email protected]("openclaw")
- "--plugin-path",
- type=click.Path(path_type=Path, file_okay=False, dir_okay=True),
- default=None,
- help="Path to local OpenClaw plugin source directory (advanced/dev override)",
-)
- "--plugin-spec",
- default="headroom-ai/openclaw",
- show_default=True,
- help="NPM plugin spec for OpenClaw install (used when --plugin-path is omitted)",
-)
- "--skip-build",
- is_flag=True,
- help="Skip npm install/build in local source mode (--plugin-path)",
-)
- "--copy",
- is_flag=True,
- help="Install by copying plugin path instead of using --link",
-)
[email protected]("--proxy-port", default=8787, type=int, help="Headroom proxy port")
[email protected]("--startup-timeout-ms", default=20000, type=int, help="Proxy startup timeout")
- "--gateway-provider-id",
- "gateway_provider_ids",
- multiple=True,
- help="OpenClaw provider id to route through Headroom (repeatable; default: openai-codex)",
-)
- "--python-path",
- default=None,
- help="Optional Python executable for proxy launcher fallback",
-)
- "--no-auto-start",
- is_flag=True,
- help="Disable plugin auto-start of local headroom proxy",
-)
- "--no-restart",
- is_flag=True,
- help="Do not restart OpenClaw gateway at the end",
-)
[email protected]("--verbose", "-v", is_flag=True, help="Verbose output")
[email protected]("--prepare-only", is_flag=True, hidden=True)
[email protected]("--existing-entry-json", default=None, hidden=True)
-def openclaw(
- plugin_path: Path | None,
- plugin_spec: str,
- skip_build: bool,
- copy: bool,
- proxy_port: int,
- startup_timeout_ms: int,
- gateway_provider_ids: tuple[str, ...],
- python_path: str | None,
- no_auto_start: bool,
- no_restart: bool,
- verbose: bool,
- prepare_only: bool,
- existing_entry_json: str | None,
-) -> None:
- """Install and configure Headroom OpenClaw plugin in one command.
-
- \b
- What this command does:
- 1. Installs OpenClaw plugin from npm (or local --plugin-path)
- 2. Builds plugin source if --plugin-path is used
- 3. Writes minimal plugin config and sets contextEngine slot
- 4. Validates config
- 5. Restarts OpenClaw gateway (unless --no-restart)
-
- \b
- Example:
- headroom wrap openclaw
- headroom wrap openclaw --plugin-path C:\\git\\headroom\\plugins\\openclaw
- """
- if prepare_only:
- entry = _build_openclaw_plugin_entry(
- existing_entry=_decode_openclaw_entry_json(existing_entry_json),
- proxy_port=proxy_port,
- startup_timeout_ms=startup_timeout_ms,
- python_path=python_path,
- no_auto_start=no_auto_start,
- gateway_provider_ids=gateway_provider_ids,
- enabled=True,
- )
- click.echo(json.dumps(entry, separators=(",", ":")))
- return
-
- openclaw_bin = shutil.which("openclaw")
- if not openclaw_bin:
- raise click.ClickException("'openclaw' not found in PATH. Install OpenClaw CLI first.")
-
- plugin_dir = plugin_path.resolve() if plugin_path else None
- local_source_mode = plugin_dir is not None
- if plugin_dir:
- if not plugin_dir.exists():
- raise click.ClickException(f"Plugin path not found: {plugin_dir}.")
- if not (plugin_dir / "package.json").exists():
- raise click.ClickException(f"Invalid plugin path (missing package.json): {plugin_dir}")
- if not (plugin_dir / "openclaw.plugin.json").exists():
- raise click.ClickException(
- f"Invalid plugin path (missing openclaw.plugin.json): {plugin_dir}"
- )
-
- npm_bin = shutil.which("npm")
- if local_source_mode and not skip_build and not npm_bin:
- raise click.ClickException(
- "'npm' not found in PATH. Install Node/npm or rerun with --skip-build."
- )
-
- click.echo()
- click.echo(" ╔═══════════════════════════════════════════════╗")
- click.echo(" ║ HEADROOM WRAP: OPENCLAW ║")
- click.echo(" ╚═══════════════════════════════════════════════╝")
- click.echo()
- if local_source_mode:
- click.echo(f" Plugin source: local ({plugin_dir})")
- else:
- click.echo(f" Plugin source: npm ({plugin_spec})")
-
- if local_source_mode and not skip_build:
- click.echo(" Building OpenClaw plugin (npm install + npm run build)...")
- _run_checked([npm_bin or "npm", "install"], cwd=plugin_dir, action="npm install")
- _run_checked([npm_bin or "npm", "run", "build"], cwd=plugin_dir, action="npm run build")
- elif not local_source_mode and skip_build:
- click.echo(" Skipping build: npm install mode does not build local source.")
-
- effective_python_path = python_path
- if effective_python_path is None and not no_auto_start and sys.executable:
- effective_python_path = sys.executable
-
- existing_entry = _read_openclaw_config_value(openclaw_bin, "plugins.entries.headroom")
- entry = _build_openclaw_plugin_entry(
- existing_entry=existing_entry,
- proxy_port=proxy_port,
- startup_timeout_ms=startup_timeout_ms,
- python_path=effective_python_path,
- no_auto_start=no_auto_start,
- gateway_provider_ids=gateway_provider_ids,
- enabled=True,
- )
-
- click.echo(" Writing plugin configuration...")
- _write_openclaw_plugin_entry(openclaw_bin, entry)
-
- install_cmd = [
- openclaw_bin,
- "plugins",
- "install",
- "--dangerously-force-unsafe-install",
- ]
- if local_source_mode:
- if copy:
- install_cmd.append(str(plugin_dir))
- install_cwd = None
- else:
- install_cmd.extend(["--link", "."])
- install_cwd = plugin_dir
- else:
- install_cmd.append(plugin_spec)
- install_cwd = None
-
- click.echo(" Installing OpenClaw plugin with required unsafe-install flag...")
- install_result = subprocess.run(
- install_cmd,
- cwd=str(install_cwd) if install_cwd else None,
- capture_output=True,
- text=True,
- encoding="utf-8",
- errors="replace",
- )
- if install_result.returncode != 0:
- combined_error = "\n".join(
- x for x in [install_result.stderr.strip(), install_result.stdout.strip()] if x
- )
- plugin_already_exists = "plugin already exists" in combined_error.lower()
- linked_install_bug = (
- "also not a valid hook pack" in combined_error.lower()
- and "--dangerously-force-unsafe-install" in " ".join(install_cmd)
- )
- if plugin_already_exists:
- click.echo(" Plugin already installed; continuing with configuration/update steps.")
- elif linked_install_bug and local_source_mode and plugin_dir is not None:
- click.echo(
- " OpenClaw linked-path install bug detected; applying extension-path fallback..."
- )
- target_dir = _copy_openclaw_plugin_into_extensions(
- plugin_dir=plugin_dir,
- openclaw_bin=openclaw_bin,
- )
- click.echo(f" Fallback plugin copy completed: {target_dir}")
- else:
- details = combined_error or f"exit code {install_result.returncode}"
- raise click.ClickException(f"openclaw plugins install failed: {details}")
- elif verbose and install_result.stdout.strip():
- click.echo(install_result.stdout.strip())
-
- _set_openclaw_context_engine_slot(openclaw_bin, "headroom")
- _run_checked(
- [openclaw_bin, "config", "validate"],
- action="openclaw config validate",
- )
-
- if no_restart:
- click.echo(" Skipping gateway restart (--no-restart).")
- click.echo(
- " Run `openclaw gateway restart` (or `openclaw gateway start`) to apply plugin changes."
- )
- else:
- click.echo(" Applying plugin changes to OpenClaw gateway...")
- gateway_action, gateway_output = _restart_or_start_openclaw_gateway(openclaw_bin)
- click.echo(f" Gateway {gateway_action}.")
- if verbose and gateway_output:
- click.echo(gateway_output)
-
- inspect_result = _run_checked(
- [openclaw_bin, "plugins", "inspect", "headroom"],
- action="openclaw plugins inspect headroom",
- )
- if verbose and inspect_result.stdout.strip():
- click.echo(inspect_result.stdout.strip())
-
- click.echo()
- click.echo("✓ OpenClaw is configured to use Headroom context compression.")
- click.echo(" Plugin: headroom")
- click.echo(" Slot: plugins.slots.contextEngine = headroom")
- click.echo()
-
-
[email protected]("openclaw")
[email protected]("--no-restart", is_flag=True, help="Do not restart OpenClaw gateway at the end")
[email protected]("--verbose", "-v", is_flag=True, help="Verbose output")
[email protected]("--prepare-only", is_flag=True, hidden=True)
[email protected]("--existing-entry-json", default=None, hidden=True)
-def unwrap_openclaw(
- no_restart: bool,
- verbose: bool,
- prepare_only: bool,
- existing_entry_json: str | None,
-) -> None:
- """Disable the Headroom OpenClaw plugin and restore the legacy engine slot."""
- if prepare_only:
- click.echo(
- json.dumps(
- _build_openclaw_unwrap_entry(_decode_openclaw_entry_json(existing_entry_json)),
- separators=(",", ":"),
- )
- )
- return
-
- openclaw_bin = shutil.which("openclaw")
- if not openclaw_bin:
- raise click.ClickException("'openclaw' not found in PATH. Install OpenClaw CLI first.")
-
- click.echo()
- click.echo(" ╔═══════════════════════════════════════════════╗")
- click.echo(" ║ HEADROOM UNWRAP: OPENCLAW ║")
- click.echo(" ╚═══════════════════════════════════════════════╝")
- click.echo()
- click.echo(" Disabling Headroom plugin and removing engine mapping...")
-
- existing_entry = _read_openclaw_config_value(openclaw_bin, "plugins.entries.headroom")
- entry = _build_openclaw_unwrap_entry(existing_entry)
- _write_openclaw_plugin_entry(openclaw_bin, entry)
- _set_openclaw_context_engine_slot(openclaw_bin, "legacy")
- _run_checked(
- [openclaw_bin, "config", "validate"],
- action="openclaw config validate",
- )
-
- if no_restart:
- click.echo(" Skipping gateway restart (--no-restart).")
- click.echo(
- " Run `openclaw gateway restart` (or `openclaw gateway start`) to apply unwrap changes."
- )
- else:
- click.echo(" Applying unwrap changes to OpenClaw gateway...")
- gateway_action, gateway_output = _restart_or_start_openclaw_gateway(openclaw_bin)
- click.echo(f" Gateway {gateway_action}.")
- if verbose and gateway_output:
- click.echo(gateway_output)
-
- if verbose:
- inspect_result = _run_checked(
- [openclaw_bin, "plugins", "inspect", "headroom"],
- action="openclaw plugins inspect headroom",
- )
- if inspect_result.stdout.strip():
- click.echo(inspect_result.stdout.strip())
-
- click.echo()
- click.echo("✓ OpenClaw Headroom wrap removed.")
- click.echo(" Plugin: headroom (installed, disabled)")
- click.echo(" Slot: plugins.slots.contextEngine = legacy")
- click.echo()
+"""Wrap CLI commands to run through Headroom proxy.
+
+Usage:
+ headroom wrap claude # Start proxy + rtk + claude
+ headroom wrap copilot -- --model ... # Start proxy + launch GitHub Copilot CLI
+ headroom wrap codex # Start proxy + OpenAI Codex CLI
+ headroom wrap aider # Start proxy + aider
+ headroom wrap cursor # Start proxy + print Cursor config instructions
+ headroom wrap openclaw # Install + configure OpenClaw plugin
+ headroom wrap claude --no-rtk # Without rtk hooks
+ headroom wrap claude --port 9999 # Custom proxy port
+ headroom wrap claude -- --model opus # Pass args to claude
+"""
+
+from __future__ import annotations
+
+import io
+import json
+import os
+import shutil
+import signal
+import socket
+import subprocess
+import sys
+import time
+from pathlib import Path
+from typing import Any
+
+# Fix Windows cp1252 encoding — box-drawing characters require UTF-8
+if sys.platform == "win32" and hasattr(sys.stdout, "buffer"):
+ if sys.stdout.encoding and sys.stdout.encoding.lower().replace("-", "") != "utf8":
+ sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8", errors="replace")
+ sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding="utf-8", errors="replace")
+
+import click
+
+from headroom.providers.aider import build_launch_env as _build_aider_launch_env
+from headroom.providers.claude import proxy_base_url as _claude_proxy_base_url
+from headroom.providers.codex import build_launch_env as _build_codex_launch_env
+from headroom.providers.copilot import (
+ build_launch_env as _build_copilot_launch_env,
+)
+from headroom.providers.copilot import (
+ detect_running_proxy_backend as _copilot_detect_running_proxy_backend,
+)
+from headroom.providers.copilot import (
+ model_configured as _copilot_model_configured_impl,
+)
+from headroom.providers.copilot import (
+ provider_key_source as _copilot_provider_key_source,
+)
+from headroom.providers.copilot import (
+ query_proxy_config as _copilot_query_proxy_config,
+)
+from headroom.providers.copilot import (
+ resolve_provider_type as _copilot_resolve_provider_type,
+)
+from headroom.providers.copilot import (
+ validate_configuration as _validate_copilot_configuration,
+)
+from headroom.providers.cursor import render_setup_lines as _render_cursor_setup_lines
+from headroom.providers.openclaw import (
+ build_plugin_entry as _build_openclaw_plugin_entry_impl,
+)
+from headroom.providers.openclaw import (
+ build_unwrap_entry as _build_openclaw_unwrap_entry_impl,
+)
+from headroom.providers.openclaw import (
+ decode_entry_json as _decode_openclaw_entry_json_impl,
+)
+from headroom.providers.openclaw import (
+ normalize_gateway_provider_ids as _normalize_openclaw_gateway_provider_ids_impl,
+)
+
+from .main import main
+
+
+def _print_telemetry_notice() -> None:
+ """Print a telemetry notice when anonymous telemetry is enabled.
+
+ Respects the HEADROOM_TELEMETRY and HEADROOM_TELEMETRY_WARN feature flags.
+ Does nothing when telemetry or warnings are disabled.
+ """
+ from headroom.telemetry.beacon import format_telemetry_notice
+
+ notice = format_telemetry_notice(prefix=" ")
+ if notice:
+ click.echo(notice)
+
+
+# Proxy health check (reused from evals/suite_runner.py pattern)
+
+
+def _check_proxy(port: int) -> bool:
+ """Check if Headroom proxy is running on given port."""
+ try:
+ with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+ s.settimeout(1)
+ s.connect(("127.0.0.1", port))
+ return True
+ except (TimeoutError, ConnectionRefusedError, OSError):
+ return False
+
+
+def _get_log_path() -> Path:
+ """Get path for proxy log file."""
+ from headroom import paths as _paths
+
+ log_dir = _paths.log_dir()
+ log_dir.mkdir(parents=True, exist_ok=True)
+ return log_dir / "proxy.log"
+
+
+def _start_proxy(
+ port: int,
+ *,
+ learn: bool = False,
+ memory: bool = False,
+ agent_type: str = "unknown",
+ code_graph: bool = False,
+ backend: str | None = None,
+ anyllm_provider: str | None = None,
+ region: str | None = None,
+) -> subprocess.Popen:
+ """Start Headroom proxy as a background subprocess.
+
+ Logs are written to ~/.headroom/logs/proxy.log to avoid pipe buffer
+ deadlocks (macOS pipe buffer is ~64KB — a busy proxy fills it quickly,
+ blocking the process).
+ """
+ cmd = [sys.executable, "-m", "headroom.cli", "proxy", "--port", str(port)]
+
+ # Forward HEADROOM_MODE env var so the proxy respects the user's mode choice
+ headroom_mode = os.environ.get("HEADROOM_MODE")
+ if headroom_mode:
+ cmd.extend(["--mode", headroom_mode])
+
+ # Forward --learn flag to proxy subprocess
+ if learn:
+ cmd.append("--learn")
+
+ # Forward --memory flag to proxy subprocess
+ if memory:
+ cmd.append("--memory")
+
+ # Forward --code-graph flag to proxy subprocess (live file watcher)
+ if code_graph:
+ cmd.append("--code-graph")
+
+ # Forward backend configuration to proxy subprocess
+ _backend = backend or os.environ.get("HEADROOM_BACKEND")
+ if _backend:
+ cmd.extend(["--backend", _backend])
+
+ _anyllm = anyllm_provider or os.environ.get("HEADROOM_ANYLLM_PROVIDER")
+ if _anyllm:
+ cmd.extend(["--anyllm-provider", _anyllm])
+
+ _region = region or os.environ.get("HEADROOM_REGION")
+ if _region:
+ cmd.extend(["--region", _region])
+
+ log_path = _get_log_path()
+ log_file = open(log_path, "a") # noqa: SIM115
+
+ # Ensure proxy subprocess uses UTF-8 (Windows defaults to cp1252)
+ proxy_env = os.environ.copy()
+ proxy_env["PYTHONIOENCODING"] = "utf-8"
+
+ # Tell the proxy which agent is being wrapped (for traffic learning output)
+ if agent_type != "unknown":
+ proxy_env["HEADROOM_AGENT_TYPE"] = agent_type
+ proxy_env.setdefault("HEADROOM_STACK", f"wrap_{agent_type}")
+
+ proc = subprocess.Popen(
+ cmd,
+ stdout=log_file,
+ stderr=log_file,
+ env=proxy_env,
+ )
+
+ # Wait for proxy to be ready (up to 45 seconds).
+ # ML components (Kompress, Magika, Tree-sitter) load synchronously before
+ # uvicorn binds the port. On slower machines this can take 20-30 seconds.
+ for _i in range(45):
+ time.sleep(1)
+ if _check_proxy(port):
+ click.echo(f" Logs: {log_path}")
+ return proc
+ # Check if process died
+ if proc.poll() is not None:
+ log_file.close()
+ # Read last few lines of log for error context
+ try:
+ tail = log_path.read_text()[-500:]
+ except Exception:
+ tail = "(no log output)"
+ raise RuntimeError(f"Proxy exited with code {proc.returncode}: {tail}")
+
+ proc.kill()
+ log_file.close()
+ raise RuntimeError(f"Proxy failed to start on port {port} within 45 seconds")
+
+
+def _setup_rtk(verbose: bool = False) -> Path | None:
+ """Ensure rtk is installed and hooks are registered."""
+ from headroom.rtk import get_rtk_path
+ from headroom.rtk.installer import ensure_rtk, register_claude_hooks
+
+ rtk_path = get_rtk_path()
+
+ if rtk_path:
+ if verbose:
+ click.echo(f" rtk found at {rtk_path}")
+ else:
+ click.echo(" Downloading rtk (Rust Token Killer)...")
+ rtk_path = ensure_rtk()
+ if rtk_path:
+ click.echo(f" rtk installed at {rtk_path}")
+ else:
+ click.echo(" rtk download failed — continuing without it")
+ return None
+
+ # Register hooks (idempotent)
+ if register_claude_hooks(rtk_path):
+ if verbose:
+ click.echo(" rtk hooks registered in Claude Code")
+ else:
+ click.echo(" rtk hook registration failed — continuing without it")
+
+ return rtk_path
+
+
+_CBM_MCP_SERVER_NAME = "codebase-memory-mcp"
+
+
+def _register_cbm_mcp_server(cbm_bin: str) -> None:
+ """Register codebase-memory-mcp as an MCP server in Claude Code.
+
+ Uses ``claude mcp add`` so the tools appear in ``/mcp`` automatically.
+ Idempotent — skips if already registered.
+ """
+ claude_cli = shutil.which("claude")
+ if not claude_cli:
+ return
+
+ # Check if already registered
+ check = subprocess.run(
+ [claude_cli, "mcp", "get", _CBM_MCP_SERVER_NAME],
+ capture_output=True,
+ text=True,
+ )
+ if check.returncode == 0:
+ return # Already registered
+
+ result = subprocess.run(
+ [claude_cli, "mcp", "add", _CBM_MCP_SERVER_NAME, "-s", "user", "--", cbm_bin],
+ capture_output=True,
+ text=True,
+ )
+ if result.returncode == 0:
+ click.echo(f" Code graph: registered {_CBM_MCP_SERVER_NAME} MCP server")
+ else:
+ pass # Non-critical — tools won't appear in /mcp but graph still works
+
+
+def _setup_code_graph(verbose: bool = False) -> bool:
+ """Ensure codebase-memory-mcp is installed, registered as MCP server, and project is indexed.
+
+ codebase-memory-mcp builds a knowledge graph of the codebase using
+ tree-sitter, enabling the LLM to query code structure (call chains,
+ function definitions, impact analysis) instead of reading entire files.
+
+ Steps:
+ 1. Download the binary if not already present.
+ 2. Register as an MCP server in Claude Code (``claude mcp add``).
+ 3. Index the current project (fast, idempotent).
+
+ With Claude Code's MCP Tool Search, the 14 graph tools add ~200 tokens
+ overhead per request (not the full ~1,915) — they're lazy-loaded.
+
+ Returns True if graph is ready, False if setup failed.
+ """
+ from headroom.graph.installer import ensure_cbm, get_cbm_path
+
+ cbm_path = get_cbm_path()
+ if not cbm_path:
+ click.echo(" Code graph: downloading codebase-memory-mcp...")
+ cbm_path = ensure_cbm()
+ if cbm_path:
+ click.echo(f" Code graph: installed at {cbm_path}")
+ else:
+ click.echo(" Code graph: download failed — skipping")
+ return False
+
+ cbm_bin = str(cbm_path)
+
+ # Register as MCP server so tools appear in /mcp
+ _register_cbm_mcp_server(cbm_bin)
+
+ # Index current project (fast — ~1s for most repos, idempotent)
+ project_dir = str(Path.cwd())
+ try:
+ result = subprocess.run(
+ [
+ cbm_bin,
+ "cli",
+ "index_repository",
+ json.dumps({"repo_path": project_dir, "mode": "fast"}),
+ ],
+ capture_output=True,
+ text=True,
+ timeout=30,
+ )
+ if result.returncode == 0:
+ # Parse node/edge counts from output
+ for line in result.stdout.splitlines():
+ if '"nodes"' in line:
+ try:
+ # Parse the JSON response to extract node/edge counts
+ import re
+
+ m_nodes = re.search(r'"nodes":(\d+)', line)
+ m_edges = re.search(r'"edges":(\d+)', line)
+ if m_nodes and m_edges:
+ nodes = int(m_nodes.group(1))
+ edges = int(m_edges.group(1))
+ click.echo(
+ f" Code graph: indexed ({nodes:,} symbols, "
+ f"{edges:,} relationships)"
+ )
+ else:
+ click.echo(" Code graph: indexed")
+ except (ValueError, AttributeError):
+ click.echo(" Code graph: indexed")
+ return True
+ click.echo(" Code graph: indexed")
+ return True
+ else:
+ if verbose:
+ click.echo(f" Code graph: indexing failed ({result.stderr[:100]})")
+ return False
+ except subprocess.TimeoutExpired:
+ click.echo(" Code graph: indexing timed out (will complete in background)")
+ return False
+ except Exception as e:
+ if verbose:
+ click.echo(f" Code graph: setup failed ({e})")
+ return False
+
+
+# rtk instructions for tools without hook support (Codex, Cursor, Aider).
+# These get injected into AGENTS.md / .cursorrules so the LLM voluntarily
+# uses rtk-prefixed commands. Kept concise to minimize instruction overhead.
+RTK_INSTRUCTIONS_BLOCK = """\
+<!-- headroom:rtk-instructions -->
+# RTK (Rust Token Killer) - Token-Optimized Commands
+
+When running shell commands, **always prefix with `rtk`**. This reduces context
+usage by 60-90% with zero behavior change. If rtk has no filter for a command,
+it passes through unchanged — so it is always safe to use.
+
+## Key Commands
+```bash
+# Git (59-80% savings)
+rtk git status rtk git diff rtk git log
+
+# Files & Search (60-75% savings)
+rtk ls <path> rtk read <file> rtk grep <pattern>
+rtk find <pattern> rtk diff <file>
+
+# Test (90-99% savings) — shows failures only
+rtk pytest tests/ rtk cargo test rtk test <cmd>
+
+# Build & Lint (80-90% savings) — shows errors only
+rtk tsc rtk lint rtk cargo build
+rtk prettier --check rtk mypy rtk ruff check
+
+# Analysis (70-90% savings)
+rtk err <cmd> rtk log <file> rtk json <file>
+rtk summary <cmd> rtk deps rtk env
+
+# GitHub (26-87% savings)
+rtk gh pr view <n> rtk gh run list rtk gh issue list
+
+# Infrastructure (85% savings)
+rtk docker ps rtk kubectl get rtk docker logs <c>
+
+# Package managers (70-90% savings)
+rtk pip list rtk pnpm install rtk npm run <script>
+```
+
+## Rules
+- In command chains, prefix each segment: `rtk git add . && rtk git commit -m "msg"`
+- For debugging, use raw command without rtk prefix
+- `rtk proxy <cmd>` runs command without filtering but tracks usage
+<!-- /headroom:rtk-instructions -->
+"""
+
+# Marker used to detect if instructions are already injected
+_RTK_MARKER = "<!-- headroom:rtk-instructions -->"
+
+# Memory MCP markers
+_MEMORY_MCP_MARKER = "# --- Headroom memory MCP (auto-injected) ---"
+_MEMORY_MCP_END = "# --- end Headroom memory ---"
+_MEMORY_AGENTS_MARKER = "<!-- headroom:memory-instructions -->"
+
+
+def _ensure_rtk_binary(verbose: bool = False) -> Path | None:
+ """Ensure rtk binary is installed (download if needed). No hook registration."""
+ from headroom.rtk import get_rtk_path
+ from headroom.rtk.installer import ensure_rtk
+
+ rtk_path = get_rtk_path()
+
+ if rtk_path:
+ if verbose:
+ click.echo(f" rtk found at {rtk_path}")
+ return rtk_path
+
+ click.echo(" Downloading rtk (Rust Token Killer)...")
+ rtk_path = ensure_rtk()
+ if rtk_path:
+ click.echo(f" rtk installed at {rtk_path}")
+ return rtk_path
+
+ click.echo(" rtk download failed — continuing without it")
+ return None
+
+
+def _prepare_wrap_rtk(verbose: bool = False, *, label: str | None = None) -> Path | None:
+ """Ensure rtk is present for host-bridged wrap flows without host-specific setup."""
+ if label:
+ click.echo(f" Preparing rtk for {label}...")
+ return _ensure_rtk_binary(verbose=verbose)
+
+
+def _inject_codex_provider_config(port: int) -> None:
+ """Inject a Headroom model provider into Codex's config.toml.
+
+ Codex ignores OPENAI_BASE_URL for WebSocket transport unless a custom
+ provider declares ``supports_websockets = true``. This writes a
+ ``[model_providers.headroom]`` section that routes both HTTP and WS
+ through the proxy, and sets ``model_provider = "headroom"``.
+
+ Safe to call multiple times — only writes if the section is missing
+ or the port changed.
+ """
+ config_dir = Path.home() / ".codex"
+ config_file = config_dir / "config.toml"
+
+ # model_provider must be a top-level TOML key (before any [section]).
+ # The [model_providers.headroom] table can go at the end.
+ top_level_marker = "# --- Headroom proxy (auto-injected by headroom wrap codex) ---"
+ top_level_block = f'{top_level_marker}\nmodel_provider = "headroom"\n'
+ provider_section = (
+ f"\n[model_providers.headroom]\n"
+ f'name = "OpenAI via Headroom proxy"\n'
+ f'base_url = "http://127.0.0.1:{port}/v1"\n'
+ f'env_key = "OPENAI_API_KEY"\n'
+ f"requires_openai_auth = true\n"
+ f"supports_websockets = true\n"
+ f"# --- end Headroom ---\n"
+ )
+
+ marker = top_level_marker
+ end_marker = "# --- end Headroom ---"
+
+ try:
+ config_dir.mkdir(parents=True, exist_ok=True)
+
+ if config_file.exists():
+ content = config_file.read_text()
+ if marker in content:
+ # Remove existing Headroom blocks entirely
+ start = content.index(marker)
+ end = content.index(end_marker) + len(end_marker)
+ content = content[:start].rstrip("\n") + content[end:].lstrip("\n")
+
+ # Strip any stale top-level model_provider left behind
+ import re
+
+ content = re.sub(r'\nmodel_provider\s*=\s*"headroom"\n', "\n", content)
+
+ # Place top-level key at the very beginning, provider table at the end
+ content = top_level_block + "\n" + content.strip() + "\n" + provider_section
+ else:
+ content = top_level_block + "\n" + provider_section
+
+ config_file.write_text(content)
+ click.echo(f" Codex config: injected Headroom provider (WS + HTTP) into {config_file}")
+ except Exception as e:
+ click.echo(f" Warning: could not update Codex config: {e}")
+
+
+def _inject_rtk_instructions(file_path: Path, verbose: bool = False) -> bool:
+ """Inject rtk instructions into a file (AGENTS.md, .cursorrules, etc.).
+
+ Idempotent — skips if marker already present. Appends to existing content.
+ Returns True if instructions were written.
+ """
+ if file_path.exists():
+ existing = file_path.read_text()
+ if _RTK_MARKER in existing:
+ if verbose:
+ click.echo(f" rtk instructions already in {file_path.name}")
+ return True
+ # Append to existing file
+ with open(file_path, "a") as f:
+ f.write("\n\n" + RTK_INSTRUCTIONS_BLOCK)
+ else:
+ file_path.parent.mkdir(parents=True, exist_ok=True)
+ file_path.write_text(RTK_INSTRUCTIONS_BLOCK)
+
+ click.echo(f" rtk instructions injected into {file_path}")
+ return True
+
+
+def _inject_memory_mcp_config(db_path: str, user_id: str) -> None:
+ """Register headroom memory as an MCP server in Codex's config.toml.
+
+ Idempotent — replaces existing section if present.
+ """
+ import sys
+
+ config_dir = Path.home() / ".codex"
+ config_file = config_dir / "config.toml"
+
+ # Use forward slashes in TOML paths (works on all platforms, avoids
+ # backslash escaping issues on Windows)
+ python_bin = sys.executable.replace("\\", "/")
+ db_path_toml = db_path.replace("\\", "/")
+ mcp_section = (
+ f"\n{_MEMORY_MCP_MARKER}\n"
+ f"[mcp_servers.headroom_memory]\n"
+ f'command = "{python_bin}"\n'
+ f'args = ["-m", "headroom.memory.mcp_server", "--db", "{db_path_toml}", "--user", "{user_id}"]\n'
+ f"startup_timeout_sec = 30\n"
+ f"tool_timeout_sec = 30\n"
+ f"{_MEMORY_MCP_END}\n"
+ )
+
+ try:
+ config_dir.mkdir(parents=True, exist_ok=True)
+
+ if config_file.exists():
+ content = config_file.read_text()
+ if _MEMORY_MCP_MARKER in content:
+ start = content.index(_MEMORY_MCP_MARKER)
+ end = content.index(_MEMORY_MCP_END) + len(_MEMORY_MCP_END)
+ content = content[:start].rstrip("\n") + mcp_section + content[end:].lstrip("\n")
+ else:
+ content = content.rstrip() + "\n" + mcp_section
+ else:
+ content = mcp_section
+
+ config_file.write_text(content)
+ click.echo(f" Memory MCP: registered in {config_file}")
+ except Exception as e:
+ click.echo(f" Warning: could not register memory MCP: {e}")
+
+
+def _inject_memory_agents_md(file_path: Path) -> bool:
+ """Inject memory usage guidance into AGENTS.md.
+
+ Idempotent — skips if marker already present.
+ """
+ memory_block = (
+ f"{_MEMORY_AGENTS_MARKER}\n"
+ "## Memory\n\n"
+ "Use the `headroom_memory` MCP server for persistent cross-session knowledge.\n\n"
+ "**Before** answering questions about prior decisions, conventions, project context,\n"
+ "architecture, user preferences, org info, codenames, debugging history, or anything\n"
+ "from past sessions — call `memory_search` first.\n\n"
+ "**After** making durable decisions, discovering conventions, or learning important\n"
+ "facts — call `memory_save` to persist them for future sessions.\n\n"
+ "Memory is your first source of truth for anything not visible in the current conversation.\n"
+ )
+
+ if file_path.exists():
+ existing = file_path.read_text()
+ if _MEMORY_AGENTS_MARKER in existing:
+ return True # Already injected
+ with open(file_path, "a") as f:
+ f.write("\n\n" + memory_block)
+ else:
+ file_path.parent.mkdir(parents=True, exist_ok=True)
+ file_path.write_text(memory_block)
+
+ click.echo(f" Memory guidance injected into {file_path.name}")
+ return True
+
+
+def _resolve_copilot_provider_type(backend: str | None, provider_type: str) -> str:
+ """Resolve Copilot BYOK provider type for the current proxy backend."""
+ return _copilot_resolve_provider_type(backend, provider_type)
+
+
+def _query_proxy_config(port: int) -> dict[str, Any] | None:
+ """Query the running proxy's feature configuration via /health.
+
+ Returns a dict with keys like backend, optimize, cache, rate_limit,
+ memory, learn, code_graph, pid. Returns None if unreachable or the
+ response lacks a config block.
+ """
+ return _copilot_query_proxy_config(port)
+
+
+def _detect_running_proxy_backend(port: int) -> str | None:
+ """Read the backend of an already-running proxy from its health endpoint."""
+ return _copilot_detect_running_proxy_backend(port)
+
+
+def _kill_proxy_by_pid(pid: int, port: int) -> bool:
+ """Terminate a proxy process by PID and wait for the port to free up.
+
+ Sends SIGTERM first, falls back to SIGKILL after 5 seconds.
+ Returns True if the port is free afterwards, False otherwise.
+ """
+ try:
+ os.kill(pid, signal.SIGTERM)
+ except ProcessLookupError:
+ pass # Already gone
+ except PermissionError:
+ click.echo(f" Warning: No permission to kill proxy PID {pid}")
+ return False
+
+ # Wait for port to free (up to 5 seconds)
+ for _ in range(50):
+ time.sleep(0.1)
+ if not _check_proxy(port):
+ return True
+
+ # SIGTERM didn't work — escalate to SIGKILL (Unix) or terminate (Windows)
+ try:
+ _kill_signal = getattr(signal, "SIGKILL", signal.SIGTERM)
+ os.kill(pid, _kill_signal)
+ except (ProcessLookupError, PermissionError):
+ pass
+
+ for _ in range(20):
+ time.sleep(0.1)
+ if not _check_proxy(port):
+ return True
+
+ return False
+
+
+def _find_persistent_manifest(port: int) -> Any:
+ """Return a matching persistent deployment manifest for the requested port."""
+ from headroom.install.state import list_manifests
+
+ manifests = [manifest for manifest in list_manifests() if manifest.port == port]
+ manifests.sort(key=lambda manifest: (manifest.profile != "default", manifest.profile))
+ return manifests[0] if manifests else None
+
+
+def _recover_persistent_proxy(port: int) -> bool:
+ """Start or recover a matching persistent deployment for the requested port."""
+ from headroom.install.health import probe_ready
+ from headroom.install.models import InstallPreset, SupervisorKind
+ from headroom.install.runtime import start_detached_agent, start_persistent_docker, wait_ready
+ from headroom.install.supervisors import start_supervisor
+
+ manifest = _find_persistent_manifest(port)
+ if manifest is None:
+ return False
+
+ if probe_ready(manifest.health_url):
+ click.echo(f" Reusing persistent deployment '{manifest.profile}' on port {port}")
+ return True
+
+ if manifest.supervisor_kind == SupervisorKind.TASK.value:
+ click.echo(
+ f" Warning: task-based deployment '{manifest.profile}' cannot be auto-recovered via wrap"
+ )
+ return False
+
+ click.echo(f" Recovering persistent deployment '{manifest.profile}' on port {port}...")
+ try:
+ if manifest.preset == InstallPreset.PERSISTENT_DOCKER.value:
+ start_persistent_docker(manifest)
+ elif manifest.supervisor_kind == SupervisorKind.SERVICE.value:
+ start_supervisor(manifest)
+ else:
+ start_detached_agent(manifest.profile)
+ except Exception as exc:
+ click.echo(
+ f" Warning: could not recover persistent deployment '{manifest.profile}': {exc}"
+ )
+ return False
+
+ if wait_ready(manifest, timeout_seconds=45):
+ click.echo(f" Recovered persistent deployment '{manifest.profile}' on port {port}")
+ return True
+
+ click.echo(f" Warning: persistent deployment '{manifest.profile}' did not become ready")
+ return False
+
+
+def _copilot_model_configured(copilot_args: tuple[str, ...], env: dict[str, str]) -> bool:
+ """Return True when Copilot BYOK model selection is configured."""
+ return _copilot_model_configured_impl(copilot_args, env)
+
+
+def _ensure_proxy(
+ port: int,
+ no_proxy: bool,
+ *,
+ learn: bool = False,
+ memory: bool = False,
+ agent_type: str = "unknown",
+ code_graph: bool = False,
+ backend: str | None = None,
+ anyllm_provider: str | None = None,
+ region: str | None = None,
+) -> subprocess.Popen | None:
+ """Start or verify proxy. Returns process handle if we started it."""
+ if not no_proxy:
+ manifest = _find_persistent_manifest(port)
+ if manifest is not None:
+ from headroom.install.health import probe_ready
+
+ if probe_ready(manifest.health_url):
+ click.echo(f" Proxy already running on port {port}")
+ return None
+ if _recover_persistent_proxy(port):
+ return None
+ raise click.ClickException(
+ f"Persistent deployment '{manifest.profile}' on port {port} is not healthy."
+ )
+
+ if _check_proxy(port):
+ # Proxy is running — check if it has the features we need
+ needs_restart = False
+ running_config = _query_proxy_config(port)
+
+ if running_config is not None:
+ missing = []
+ if memory and not running_config.get("memory"):
+ missing.append("memory")
+ if learn and not running_config.get("learn"):
+ missing.append("learn")
+ if code_graph and not running_config.get("code_graph"):
+ missing.append("code_graph")
+
+ if missing:
+ needs_restart = True
+ flags_str = ", ".join(f"--{f.replace('_', '-')}" for f in missing)
+ click.echo(f" Proxy on port {port} is missing: {flags_str}")
+ click.echo(" Restarting proxy with upgraded configuration...")
+
+ # Merge: keep features the running proxy already has
+ memory = memory or bool(running_config.get("memory"))
+ learn = learn or bool(running_config.get("learn"))
+ code_graph = code_graph or bool(running_config.get("code_graph"))
+
+ proxy_pid = running_config.get("pid")
+ if proxy_pid is not None:
+ if not _kill_proxy_by_pid(int(proxy_pid), port):
+ raise click.ClickException(
+ f"Failed to stop existing proxy (PID {proxy_pid}) on port {port}. "
+ "Stop it manually and retry."
+ )
+ else:
+ click.echo(
+ " Warning: Running proxy does not expose PID. "
+ "Cannot restart automatically."
+ )
+ click.echo(
+ f" Please stop the proxy on port {port} manually "
+ f"and rerun with {flags_str}."
+ )
+ return None
+
+ if not needs_restart:
+ click.echo(f" Proxy already running on port {port}")
+ return None
+
+ # Start (or restart) the proxy with the requested flags
+ click.echo(f" Starting Headroom proxy on port {port}...")
+ try:
+ proc = _start_proxy(
+ port,
+ learn=learn,
+ memory=memory,
+ agent_type=agent_type,
+ code_graph=code_graph,
+ backend=backend,
+ anyllm_provider=anyllm_provider,
+ region=region,
+ )
+ click.echo(f" Proxy ready on http://127.0.0.1:{port}")
+ return proc
+ except RuntimeError as e:
+ click.echo(f" Error: {e}")
+ raise SystemExit(1) from e
+ else:
+ if not _check_proxy(port):
+ click.echo(f" Warning: No proxy detected on port {port}")
+ return None
+
+
+def _make_cleanup(proxy_proc_holder: list, port: int = 8787) -> Any:
+ """Create a cleanup function that terminates the proxy on exit.
+
+ Only kills the proxy if no other headroom-wrapped clients are using it.
+ Checks by looking for other processes with ANTHROPIC_BASE_URL or
+ OPENAI_BASE_URL pointing at our port.
+ """
+
+ def _other_clients_exist() -> bool:
+ """Check if other processes are using this proxy."""
+ try:
+ # Count headroom wrap processes (excluding ourselves)
+ result = subprocess.run(
+ ["pgrep", "-f", f"127.0.0.1:{port}"],
+ capture_output=True,
+ text=True,
+ )
+ pids = [p.strip() for p in result.stdout.strip().split("\n") if p.strip()]
+ my_pid = str(os.getpid())
+ other_pids = [p for p in pids if p != my_pid]
+ return len(other_pids) > 0
+ except Exception:
+ return False # If we can't check, assume no others
+
+ def cleanup(signum: int | None = None, frame: Any = None) -> None:
+ proc = proxy_proc_holder[0] if proxy_proc_holder else None
+ if proc and proc.poll() is None:
+ if _other_clients_exist():
+ # Other clients still using the proxy — leave it running
+ return
+ proc.terminate()
+ try:
+ proc.wait(timeout=5)
+ except subprocess.TimeoutExpired:
+ proc.kill()
+
+ return cleanup
+
+
+def _launch_tool(
+ binary: str,
+ args: tuple,
+ env: dict[str, str],
+ port: int,
+ no_proxy: bool,
+ tool_label: str,
+ env_vars_display: list[str],
+ *,
+ learn: bool = False,
+ memory: bool = False,
+ agent_type: str = "unknown",
+ code_graph: bool = False,
+ backend: str | None = None,
+ anyllm_provider: str | None = None,
+ region: str | None = None,
+) -> None:
+ """Common logic: start proxy, launch tool, clean up."""
+ proxy_holder: list[subprocess.Popen | None] = [None]
+ cleanup = _make_cleanup(proxy_holder, port)
+ signal.signal(signal.SIGINT, cleanup)
+ signal.signal(signal.SIGTERM, cleanup)
+
+ try:
+ click.echo()
+ padded = f"HEADROOM WRAP: {tool_label}".center(47)
+ click.echo(" ╔═══════════════════════════════════════════════╗")
+ click.echo(f" ║{padded}║")
+ click.echo(" ╚═══════════════════════════════════════════════╝")
+ click.echo()
+
+ proxy_holder[0] = _ensure_proxy(
+ port,
+ no_proxy,
+ learn=learn,
+ memory=memory,
+ agent_type=agent_type,
+ code_graph=code_graph,
+ backend=backend,
+ anyllm_provider=anyllm_provider,
+ region=region,
+ )
+
+ if code_graph:
+ _setup_code_graph(verbose=False)
+
+ click.echo()
+ click.echo(f" Launching {tool_label} (API routed through Headroom)...")
+ for var in env_vars_display:
+ click.echo(f" {var}")
+ if args:
+ click.echo(f" Extra args: {' '.join(args)}")
+ _print_telemetry_notice()
+ click.echo()
+
+ result = subprocess.run([binary, *args], env=env)
+ raise SystemExit(result.returncode)
+
+ except SystemExit:
+ raise
+ except Exception as e:
+ click.echo(f" Error: {e}")
+ raise SystemExit(1) from e
+ finally:
+ cleanup()
+
+
+def _run_checked(
+ cmd: list[str],
+ *,
+ cwd: Path | None = None,
+ action: str,
+) -> subprocess.CompletedProcess[str]:
+ """Run subprocess and raise a ClickException with actionable context on failure."""
+ try:
+ return subprocess.run(
+ cmd,
+ cwd=str(cwd) if cwd else None,
+ check=True,
+ capture_output=True,
+ text=True,
+ encoding="utf-8",
+ errors="replace",
+ )
+ except FileNotFoundError as e:
+ raise click.ClickException(f"{action} failed: command not found: {cmd[0]}") from e
+ except subprocess.CalledProcessError as e:
+ stderr = (e.stderr or "").strip()
+ stdout = (e.stdout or "").strip()
+ details = stderr or stdout or f"exit code {e.returncode}"
+ raise click.ClickException(f"{action} failed: {details}") from e
+
+
+def _resolve_openclaw_extensions_dir(openclaw_bin: str) -> Path:
+ """Resolve OpenClaw extension root from active config file path."""
+ result = _run_checked([openclaw_bin, "config", "file"], action="openclaw config file")
+ lines = result.stdout.strip().splitlines()
+ config_path_str = lines[-1].strip() if lines else ""
+ if not config_path_str:
+ raise click.ClickException(
+ "Unable to resolve OpenClaw config path from `openclaw config file`."
+ )
+ config_path = Path(config_path_str).expanduser()
+ return config_path.parent / "extensions"
+
+
+def _normalize_openclaw_gateway_provider_ids(provider_ids: tuple[str, ...] | None) -> list[str]:
+ """Normalize configured OpenClaw provider ids, defaulting to openai-codex."""
+ return _normalize_openclaw_gateway_provider_ids_impl(provider_ids)
+
+
+def _read_openclaw_config_value(openclaw_bin: str, path: str) -> Any | None:
+ """Read an OpenClaw config value when present, returning None on missing paths."""
+ result = subprocess.run(
+ [openclaw_bin, "config", "get", path],
+ capture_output=True,
+ text=True,
+ encoding="utf-8",
+ errors="replace",
+ )
+ if result.returncode != 0:
+ return None
+
+ output = result.stdout.strip()
+ if not output:
+ return None
+
+ try:
+ return json.loads(output)
+ except json.JSONDecodeError:
+ return output
+
+
+def _decode_openclaw_entry_json(raw_value: str | None) -> Any | None:
+ """Decode a JSON payload captured from `openclaw config get` when available."""
+ return _decode_openclaw_entry_json_impl(raw_value)
+
+
+def _build_openclaw_plugin_entry(
+ *,
+ existing_entry: Any,
+ proxy_port: int,
+ startup_timeout_ms: int,
+ python_path: str | None,
+ no_auto_start: bool,
+ gateway_provider_ids: tuple[str, ...] | None,
+ enabled: bool,
+) -> dict[str, object]:
+ """Merge managed Headroom plugin settings with any existing entry payload."""
+ return _build_openclaw_plugin_entry_impl(
+ existing_entry=existing_entry,
+ proxy_port=proxy_port,
+ startup_timeout_ms=startup_timeout_ms,
+ python_path=python_path,
+ no_auto_start=no_auto_start,
+ gateway_provider_ids=gateway_provider_ids,
+ enabled=enabled,
+ )
+
+
+def _build_openclaw_unwrap_entry(existing_entry: Any) -> dict[str, object]:
+ """Disable the managed plugin while preserving unrelated user config."""
+ return _build_openclaw_unwrap_entry_impl(existing_entry)
+
+
+def _write_openclaw_plugin_entry(openclaw_bin: str, entry: dict[str, object]) -> None:
+ """Persist the Headroom plugin config entry."""
+ _run_checked(
+ [
+ openclaw_bin,
+ "config",
+ "set",
+ "plugins.entries.headroom",
+ json.dumps(entry, separators=(",", ":")),
+ "--strict-json",
+ ],
+ action="openclaw config set plugins.entries.headroom",
+ )
+
+
+def _set_openclaw_context_engine_slot(openclaw_bin: str, engine_id: str) -> None:
+ """Persist the selected OpenClaw context engine slot."""
+ _run_checked(
+ [
+ openclaw_bin,
+ "config",
+ "set",
+ "plugins.slots.contextEngine",
+ json.dumps(engine_id),
+ "--strict-json",
+ ],
+ action="openclaw config set plugins.slots.contextEngine",
+ )
+
+
+def _restart_or_start_openclaw_gateway(openclaw_bin: str) -> tuple[str, str]:
+ """Restart the gateway when running, otherwise start it."""
+ restart_result = subprocess.run(
+ [openclaw_bin, "gateway", "restart"],
+ capture_output=True,
+ text=True,
+ encoding="utf-8",
+ errors="replace",
+ )
+ if restart_result.returncode == 0:
+ output = restart_result.stdout.strip() or restart_result.stderr.strip()
+ return "restarted", output
+
+ start_result = _run_checked(
+ [openclaw_bin, "gateway", "start"],
+ action="openclaw gateway start",
+ )
+ output = start_result.stdout.strip() or start_result.stderr.strip()
+ return "started", output
+
+
+def _copy_openclaw_plugin_into_extensions(
+ *,
+ plugin_dir: Path,
+ openclaw_bin: str,
+) -> Path:
+ """Fallback install path when `openclaw plugins install` is blocked on linked source."""
+ dist_dir = plugin_dir / "dist"
+ if not dist_dir.exists():
+ raise click.ClickException(
+ f"Plugin dist folder missing at {dist_dir}. Build the plugin first."
+ )
+ hook_shim_dir = plugin_dir / "hook-shim"
+ if not hook_shim_dir.exists():
+ raise click.ClickException(
+ f"Plugin hook-shim folder missing at {hook_shim_dir}. Build the plugin first."
+ )
+
+ extensions_dir = _resolve_openclaw_extensions_dir(openclaw_bin)
+ target_dir = extensions_dir / "headroom"
+ target_dist = target_dir / "dist"
+ target_hook_shim = target_dir / "hook-shim"
+ target_dir.mkdir(parents=True, exist_ok=True)
+ if target_dist.exists():
+ shutil.rmtree(target_dist)
+ if target_hook_shim.exists():
+ shutil.rmtree(target_hook_shim)
+ shutil.copytree(dist_dir, target_dist)
+ shutil.copytree(hook_shim_dir, target_hook_shim)
+
+ for filename in ("openclaw.plugin.json", "package.json", "README.md"):
+ source = plugin_dir / filename
+ if source.exists():
+ shutil.copy2(source, target_dir / filename)
+
+ return target_dir
+
+
+def wrap() -> None:
+ """Wrap CLI tools to run through Headroom.
+
+ \b
+ Starts a Headroom proxy, configures the environment, and launches
+ the target tool so all API calls route through Headroom automatically.
+
+ \b
+ Supported tools:
+ headroom wrap claude # Claude Code (Anthropic)
+ headroom wrap copilot -- --model claude-sonnet-4-20250514
+ headroom wrap codex # OpenAI Codex CLI
+ headroom wrap aider # Aider
+ headroom wrap cursor # Cursor (prints config instructions)
+ headroom wrap openclaw # OpenClaw plugin bootstrap
+ """
+
+
+def unwrap() -> None:
+ """Undo durable Headroom wrapping for supported tools."""
+
+
+# =============================================================================
+# Claude Code
+# =============================================================================
+
+
[email protected](context_settings={"ignore_unknown_options": True})
[email protected]("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)")
[email protected]("--no-rtk", is_flag=True, help="Skip rtk installation and hook registration")
+ "--code-graph",
+ is_flag=True,
+ help="Enable code graph indexing via codebase-memory-mcp (optional)",
+)
[email protected]("--no-proxy", is_flag=True, help="Skip proxy startup (use existing proxy)")
+ "--learn", is_flag=True, help="Enable live traffic learning (patterns saved to MEMORY.md)"
+)
[email protected]("--memory", is_flag=True, help="Enable persistent cross-session memory")
[email protected]("--verbose", "-v", is_flag=True, help="Verbose output")
[email protected]("--prepare-only", is_flag=True, hidden=True)
[email protected]("claude_args", nargs=-1, type=click.UNPROCESSED)
+def claude(
+ port: int,
+ no_rtk: bool,
+ code_graph: bool,
+ no_proxy: bool,
+ learn: bool,
+ memory: bool,
+ verbose: bool,
+ prepare_only: bool,
+ claude_args: tuple,
+) -> None:
+ """Launch Claude Code through Headroom proxy.
+
+ \b
+ Sets ANTHROPIC_BASE_URL to route all Anthropic API calls through Headroom.
+ All unknown flags are passed through to claude (e.g. --resume, --model).
+
+ \b
+ Examples:
+ headroom wrap claude # Start everything
+ headroom wrap claude --memory # With persistent memory
+ headroom wrap claude --resume <id> # Resume a session
+ headroom wrap claude -- -p # Claude in print mode
+ headroom wrap claude --code-graph # With code graph intelligence
+ headroom wrap claude --no-rtk # Skip rtk (proxy only)
+ """
+ if prepare_only:
+ if not no_rtk:
+ _prepare_wrap_rtk(verbose=verbose, label="Claude")
+ return
+
+ claude_bin = shutil.which("claude")
+ if not claude_bin:
+ click.echo("Error: 'claude' not found in PATH.")
+ click.echo("Install Claude Code: https://docs.anthropic.com/en/docs/claude-code")
+ raise SystemExit(1)
+
+ # Setup rtk before launching (Claude-specific)
+ proxy_holder: list[subprocess.Popen | None] = [None]
+ cleanup = _make_cleanup(proxy_holder, port)
+ signal.signal(signal.SIGINT, cleanup)
+ signal.signal(signal.SIGTERM, cleanup)
+
+ # Memory sync BEFORE proxy startup — sync headroom DB ↔ Claude's files
+ if memory:
+ try:
+ import subprocess as _sp
+
+ mem_dir = Path.cwd() / ".headroom"
+ mem_dir.mkdir(parents=True, exist_ok=True)
+ _sync_db = str(mem_dir / "memory.db")
+ _sync_user = os.environ.get("USER", os.environ.get("USERNAME", "default"))
+
+ click.echo(f" Syncing memory (user={_sync_user})...")
+ sync_result = _sp.run(
+ [
+ sys.executable,
+ "-m",
+ "headroom.memory.sync",
+ "--db",
+ _sync_db,
+ "--user",
+ _sync_user,
+ "--agent",
+ "claude",
+ "--force",
+ ],
+ capture_output=True,
+ text=True,
+ timeout=30,
+ )
+ if sync_result.returncode == 0 and sync_result.stdout.strip():
+ import json as _json
+
+ stats = _json.loads(sync_result.stdout.strip().split("\n")[-1])
+ imp, exp, ms = stats["imported"], stats["exported"], stats["ms"]
+ if imp or exp:
+ click.echo(f" Memory synced: {imp} imported, {exp} exported ({ms}ms)")
+ else:
+ click.echo(f" Memory: up to date ({ms}ms)")
+ elif sync_result.returncode != 0:
+ click.echo(f" Warning: memory sync error: {sync_result.stderr[-200:]}")
+ except Exception as e:
+ click.echo(f" Warning: memory sync failed: {e}")
+
+ try:
+ click.echo()
+ click.echo(" ╔═══════════════════════════════════════════════╗")
+ click.echo(" ║ HEADROOM WRAP: CLAUDE ║")
+ click.echo(" ╚═══════════════════════════════════════════════╝")
+ click.echo()
+
+ proxy_holder[0] = _ensure_proxy(
+ port, no_proxy, learn=learn, memory=memory, agent_type="claude", code_graph=code_graph
+ )
+
+ if not no_rtk:
+ click.echo(" Setting up rtk...")
+ _setup_rtk(verbose=verbose)
+ elif verbose:
+ click.echo(" Skipping rtk (--no-rtk)")
+
+ if code_graph:
+ _setup_code_graph(verbose=verbose)
+
+ click.echo()
+ click.echo(" Launching Claude Code (API routed through Headroom)...")
+ click.echo(f" ANTHROPIC_BASE_URL={_claude_proxy_base_url(port)}")
+ if claude_args:
+ click.echo(f" Extra args: {' '.join(claude_args)}")
+ _print_telemetry_notice()
+ click.echo()
+
+ env = os.environ.copy()
+ env["ANTHROPIC_BASE_URL"] = _claude_proxy_base_url(port)
+
+ result = subprocess.run([claude_bin, *claude_args], env=env)
+ raise SystemExit(result.returncode)
+
+ except SystemExit:
+ raise
+ except Exception as e:
+ click.echo(f" Error: {e}")
+ raise SystemExit(1) from e
+ finally:
+ cleanup()
+
+
+# =============================================================================
+# GitHub Copilot CLI
+# =============================================================================
+
+
[email protected](context_settings={"ignore_unknown_options": True})
[email protected]("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)")
+ "--no-rtk",
+ is_flag=True,
+ help="Skip rtk installation and Copilot instructions injection",
+)
[email protected]("--no-proxy", is_flag=True, help="Skip proxy startup (use existing proxy)")
+ "--backend",
+ default=None,
+ help="API backend for the proxy: 'anthropic', 'anyllm', 'litellm-vertex', etc. (env: HEADROOM_BACKEND)",
+)
+ "--anyllm-provider",
+ default=None,
+ help="Provider for any-llm backend: openai, mistral, groq, etc. (env: HEADROOM_ANYLLM_PROVIDER)",
+)
+ "--region", default=None, help="Cloud region for Bedrock/Vertex (env: HEADROOM_REGION)"
+)
+ "--provider-type",
+ type=click.Choice(["auto", "anthropic", "openai"]),
+ default="auto",
+ show_default=True,
+ help="Copilot BYOK provider mode. 'auto' uses anthropic for the default proxy backend and openai for translated backends.",
+)
+ "--wire-api",
+ type=click.Choice(["completions", "responses"]),
+ default=None,
+ help="OpenAI-compatible Copilot wire API. Defaults to 'completions' when provider-type resolves to openai.",
+)
[email protected]("--memory", is_flag=True, help="Enable persistent cross-session memory")
[email protected]("--verbose", "-v", is_flag=True, help="Verbose output")
[email protected]("copilot_args", nargs=-1, type=click.UNPROCESSED)
+def copilot(
+ port: int,
+ no_rtk: bool,
+ no_proxy: bool,
+ backend: str | None,
+ anyllm_provider: str | None,
+ region: str | None,
+ provider_type: str,
+ wire_api: str | None,
+ memory: bool,
+ verbose: bool,
+ copilot_args: tuple[str, ...],
+) -> None:
+ """Launch GitHub Copilot CLI through Headroom proxy.
+
+ \b
+ Configures Copilot CLI BYOK provider variables so Copilot routes through
+ the local Headroom proxy. In auto mode, the wrapper uses Anthropic-style
+ routing for the stock proxy backend and OpenAI-compatible routing for
+ translated backends such as any-llm and LiteLLM.
+
+ \b
+ Examples:
+ headroom wrap copilot -- --model claude-sonnet-4-20250514
+ headroom wrap copilot --backend anyllm --anyllm-provider groq -- --model gpt-4o
+ headroom wrap copilot --provider-type openai --wire-api responses -- --model gpt-5.4
+ headroom wrap copilot --no-rtk -- --prompt "explain this file"
+ """
+ copilot_bin = shutil.which("copilot")
+ if not copilot_bin:
+ click.echo("Error: 'copilot' not found in PATH.")
+ click.echo(
+ "Install GitHub Copilot CLI: "
+ "https://docs.github.com/en/copilot/how-tos/copilot-cli/set-up-copilot-cli/install-copilot-cli"
+ )
+ raise SystemExit(1)
+
+ effective_backend = backend or os.environ.get("HEADROOM_BACKEND")
+ if _check_proxy(port):
+ running_backend = _detect_running_proxy_backend(port)
+ if effective_backend and running_backend and effective_backend != running_backend:
+ raise click.ClickException(
+ f"Proxy already running on port {port} with backend '{running_backend}'. "
+ f"Stop it or rerun with --backend {running_backend}."
+ )
+ effective_backend = running_backend or effective_backend
+
+ effective_provider_type = _resolve_copilot_provider_type(effective_backend, provider_type)
+ _validate_copilot_configuration(
+ provider_type=effective_provider_type,
+ wire_api=wire_api,
+ backend=effective_backend,
+ )
+
+ if not no_rtk:
+ click.echo(" Setting up rtk for Copilot...")
+ rtk_path = _ensure_rtk_binary(verbose=verbose)
+ if rtk_path:
+ copilot_instructions = Path.cwd() / ".github" / "copilot-instructions.md"
+ _inject_rtk_instructions(copilot_instructions, verbose=verbose)
+
+ env, env_vars_display = _build_copilot_launch_env(
+ port=port,
+ provider_type=effective_provider_type,
+ wire_api=wire_api,
+ environ=os.environ,
+ )
+
+ if not env.get("COPILOT_PROVIDER_API_KEY"):
+ src = _copilot_provider_key_source(effective_provider_type)
+ click.echo(
+ f"\n Error: Copilot BYOK mode requires a provider API key.\n"
+ f" `headroom wrap copilot` uses Copilot's BYOK mode, which bypasses GitHub's\n"
+ f" Copilot API and routes requests directly to the model provider through the\n"
+ f" Headroom proxy. A GitHub Copilot subscription alone is not sufficient.\n\n"
+ f" Set one of:\n"
+ f" export {src}=sk-... # recommended\n"
+ f" export COPILOT_PROVIDER_API_KEY=sk-... # also works\n"
+ )
+ raise SystemExit(1)
+
+ if not _copilot_model_configured(copilot_args, env):
+ click.echo(
+ " Note: Copilot BYOK requires a model. Pass `--model <name>` "
+ "or set `COPILOT_MODEL` / `COPILOT_PROVIDER_MODEL_ID`."
+ )
+
+ _launch_tool(
+ binary=copilot_bin,
+ args=copilot_args,
+ env=env,
+ port=port,
+ no_proxy=no_proxy,
+ tool_label="COPILOT",
+ env_vars_display=env_vars_display,
+ learn=False,
+ memory=memory,
+ agent_type="copilot",
+ backend=backend,
+ anyllm_provider=anyllm_provider,
+ region=region,
+ )
+
+
+# =============================================================================
+# OpenAI Codex CLI
+# =============================================================================
+
+
[email protected](context_settings={"ignore_unknown_options": True})
[email protected]("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)")
[email protected]("--no-rtk", is_flag=True, help="Skip rtk installation and AGENTS.md injection")
+ "--code-graph",
+ is_flag=True,
+ help="Enable code graph indexing via codebase-memory-mcp (optional)",
+)
[email protected]("--no-proxy", is_flag=True, help="Skip proxy startup (use existing proxy)")
+ "--learn", is_flag=True, help="Enable live traffic learning (patterns saved to AGENTS.md)"
+)
+ "--backend",
+ default=None,
+ help="API backend for the proxy: 'anthropic', 'anyllm', 'litellm-vertex', etc. (env: HEADROOM_BACKEND)",
+)
+ "--anyllm-provider",
+ default=None,
+ help="Provider for any-llm backend: openai, mistral, groq, etc. (env: HEADROOM_ANYLLM_PROVIDER)",
+)
+ "--region", default=None, help="Cloud region for Bedrock/Vertex (env: HEADROOM_REGION)"
+)
[email protected]("--memory", is_flag=True, help="Enable persistent cross-session memory")
[email protected]("--verbose", "-v", is_flag=True, help="Verbose output")
[email protected]("--prepare-only", is_flag=True, hidden=True)
[email protected]("codex_args", nargs=-1, type=click.UNPROCESSED)
+def codex(
+ port: int,
+ no_rtk: bool,
+ code_graph: bool,
+ no_proxy: bool,
+ learn: bool,
+ memory: bool,
+ backend: str | None,
+ anyllm_provider: str | None,
+ region: str | None,
+ verbose: bool,
+ prepare_only: bool,
+ codex_args: tuple,
+) -> None:
+ """Launch OpenAI Codex CLI through Headroom proxy.
+
+ \b
+ Sets OPENAI_BASE_URL to route all OpenAI API calls through Headroom.
+ Installs rtk and injects instructions into AGENTS.md so Codex uses
+ token-optimized commands (60-90% savings on shell output).
+
+ \b
+ Examples:
+ headroom wrap codex # Start proxy + rtk + codex
+ headroom wrap codex -- "fix the bug" # Pass prompt to codex
+ headroom wrap codex --no-rtk # Skip rtk setup
+ headroom wrap codex --port 9999 # Custom proxy port
+ headroom wrap codex --backend anyllm --anyllm-provider groq
+ """
+ # Setup rtk for Codex (binary + AGENTS.md instructions, no hooks)
+ if not no_rtk:
+ click.echo(" Setting up rtk for Codex...")
+ rtk_path = _ensure_rtk_binary(verbose=verbose)
+ if rtk_path:
+ # Inject into project AGENTS.md (Codex reads this automatically)
+ agents_md = Path.cwd() / "AGENTS.md"
+ _inject_rtk_instructions(agents_md, verbose=verbose)
+
+ # Also inject into global ~/.codex/AGENTS.md
+ global_agents = Path.home() / ".codex" / "AGENTS.md"
+ _inject_rtk_instructions(global_agents, verbose=verbose)
+
+ # Setup memory MCP server for Codex (native tool integration)
+ if memory:
+ click.echo(" Setting up memory for Codex...")
+ mem_dir = Path.cwd() / ".headroom"
+ mem_dir.mkdir(parents=True, exist_ok=True)
+ db_path = str(mem_dir / "memory.db")
+ mem_user = os.environ.get("USER", os.environ.get("USERNAME", "default"))
+
+ # Register MCP server in Codex config
+ _inject_memory_mcp_config(db_path, mem_user)
+
+ # Inject memory guidance into project AGENTS.md
+ agents_md = Path.cwd() / "AGENTS.md"
+ _inject_memory_agents_md(agents_md)
+
+ # Sync Claude's memories → DB so MCP search finds them
+ try:
+ import asyncio
+
+ from headroom.memory.backends.local import LocalBackend, LocalBackendConfig
+ from headroom.memory.sync import sync_import
+ from headroom.memory.sync_adapters.claude_code import (
+ ClaudeCodeAdapter,
+ get_claude_memory_dir,
+ )
+
+ claude_memory_dir = get_claude_memory_dir()
+
+ async def _import_claude_memories() -> int:
+ config = LocalBackendConfig(db_path=db_path)
+ backend = LocalBackend(config)
+ await backend._ensure_initialized()
+ adapter = ClaudeCodeAdapter(claude_memory_dir)
+ count = await sync_import(backend, adapter, mem_user)
+ await backend.close()
+ return count
+
+ imported = asyncio.run(_import_claude_memories())
+ if imported:
+ click.echo(f" Memory: imported {imported} memories from Claude")
+ except Exception as e:
+ click.echo(f" Warning: Claude memory import failed: {e}")
+
+ if prepare_only:
+ _inject_codex_provider_config(port)
+ return
+
+ codex_bin = shutil.which("codex")
+ if not codex_bin:
+ click.echo("Error: 'codex' not found in PATH.")
+ click.echo("Install Codex CLI: npm install -g @openai/codex")
+ raise SystemExit(1)
+
+ env, env_vars_display = _build_codex_launch_env(port, os.environ)
+
+ # Inject Headroom provider into Codex config so WebSocket traffic also
+ # routes through the proxy. Codex ignores OPENAI_BASE_URL for its WS
+ # transport unless a custom provider declares supports_websockets = true.
+ # NOTE: this must run BEFORE _inject_memory_mcp_config because it rewrites
+ # the config file. Re-inject MCP config after if memory is enabled.
+ _inject_codex_provider_config(port)
+ if memory:
+ mem_dir = Path.cwd() / ".headroom"
+ _inject_memory_mcp_config(
+ str(mem_dir / "memory.db"),
+ os.environ.get("USER", os.environ.get("USERNAME", "default")),
+ )
+
+ _launch_tool(
+ binary=codex_bin,
+ args=codex_args,
+ env=env,
+ port=port,
+ no_proxy=no_proxy,
+ tool_label="CODEX",
+ env_vars_display=env_vars_display,
+ learn=learn,
+ memory=memory,
+ agent_type="codex",
+ code_graph=code_graph,
+ backend=backend,
+ anyllm_provider=anyllm_provider,
+ region=region,
+ )
+
+
+# =============================================================================
+# Aider
+# =============================================================================
+
+
[email protected](context_settings={"ignore_unknown_options": True})
[email protected]("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)")
[email protected]("--no-rtk", is_flag=True, help="Skip rtk installation and conventions injection")
+ "--code-graph",
+ is_flag=True,
+ help="Enable code graph indexing via codebase-memory-mcp (optional)",
+)
[email protected]("--no-proxy", is_flag=True, help="Skip proxy startup (use existing proxy)")
[email protected]("--learn", is_flag=True, help="Enable live traffic learning")
[email protected]("--memory", is_flag=True, help="Enable persistent cross-session memory")
+ "--backend", default=None, help="API backend: 'anthropic', 'anyllm', 'litellm-vertex', etc."
+)
[email protected]("--anyllm-provider", default=None, help="Provider for any-llm backend")
[email protected]("--region", default=None, help="Cloud region for Bedrock/Vertex")
[email protected]("--verbose", "-v", is_flag=True, help="Verbose output")
[email protected]("--prepare-only", is_flag=True, hidden=True)
[email protected]("aider_args", nargs=-1, type=click.UNPROCESSED)
+def aider(
+ port: int,
+ no_rtk: bool,
+ code_graph: bool,
+ no_proxy: bool,
+ learn: bool,
+ memory: bool,
+ backend: str | None,
+ anyllm_provider: str | None,
+ region: str | None,
+ verbose: bool,
+ prepare_only: bool,
+ aider_args: tuple,
+) -> None:
+ """Launch aider through Headroom proxy.
+
+ \b
+ Sets OPENAI_API_BASE to route all API calls through Headroom.
+ Installs rtk and injects instructions into .aider.conf.yml conventions
+ so aider uses token-optimized commands.
+
+ \b
+ Examples:
+ headroom wrap aider # Start proxy + rtk + aider
+ headroom wrap aider -- --model gpt-4o # Use GPT-4o
+ headroom wrap aider -- --model claude-sonnet-4 # Use Claude
+ headroom wrap aider --no-rtk # Skip rtk setup
+ headroom wrap aider --backend litellm-vertex --region us-central1
+ """
+ # Setup rtk for aider (binary + CONVENTIONS.md instructions)
+ if not no_rtk:
+ click.echo(" Setting up rtk for aider...")
+ rtk_path = _ensure_rtk_binary(verbose=verbose)
+ if rtk_path:
+ # aider reads CONVENTIONS.md from project root
+ conventions = Path.cwd() / "CONVENTIONS.md"
+ _inject_rtk_instructions(conventions, verbose=verbose)
+
+ if prepare_only:
+ return
+
+ aider_bin = shutil.which("aider")
+ if not aider_bin:
+ click.echo("Error: 'aider' not found in PATH.")
+ click.echo("Install aider: pip install aider-chat")
+ raise SystemExit(1)
+
+ env, env_vars_display = _build_aider_launch_env(port, os.environ)
+
+ _launch_tool(
+ binary=aider_bin,
+ args=aider_args,
+ env=env,
+ port=port,
+ no_proxy=no_proxy,
+ tool_label="AIDER",
+ env_vars_display=env_vars_display,
+ learn=learn,
+ memory=memory,
+ agent_type="aider",
+ code_graph=code_graph,
+ backend=backend,
+ anyllm_provider=anyllm_provider,
+ region=region,
+ )
+
+
+# =============================================================================
+# Cursor
+# =============================================================================
+
+
[email protected](context_settings={"ignore_unknown_options": True})
[email protected]("--port", "-p", default=8787, type=int, help="Proxy port (default: 8787)")
[email protected]("--no-rtk", is_flag=True, help="Skip rtk installation and .cursorrules injection")
[email protected]("--no-proxy", is_flag=True, help="Skip proxy startup (use existing proxy)")
+ "--learn", is_flag=True, help="Enable live traffic learning (patterns saved to .cursor/rules/)"
+)
[email protected]("--memory", is_flag=True, help="Enable persistent cross-session memory")
[email protected]("--verbose", "-v", is_flag=True, help="Verbose output")
[email protected]("--prepare-only", is_flag=True, hidden=True)
+def cursor(
+ port: int,
+ no_rtk: bool,
+ no_proxy: bool,
+ learn: bool,
+ memory: bool,
+ verbose: bool,
+ prepare_only: bool,
+) -> None:
+ """Start Headroom proxy for use with Cursor.
+
+ \b
+ Cursor reads its API configuration from its settings UI, not from
+ environment variables. This command starts the proxy, installs rtk
+ with .cursorrules instructions, and prints the Cursor settings.
+
+ \b
+ After running this command, open Cursor and configure:
+ Settings > Models > OpenAI API Key > Advanced > Override Base URL
+
+ \b
+ Example:
+ headroom wrap cursor # Start proxy + rtk + instructions
+ headroom wrap cursor --no-rtk # Proxy only, no rtk
+ headroom wrap cursor --port 9999 # Custom proxy port
+ """
+ if not no_rtk:
+ click.echo(" Setting up rtk for Cursor...")
+ rtk_path = _ensure_rtk_binary(verbose=verbose)
+ if rtk_path:
+ cursorrules = Path.cwd() / ".cursorrules"
+ _inject_rtk_instructions(cursorrules, verbose=verbose)
+
+ if prepare_only:
+ return
+
+ proxy_holder: list[subprocess.Popen | None] = [None]
+ cleanup = _make_cleanup(proxy_holder, port)
+ signal.signal(signal.SIGINT, cleanup)
+ signal.signal(signal.SIGTERM, cleanup)
+
+ try:
+ click.echo()
+ click.echo(" ╔═══════════════════════════════════════════════╗")
+ click.echo(" ║ HEADROOM WRAP: CURSOR ║")
+ click.echo(" ╚═══════════════════════════════════════════════╝")
+ click.echo()
+
+ proxy_holder[0] = _ensure_proxy(
+ port, no_proxy, learn=learn, memory=memory, agent_type="cursor"
+ )
+
+ click.echo()
+ for line in _render_cursor_setup_lines(port):
+ click.echo(line)
+ if not no_rtk:
+ click.echo()
+ click.echo(" rtk instructions injected into .cursorrules")
+ click.echo(" Cursor will use token-optimized commands automatically.")
+ click.echo()
+ click.echo(" Press Ctrl+C to stop the proxy.")
+ click.echo()
+
+ # Block until Ctrl+C
+ try:
+ while True:
+ time.sleep(1)
+ proc = proxy_holder[0]
+ if proc and proc.poll() is not None:
+ click.echo(" Proxy process exited unexpectedly.")
+ raise SystemExit(1)
+ except KeyboardInterrupt:
+ click.echo("\n Shutting down...")
+
+ except SystemExit:
+ raise
+ except Exception as e:
+ click.echo(f" Error: {e}")
+ raise SystemExit(1) from e
+ finally:
+ cleanup()
+
+
+# =============================================================================
+# OpenClaw
+# =============================================================================
+
+
[email protected]("openclaw")
+ "--plugin-path",
+ type=click.Path(path_type=Path, file_okay=False, dir_okay=True),
+ default=None,
+ help="Path to local OpenClaw plugin source directory (advanced/dev override)",
+)
+ "--plugin-spec",
+ default="headroom-ai/openclaw",
+ show_default=True,
+ help="NPM plugin spec for OpenClaw install (used when --plugin-path is omitted)",
+)
+ "--skip-build",
+ is_flag=True,
+ help="Skip npm install/build in local source mode (--plugin-path)",
+)
+ "--copy",
+ is_flag=True,
+ help="Install by copying plugin path instead of using --link",
+)
[email protected]("--proxy-port", default=8787, type=int, help="Headroom proxy port")
[email protected]("--startup-timeout-ms", default=20000, type=int, help="Proxy startup timeout")
+ "--gateway-provider-id",
+ "gateway_provider_ids",
+ multiple=True,
+ help="OpenClaw provider id to route through Headroom (repeatable; default: openai-codex)",
+)
+ "--python-path",
+ default=None,
+ help="Optional Python executable for proxy launcher fallback",
+)
+ "--no-auto-start",
+ is_flag=True,
+ help="Disable plugin auto-start of local headroom proxy",
+)
+ "--no-restart",
+ is_flag=True,
+ help="Do not restart OpenClaw gateway at the end",
+)
[email protected]("--verbose", "-v", is_flag=True, help="Verbose output")
[email protected]("--prepare-only", is_flag=True, hidden=True)
[email protected]("--existing-entry-json", default=None, hidden=True)
+def openclaw(
+ plugin_path: Path | None,
+ plugin_spec: str,
+ skip_build: bool,
+ copy: bool,
+ proxy_port: int,
+ startup_timeout_ms: int,
+ gateway_provider_ids: tuple[str, ...],
+ python_path: str | None,
+ no_auto_start: bool,
+ no_restart: bool,
+ verbose: bool,
+ prepare_only: bool,
+ existing_entry_json: str | None,
+) -> None:
+ """Install and configure Headroom OpenClaw plugin in one command.
+
+ \b
+ What this command does:
+ 1. Installs OpenClaw plugin from npm (or local --plugin-path)
+ 2. Builds plugin source if --plugin-path is used
+ 3. Writes minimal plugin config and sets contextEngine slot
+ 4. Validates config
+ 5. Restarts OpenClaw gateway (unless --no-restart)
+
+ \b
+ Example:
+ headroom wrap openclaw
+ headroom wrap openclaw --plugin-path C:\\git\\headroom\\plugins\\openclaw
+ """
+ if prepare_only:
+ entry = _build_openclaw_plugin_entry(
+ existing_entry=_decode_openclaw_entry_json(existing_entry_json),
+ proxy_port=proxy_port,
+ startup_timeout_ms=startup_timeout_ms,
+ python_path=python_path,
+ no_auto_start=no_auto_start,
+ gateway_provider_ids=gateway_provider_ids,
+ enabled=True,
+ )
+ click.echo(json.dumps(entry, separators=(",", ":")))
+ return
+
+ openclaw_bin = shutil.which("openclaw")
+ if not openclaw_bin:
+ raise click.ClickException("'openclaw' not found in PATH. Install OpenClaw CLI first.")
+
+ plugin_dir = plugin_path.resolve() if plugin_path else None
+ local_source_mode = plugin_dir is not None
+ if plugin_dir:
+ if not plugin_dir.exists():
+ raise click.ClickException(f"Plugin path not found: {plugin_dir}.")
+ if not (plugin_dir / "package.json").exists():
+ raise click.ClickException(f"Invalid plugin path (missing package.json): {plugin_dir}")
+ if not (plugin_dir / "openclaw.plugin.json").exists():
+ raise click.ClickException(
+ f"Invalid plugin path (missing openclaw.plugin.json): {plugin_dir}"
+ )
+
+ npm_bin = shutil.which("npm")
+ if local_source_mode and not skip_build and not npm_bin:
+ raise click.ClickException(
+ "'npm' not found in PATH. Install Node/npm or rerun with --skip-build."
+ )
+
+ click.echo()
+ click.echo(" ╔═══════════════════════════════════════════════╗")
+ click.echo(" ║ HEADROOM WRAP: OPENCLAW ║")
+ click.echo(" ╚═══════════════════════════════════════════════╝")
+ click.echo()
+ if local_source_mode:
+ click.echo(f" Plugin source: local ({plugin_dir})")
+ else:
+ click.echo(f" Plugin source: npm ({plugin_spec})")
+
+ if local_source_mode and not skip_build:
+ click.echo(" Building OpenClaw plugin (npm install + npm run build)...")
+ _run_checked([npm_bin or "npm", "install"], cwd=plugin_dir, action="npm install")
+ _run_checked([npm_bin or "npm", "run", "build"], cwd=plugin_dir, action="npm run build")
+ elif not local_source_mode and skip_build:
+ click.echo(" Skipping build: npm install mode does not build local source.")
+
+ effective_python_path = python_path
+ if effective_python_path is None and not no_auto_start and sys.executable:
+ effective_python_path = sys.executable
+
+ existing_entry = _read_openclaw_config_value(openclaw_bin, "plugins.entries.headroom")
+ entry = _build_openclaw_plugin_entry(
+ existing_entry=existing_entry,
+ proxy_port=proxy_port,
+ startup_timeout_ms=startup_timeout_ms,
+ python_path=effective_python_path,
+ no_auto_start=no_auto_start,
+ gateway_provider_ids=gateway_provider_ids,
+ enabled=True,
+ )
+
+ click.echo(" Writing plugin configuration...")
+ _write_openclaw_plugin_entry(openclaw_bin, entry)
+
+ install_cmd = [
+ openclaw_bin,
+ "plugins",
+ "install",
+ "--dangerously-force-unsafe-install",
+ ]
+ if local_source_mode:
+ if copy:
+ install_cmd.append(str(plugin_dir))
+ install_cwd = None
+ else:
+ install_cmd.extend(["--link", "."])
+ install_cwd = plugin_dir
+ else:
+ install_cmd.append(plugin_spec)
+ install_cwd = None
+
+ click.echo(" Installing OpenClaw plugin with required unsafe-install flag...")
+ install_result = subprocess.run(
+ install_cmd,
+ cwd=str(install_cwd) if install_cwd else None,
+ capture_output=True,
+ text=True,
+ encoding="utf-8",
+ errors="replace",
+ )
+ if install_result.returncode != 0:
+ combined_error = "\n".join(
+ x for x in [install_result.stderr.strip(), install_result.stdout.strip()] if x
+ )
+ plugin_already_exists = "plugin already exists" in combined_error.lower()
+ linked_install_bug = (
+ "also not a valid hook pack" in combined_error.lower()
+ and "--dangerously-force-unsafe-install" in " ".join(install_cmd)
+ )
+ if plugin_already_exists:
+ click.echo(" Plugin already installed; continuing with configuration/update steps.")
+ elif linked_install_bug and local_source_mode and plugin_dir is not None:
+ click.echo(
+ " OpenClaw linked-path install bug detected; applying extension-path fallback..."
+ )
+ target_dir = _copy_openclaw_plugin_into_extensions(
+ plugin_dir=plugin_dir,
+ openclaw_bin=openclaw_bin,
+ )
+ click.echo(f" Fallback plugin copy completed: {target_dir}")
+ else:
+ details = combined_error or f"exit code {install_result.returncode}"
+ raise click.ClickException(f"openclaw plugins install failed: {details}")
+ elif verbose and install_result.stdout.strip():
+ click.echo(install_result.stdout.strip())
+
+ _set_openclaw_context_engine_slot(openclaw_bin, "headroom")
+ _run_checked(
+ [openclaw_bin, "config", "validate"],
+ action="openclaw config validate",
+ )
+
+ if no_restart:
+ click.echo(" Skipping gateway restart (--no-restart).")
+ click.echo(
+ " Run `openclaw gateway restart` (or `openclaw gateway start`) to apply plugin changes."
+ )
+ else:
+ click.echo(" Applying plugin changes to OpenClaw gateway...")
+ gateway_action, gateway_output = _restart_or_start_openclaw_gateway(openclaw_bin)
+ click.echo(f" Gateway {gateway_action}.")
+ if verbose and gateway_output:
+ click.echo(gateway_output)
+
+ inspect_result = _run_checked(
+ [openclaw_bin, "plugins", "inspect", "headroom"],
+ action="openclaw plugins inspect headroom",
+ )
+ if verbose and inspect_result.stdout.strip():
+ click.echo(inspect_result.stdout.strip())
+
+ click.echo()
+ click.echo("✓ OpenClaw is configured to use Headroom context compression.")
+ click.echo(" Plugin: headroom")
+ click.echo(" Slot: plugins.slots.contextEngine = headroom")
+ click.echo()
+
+
[email protected]("openclaw")
[email protected]("--no-restart", is_flag=True, help="Do not restart OpenClaw gateway at the end")
[email protected]("--verbose", "-v", is_flag=True, help="Verbose output")
[email protected]("--prepare-only", is_flag=True, hidden=True)
[email protected]("--existing-entry-json", default=None, hidden=True)
+def unwrap_openclaw(
+ no_restart: bool,
+ verbose: bool,
+ prepare_only: bool,
+ existing_entry_json: str | None,
+) -> None:
+ """Disable the Headroom OpenClaw plugin and restore the legacy engine slot."""
+ if prepare_only:
+ click.echo(
+ json.dumps(
+ _build_openclaw_unwrap_entry(_decode_openclaw_entry_json(existing_entry_json)),
+ separators=(",", ":"),
+ )
+ )
+ return
+
+ openclaw_bin = shutil.which("openclaw")
+ if not openclaw_bin:
+ raise click.ClickException("'openclaw' not found in PATH. Install OpenClaw CLI first.")
+
+ click.echo()
+ click.echo(" ╔═══════════════════════════════════════════════╗")
+ click.echo(" ║ HEADROOM UNWRAP: OPENCLAW ║")
+ click.echo(" ╚═══════════════════════════════════════════════╝")
+ click.echo()
+ click.echo(" Disabling Headroom plugin and removing engine mapping...")
+
+ existing_entry = _read_openclaw_config_value(openclaw_bin, "plugins.entries.headroom")
+ entry = _build_openclaw_unwrap_entry(existing_entry)
+ _write_openclaw_plugin_entry(openclaw_bin, entry)
+ _set_openclaw_context_engine_slot(openclaw_bin, "legacy")
+ _run_checked(
+ [openclaw_bin, "config", "validate"],
+ action="openclaw config validate",
+ )
+
+ if no_restart:
+ click.echo(" Skipping gateway restart (--no-restart).")
+ click.echo(
+ " Run `openclaw gateway restart` (or `openclaw gateway start`) to apply unwrap changes."
+ )
+ else:
+ click.echo(" Applying unwrap changes to OpenClaw gateway...")
+ gateway_action, gateway_output = _restart_or_start_openclaw_gateway(openclaw_bin)
+ click.echo(f" Gateway {gateway_action}.")
+ if verbose and gateway_output:
+ click.echo(gateway_output)
+
+ if verbose:
+ inspect_result = _run_checked(
+ [openclaw_bin, "plugins", "inspect", "headroom"],
+ action="openclaw plugins inspect headroom",
+ )
+ if inspect_result.stdout.strip():
+ click.echo(inspect_result.stdout.strip())
+
+ click.echo()
+ click.echo("✓ OpenClaw Headroom wrap removed.")
+ click.echo(" Plugin: headroom (installed, disabled)")
+ click.echo(" Slot: plugins.slots.contextEngine = legacy")
+ click.echo()
diff --git a/headroom/client.py b/headroom/client.py
index 38812fc30..8e9b3030e 100644
--- a/headroom/client.py
+++ b/headroom/client.py
@@ -1,1090 +1,1049 @@
-"""Main HeadroomClient implementation for Headroom SDK."""
-
-from __future__ import annotations
-
-from collections.abc import Iterator
-from datetime import datetime, timezone
-from typing import Any
-
-from .cache import (
- BaseCacheOptimizer,
- CacheConfig,
- CacheOptimizerRegistry,
- OptimizationContext,
- SemanticCacheLayer,
-)
-from .config import (
- HeadroomConfig,
- HeadroomMode,
- RequestMetrics,
- SimulationResult,
-)
-from .parser import parse_messages
-from .pipeline import PipelineExtensionManager, PipelineStage, summarize_routing_markers
-from .providers.base import Provider
-from .storage import create_storage
-from .tokenizer import Tokenizer
-from .transforms import CacheAligner, TransformPipeline
-from .utils import (
- compute_messages_hash,
- compute_prefix_hash,
- estimate_cost,
- format_cost,
- generate_request_id,
-)
-
-
-class ChatCompletions:
- """Wrapper for chat.completions API (OpenAI-style)."""
-
- def __init__(self, client: HeadroomClient):
- self._client = client
-
- def create(
- self,
- *,
- model: str,
- messages: list[dict[str, Any]],
- stream: bool = False,
- # Headroom-specific parameters
- headroom_mode: str | None = None,
- headroom_cache_prefix_tokens: int | None = None,
- headroom_output_buffer_tokens: int | None = None,
- headroom_keep_turns: int | None = None,
- headroom_tool_profiles: dict[str, dict[str, Any]] | None = None,
- # Pass through all other kwargs
- **kwargs: Any,
- ) -> Any:
- """
- Create a chat completion with optional Headroom optimization.
-
- Args:
- model: Model name.
- messages: List of messages.
- stream: Whether to stream the response.
- headroom_mode: Override default mode ("audit" | "optimize").
- headroom_cache_prefix_tokens: Target cache-aligned prefix size.
- headroom_output_buffer_tokens: Reserve tokens for output.
- headroom_keep_turns: Never drop last N turns.
- headroom_tool_profiles: Per-tool compression config.
- **kwargs: Additional arguments passed to underlying client.
-
- Returns:
- Chat completion response (or stream iterator).
- """
- return self._client._create(
- model=model,
- messages=messages,
- stream=stream,
- headroom_mode=headroom_mode,
- headroom_cache_prefix_tokens=headroom_cache_prefix_tokens,
- headroom_output_buffer_tokens=headroom_output_buffer_tokens,
- headroom_keep_turns=headroom_keep_turns,
- headroom_tool_profiles=headroom_tool_profiles,
- api_style="openai",
- **kwargs,
- )
-
- def simulate(
- self,
- *,
- model: str,
- messages: list[dict[str, Any]],
- headroom_mode: str = "optimize",
- headroom_output_buffer_tokens: int | None = None,
- headroom_tool_profiles: dict[str, dict[str, Any]] | None = None,
- **kwargs: Any,
- ) -> SimulationResult:
- """
- Simulate optimization without calling the API.
-
- Args:
- model: Model name.
- messages: List of messages.
- headroom_mode: Mode to simulate.
- headroom_output_buffer_tokens: Output buffer to use.
- headroom_tool_profiles: Tool profiles to use.
- **kwargs: Additional arguments (ignored).
-
- Returns:
- SimulationResult with projected changes.
- """
- return self._client._simulate(
- model=model,
- messages=messages,
- headroom_mode=headroom_mode,
- headroom_output_buffer_tokens=headroom_output_buffer_tokens,
- headroom_tool_profiles=headroom_tool_profiles,
- )
-
-
-class Messages:
- """Wrapper for messages API (Anthropic-style)."""
-
- def __init__(self, client: HeadroomClient):
- self._client = client
-
- def create(
- self,
- *,
- model: str,
- messages: list[dict[str, Any]],
- max_tokens: int = 1024,
- # Headroom-specific parameters
- headroom_mode: str | None = None,
- headroom_cache_prefix_tokens: int | None = None,
- headroom_output_buffer_tokens: int | None = None,
- headroom_keep_turns: int | None = None,
- headroom_tool_profiles: dict[str, dict[str, Any]] | None = None,
- # Pass through all other kwargs
- **kwargs: Any,
- ) -> Any:
- """
- Create a message with optional Headroom optimization.
-
- Args:
- model: Model name.
- messages: List of messages.
- max_tokens: Maximum tokens in response.
- headroom_mode: Override default mode ("audit" | "optimize").
- headroom_cache_prefix_tokens: Target cache-aligned prefix size.
- headroom_output_buffer_tokens: Reserve tokens for output.
- headroom_keep_turns: Never drop last N turns.
- headroom_tool_profiles: Per-tool compression config.
- **kwargs: Additional arguments passed to underlying client.
-
- Returns:
- Message response.
- """
- return self._client._create(
- model=model,
- messages=messages,
- stream=False,
- headroom_mode=headroom_mode,
- headroom_cache_prefix_tokens=headroom_cache_prefix_tokens,
- headroom_output_buffer_tokens=headroom_output_buffer_tokens,
- headroom_keep_turns=headroom_keep_turns,
- headroom_tool_profiles=headroom_tool_profiles,
- api_style="anthropic",
- max_tokens=max_tokens,
- **kwargs,
- )
-
- def stream(
- self,
- *,
- model: str,
- messages: list[dict[str, Any]],
- max_tokens: int = 1024,
- # Headroom-specific parameters
- headroom_mode: str | None = None,
- headroom_cache_prefix_tokens: int | None = None,
- headroom_output_buffer_tokens: int | None = None,
- headroom_keep_turns: int | None = None,
- headroom_tool_profiles: dict[str, dict[str, Any]] | None = None,
- # Pass through all other kwargs
- **kwargs: Any,
- ) -> Any:
- """
- Stream a message with optional Headroom optimization.
-
- Args:
- model: Model name.
- messages: List of messages.
- max_tokens: Maximum tokens in response.
- headroom_mode: Override default mode ("audit" | "optimize").
- headroom_cache_prefix_tokens: Target cache-aligned prefix size.
- headroom_output_buffer_tokens: Reserve tokens for output.
- headroom_keep_turns: Never drop last N turns.
- headroom_tool_profiles: Per-tool compression config.
- **kwargs: Additional arguments passed to underlying client.
-
- Returns:
- Stream context manager.
- """
- return self._client._create(
- model=model,
- messages=messages,
- stream=True,
- headroom_mode=headroom_mode,
- headroom_cache_prefix_tokens=headroom_cache_prefix_tokens,
- headroom_output_buffer_tokens=headroom_output_buffer_tokens,
- headroom_keep_turns=headroom_keep_turns,
- headroom_tool_profiles=headroom_tool_profiles,
- api_style="anthropic",
- max_tokens=max_tokens,
- **kwargs,
- )
-
- def simulate(
- self,
- *,
- model: str,
- messages: list[dict[str, Any]],
- headroom_mode: str = "optimize",
- headroom_output_buffer_tokens: int | None = None,
- headroom_tool_profiles: dict[str, dict[str, Any]] | None = None,
- **kwargs: Any,
- ) -> SimulationResult:
- """
- Simulate optimization without calling the API.
-
- Args:
- model: Model name.
- messages: List of messages.
- headroom_mode: Mode to simulate.
- headroom_output_buffer_tokens: Output buffer to use.
- headroom_tool_profiles: Tool profiles to use.
- **kwargs: Additional arguments (ignored).
-
- Returns:
- SimulationResult with projected changes.
- """
- return self._client._simulate(
- model=model,
- messages=messages,
- headroom_mode=headroom_mode,
- headroom_output_buffer_tokens=headroom_output_buffer_tokens,
- headroom_tool_profiles=headroom_tool_profiles,
- )
-
-
-class HeadroomClient:
- """
- Context Budget Controller wrapper for LLM API clients.
-
- Provides automatic context optimization, waste detection, and
- cache alignment while maintaining API compatibility.
- """
-
- def __init__(
- self,
- original_client: Any,
- provider: Provider,
- store_url: str | None = None,
- default_mode: str = "audit",
- model_context_limits: dict[str, int] | None = None,
- cache_optimizer: BaseCacheOptimizer | None = None,
- enable_cache_optimizer: bool = True,
- enable_semantic_cache: bool = False,
- config: HeadroomConfig | None = None,
- ):
- """
- Initialize HeadroomClient.
-
- Args:
- original_client: The underlying LLM client (OpenAI-compatible).
- provider: Provider instance for model-specific behavior.
- store_url: Storage URL (sqlite:// or jsonl://). Defaults to temp dir.
- default_mode: Default mode ("audit" | "optimize").
- model_context_limits: Override context limits for models.
- cache_optimizer: Optional custom cache optimizer. If None and
- enable_cache_optimizer=True, auto-detects from provider.
- enable_cache_optimizer: Enable provider-specific cache optimization.
- enable_semantic_cache: Enable query-level semantic caching.
- config: Optional HeadroomConfig for full control over all settings
- including intelligent_context. When provided, takes precedence
- over individual settings like store_url, default_mode, etc.
- """
- self._original = original_client
- self._provider = provider
-
- # Set default store_url to temp directory for better DevEx
- if store_url is None:
- import os
- import tempfile
-
- db_path = os.path.join(tempfile.gettempdir(), "headroom.db")
- store_url = f"sqlite:///{db_path}"
-
- self._store_url = store_url
- self._default_mode = HeadroomMode(default_mode)
-
- # Use provided config or build from individual parameters
- if config is not None:
- self._config = config
- # Override store_url and mode if explicitly provided in config
- if config.store_url:
- self._store_url = config.store_url
- else:
- self._config.store_url = store_url
- self._default_mode = config.default_mode
- else:
- # Build config from individual parameters
- self._config = HeadroomConfig()
- self._config.store_url = store_url
- self._config.default_mode = self._default_mode
- self._config.cache_optimizer.enabled = enable_cache_optimizer
- self._config.cache_optimizer.enable_semantic_cache = enable_semantic_cache
-
- if model_context_limits:
- self._config.model_context_limits.update(model_context_limits)
-
- # Initialize storage
- self._storage = create_storage(store_url)
-
- # Initialize transform pipeline
- self._pipeline = TransformPipeline(self._config, provider=self._provider)
- self._pipeline_extensions = PipelineExtensionManager(
- extensions=self._config.pipeline_extensions,
- discover=self._config.discover_pipeline_extensions,
- )
-
- # Initialize cache optimizer
- self._cache_optimizer: BaseCacheOptimizer | None = None
- self._semantic_cache_layer: SemanticCacheLayer | None = None
-
- if enable_cache_optimizer:
- if cache_optimizer is not None:
- self._cache_optimizer = cache_optimizer
- else:
- # Auto-detect from provider
- provider_name = self._provider.name.lower()
- if CacheOptimizerRegistry.is_registered(provider_name):
- cache_config = CacheConfig(
- min_cacheable_tokens=self._config.cache_optimizer.min_cacheable_tokens,
- )
- self._cache_optimizer = CacheOptimizerRegistry.get(
- provider_name,
- config=cache_config,
- )
-
- # Wrap with semantic cache if enabled
- if enable_semantic_cache and self._cache_optimizer is not None:
- self._semantic_cache_layer = SemanticCacheLayer(
- self._cache_optimizer,
- similarity_threshold=self._config.cache_optimizer.semantic_cache_similarity,
- max_entries=self._config.cache_optimizer.semantic_cache_max_entries,
- ttl_seconds=self._config.cache_optimizer.semantic_cache_ttl_seconds,
- )
-
- # Public API - OpenAI style
- self.chat = type("Chat", (), {"completions": ChatCompletions(self)})()
- # Public API - Anthropic style
- self.messages = Messages(self)
- self._pipeline_extensions.emit(
- PipelineStage.SETUP,
- operation="sdk.setup",
- provider=self._provider.name.lower(),
- metadata={
- "default_mode": self._default_mode.value,
- "cache_optimizer_enabled": enable_cache_optimizer,
- "semantic_cache_enabled": enable_semantic_cache,
- },
- )
-
- def _get_tokenizer(self, model: str) -> Tokenizer:
- """Get tokenizer for model using provider."""
- token_counter = self._provider.get_token_counter(model)
- return Tokenizer(token_counter, model)
-
- def _get_context_limit(self, model: str) -> int:
- """Get context limit from user config or provider."""
- # User override takes precedence
- limit = self._config.get_context_limit(model)
- if limit is not None:
- return limit
- # Fall back to provider
- return self._provider.get_context_limit(model)
-
- def _create(
- self,
- *,
- model: str,
- messages: list[dict[str, Any]],
- stream: bool = False,
- headroom_mode: str | None = None,
- headroom_cache_prefix_tokens: int | None = None,
- headroom_output_buffer_tokens: int | None = None,
- headroom_keep_turns: int | None = None,
- headroom_tool_profiles: dict[str, dict[str, Any]] | None = None,
- api_style: str = "openai",
- **kwargs: Any,
- ) -> Any:
- """Internal implementation of create."""
- request_id = generate_request_id()
- timestamp = datetime.now(timezone.utc).replace(tzinfo=None)
- mode = HeadroomMode(headroom_mode) if headroom_mode else self._default_mode
-
- input_event = self._pipeline_extensions.emit(
- PipelineStage.INPUT_RECEIVED,
- operation="sdk.request",
- request_id=request_id,
- provider=self._provider.name.lower(),
- model=model,
- messages=messages,
- metadata={"api_style": api_style, "stream": stream, "mode": mode.value},
- )
- if input_event.messages is not None:
- messages = input_event.messages
-
- tokenizer = self._get_tokenizer(model)
-
- # Analyze original messages
- blocks, block_breakdown, waste_signals = parse_messages(messages, tokenizer)
- tokens_before = tokenizer.count_messages(messages)
-
- # Compute cache alignment score
- aligner = CacheAligner(self._config.cache_aligner)
- cache_alignment_score = aligner.get_alignment_score(messages)
-
- # Compute stable prefix hash
- stable_prefix_hash = compute_prefix_hash(messages)
-
- # Cache optimizer metrics (populated later if optimizer is used)
- cache_optimizer_used = None
- cache_optimizer_strategy = None
- cacheable_tokens = 0
- breakpoints_inserted = 0
- estimated_cache_hit = False
- estimated_savings_percent = 0.0
- semantic_cache_hit = False
- cached_response = None
-
- # Apply transforms if in optimize mode
- if mode == HeadroomMode.OPTIMIZE:
- output_buffer = (
- headroom_output_buffer_tokens or self._config.rolling_window.output_buffer_tokens
- )
- model_limit = self._get_context_limit(model)
-
- result = self._pipeline.apply(
- messages,
- model,
- model_limit=model_limit,
- output_buffer=output_buffer,
- tool_profiles=headroom_tool_profiles or {},
- )
-
- optimized_messages = result.messages
- tokens_after = result.tokens_after
- transforms_applied = result.transforms_applied
-
- routing_markers = summarize_routing_markers(transforms_applied)
- if routing_markers:
- routed_event = self._pipeline_extensions.emit(
- PipelineStage.INPUT_ROUTED,
- operation="sdk.request",
- request_id=request_id,
- provider=self._provider.name.lower(),
- model=model,
- messages=optimized_messages,
- metadata={
- "routing_markers": routing_markers,
- "transforms_applied": transforms_applied,
- },
- )
- if routed_event.messages is not None:
- optimized_messages = routed_event.messages
-
- # Apply provider-specific cache optimization
- if self._cache_optimizer is not None or self._semantic_cache_layer is not None:
- cache_context = OptimizationContext(
- provider=self._provider.name.lower(),
- model=model,
- query=self._extract_query(optimized_messages),
- )
-
- # Check semantic cache first (if enabled)
- if self._semantic_cache_layer is not None:
- cache_result = self._semantic_cache_layer.process(
- optimized_messages, cache_context
- )
- semantic_cache_hit = cache_result.semantic_cache_hit
- if semantic_cache_hit:
- cached_response = cache_result.cached_response
-
- # Update metrics from cache result
- cache_optimizer_used = getattr(
- cache_result.metrics, "optimizer_name", None
- ) or (self._cache_optimizer.name if self._cache_optimizer else "")
- cache_optimizer_strategy = getattr(cache_result.metrics, "strategy", "")
- cacheable_tokens = cache_result.metrics.cacheable_tokens
- breakpoints_inserted = cache_result.metrics.breakpoints_inserted
- estimated_cache_hit = cache_result.metrics.estimated_cache_hit
- estimated_savings_percent = cache_result.metrics.estimated_savings_percent
-
- # Apply optimized messages (with cache_control blocks for Anthropic)
- if cache_result.messages:
- optimized_messages = cache_result.messages
-
- elif self._cache_optimizer is not None:
- # Direct cache optimizer (no semantic layer)
- cache_result = self._cache_optimizer.optimize(optimized_messages, cache_context)
- cache_optimizer_used = self._cache_optimizer.name
- cache_optimizer_strategy = self._cache_optimizer.strategy.value
- cacheable_tokens = cache_result.metrics.cacheable_tokens
- breakpoints_inserted = cache_result.metrics.breakpoints_inserted
- estimated_cache_hit = cache_result.metrics.estimated_cache_hit
- estimated_savings_percent = cache_result.metrics.estimated_savings_percent
-
- if cache_result.messages:
- optimized_messages = cache_result.messages
-
- transforms_applied.extend(
- f"cache_optimizer:{t}" for t in (cache_result.transforms_applied or [])
- )
-
- compressed_event = self._pipeline_extensions.emit(
- PipelineStage.INPUT_COMPRESSED,
- operation="sdk.request",
- request_id=request_id,
- provider=self._provider.name.lower(),
- model=model,
- messages=optimized_messages,
- metadata={
- "tokens_before": tokens_before,
- "tokens_after": tokens_after,
- "transforms_applied": transforms_applied,
- },
- )
- if compressed_event.messages is not None:
- optimized_messages = compressed_event.messages
- tokens_after = tokenizer.count_messages(optimized_messages)
-
- # Recalculate prefix hash after optimization
- stable_prefix_hash = compute_prefix_hash(optimized_messages)
- else:
- # Audit mode - no changes
- optimized_messages = messages
- tokens_after = tokens_before
- transforms_applied = []
-
- presend_event = self._pipeline_extensions.emit(
- PipelineStage.PRE_SEND,
- operation="sdk.request",
- request_id=request_id,
- provider=self._provider.name.lower(),
- model=model,
- messages=optimized_messages,
- metadata={"api_style": api_style, "stream": stream},
- )
- if presend_event.messages is not None:
- optimized_messages = presend_event.messages
- tokens_after = tokenizer.count_messages(optimized_messages)
- stable_prefix_hash = compute_prefix_hash(optimized_messages)
-
- # Create metrics
- metrics = RequestMetrics(
- request_id=request_id,
- timestamp=timestamp,
- model=model,
- stream=stream,
- mode=mode.value,
- tokens_input_before=tokens_before,
- tokens_input_after=tokens_after,
- block_breakdown=block_breakdown,
- waste_signals=waste_signals.to_dict(),
- stable_prefix_hash=stable_prefix_hash,
- cache_alignment_score=cache_alignment_score,
- transforms_applied=transforms_applied,
- messages_hash=compute_messages_hash(messages),
- # Cache optimizer metrics
- cache_optimizer_used=cache_optimizer_used,
- cache_optimizer_strategy=cache_optimizer_strategy,
- cacheable_tokens=cacheable_tokens,
- breakpoints_inserted=breakpoints_inserted,
- estimated_cache_hit=estimated_cache_hit,
- estimated_savings_percent=estimated_savings_percent,
- semantic_cache_hit=semantic_cache_hit,
- )
-
- # Update session stats
- self._update_session_stats(
- mode=mode,
- tokens_before=tokens_before,
- tokens_after=tokens_after,
- cache_hit=semantic_cache_hit,
- )
-
- # Return cached response if semantic cache hit
- if semantic_cache_hit and cached_response is not None:
- self._storage.save(metrics)
- return cached_response
-
- # Call underlying client based on API style
- try:
- if api_style == "anthropic":
- response = self._call_anthropic(
- model=model,
- messages=optimized_messages,
- stream=stream,
- metrics=metrics,
- **kwargs,
- )
- else:
- response = self._call_openai(
- model=model,
- messages=optimized_messages,
- stream=stream,
- metrics=metrics,
- **kwargs,
- )
-
- self._pipeline_extensions.emit(
- PipelineStage.POST_SEND,
- operation="sdk.request",
- request_id=request_id,
- provider=self._provider.name.lower(),
- model=model,
- messages=optimized_messages,
- response=response,
- metadata={"api_style": api_style, "stream": stream},
- )
- self._pipeline_extensions.emit(
- PipelineStage.RESPONSE_RECEIVED,
- operation="sdk.request",
- request_id=request_id,
- provider=self._provider.name.lower(),
- model=model,
- response=response,
- metadata={"api_style": api_style, "stream": stream},
- )
- return response
-
- except Exception as e:
- metrics.error = str(e)
- self._storage.save(metrics)
- raise
-
- def _call_openai(
- self,
- *,
- model: str,
- messages: list[dict[str, Any]],
- stream: bool,
- metrics: RequestMetrics,
- **kwargs: Any,
- ) -> Any:
- """Call OpenAI-style API."""
- if stream:
- response = self._original.chat.completions.create(
- model=model,
- messages=messages,
- stream=True,
- **kwargs,
- )
- return self._wrap_stream(response, metrics)
- else:
- response = self._original.chat.completions.create(
- model=model,
- messages=messages,
- stream=False,
- **kwargs,
- )
-
- # Extract output tokens from response
- if hasattr(response, "usage") and response.usage:
- metrics.tokens_output = response.usage.completion_tokens
- # Check for cached tokens in usage
- if hasattr(response.usage, "prompt_tokens_details"):
- details = response.usage.prompt_tokens_details
- if hasattr(details, "cached_tokens"):
- metrics.cached_tokens = details.cached_tokens
-
- self._storage.save(metrics)
- return response
-
- def _call_anthropic(
- self,
- *,
- model: str,
- messages: list[dict[str, Any]],
- stream: bool,
- metrics: RequestMetrics,
- **kwargs: Any,
- ) -> Any:
- """Call Anthropic-style API."""
- if stream:
- # Anthropic streaming returns a context manager
- stream_manager = self._original.messages.stream(
- model=model,
- messages=messages,
- **kwargs,
- )
- # Save metrics when stream is created
- self._storage.save(metrics)
- return stream_manager
- else:
- response = self._original.messages.create(
- model=model,
- messages=messages,
- **kwargs,
- )
-
- # Extract output tokens from Anthropic response
- if hasattr(response, "usage") and response.usage:
- metrics.tokens_output = response.usage.output_tokens
- # Check for cached tokens in Anthropic usage
- if hasattr(response.usage, "cache_read_input_tokens"):
- metrics.cached_tokens = response.usage.cache_read_input_tokens
-
- self._storage.save(metrics)
- return response
-
- def _wrap_stream(
- self,
- stream: Iterator[Any],
- metrics: RequestMetrics,
- ) -> Iterator[Any]:
- """Wrap stream to pass through chunks and save metrics at end."""
- try:
- yield from stream
- finally:
- # Save metrics when stream completes
- # Note: output tokens unknown for streams
- self._storage.save(metrics)
-
- def _extract_query(self, messages: list[dict[str, Any]]) -> str:
- """Extract query from messages for semantic caching.
-
- Returns the last user message content as the query.
- """
- for msg in reversed(messages):
- if msg.get("role") == "user":
- content = msg.get("content", "")
- if isinstance(content, str):
- return content
- elif isinstance(content, list):
- # Content block format
- for block in content:
- if isinstance(block, dict) and block.get("type") == "text":
- text_val = block.get("text", "")
- return str(text_val) if text_val else ""
- return ""
- return ""
-
- def _store_response_in_semantic_cache(
- self,
- messages: list[dict[str, Any]],
- response: Any,
- model: str,
- ) -> None:
- """Store response in semantic cache for future hits."""
- if self._semantic_cache_layer is not None:
- cache_context = OptimizationContext(
- provider=self._provider.name.lower(),
- model=model,
- query=self._extract_query(messages),
- )
- # Extract response content for caching
- response_data = self._extract_response_content(response)
- if response_data:
- self._semantic_cache_layer.store_response(messages, response_data, cache_context)
-
- def _extract_response_content(self, response: Any) -> dict[str, Any] | None:
- """Extract cacheable content from API response."""
- try:
- # OpenAI format
- if hasattr(response, "choices") and response.choices:
- choice = response.choices[0]
- if hasattr(choice, "message"):
- return {
- "role": "assistant",
- "content": choice.message.content,
- }
- # Anthropic format
- elif hasattr(response, "content"):
- return {
- "role": "assistant",
- "content": response.content,
- }
- except Exception:
- pass
- return None
-
- def _simulate(
- self,
- *,
- model: str,
- messages: list[dict[str, Any]],
- headroom_mode: str = "optimize",
- headroom_output_buffer_tokens: int | None = None,
- headroom_tool_profiles: dict[str, dict[str, Any]] | None = None,
- ) -> SimulationResult:
- """Internal implementation of simulate."""
- tokenizer = self._get_tokenizer(model)
-
- # Analyze original
- blocks, block_breakdown, waste_signals = parse_messages(messages, tokenizer)
- tokens_before = tokenizer.count_messages(messages)
-
- # Compute original cache alignment
- aligner = CacheAligner(self._config.cache_aligner)
- cache_alignment_score = aligner.get_alignment_score(messages)
- compute_prefix_hash(messages)
-
- # Apply transforms
- output_buffer = (
- headroom_output_buffer_tokens or self._config.rolling_window.output_buffer_tokens
- )
- model_limit = self._get_context_limit(model)
-
- result = self._pipeline.simulate(
- messages,
- model,
- model_limit=model_limit,
- output_buffer=output_buffer,
- tool_profiles=headroom_tool_profiles or {},
- )
-
- tokens_saved = tokens_before - result.tokens_after
-
- # Estimate cost savings using provider (use output_buffer tokens)
- # Note: output_buffer reserves tokens for expected model output
- cost_before = estimate_cost(tokens_before, output_buffer, model, provider=self._provider)
- cost_after = estimate_cost(
- result.tokens_after, output_buffer, model, provider=self._provider
- )
-
- if cost_before is not None and cost_after is not None:
- savings = format_cost(cost_before - cost_after)
- else:
- savings = "N/A"
-
- # Recalculate prefix hash after optimization
- optimized_prefix_hash = compute_prefix_hash(result.messages)
-
- return SimulationResult(
- tokens_before=tokens_before,
- tokens_after=result.tokens_after,
- tokens_saved=tokens_saved,
- transforms=result.transforms_applied,
- estimated_savings=f"{savings} per request",
- messages_optimized=result.messages,
- block_breakdown=block_breakdown,
- waste_signals=waste_signals.to_dict(),
- stable_prefix_hash=optimized_prefix_hash,
- cache_alignment_score=cache_alignment_score,
- )
-
- def get_metrics(
- self,
- start_time: datetime | None = None,
- end_time: datetime | None = None,
- model: str | None = None,
- mode: str | None = None,
- limit: int = 100,
- ) -> list[RequestMetrics]:
- """
- Query stored metrics.
-
- Args:
- start_time: Filter by timestamp >= start_time.
- end_time: Filter by timestamp <= end_time.
- model: Filter by model name.
- mode: Filter by mode.
- limit: Maximum results.
-
- Returns:
- List of RequestMetrics.
- """
- return self._storage.query(
- start_time=start_time,
- end_time=end_time,
- model=model,
- mode=mode,
- limit=limit,
- )
-
- def get_summary(
- self,
- start_time: datetime | None = None,
- end_time: datetime | None = None,
- ) -> dict[str, Any]:
- """
- Get summary statistics.
-
- Args:
- start_time: Filter by timestamp >= start_time.
- end_time: Filter by timestamp <= end_time.
-
- Returns:
- Summary statistics dict.
- """
- return self._storage.get_summary_stats(start_time, end_time)
-
- def close(self) -> None:
- """Close storage connection."""
- self._storage.close()
-
- def __enter__(self) -> HeadroomClient:
- """Context manager entry."""
- return self
-
- def __exit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None:
- """Context manager exit."""
- self.close()
-
- def validate_setup(self) -> dict[str, Any]:
- """Validate that Headroom is properly configured.
-
- This method checks:
- - Provider is valid and can count tokens
- - Storage is accessible and writable
- - Configuration is valid
- - Cache optimizer (if enabled) is working
-
- Returns:
- dict with validation results:
- {
- "valid": True/False,
- "provider": {"ok": bool, "name": str, "error": str | None},
- "storage": {"ok": bool, "url": str, "error": str | None},
- "config": {"ok": bool, "mode": str, "error": str | None},
- "cache_optimizer": {"ok": bool, "name": str | None, "error": str | None},
- }
-
- Raises:
- ValidationError: If validation fails and raise_on_error=True.
-
- Example:
- client = HeadroomClient(...)
- result = client.validate_setup()
- if not result["valid"]:
- print("Setup issues:", result)
- """
- result: dict[str, Any] = {
- "valid": True,
- "provider": {"ok": False, "name": None, "error": None},
- "storage": {"ok": False, "url": self._store_url, "error": None},
- "config": {"ok": False, "mode": self._default_mode.value, "error": None},
- "cache_optimizer": {"ok": True, "name": None, "error": None},
- }
-
- # Validate provider
- try:
- result["provider"]["name"] = self._provider.name
- # Test token counting
- test_messages = [{"role": "user", "content": "test"}]
- tokenizer = self._get_tokenizer("gpt-4")
- count = tokenizer.count_messages(test_messages)
- if count > 0:
- result["provider"]["ok"] = True
- else:
- result["provider"]["error"] = "Token count returned 0"
- result["valid"] = False
- except Exception as e:
- result["provider"]["error"] = str(e)
- result["valid"] = False
-
- # Validate storage
- try:
- # Try to get summary (tests read)
- self._storage.get_summary_stats()
- result["storage"]["ok"] = True
- except Exception as e:
- result["storage"]["error"] = str(e)
- result["valid"] = False
-
- # Validate config
- try:
- # Check mode is valid
- if self._default_mode in (HeadroomMode.AUDIT, HeadroomMode.OPTIMIZE):
- result["config"]["ok"] = True
- else:
- result["config"]["error"] = f"Invalid mode: {self._default_mode}"
- result["valid"] = False
- except Exception as e:
- result["config"]["error"] = str(e)
- result["valid"] = False
-
- # Validate cache optimizer (if enabled)
- if self._cache_optimizer is not None:
- try:
- result["cache_optimizer"]["name"] = self._cache_optimizer.name
- result["cache_optimizer"]["ok"] = True
- except Exception as e:
- result["cache_optimizer"]["error"] = str(e)
- # Don't fail validation for cache optimizer issues
- elif self._config.cache_optimizer.enabled:
- result["cache_optimizer"]["error"] = "Enabled but no optimizer loaded"
- # Don't fail validation, just warn
-
- return result
-
- def get_stats(self) -> dict[str, Any]:
- """Get quick statistics without database query.
-
- This returns in-memory stats tracked during this session.
- For historical metrics, use get_metrics() or get_summary().
-
- Returns:
- dict with session statistics:
- {
- "session": {
- "requests_total": int,
- "requests_optimized": int,
- "requests_audit": int,
- "tokens_saved_total": int,
- "cache_hits": int,
- },
- "config": {
- "mode": str,
- "provider": str,
- "cache_optimizer": str | None,
- "semantic_cache": bool,
- },
- "transforms": {
- "smart_crusher_enabled": bool,
- "rolling_window_enabled": bool,
- "cache_aligner_enabled": bool,
- },
- }
-
- Example:
- stats = client.get_stats()
- print(f"Saved {stats['session']['tokens_saved_total']} tokens this session")
- """
- # Initialize session stats if not present
- if not hasattr(self, "_session_stats"):
- self._session_stats = {
- "requests_total": 0,
- "requests_optimized": 0,
- "requests_audit": 0,
- "tokens_saved_total": 0,
- "cache_hits": 0,
- }
-
- return {
- "session": dict(self._session_stats),
- "config": {
- "mode": self._default_mode.value,
- "provider": self._provider.name,
- "cache_optimizer": (self._cache_optimizer.name if self._cache_optimizer else None),
- "semantic_cache": self._semantic_cache_layer is not None,
- },
- "transforms": {
- "smart_crusher_enabled": self._config.smart_crusher.enabled,
- "rolling_window_enabled": self._config.rolling_window.enabled,
- "cache_aligner_enabled": self._config.cache_aligner.enabled,
- },
- }
-
- def _update_session_stats(
- self,
- mode: HeadroomMode,
- tokens_before: int,
- tokens_after: int,
- cache_hit: bool = False,
- ) -> None:
- """Update in-memory session statistics."""
- if not hasattr(self, "_session_stats"):
- self._session_stats = {
- "requests_total": 0,
- "requests_optimized": 0,
- "requests_audit": 0,
- "tokens_saved_total": 0,
- "cache_hits": 0,
- }
-
- self._session_stats["requests_total"] += 1
-
- if mode == HeadroomMode.OPTIMIZE:
- self._session_stats["requests_optimized"] += 1
- self._session_stats["tokens_saved_total"] += max(0, tokens_before - tokens_after)
- else:
- self._session_stats["requests_audit"] += 1
-
- if cache_hit:
- self._session_stats["cache_hits"] += 1
+"""Main HeadroomClient implementation for Headroom SDK."""
+
+from __future__ import annotations
+
+from collections.abc import Iterator
+from datetime import datetime, timezone
+from typing import Any
+
+from .cache import (
+ BaseCacheOptimizer,
+ CacheConfig,
+ CacheOptimizerRegistry,
+ OptimizationContext,
+ SemanticCacheLayer,
+)
+from .config import (
+ HeadroomConfig,
+ HeadroomMode,
+ RequestMetrics,
+ SimulationResult,
+)
+from .parser import parse_messages
+from .pipeline import PipelineExtensionManager, PipelineStage, summarize_routing_markers
+from .providers.base import Provider
+from .providers.registry import call_client_transport
+from .storage import create_storage
+from .tokenizer import Tokenizer
+from .transforms import CacheAligner, TransformPipeline
+from .utils import (
+ compute_messages_hash,
+ compute_prefix_hash,
+ estimate_cost,
+ format_cost,
+ generate_request_id,
+)
+
+
+class ChatCompletions:
+ """Wrapper for chat.completions API (OpenAI-style)."""
+
+ def __init__(self, client: HeadroomClient):
+ self._client = client
+
+ def create(
+ self,
+ *,
+ model: str,
+ messages: list[dict[str, Any]],
+ stream: bool = False,
+ # Headroom-specific parameters
+ headroom_mode: str | None = None,
+ headroom_cache_prefix_tokens: int | None = None,
+ headroom_output_buffer_tokens: int | None = None,
+ headroom_keep_turns: int | None = None,
+ headroom_tool_profiles: dict[str, dict[str, Any]] | None = None,
+ # Pass through all other kwargs
+ **kwargs: Any,
+ ) -> Any:
+ """
+ Create a chat completion with optional Headroom optimization.
+
+ Args:
+ model: Model name.
+ messages: List of messages.
+ stream: Whether to stream the response.
+ headroom_mode: Override default mode ("audit" | "optimize").
+ headroom_cache_prefix_tokens: Target cache-aligned prefix size.
+ headroom_output_buffer_tokens: Reserve tokens for output.
+ headroom_keep_turns: Never drop last N turns.
+ headroom_tool_profiles: Per-tool compression config.
+ **kwargs: Additional arguments passed to underlying client.
+
+ Returns:
+ Chat completion response (or stream iterator).
+ """
+ return self._client._create(
+ model=model,
+ messages=messages,
+ stream=stream,
+ headroom_mode=headroom_mode,
+ headroom_cache_prefix_tokens=headroom_cache_prefix_tokens,
+ headroom_output_buffer_tokens=headroom_output_buffer_tokens,
+ headroom_keep_turns=headroom_keep_turns,
+ headroom_tool_profiles=headroom_tool_profiles,
+ api_style="openai",
+ **kwargs,
+ )
+
+ def simulate(
+ self,
+ *,
+ model: str,
+ messages: list[dict[str, Any]],
+ headroom_mode: str = "optimize",
+ headroom_output_buffer_tokens: int | None = None,
+ headroom_tool_profiles: dict[str, dict[str, Any]] | None = None,
+ **kwargs: Any,
+ ) -> SimulationResult:
+ """
+ Simulate optimization without calling the API.
+
+ Args:
+ model: Model name.
+ messages: List of messages.
+ headroom_mode: Mode to simulate.
+ headroom_output_buffer_tokens: Output buffer to use.
+ headroom_tool_profiles: Tool profiles to use.
+ **kwargs: Additional arguments (ignored).
+
+ Returns:
+ SimulationResult with projected changes.
+ """
+ return self._client._simulate(
+ model=model,
+ messages=messages,
+ headroom_mode=headroom_mode,
+ headroom_output_buffer_tokens=headroom_output_buffer_tokens,
+ headroom_tool_profiles=headroom_tool_profiles,
+ )
+
+
+class Messages:
+ """Wrapper for messages API (Anthropic-style)."""
+
+ def __init__(self, client: HeadroomClient):
+ self._client = client
+
+ def create(
+ self,
+ *,
+ model: str,
+ messages: list[dict[str, Any]],
+ max_tokens: int = 1024,
+ # Headroom-specific parameters
+ headroom_mode: str | None = None,
+ headroom_cache_prefix_tokens: int | None = None,
+ headroom_output_buffer_tokens: int | None = None,
+ headroom_keep_turns: int | None = None,
+ headroom_tool_profiles: dict[str, dict[str, Any]] | None = None,
+ # Pass through all other kwargs
+ **kwargs: Any,
+ ) -> Any:
+ """
+ Create a message with optional Headroom optimization.
+
+ Args:
+ model: Model name.
+ messages: List of messages.
+ max_tokens: Maximum tokens in response.
+ headroom_mode: Override default mode ("audit" | "optimize").
+ headroom_cache_prefix_tokens: Target cache-aligned prefix size.
+ headroom_output_buffer_tokens: Reserve tokens for output.
+ headroom_keep_turns: Never drop last N turns.
+ headroom_tool_profiles: Per-tool compression config.
+ **kwargs: Additional arguments passed to underlying client.
+
+ Returns:
+ Message response.
+ """
+ return self._client._create(
+ model=model,
+ messages=messages,
+ stream=False,
+ headroom_mode=headroom_mode,
+ headroom_cache_prefix_tokens=headroom_cache_prefix_tokens,
+ headroom_output_buffer_tokens=headroom_output_buffer_tokens,
+ headroom_keep_turns=headroom_keep_turns,
+ headroom_tool_profiles=headroom_tool_profiles,
+ api_style="anthropic",
+ max_tokens=max_tokens,
+ **kwargs,
+ )
+
+ def stream(
+ self,
+ *,
+ model: str,
+ messages: list[dict[str, Any]],
+ max_tokens: int = 1024,
+ # Headroom-specific parameters
+ headroom_mode: str | None = None,
+ headroom_cache_prefix_tokens: int | None = None,
+ headroom_output_buffer_tokens: int | None = None,
+ headroom_keep_turns: int | None = None,
+ headroom_tool_profiles: dict[str, dict[str, Any]] | None = None,
+ # Pass through all other kwargs
+ **kwargs: Any,
+ ) -> Any:
+ """
+ Stream a message with optional Headroom optimization.
+
+ Args:
+ model: Model name.
+ messages: List of messages.
+ max_tokens: Maximum tokens in response.
+ headroom_mode: Override default mode ("audit" | "optimize").
+ headroom_cache_prefix_tokens: Target cache-aligned prefix size.
+ headroom_output_buffer_tokens: Reserve tokens for output.
+ headroom_keep_turns: Never drop last N turns.
+ headroom_tool_profiles: Per-tool compression config.
+ **kwargs: Additional arguments passed to underlying client.
+
+ Returns:
+ Stream context manager.
+ """
+ return self._client._create(
+ model=model,
+ messages=messages,
+ stream=True,
+ headroom_mode=headroom_mode,
+ headroom_cache_prefix_tokens=headroom_cache_prefix_tokens,
+ headroom_output_buffer_tokens=headroom_output_buffer_tokens,
+ headroom_keep_turns=headroom_keep_turns,
+ headroom_tool_profiles=headroom_tool_profiles,
+ api_style="anthropic",
+ max_tokens=max_tokens,
+ **kwargs,
+ )
+
+ def simulate(
+ self,
+ *,
+ model: str,
+ messages: list[dict[str, Any]],
+ headroom_mode: str = "optimize",
+ headroom_output_buffer_tokens: int | None = None,
+ headroom_tool_profiles: dict[str, dict[str, Any]] | None = None,
+ **kwargs: Any,
+ ) -> SimulationResult:
+ """
+ Simulate optimization without calling the API.
+
+ Args:
+ model: Model name.
+ messages: List of messages.
+ headroom_mode: Mode to simulate.
+ headroom_output_buffer_tokens: Output buffer to use.
+ headroom_tool_profiles: Tool profiles to use.
+ **kwargs: Additional arguments (ignored).
+
+ Returns:
+ SimulationResult with projected changes.
+ """
+ return self._client._simulate(
+ model=model,
+ messages=messages,
+ headroom_mode=headroom_mode,
+ headroom_output_buffer_tokens=headroom_output_buffer_tokens,
+ headroom_tool_profiles=headroom_tool_profiles,
+ )
+
+
+class HeadroomClient:
+ """
+ Context Budget Controller wrapper for LLM API clients.
+
+ Provides automatic context optimization, waste detection, and
+ cache alignment while maintaining API compatibility.
+ """
+
+ def __init__(
+ self,
+ original_client: Any,
+ provider: Provider,
+ store_url: str | None = None,
+ default_mode: str = "audit",
+ model_context_limits: dict[str, int] | None = None,
+ cache_optimizer: BaseCacheOptimizer | None = None,
+ enable_cache_optimizer: bool = True,
+ enable_semantic_cache: bool = False,
+ config: HeadroomConfig | None = None,
+ ):
+ """
+ Initialize HeadroomClient.
+
+ Args:
+ original_client: The underlying LLM client (OpenAI-compatible).
+ provider: Provider instance for model-specific behavior.
+ store_url: Storage URL (sqlite:// or jsonl://). Defaults to temp dir.
+ default_mode: Default mode ("audit" | "optimize").
+ model_context_limits: Override context limits for models.
+ cache_optimizer: Optional custom cache optimizer. If None and
+ enable_cache_optimizer=True, auto-detects from provider.
+ enable_cache_optimizer: Enable provider-specific cache optimization.
+ enable_semantic_cache: Enable query-level semantic caching.
+ config: Optional HeadroomConfig for full control over all settings
+ including intelligent_context. When provided, takes precedence
+ over individual settings like store_url, default_mode, etc.
+ """
+ self._original = original_client
+ self._provider = provider
+
+ # Set default store_url to temp directory for better DevEx
+ if store_url is None:
+ import os
+ import tempfile
+
+ db_path = os.path.join(tempfile.gettempdir(), "headroom.db")
+ store_url = f"sqlite:///{db_path}"
+
+ self._store_url = store_url
+ self._default_mode = HeadroomMode(default_mode)
+
+ # Use provided config or build from individual parameters
+ if config is not None:
+ self._config = config
+ # Override store_url and mode if explicitly provided in config
+ if config.store_url:
+ self._store_url = config.store_url
+ else:
+ self._config.store_url = store_url
+ self._default_mode = config.default_mode
+ else:
+ # Build config from individual parameters
+ self._config = HeadroomConfig()
+ self._config.store_url = store_url
+ self._config.default_mode = self._default_mode
+ self._config.cache_optimizer.enabled = enable_cache_optimizer
+ self._config.cache_optimizer.enable_semantic_cache = enable_semantic_cache
+
+ if model_context_limits:
+ self._config.model_context_limits.update(model_context_limits)
+
+ # Initialize storage
+ self._storage = create_storage(store_url)
+
+ # Initialize transform pipeline
+ self._pipeline = TransformPipeline(self._config, provider=self._provider)
+ self._pipeline_extensions = PipelineExtensionManager(
+ extensions=self._config.pipeline_extensions,
+ discover=self._config.discover_pipeline_extensions,
+ )
+
+ # Initialize cache optimizer
+ self._cache_optimizer: BaseCacheOptimizer | None = None
+ self._semantic_cache_layer: SemanticCacheLayer | None = None
+
+ if enable_cache_optimizer:
+ if cache_optimizer is not None:
+ self._cache_optimizer = cache_optimizer
+ else:
+ # Auto-detect from provider
+ provider_name = self._provider.name.lower()
+ if CacheOptimizerRegistry.is_registered(provider_name):
+ cache_config = CacheConfig(
+ min_cacheable_tokens=self._config.cache_optimizer.min_cacheable_tokens,
+ )
+ self._cache_optimizer = CacheOptimizerRegistry.get(
+ provider_name,
+ config=cache_config,
+ )
+
+ # Wrap with semantic cache if enabled
+ if enable_semantic_cache and self._cache_optimizer is not None:
+ self._semantic_cache_layer = SemanticCacheLayer(
+ self._cache_optimizer,
+ similarity_threshold=self._config.cache_optimizer.semantic_cache_similarity,
+ max_entries=self._config.cache_optimizer.semantic_cache_max_entries,
+ ttl_seconds=self._config.cache_optimizer.semantic_cache_ttl_seconds,
+ )
+
+ # Public API - OpenAI style
+ self.chat = type("Chat", (), {"completions": ChatCompletions(self)})()
+ # Public API - Anthropic style
+ self.messages = Messages(self)
+ self._pipeline_extensions.emit(
+ PipelineStage.SETUP,
+ operation="sdk.setup",
+ provider=self._provider.name.lower(),
+ metadata={
+ "default_mode": self._default_mode.value,
+ "cache_optimizer_enabled": enable_cache_optimizer,
+ "semantic_cache_enabled": enable_semantic_cache,
+ },
+ )
+
+ def _get_tokenizer(self, model: str) -> Tokenizer:
+ """Get tokenizer for model using provider."""
+ token_counter = self._provider.get_token_counter(model)
+ return Tokenizer(token_counter, model)
+
+ def _get_context_limit(self, model: str) -> int:
+ """Get context limit from user config or provider."""
+ # User override takes precedence
+ limit = self._config.get_context_limit(model)
+ if limit is not None:
+ return limit
+ # Fall back to provider
+ return self._provider.get_context_limit(model)
+
+ def _create(
+ self,
+ *,
+ model: str,
+ messages: list[dict[str, Any]],
+ stream: bool = False,
+ headroom_mode: str | None = None,
+ headroom_cache_prefix_tokens: int | None = None,
+ headroom_output_buffer_tokens: int | None = None,
+ headroom_keep_turns: int | None = None,
+ headroom_tool_profiles: dict[str, dict[str, Any]] | None = None,
+ api_style: str = "openai",
+ **kwargs: Any,
+ ) -> Any:
+ """Internal implementation of create."""
+ request_id = generate_request_id()
+ timestamp = datetime.now(timezone.utc).replace(tzinfo=None)
+ mode = HeadroomMode(headroom_mode) if headroom_mode else self._default_mode
+
+ input_event = self._pipeline_extensions.emit(
+ PipelineStage.INPUT_RECEIVED,
+ operation="sdk.request",
+ request_id=request_id,
+ provider=self._provider.name.lower(),
+ model=model,
+ messages=messages,
+ metadata={"api_style": api_style, "stream": stream, "mode": mode.value},
+ )
+ if input_event.messages is not None:
+ messages = input_event.messages
+
+ tokenizer = self._get_tokenizer(model)
+
+ # Analyze original messages
+ blocks, block_breakdown, waste_signals = parse_messages(messages, tokenizer)
+ tokens_before = tokenizer.count_messages(messages)
+
+ # Compute cache alignment score
+ aligner = CacheAligner(self._config.cache_aligner)
+ cache_alignment_score = aligner.get_alignment_score(messages)
+
+ # Compute stable prefix hash
+ stable_prefix_hash = compute_prefix_hash(messages)
+
+ # Cache optimizer metrics (populated later if optimizer is used)
+ cache_optimizer_used = None
+ cache_optimizer_strategy = None
+ cacheable_tokens = 0
+ breakpoints_inserted = 0
+ estimated_cache_hit = False
+ estimated_savings_percent = 0.0
+ semantic_cache_hit = False
+ cached_response = None
+
+ # Apply transforms if in optimize mode
+ if mode == HeadroomMode.OPTIMIZE:
+ output_buffer = (
+ headroom_output_buffer_tokens or self._config.rolling_window.output_buffer_tokens
+ )
+ model_limit = self._get_context_limit(model)
+
+ result = self._pipeline.apply(
+ messages,
+ model,
+ model_limit=model_limit,
+ output_buffer=output_buffer,
+ tool_profiles=headroom_tool_profiles or {},
+ )
+
+ optimized_messages = result.messages
+ tokens_after = result.tokens_after
+ transforms_applied = result.transforms_applied
+
+ routing_markers = summarize_routing_markers(transforms_applied)
+ if routing_markers:
+ routed_event = self._pipeline_extensions.emit(
+ PipelineStage.INPUT_ROUTED,
+ operation="sdk.request",
+ request_id=request_id,
+ provider=self._provider.name.lower(),
+ model=model,
+ messages=optimized_messages,
+ metadata={
+ "routing_markers": routing_markers,
+ "transforms_applied": transforms_applied,
+ },
+ )
+ if routed_event.messages is not None:
+ optimized_messages = routed_event.messages
+
+ # Apply provider-specific cache optimization
+ if self._cache_optimizer is not None or self._semantic_cache_layer is not None:
+ cache_context = OptimizationContext(
+ provider=self._provider.name.lower(),
+ model=model,
+ query=self._extract_query(optimized_messages),
+ )
+
+ # Check semantic cache first (if enabled)
+ if self._semantic_cache_layer is not None:
+ cache_result = self._semantic_cache_layer.process(
+ optimized_messages, cache_context
+ )
+ semantic_cache_hit = cache_result.semantic_cache_hit
+ if semantic_cache_hit:
+ cached_response = cache_result.cached_response
+
+ # Update metrics from cache result
+ cache_optimizer_used = getattr(
+ cache_result.metrics, "optimizer_name", None
+ ) or (self._cache_optimizer.name if self._cache_optimizer else "")
+ cache_optimizer_strategy = getattr(cache_result.metrics, "strategy", "")
+ cacheable_tokens = cache_result.metrics.cacheable_tokens
+ breakpoints_inserted = cache_result.metrics.breakpoints_inserted
+ estimated_cache_hit = cache_result.metrics.estimated_cache_hit
+ estimated_savings_percent = cache_result.metrics.estimated_savings_percent
+
+ # Apply optimized messages (with cache_control blocks for Anthropic)
+ if cache_result.messages:
+ optimized_messages = cache_result.messages
+
+ elif self._cache_optimizer is not None:
+ # Direct cache optimizer (no semantic layer)
+ cache_result = self._cache_optimizer.optimize(optimized_messages, cache_context)
+ cache_optimizer_used = self._cache_optimizer.name
+ cache_optimizer_strategy = self._cache_optimizer.strategy.value
+ cacheable_tokens = cache_result.metrics.cacheable_tokens
+ breakpoints_inserted = cache_result.metrics.breakpoints_inserted
+ estimated_cache_hit = cache_result.metrics.estimated_cache_hit
+ estimated_savings_percent = cache_result.metrics.estimated_savings_percent
+
+ if cache_result.messages:
+ optimized_messages = cache_result.messages
+
+ transforms_applied.extend(
+ f"cache_optimizer:{t}" for t in (cache_result.transforms_applied or [])
+ )
+
+ compressed_event = self._pipeline_extensions.emit(
+ PipelineStage.INPUT_COMPRESSED,
+ operation="sdk.request",
+ request_id=request_id,
+ provider=self._provider.name.lower(),
+ model=model,
+ messages=optimized_messages,
+ metadata={
+ "tokens_before": tokens_before,
+ "tokens_after": tokens_after,
+ "transforms_applied": transforms_applied,
+ },
+ )
+ if compressed_event.messages is not None:
+ optimized_messages = compressed_event.messages
+ tokens_after = tokenizer.count_messages(optimized_messages)
+
+ # Recalculate prefix hash after optimization
+ stable_prefix_hash = compute_prefix_hash(optimized_messages)
+ else:
+ # Audit mode - no changes
+ optimized_messages = messages
+ tokens_after = tokens_before
+ transforms_applied = []
+
+ presend_event = self._pipeline_extensions.emit(
+ PipelineStage.PRE_SEND,
+ operation="sdk.request",
+ request_id=request_id,
+ provider=self._provider.name.lower(),
+ model=model,
+ messages=optimized_messages,
+ metadata={"api_style": api_style, "stream": stream},
+ )
+ if presend_event.messages is not None:
+ optimized_messages = presend_event.messages
+ tokens_after = tokenizer.count_messages(optimized_messages)
+ stable_prefix_hash = compute_prefix_hash(optimized_messages)
+
+ # Create metrics
+ metrics = RequestMetrics(
+ request_id=request_id,
+ timestamp=timestamp,
+ model=model,
+ stream=stream,
+ mode=mode.value,
+ tokens_input_before=tokens_before,
+ tokens_input_after=tokens_after,
+ block_breakdown=block_breakdown,
+ waste_signals=waste_signals.to_dict(),
+ stable_prefix_hash=stable_prefix_hash,
+ cache_alignment_score=cache_alignment_score,
+ transforms_applied=transforms_applied,
+ messages_hash=compute_messages_hash(messages),
+ # Cache optimizer metrics
+ cache_optimizer_used=cache_optimizer_used,
+ cache_optimizer_strategy=cache_optimizer_strategy,
+ cacheable_tokens=cacheable_tokens,
+ breakpoints_inserted=breakpoints_inserted,
+ estimated_cache_hit=estimated_cache_hit,
+ estimated_savings_percent=estimated_savings_percent,
+ semantic_cache_hit=semantic_cache_hit,
+ )
+
+ # Update session stats
+ self._update_session_stats(
+ mode=mode,
+ tokens_before=tokens_before,
+ tokens_after=tokens_after,
+ cache_hit=semantic_cache_hit,
+ )
+
+ # Return cached response if semantic cache hit
+ if semantic_cache_hit and cached_response is not None:
+ self._storage.save(metrics)
+ return cached_response
+
+ # Call underlying client based on API style
+ try:
+ response = call_client_transport(
+ api_style,
+ self,
+ model=model,
+ messages=optimized_messages,
+ stream=stream,
+ metrics=metrics,
+ **kwargs,
+ )
+
+ self._pipeline_extensions.emit(
+ PipelineStage.POST_SEND,
+ operation="sdk.request",
+ request_id=request_id,
+ provider=self._provider.name.lower(),
+ model=model,
+ messages=optimized_messages,
+ response=response,
+ metadata={"api_style": api_style, "stream": stream},
+ )
+ self._pipeline_extensions.emit(
+ PipelineStage.RESPONSE_RECEIVED,
+ operation="sdk.request",
+ request_id=request_id,
+ provider=self._provider.name.lower(),
+ model=model,
+ response=response,
+ metadata={"api_style": api_style, "stream": stream},
+ )
+ return response
+
+ except Exception as e:
+ metrics.error = str(e)
+ self._storage.save(metrics)
+ raise
+
+ def _call_openai(
+ self,
+ *,
+ model: str,
+ messages: list[dict[str, Any]],
+ stream: bool,
+ metrics: RequestMetrics,
+ **kwargs: Any,
+ ) -> Any:
+ """Call OpenAI-style API."""
+ return call_client_transport(
+ "openai",
+ self,
+ model=model,
+ messages=messages,
+ stream=stream,
+ metrics=metrics,
+ **kwargs,
+ )
+
+ def _call_anthropic(
+ self,
+ *,
+ model: str,
+ messages: list[dict[str, Any]],
+ stream: bool,
+ metrics: RequestMetrics,
+ **kwargs: Any,
+ ) -> Any:
+ """Call Anthropic-style API."""
+ return call_client_transport(
+ "anthropic",
+ self,
+ model=model,
+ messages=messages,
+ stream=stream,
+ metrics=metrics,
+ **kwargs,
+ )
+
+ def _wrap_stream(
+ self,
+ stream: Iterator[Any],
+ metrics: RequestMetrics,
+ ) -> Iterator[Any]:
+ """Wrap stream to pass through chunks and save metrics at end."""
+ try:
+ yield from stream
+ finally:
+ # Save metrics when stream completes
+ # Note: output tokens unknown for streams
+ self._storage.save(metrics)
+
+ def _extract_query(self, messages: list[dict[str, Any]]) -> str:
+ """Extract query from messages for semantic caching.
+
+ Returns the last user message content as the query.
+ """
+ for msg in reversed(messages):
+ if msg.get("role") == "user":
+ content = msg.get("content", "")
+ if isinstance(content, str):
+ return content
+ elif isinstance(content, list):
+ # Content block format
+ for block in content:
+ if isinstance(block, dict) and block.get("type") == "text":
+ text_val = block.get("text", "")
+ return str(text_val) if text_val else ""
+ return ""
+ return ""
+
+ def _store_response_in_semantic_cache(
+ self,
+ messages: list[dict[str, Any]],
+ response: Any,
+ model: str,
+ ) -> None:
+ """Store response in semantic cache for future hits."""
+ if self._semantic_cache_layer is not None:
+ cache_context = OptimizationContext(
+ provider=self._provider.name.lower(),
+ model=model,
+ query=self._extract_query(messages),
+ )
+ # Extract response content for caching
+ response_data = self._extract_response_content(response)
+ if response_data:
+ self._semantic_cache_layer.store_response(messages, response_data, cache_context)
+
+ def _extract_response_content(self, response: Any) -> dict[str, Any] | None:
+ """Extract cacheable content from API response."""
+ try:
+ # OpenAI format
+ if hasattr(response, "choices") and response.choices:
+ choice = response.choices[0]
+ if hasattr(choice, "message"):
+ return {
+ "role": "assistant",
+ "content": choice.message.content,
+ }
+ # Anthropic format
+ elif hasattr(response, "content"):
+ return {
+ "role": "assistant",
+ "content": response.content,
+ }
+ except Exception:
+ pass
+ return None
+
+ def _simulate(
+ self,
+ *,
+ model: str,
+ messages: list[dict[str, Any]],
+ headroom_mode: str = "optimize",
+ headroom_output_buffer_tokens: int | None = None,
+ headroom_tool_profiles: dict[str, dict[str, Any]] | None = None,
+ ) -> SimulationResult:
+ """Internal implementation of simulate."""
+ tokenizer = self._get_tokenizer(model)
+
+ # Analyze original
+ blocks, block_breakdown, waste_signals = parse_messages(messages, tokenizer)
+ tokens_before = tokenizer.count_messages(messages)
+
+ # Compute original cache alignment
+ aligner = CacheAligner(self._config.cache_aligner)
+ cache_alignment_score = aligner.get_alignment_score(messages)
+ compute_prefix_hash(messages)
+
+ # Apply transforms
+ output_buffer = (
+ headroom_output_buffer_tokens or self._config.rolling_window.output_buffer_tokens
+ )
+ model_limit = self._get_context_limit(model)
+
+ result = self._pipeline.simulate(
+ messages,
+ model,
+ model_limit=model_limit,
+ output_buffer=output_buffer,
+ tool_profiles=headroom_tool_profiles or {},
+ )
+
+ tokens_saved = tokens_before - result.tokens_after
+
+ # Estimate cost savings using provider (use output_buffer tokens)
+ # Note: output_buffer reserves tokens for expected model output
+ cost_before = estimate_cost(tokens_before, output_buffer, model, provider=self._provider)
+ cost_after = estimate_cost(
+ result.tokens_after, output_buffer, model, provider=self._provider
+ )
+
+ if cost_before is not None and cost_after is not None:
+ savings = format_cost(cost_before - cost_after)
+ else:
+ savings = "N/A"
+
+ # Recalculate prefix hash after optimization
+ optimized_prefix_hash = compute_prefix_hash(result.messages)
+
+ return SimulationResult(
+ tokens_before=tokens_before,
+ tokens_after=result.tokens_after,
+ tokens_saved=tokens_saved,
+ transforms=result.transforms_applied,
+ estimated_savings=f"{savings} per request",
+ messages_optimized=result.messages,
+ block_breakdown=block_breakdown,
+ waste_signals=waste_signals.to_dict(),
+ stable_prefix_hash=optimized_prefix_hash,
+ cache_alignment_score=cache_alignment_score,
+ )
+
+ def get_metrics(
+ self,
+ start_time: datetime | None = None,
+ end_time: datetime | None = None,
+ model: str | None = None,
+ mode: str | None = None,
+ limit: int = 100,
+ ) -> list[RequestMetrics]:
+ """
+ Query stored metrics.
+
+ Args:
+ start_time: Filter by timestamp >= start_time.
+ end_time: Filter by timestamp <= end_time.
+ model: Filter by model name.
+ mode: Filter by mode.
+ limit: Maximum results.
+
+ Returns:
+ List of RequestMetrics.
+ """
+ return self._storage.query(
+ start_time=start_time,
+ end_time=end_time,
+ model=model,
+ mode=mode,
+ limit=limit,
+ )
+
+ def get_summary(
+ self,
+ start_time: datetime | None = None,
+ end_time: datetime | None = None,
+ ) -> dict[str, Any]:
+ """
+ Get summary statistics.
+
+ Args:
+ start_time: Filter by timestamp >= start_time.
+ end_time: Filter by timestamp <= end_time.
+
+ Returns:
+ Summary statistics dict.
+ """
+ return self._storage.get_summary_stats(start_time, end_time)
+
+ def close(self) -> None:
+ """Close storage connection."""
+ self._storage.close()
+
+ def __enter__(self) -> HeadroomClient:
+ """Context manager entry."""
+ return self
+
+ def __exit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None:
+ """Context manager exit."""
+ self.close()
+
+ def validate_setup(self) -> dict[str, Any]:
+ """Validate that Headroom is properly configured.
+
+ This method checks:
+ - Provider is valid and can count tokens
+ - Storage is accessible and writable
+ - Configuration is valid
+ - Cache optimizer (if enabled) is working
+
+ Returns:
+ dict with validation results:
+ {
+ "valid": True/False,
+ "provider": {"ok": bool, "name": str, "error": str | None},
+ "storage": {"ok": bool, "url": str, "error": str | None},
+ "config": {"ok": bool, "mode": str, "error": str | None},
+ "cache_optimizer": {"ok": bool, "name": str | None, "error": str | None},
+ }
+
+ Raises:
+ ValidationError: If validation fails and raise_on_error=True.
+
+ Example:
+ client = HeadroomClient(...)
+ result = client.validate_setup()
+ if not result["valid"]:
+ print("Setup issues:", result)
+ """
+ result: dict[str, Any] = {
+ "valid": True,
+ "provider": {"ok": False, "name": None, "error": None},
+ "storage": {"ok": False, "url": self._store_url, "error": None},
+ "config": {"ok": False, "mode": self._default_mode.value, "error": None},
+ "cache_optimizer": {"ok": True, "name": None, "error": None},
+ }
+
+ # Validate provider
+ try:
+ result["provider"]["name"] = self._provider.name
+ # Test token counting
+ test_messages = [{"role": "user", "content": "test"}]
+ tokenizer = self._get_tokenizer("gpt-4")
+ count = tokenizer.count_messages(test_messages)
+ if count > 0:
+ result["provider"]["ok"] = True
+ else:
+ result["provider"]["error"] = "Token count returned 0"
+ result["valid"] = False
+ except Exception as e:
+ result["provider"]["error"] = str(e)
+ result["valid"] = False
+
+ # Validate storage
+ try:
+ # Try to get summary (tests read)
+ self._storage.get_summary_stats()
+ result["storage"]["ok"] = True
+ except Exception as e:
+ result["storage"]["error"] = str(e)
+ result["valid"] = False
+
+ # Validate config
+ try:
+ # Check mode is valid
+ if self._default_mode in (HeadroomMode.AUDIT, HeadroomMode.OPTIMIZE):
+ result["config"]["ok"] = True
+ else:
+ result["config"]["error"] = f"Invalid mode: {self._default_mode}"
+ result["valid"] = False
+ except Exception as e:
+ result["config"]["error"] = str(e)
+ result["valid"] = False
+
+ # Validate cache optimizer (if enabled)
+ if self._cache_optimizer is not None:
+ try:
+ result["cache_optimizer"]["name"] = self._cache_optimizer.name
+ result["cache_optimizer"]["ok"] = True
+ except Exception as e:
+ result["cache_optimizer"]["error"] = str(e)
+ # Don't fail validation for cache optimizer issues
+ elif self._config.cache_optimizer.enabled:
+ result["cache_optimizer"]["error"] = "Enabled but no optimizer loaded"
+ # Don't fail validation, just warn
+
+ return result
+
+ def get_stats(self) -> dict[str, Any]:
+ """Get quick statistics without database query.
+
+ This returns in-memory stats tracked during this session.
+ For historical metrics, use get_metrics() or get_summary().
+
+ Returns:
+ dict with session statistics:
+ {
+ "session": {
+ "requests_total": int,
+ "requests_optimized": int,
+ "requests_audit": int,
+ "tokens_saved_total": int,
+ "cache_hits": int,
+ },
+ "config": {
+ "mode": str,
+ "provider": str,
+ "cache_optimizer": str | None,
+ "semantic_cache": bool,
+ },
+ "transforms": {
+ "smart_crusher_enabled": bool,
+ "rolling_window_enabled": bool,
+ "cache_aligner_enabled": bool,
+ },
+ }
+
+ Example:
+ stats = client.get_stats()
+ print(f"Saved {stats['session']['tokens_saved_total']} tokens this session")
+ """
+ # Initialize session stats if not present
+ if not hasattr(self, "_session_stats"):
+ self._session_stats = {
+ "requests_total": 0,
+ "requests_optimized": 0,
+ "requests_audit": 0,
+ "tokens_saved_total": 0,
+ "cache_hits": 0,
+ }
+
+ return {
+ "session": dict(self._session_stats),
+ "config": {
+ "mode": self._default_mode.value,
+ "provider": self._provider.name,
+ "cache_optimizer": (self._cache_optimizer.name if self._cache_optimizer else None),
+ "semantic_cache": self._semantic_cache_layer is not None,
+ },
+ "transforms": {
+ "smart_crusher_enabled": self._config.smart_crusher.enabled,
+ "rolling_window_enabled": self._config.rolling_window.enabled,
+ "cache_aligner_enabled": self._config.cache_aligner.enabled,
+ },
+ }
+
+ def _update_session_stats(
+ self,
+ mode: HeadroomMode,
+ tokens_before: int,
+ tokens_after: int,
+ cache_hit: bool = False,
+ ) -> None:
+ """Update in-memory session statistics."""
+ if not hasattr(self, "_session_stats"):
+ self._session_stats = {
+ "requests_total": 0,
+ "requests_optimized": 0,
+ "requests_audit": 0,
+ "tokens_saved_total": 0,
+ "cache_hits": 0,
+ }
+
+ self._session_stats["requests_total"] += 1
+
+ if mode == HeadroomMode.OPTIMIZE:
+ self._session_stats["requests_optimized"] += 1
+ self._session_stats["tokens_saved_total"] += max(0, tokens_before - tokens_after)
+ else:
+ self._session_stats["requests_audit"] += 1
+
+ if cache_hit:
+ self._session_stats["cache_hits"] += 1
diff --git a/headroom/providers/aider/__init__.py b/headroom/providers/aider/__init__.py
new file mode 100644
index 000000000..681d720aa
--- /dev/null
+++ b/headroom/providers/aider/__init__.py
@@ -0,0 +1,5 @@
+"""Aider-specific provider helpers."""
+
+from .runtime import build_launch_env
+
+__all__ = ["build_launch_env"]
diff --git a/headroom/providers/aider/runtime.py b/headroom/providers/aider/runtime.py
new file mode 100644
index 000000000..d6253ad93
--- /dev/null
+++ b/headroom/providers/aider/runtime.py
@@ -0,0 +1,24 @@
+"""Runtime helpers for Aider integrations."""
+
+from __future__ import annotations
+
+import os
+from collections.abc import Mapping
+
+from headroom.providers.claude import proxy_base_url as claude_proxy_base_url
+from headroom.providers.codex import proxy_base_url as codex_proxy_base_url
+
+
+def build_launch_env(
+ port: int, environ: Mapping[str, str] | None = None
+) -> tuple[dict[str, str], list[str]]:
+ """Build environment variables for Aider through the local proxy."""
+ env = dict(environ or os.environ)
+ openai_base_url = codex_proxy_base_url(port)
+ anthropic_base_url = claude_proxy_base_url(port)
+ env["OPENAI_API_BASE"] = openai_base_url
+ env["ANTHROPIC_BASE_URL"] = anthropic_base_url
+ return env, [
+ f"OPENAI_API_BASE={openai_base_url}",
+ f"ANTHROPIC_BASE_URL={anthropic_base_url}",
+ ]
diff --git a/headroom/providers/claude/__init__.py b/headroom/providers/claude/__init__.py
new file mode 100644
index 000000000..5db99094a
--- /dev/null
+++ b/headroom/providers/claude/__init__.py
@@ -0,0 +1,5 @@
+"""Claude-specific provider helpers."""
+
+from .runtime import DEFAULT_API_URL, proxy_base_url
+
+__all__ = ["DEFAULT_API_URL", "proxy_base_url"]
diff --git a/headroom/providers/claude/runtime.py b/headroom/providers/claude/runtime.py
new file mode 100644
index 000000000..fa1662103
--- /dev/null
+++ b/headroom/providers/claude/runtime.py
@@ -0,0 +1,10 @@
+"""Runtime helpers for Claude-facing integrations."""
+
+from __future__ import annotations
+
+DEFAULT_API_URL = "https://api.anthropic.com"
+
+
+def proxy_base_url(port: int) -> str:
+ """Return the local proxy base URL used by Claude integrations."""
+ return f"http://127.0.0.1:{port}"
diff --git a/headroom/providers/codex/__init__.py b/headroom/providers/codex/__init__.py
new file mode 100644
index 000000000..367e60007
--- /dev/null
+++ b/headroom/providers/codex/__init__.py
@@ -0,0 +1,5 @@
+"""Codex-specific provider helpers."""
+
+from .runtime import DEFAULT_API_URL, build_launch_env, proxy_base_url
+
+__all__ = ["DEFAULT_API_URL", "build_launch_env", "proxy_base_url"]
diff --git a/headroom/providers/codex/runtime.py b/headroom/providers/codex/runtime.py
new file mode 100644
index 000000000..e639114de
--- /dev/null
+++ b/headroom/providers/codex/runtime.py
@@ -0,0 +1,23 @@
+"""Runtime helpers for Codex/OpenAI-facing integrations."""
+
+from __future__ import annotations
+
+import os
+from collections.abc import Mapping
+
+DEFAULT_API_URL = "https://api.openai.com"
+
+
+def proxy_base_url(port: int) -> str:
+ """Return the local proxy base URL used by OpenAI-compatible integrations."""
+ return f"http://127.0.0.1:{port}/v1"
+
+
+def build_launch_env(
+ port: int, environ: Mapping[str, str] | None = None
+) -> tuple[dict[str, str], list[str]]:
+ """Build environment variables for Codex through the local proxy."""
+ env = dict(environ or os.environ)
+ base_url = proxy_base_url(port)
+ env["OPENAI_BASE_URL"] = base_url
+ return env, [f"OPENAI_BASE_URL={base_url}"]
diff --git a/headroom/providers/copilot/__init__.py b/headroom/providers/copilot/__init__.py
new file mode 100644
index 000000000..3b0c065ba
--- /dev/null
+++ b/headroom/providers/copilot/__init__.py
@@ -0,0 +1,21 @@
+"""Copilot-specific provider helpers."""
+
+from .wrap import (
+ build_launch_env,
+ detect_running_proxy_backend,
+ model_configured,
+ provider_key_source,
+ query_proxy_config,
+ resolve_provider_type,
+ validate_configuration,
+)
+
+__all__ = [
+ "build_launch_env",
+ "detect_running_proxy_backend",
+ "model_configured",
+ "provider_key_source",
+ "query_proxy_config",
+ "resolve_provider_type",
+ "validate_configuration",
+]
diff --git a/headroom/providers/copilot/wrap.py b/headroom/providers/copilot/wrap.py
new file mode 100644
index 000000000..d37beb8f5
--- /dev/null
+++ b/headroom/providers/copilot/wrap.py
@@ -0,0 +1,120 @@
+"""Copilot wrapper provider helpers."""
+
+from __future__ import annotations
+
+import json
+import os
+import urllib.error
+import urllib.request
+from collections.abc import Mapping
+from typing import Any
+
+import click
+
+
+def resolve_provider_type(
+ backend: str | None, provider_type: str, environ: Mapping[str, str] | None = None
+) -> str:
+ """Resolve Copilot BYOK provider type for the current proxy backend."""
+ if provider_type != "auto":
+ return provider_type
+
+ env = environ or os.environ
+ effective_backend = backend or env.get("HEADROOM_BACKEND") or "anthropic"
+ return "anthropic" if effective_backend == "anthropic" else "openai"
+
+
+def query_proxy_config(port: int) -> dict[str, Any] | None:
+ """Query the running proxy's feature configuration via /health."""
+ url = f"http://127.0.0.1:{port}/health"
+ try:
+ with urllib.request.urlopen(url, timeout=2) as response:
+ payload = json.loads(response.read().decode("utf-8"))
+ except (OSError, urllib.error.URLError, ValueError, json.JSONDecodeError):
+ return None
+
+ config = payload.get("config")
+ if not isinstance(config, dict):
+ return None
+ return config
+
+
+def detect_running_proxy_backend(port: int) -> str | None:
+ """Read the backend of an already-running proxy from its health endpoint."""
+ config = query_proxy_config(port)
+ if config is None:
+ return None
+ backend = config.get("backend")
+ return backend if isinstance(backend, str) else None
+
+
+def validate_configuration(
+ *,
+ provider_type: str,
+ wire_api: str | None,
+ backend: str | None,
+) -> None:
+ """Validate Copilot BYOK provider and wire-api settings."""
+ if provider_type == "anthropic" and wire_api is not None:
+ raise click.ClickException(
+ "--wire-api is only valid when Copilot is using the openai provider type."
+ )
+ if wire_api == "responses" and backend not in (None, "anthropic"):
+ raise click.ClickException(
+ "--wire-api responses is not supported with translated backends; use completions."
+ )
+
+
+def provider_key_source(provider_type: str) -> str:
+ """Return the preferred provider key variable for the selected provider type."""
+ return "ANTHROPIC_API_KEY" if provider_type == "anthropic" else "OPENAI_API_KEY"
+
+
+def build_launch_env(
+ *,
+ port: int,
+ provider_type: str,
+ wire_api: str | None,
+ environ: Mapping[str, str] | None = None,
+) -> tuple[dict[str, str], list[str]]:
+ """Build the Copilot BYOK environment for the selected provider type."""
+ env = dict(environ or os.environ)
+ env["COPILOT_PROVIDER_TYPE"] = provider_type
+ env.pop("COPILOT_PROVIDER_WIRE_API", None)
+
+ if not env.get("COPILOT_PROVIDER_API_KEY"):
+ key = env.get(provider_key_source(provider_type), "")
+ if key:
+ env["COPILOT_PROVIDER_API_KEY"] = key
+
+ if provider_type == "anthropic":
+ base_url = f"http://127.0.0.1:{port}"
+ env["COPILOT_PROVIDER_BASE_URL"] = base_url
+ return env, [
+ "COPILOT_PROVIDER_TYPE=anthropic",
+ f"COPILOT_PROVIDER_BASE_URL={base_url}",
+ ]
+
+ effective_wire_api = wire_api or "completions"
+ base_url = f"http://127.0.0.1:{port}/v1"
+ env["COPILOT_PROVIDER_BASE_URL"] = base_url
+ env["COPILOT_PROVIDER_WIRE_API"] = effective_wire_api
+ return env, [
+ "COPILOT_PROVIDER_TYPE=openai",
+ f"COPILOT_PROVIDER_BASE_URL={base_url}",
+ f"COPILOT_PROVIDER_WIRE_API={effective_wire_api}",
+ ]
+
+
+def model_configured(copilot_args: tuple[str, ...], env: Mapping[str, str]) -> bool:
+ """Return True when Copilot BYOK model selection is configured."""
+ if env.get("COPILOT_MODEL") or env.get("COPILOT_PROVIDER_MODEL_ID"):
+ return True
+
+ for idx, arg in enumerate(copilot_args):
+ if arg == "--model" and idx + 1 < len(copilot_args):
+ return True
+ if arg.startswith("--model="):
+ return True
+
+ return False
diff --git a/headroom/providers/cursor/__init__.py b/headroom/providers/cursor/__init__.py
new file mode 100644
index 000000000..e15974a58
--- /dev/null
+++ b/headroom/providers/cursor/__init__.py
@@ -0,0 +1,5 @@
+"""Cursor-specific provider helpers."""
+
+from .runtime import CursorProxyTargets, build_proxy_targets, render_setup_lines
+
+__all__ = ["CursorProxyTargets", "build_proxy_targets", "render_setup_lines"]
diff --git a/headroom/providers/cursor/runtime.py b/headroom/providers/cursor/runtime.py
new file mode 100644
index 000000000..a00794504
--- /dev/null
+++ b/headroom/providers/cursor/runtime.py
@@ -0,0 +1,44 @@
+"""Runtime helpers for Cursor integrations."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+from headroom.providers.claude import proxy_base_url as claude_proxy_base_url
+from headroom.providers.codex import proxy_base_url as codex_proxy_base_url
+
+
+@dataclass(frozen=True)
+class CursorProxyTargets:
+ """Resolved local proxy targets shown in Cursor setup instructions."""
+
+ openai_base_url: str
+ anthropic_base_url: str
+
+
+def build_proxy_targets(port: int) -> CursorProxyTargets:
+ """Build the local proxy URLs shown to Cursor users."""
+ return CursorProxyTargets(
+ openai_base_url=codex_proxy_base_url(port),
+ anthropic_base_url=claude_proxy_base_url(port),
+ )
+
+
+def render_setup_lines(port: int) -> list[str]:
+ """Render the Cursor setup instructions for the local proxy."""
+ targets = build_proxy_targets(port)
+ return [
+ " Headroom proxy is running. Configure Cursor:",
+ "",
+ " For OpenAI models:",
+ f" Base URL: {targets.openai_base_url}",
+ " API Key: your-openai-api-key",
+ "",
+ " For Anthropic models:",
+ f" Base URL: {targets.anthropic_base_url}",
+ " API Key: your-anthropic-api-key",
+ "",
+ " In Cursor:",
+ " Settings > Models > OpenAI API Key > Override OpenAI Base URL",
+ f" Set to: {targets.openai_base_url}",
+ ]
diff --git a/headroom/providers/gemini/__init__.py b/headroom/providers/gemini/__init__.py
new file mode 100644
index 000000000..6e0d83281
--- /dev/null
+++ b/headroom/providers/gemini/__init__.py
@@ -0,0 +1,5 @@
+"""Gemini-specific provider helpers."""
+
+from .runtime import DEFAULT_API_URL
+
+__all__ = ["DEFAULT_API_URL"]
diff --git a/headroom/providers/gemini/runtime.py b/headroom/providers/gemini/runtime.py
new file mode 100644
index 000000000..ce5966313
--- /dev/null
+++ b/headroom/providers/gemini/runtime.py
@@ -0,0 +1,5 @@
+"""Runtime helpers for Gemini-facing integrations."""
+
+from __future__ import annotations
+
+DEFAULT_API_URL = "https://generativelanguage.googleapis.com"
diff --git a/headroom/providers/openclaw/__init__.py b/headroom/providers/openclaw/__init__.py
new file mode 100644
index 000000000..41144e733
--- /dev/null
+++ b/headroom/providers/openclaw/__init__.py
@@ -0,0 +1,15 @@
+"""OpenClaw-specific provider helpers."""
+
+from .wrap import (
+ build_plugin_entry,
+ build_unwrap_entry,
+ decode_entry_json,
+ normalize_gateway_provider_ids,
+)
+
+__all__ = [
+ "build_plugin_entry",
+ "build_unwrap_entry",
+ "decode_entry_json",
+ "normalize_gateway_provider_ids",
+]
diff --git a/headroom/providers/openclaw/wrap.py b/headroom/providers/openclaw/wrap.py
new file mode 100644
index 000000000..a5f741a1c
--- /dev/null
+++ b/headroom/providers/openclaw/wrap.py
@@ -0,0 +1,89 @@
+"""OpenClaw wrapper provider helpers."""
+
+from __future__ import annotations
+
+import json
+from typing import Any
+
+DEFAULT_GATEWAY_PROVIDER_IDS = ["openai-codex"]
+
+
+def normalize_gateway_provider_ids(provider_ids: tuple[str, ...] | None) -> list[str]:
+ """Normalize configured OpenClaw provider ids."""
+ values = provider_ids or ()
+ seen: set[str] = set()
+ normalized: list[str] = []
+
+ for entry in values:
+ provider_id = entry.strip()
+ if not provider_id or provider_id in seen:
+ continue
+ seen.add(provider_id)
+ normalized.append(provider_id)
+
+ return normalized or DEFAULT_GATEWAY_PROVIDER_IDS.copy()
+
+
+def decode_entry_json(raw_value: str | None) -> Any | None:
+ """Decode a JSON payload captured from `openclaw config get` when available."""
+ if not raw_value:
+ return None
+
+ try:
+ return json.loads(raw_value)
+ except json.JSONDecodeError:
+ return raw_value
+
+
+def build_plugin_entry(
+ *,
+ existing_entry: Any,
+ proxy_port: int,
+ startup_timeout_ms: int,
+ python_path: str | None,
+ no_auto_start: bool,
+ gateway_provider_ids: tuple[str, ...] | None,
+ enabled: bool,
+) -> dict[str, object]:
+ """Merge managed Headroom plugin settings with any existing entry payload."""
+ base_entry = existing_entry if isinstance(existing_entry, dict) else {}
+ existing_config = base_entry.get("config")
+ next_config = dict(existing_config) if isinstance(existing_config, dict) else {}
+
+ next_config["proxyPort"] = proxy_port
+ next_config["autoStart"] = not no_auto_start
+ next_config["startupTimeoutMs"] = startup_timeout_ms
+ next_config["gatewayProviderIds"] = normalize_gateway_provider_ids(gateway_provider_ids)
+
+ if python_path:
+ next_config["pythonPath"] = python_path
+ else:
+ next_config.pop("pythonPath", None)
+
+ return {
+ **base_entry,
+ "enabled": enabled,
+ "config": next_config,
+ }
+
+
+def build_unwrap_entry(existing_entry: Any) -> dict[str, object]:
+ """Disable the managed plugin while preserving unrelated user config."""
+ base_entry = existing_entry if isinstance(existing_entry, dict) else {}
+ existing_config: dict[str, object] = {}
+ if isinstance(existing_entry, dict) and isinstance(existing_entry.get("config"), dict):
+ existing_config = {
+ key: value
+ for key, value in existing_entry["config"].items()
+ if key
+ not in {
+ "gatewayProviderIds",
+ "proxyUrl",
+ "proxyPort",
+ "autoStart",
+ "startupTimeoutMs",
+ "pythonPath",
+ }
+ }
+
+ return {**base_entry, "enabled": False, "config": existing_config}
diff --git a/headroom/providers/proxy_routes.py b/headroom/providers/proxy_routes.py
new file mode 100644
index 000000000..4b881f6c9
--- /dev/null
+++ b/headroom/providers/proxy_routes.py
@@ -0,0 +1,337 @@
+# mypy: disable-error-code=no-untyped-def
+"""Provider-specific proxy route registration."""
+
+from __future__ import annotations
+
+import logging
+from typing import Any
+
+from fastapi import FastAPI, Request, WebSocket
+from fastapi.responses import Response
+
+from headroom.proxy.handlers.openai import _resolve_codex_routing_headers
+
+logger = logging.getLogger("headroom.proxy.routes")
+
+
+def register_provider_routes(app: FastAPI, proxy: Any) -> None:
+ """Register provider-specific proxy endpoints."""
+
+ @app.post("/v1/messages")
+ async def anthropic_messages(request: Request):
+ return await proxy.handle_anthropic_messages(request)
+
+ @app.post("/v1/messages/count_tokens")
+ async def anthropic_count_tokens(request: Request):
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.api_target("anthropic"),
+ "count_tokens",
+ "anthropic",
+ )
+
+ @app.post("/v1/messages/batches")
+ async def anthropic_batch_create(request: Request):
+ return await proxy.handle_anthropic_batch_create(request)
+
+ @app.get("/v1/messages/batches")
+ async def anthropic_batch_list(request: Request):
+ return await proxy.handle_anthropic_batch_passthrough(request)
+
+ @app.get("/v1/messages/batches/{batch_id}")
+ async def anthropic_batch_get(request: Request, batch_id: str):
+ return await proxy.handle_anthropic_batch_passthrough(request, batch_id)
+
+ @app.get("/v1/messages/batches/{batch_id}/results")
+ async def anthropic_batch_results(request: Request, batch_id: str):
+ return await proxy.handle_anthropic_batch_results(request, batch_id)
+
+ @app.post("/v1/messages/batches/{batch_id}/cancel")
+ async def anthropic_batch_cancel(request: Request, batch_id: str):
+ return await proxy.handle_anthropic_batch_passthrough(request, batch_id)
+
+ @app.post("/v1/chat/completions")
+ async def openai_chat(request: Request):
+ return await proxy.handle_openai_chat(request)
+
+ @app.post("/v1/responses")
+ async def openai_responses(request: Request):
+ return await proxy.handle_openai_responses(request)
+
+ @app.post("/v1/codex/responses")
+ async def openai_v1_codex_responses(request: Request):
+ return await proxy.handle_openai_responses(request)
+
+ @app.post("/backend-api/responses")
+ async def openai_codex_responses(request: Request):
+ return await proxy.handle_openai_responses(request)
+
+ @app.post("/backend-api/codex/responses")
+ async def openai_codex_nested_responses(request: Request):
+ return await proxy.handle_openai_responses(request)
+
+ @app.websocket("/v1/responses")
+ async def openai_responses_ws(websocket: WebSocket):
+ await proxy.handle_openai_responses_ws(websocket)
+
+ @app.websocket("/v1/codex/responses")
+ async def openai_v1_codex_responses_ws(websocket: WebSocket):
+ await proxy.handle_openai_responses_ws(websocket)
+
+ @app.api_route("/v1/responses/{sub_path:path}", methods=["GET", "POST", "DELETE"])
+ async def openai_responses_sub(request: Request, sub_path: str):
+ headers = dict(request.headers.items())
+ headers.pop("host", None)
+ headers, is_chatgpt_auth = _resolve_codex_routing_headers(headers)
+
+ if is_chatgpt_auth:
+ url = f"https://chatgpt.com/backend-api/codex/responses/{sub_path}"
+ else:
+ url = f"{proxy.provider_runtime.api_target('openai')}/v1/responses/{sub_path}"
+
+ if request.url.query:
+ url = f"{url}?{request.url.query}"
+
+ body = await request.body()
+ try:
+ assert proxy.http_client is not None
+ resp = await proxy.http_client.request(
+ request.method,
+ url,
+ headers=headers,
+ content=body,
+ timeout=120.0,
+ )
+ return Response(
+ content=resp.content,
+ status_code=resp.status_code,
+ headers=dict(resp.headers),
+ )
+ except Exception as exc:
+ logger.error("Passthrough /v1/responses/%s failed: %s", sub_path, exc)
+ return Response(content=str(exc), status_code=502)
+
+ @app.api_route("/v1/codex/responses/{sub_path:path}", methods=["GET", "POST", "DELETE"])
+ async def openai_v1_codex_responses_sub(request: Request, sub_path: str):
+ return await openai_responses_sub(request, sub_path)
+
+ @app.websocket("/backend-api/responses")
+ async def openai_codex_responses_ws(websocket: WebSocket):
+ await proxy.handle_openai_responses_ws(websocket)
+
+ @app.websocket("/backend-api/codex/responses")
+ async def openai_codex_nested_responses_ws(websocket: WebSocket):
+ await proxy.handle_openai_responses_ws(websocket)
+
+ @app.api_route("/backend-api/responses/{sub_path:path}", methods=["GET", "POST", "DELETE"])
+ async def openai_codex_responses_sub(request: Request, sub_path: str):
+ return await openai_responses_sub(request, sub_path)
+
+ @app.api_route(
+ "/backend-api/codex/responses/{sub_path:path}", methods=["GET", "POST", "DELETE"]
+ )
+ async def openai_codex_nested_responses_sub(request: Request, sub_path: str):
+ return await openai_responses_sub(request, sub_path)
+
+ @app.post("/v1/batches")
+ async def create_batch(request: Request):
+ return await proxy.handle_batch_create(request)
+
+ @app.get("/v1/batches")
+ async def list_batches(request: Request):
+ return await proxy.handle_batch_list(request)
+
+ @app.get("/v1/batches/{batch_id}")
+ async def get_batch(request: Request, batch_id: str):
+ return await proxy.handle_batch_get(request, batch_id)
+
+ @app.post("/v1/batches/{batch_id}/cancel")
+ async def cancel_batch(request: Request, batch_id: str):
+ return await proxy.handle_batch_cancel(request, batch_id)
+
+ @app.post("/v1beta/models/{model}:generateContent")
+ async def gemini_generate_content(request: Request, model: str):
+ return await proxy.handle_gemini_generate_content(request, model)
+
+ @app.post("/v1beta/models/{model}:streamGenerateContent")
+ async def gemini_stream_generate_content(request: Request, model: str):
+ return await proxy.handle_gemini_stream_generate_content(request, model)
+
+ @app.post("/v1beta/models/{model}:countTokens")
+ async def gemini_count_tokens(request: Request, model: str):
+ return await proxy.handle_gemini_count_tokens(request, model)
+
+ @app.post("/v1internal:streamGenerateContent")
+ async def google_cloudcode_stream_generate_content(request: Request):
+ return await proxy.handle_google_cloudcode_stream(request)
+
+ @app.post("/v1/v1internal:streamGenerateContent")
+ async def google_cloudcode_stream_generate_content_v1(request: Request):
+ return await proxy.handle_google_cloudcode_stream(request)
+
+ @app.post("/serving-endpoints/{model}/invocations")
+ async def databricks_invocations(request: Request, model: str):
+ return await proxy.handle_databricks_invocations(request, model)
+
+ @app.get("/v1/models")
+ async def list_models(request: Request):
+ provider_name = proxy.provider_runtime.model_metadata_provider(dict(request.headers))
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.api_target(provider_name),
+ "models",
+ provider_name,
+ )
+
+ @app.get("/v1/models/{model_id}")
+ async def get_model(request: Request, model_id: str):
+ provider_name = proxy.provider_runtime.model_metadata_provider(dict(request.headers))
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.api_target(provider_name),
+ "models",
+ provider_name,
+ )
+
+ @app.post("/v1/embeddings")
+ async def openai_embeddings(request: Request):
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.api_target("openai"),
+ "embeddings",
+ "openai",
+ )
+
+ @app.post("/v1/moderations")
+ async def openai_moderations(request: Request):
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.api_target("openai"),
+ "moderations",
+ "openai",
+ )
+
+ @app.post("/v1/images/generations")
+ async def openai_images_generations(request: Request):
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.api_target("openai"),
+ "images/generations",
+ "openai",
+ )
+
+ @app.post("/v1/audio/transcriptions")
+ async def openai_audio_transcriptions(request: Request):
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.api_target("openai"),
+ "audio/transcriptions",
+ "openai",
+ )
+
+ @app.post("/v1/audio/speech")
+ async def openai_audio_speech(request: Request):
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.api_target("openai"),
+ "audio/speech",
+ "openai",
+ )
+
+ @app.get("/v1beta/models")
+ async def gemini_list_models(request: Request):
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.api_target("gemini"),
+ "models",
+ "gemini",
+ )
+
+ @app.get("/v1beta/models/{model_name}")
+ async def gemini_get_model(request: Request, model_name: str):
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.api_target("gemini"),
+ "models",
+ "gemini",
+ )
+
+ @app.post("/v1beta/models/{model}:embedContent")
+ async def gemini_embed_content(request: Request, model: str):
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.api_target("gemini"),
+ "embedContent",
+ "gemini",
+ )
+
+ @app.post("/v1beta/models/{model}:batchEmbedContents")
+ async def gemini_batch_embed_contents(request: Request, model: str):
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.api_target("gemini"),
+ "batchEmbedContents",
+ "gemini",
+ )
+
+ @app.post("/v1beta/models/{model}:batchGenerateContent")
+ async def gemini_batch_create(request: Request, model: str):
+ return await proxy.handle_google_batch_create(request, model)
+
+ @app.get("/v1beta/batches/{batch_name}")
+ async def gemini_batch_get(request: Request, batch_name: str):
+ return await proxy.handle_google_batch_results(request, batch_name)
+
+ @app.post("/v1beta/batches/{batch_name}:cancel")
+ async def gemini_batch_cancel(request: Request, batch_name: str):
+ return await proxy.handle_google_batch_passthrough(request, batch_name)
+
+ @app.delete("/v1beta/batches/{batch_name}")
+ async def gemini_batch_delete(request: Request, batch_name: str):
+ return await proxy.handle_google_batch_passthrough(request, batch_name)
+
+ @app.post("/v1beta/cachedContents")
+ async def gemini_create_cached_content(request: Request):
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.api_target("gemini"),
+ "cachedContents",
+ "gemini",
+ )
+
+ @app.get("/v1beta/cachedContents")
+ async def gemini_list_cached_contents(request: Request):
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.api_target("gemini"),
+ "cachedContents",
+ "gemini",
+ )
+
+ @app.get("/v1beta/cachedContents/{cache_id}")
+ async def gemini_get_cached_content(request: Request, cache_id: str):
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.api_target("gemini"),
+ "cachedContents",
+ "gemini",
+ )
+
+ @app.delete("/v1beta/cachedContents/{cache_id}")
+ async def gemini_delete_cached_content(request: Request, cache_id: str):
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.api_target("gemini"),
+ "cachedContents",
+ "gemini",
+ )
+
+ @app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE"])
+ async def passthrough(request: Request, path: str):
+ custom_base = request.headers.get("x-headroom-base-url")
+ if custom_base:
+ return await proxy.handle_passthrough(request, custom_base.rstrip("/"))
+ return await proxy.handle_passthrough(
+ request,
+ proxy.provider_runtime.select_passthrough_base_url(dict(request.headers)),
+ )
diff --git a/headroom/providers/registry.py b/headroom/providers/registry.py
new file mode 100644
index 000000000..7f0c82836
--- /dev/null
+++ b/headroom/providers/registry.py
@@ -0,0 +1,316 @@
+"""Provider runtime registry and transport helpers."""
+
+from __future__ import annotations
+
+import logging
+import os
+from collections.abc import Callable, Mapping
+from dataclasses import dataclass
+from typing import TYPE_CHECKING, Any, cast
+
+from headroom.providers.claude import DEFAULT_API_URL as DEFAULT_ANTHROPIC_API_URL
+from headroom.providers.codex import DEFAULT_API_URL as DEFAULT_OPENAI_API_URL
+from headroom.providers.gemini import DEFAULT_API_URL as DEFAULT_GEMINI_API_URL
+
+DEFAULT_CLOUDCODE_API_URL = "https://cloudcode-pa.googleapis.com"
+
+if TYPE_CHECKING:
+ from headroom.backends.base import Backend
+ from headroom.providers.base import Provider
+
+AnyLLMBackendType: Any = None
+LiteLLMBackendType: Any = None
+
+
+@dataclass(frozen=True)
+class ProviderApiOverrides:
+ """Optional upstream API URL overrides configured for the proxy."""
+
+ anthropic: str | None = None
+ openai: str | None = None
+ gemini: str | None = None
+ cloudcode: str | None = None
+
+
+@dataclass(frozen=True)
+class ProviderApiTargets:
+ """Resolved upstream API targets after provider normalization."""
+
+ anthropic: str = DEFAULT_ANTHROPIC_API_URL
+ openai: str = DEFAULT_OPENAI_API_URL
+ gemini: str = DEFAULT_GEMINI_API_URL
+ cloudcode: str = DEFAULT_CLOUDCODE_API_URL
+
+
+@dataclass(frozen=True)
+class ProxyProviderRuntime:
+ """Provider runtime state used by the proxy server."""
+
+ api_targets: ProviderApiTargets
+ pipeline_providers: dict[str, Provider]
+
+ def api_target(self, provider_name: str) -> str:
+ """Return the resolved upstream target for a provider."""
+ return {
+ "anthropic": self.api_targets.anthropic,
+ "openai": self.api_targets.openai,
+ "gemini": self.api_targets.gemini,
+ "cloudcode": self.api_targets.cloudcode,
+ }[provider_name]
+
+ def pipeline_provider(self, provider_name: str) -> Provider:
+ """Return the pipeline provider instance for a provider."""
+ return self.pipeline_providers[provider_name]
+
+ def model_metadata_provider(self, headers: Mapping[str, str]) -> str:
+ """Resolve the upstream provider that should serve OpenAI-style model metadata."""
+ return "anthropic" if _is_anthropic_auth(headers) else "openai"
+
+ def select_passthrough_base_url(self, headers: Mapping[str, str]) -> str:
+ """Resolve the upstream base URL for catch-all passthrough requests."""
+ if _is_anthropic_auth(headers):
+ return self.api_targets.anthropic
+ if headers.get("x-goog-api-key"):
+ return self.api_targets.gemini
+ if headers.get("api-key"):
+ azure_base = headers.get("x-headroom-base-url", "")
+ if azure_base:
+ return azure_base.rstrip("/")
+ return self.api_targets.openai
+
+
+def _normalize_api_url(url: str | None, *, default: str) -> str:
+ if not url:
+ return default
+
+ normalized = url.rstrip("/")
+ if normalized.endswith("/v1"):
+ normalized = normalized[:-3]
+ return normalized
+
+
+def resolve_api_overrides(
+ *,
+ anthropic_api_url: str | None,
+ openai_api_url: str | None,
+ gemini_api_url: str | None,
+ cloudcode_api_url: str | None,
+ environ: Mapping[str, str] | None = None,
+) -> ProviderApiOverrides:
+ """Resolve provider API URL overrides from CLI/config inputs and environment."""
+ env = environ or os.environ
+ return ProviderApiOverrides(
+ anthropic=anthropic_api_url or env.get("ANTHROPIC_TARGET_API_URL"),
+ openai=openai_api_url or env.get("OPENAI_TARGET_API_URL"),
+ gemini=gemini_api_url or env.get("GEMINI_TARGET_API_URL"),
+ cloudcode=cloudcode_api_url or env.get("CLOUDCODE_TARGET_API_URL"),
+ )
+
+
+def resolve_api_targets(overrides: ProviderApiOverrides) -> ProviderApiTargets:
+ """Resolve normalized upstream provider targets from configured overrides."""
+ return ProviderApiTargets(
+ anthropic=_normalize_api_url(overrides.anthropic, default=DEFAULT_ANTHROPIC_API_URL),
+ openai=_normalize_api_url(overrides.openai, default=DEFAULT_OPENAI_API_URL),
+ gemini=_normalize_api_url(overrides.gemini, default=DEFAULT_GEMINI_API_URL),
+ cloudcode=_normalize_api_url(overrides.cloudcode, default=DEFAULT_CLOUDCODE_API_URL),
+ )
+
+
+def build_proxy_provider_runtime(config: Any) -> ProxyProviderRuntime:
+ """Build provider runtime objects and resolved targets for the proxy."""
+ from headroom.providers.anthropic import AnthropicProvider
+ from headroom.providers.openai import OpenAIProvider
+
+ api_targets = resolve_api_targets(config.provider_api_overrides)
+ return ProxyProviderRuntime(
+ api_targets=api_targets,
+ pipeline_providers={
+ "anthropic": AnthropicProvider(),
+ "openai": OpenAIProvider(),
+ },
+ )
+
+
+def create_proxy_backend(
+ *,
+ backend: str,
+ anyllm_provider: str,
+ bedrock_region: str | None,
+ logger: logging.Logger,
+) -> Backend | None:
+ """Create the optional translated backend for Anthropic proxy requests."""
+ if backend == "anthropic":
+ return None
+
+ if backend == "anyllm" or backend.startswith("anyllm-"):
+ provider = anyllm_provider
+ try:
+ backend_cls = _load_anyllm_backend()
+ instance = cast("Backend", backend_cls(provider=provider))
+ logger.info("any-llm backend enabled (provider=%s)", provider)
+ return instance
+ except ImportError as exc:
+ logger.warning("any-llm backend not available: %s", exc)
+ return None
+ except Exception as exc: # pragma: no cover - defensive logging
+ logger.error("Failed to initialize any-llm backend: %s", exc)
+ return None
+
+ normalized_backend = backend if backend.startswith("litellm-") else f"litellm-{backend}"
+ provider = normalized_backend.replace("litellm-", "")
+ try:
+ backend_cls = _load_litellm_backend()
+ instance = cast("Backend", backend_cls(provider=provider, region=bedrock_region))
+ logger.info("LiteLLM backend enabled (provider=%s, region=%s)", provider, bedrock_region)
+ return instance
+ except ImportError as exc:
+ logger.warning("LiteLLM backend not available: %s", exc)
+ return None
+ except Exception as exc: # pragma: no cover - defensive logging
+ logger.error("Failed to initialize LiteLLM backend: %s", exc)
+ return None
+
+
+def format_backend_status(*, backend: str, anyllm_provider: str, bedrock_region: str | None) -> str:
+ """Build the human-readable backend status string shown in CLI/server output."""
+ if backend == "anthropic":
+ return "ANTHROPIC (direct API)"
+ if backend == "anyllm" or backend.startswith("anyllm-"):
+ return f"{anyllm_provider.title()} via any-llm"
+
+ from headroom.backends.litellm import get_provider_config
+
+ provider = backend.replace("litellm-", "")
+ provider_config = get_provider_config(provider)
+ if provider_config.uses_region:
+ return f"{provider_config.display_name} via LiteLLM (region={bedrock_region})"
+ return f"{provider_config.display_name} via LiteLLM"
+
+
+def call_client_transport(
+ api_style: str,
+ client: Any,
+ *,
+ model: str,
+ messages: list[dict[str, Any]],
+ stream: bool,
+ metrics: Any,
+ **kwargs: Any,
+) -> Any:
+ """Dispatch the SDK request to the provider-specific transport handler."""
+ try:
+ transport = _CLIENT_TRANSPORTS[api_style]
+ except KeyError as exc:
+ raise ValueError(f"Unsupported api_style: {api_style}") from exc
+
+ return transport(
+ client,
+ model=model,
+ messages=messages,
+ stream=stream,
+ metrics=metrics,
+ **kwargs,
+ )
+
+
+def _load_anyllm_backend() -> Any:
+ global AnyLLMBackendType
+ if AnyLLMBackendType is None:
+ from headroom.backends.anyllm import AnyLLMBackend
+
+ AnyLLMBackendType = AnyLLMBackend
+ return AnyLLMBackendType
+
+
+def _load_litellm_backend() -> Any:
+ global LiteLLMBackendType
+ if LiteLLMBackendType is None:
+ from headroom.backends.litellm import LiteLLMBackend
+
+ LiteLLMBackendType = LiteLLMBackend
+ return LiteLLMBackendType
+
+
+def _call_openai_transport(
+ client: Any,
+ *,
+ model: str,
+ messages: list[dict[str, Any]],
+ stream: bool,
+ metrics: Any,
+ **kwargs: Any,
+) -> Any:
+ if stream:
+ response = client._original.chat.completions.create(
+ model=model,
+ messages=messages,
+ stream=True,
+ **kwargs,
+ )
+ return client._wrap_stream(response, metrics)
+
+ response = client._original.chat.completions.create(
+ model=model,
+ messages=messages,
+ stream=False,
+ **kwargs,
+ )
+
+ if hasattr(response, "usage") and response.usage:
+ metrics.tokens_output = response.usage.completion_tokens
+ if hasattr(response.usage, "prompt_tokens_details"):
+ details = response.usage.prompt_tokens_details
+ if hasattr(details, "cached_tokens"):
+ metrics.cached_tokens = details.cached_tokens
+
+ client._storage.save(metrics)
+ return response
+
+
+def _call_anthropic_transport(
+ client: Any,
+ *,
+ model: str,
+ messages: list[dict[str, Any]],
+ stream: bool,
+ metrics: Any,
+ **kwargs: Any,
+) -> Any:
+ if stream:
+ stream_manager = client._original.messages.stream(
+ model=model,
+ messages=messages,
+ **kwargs,
+ )
+ client._storage.save(metrics)
+ return stream_manager
+
+ response = client._original.messages.create(
+ model=model,
+ messages=messages,
+ **kwargs,
+ )
+
+ if hasattr(response, "usage") and response.usage:
+ metrics.tokens_output = response.usage.output_tokens
+ if hasattr(response.usage, "cache_read_input_tokens"):
+ metrics.cached_tokens = response.usage.cache_read_input_tokens
+
+ client._storage.save(metrics)
+ return response
+
+
+_ClientTransport = Callable[..., Any]
+_CLIENT_TRANSPORTS: dict[str, _ClientTransport] = {
+ "anthropic": _call_anthropic_transport,
+ "openai": _call_openai_transport,
+}
+
+
+def _is_anthropic_auth(headers: Mapping[str, str]) -> bool:
+ return bool(
+ headers.get("x-api-key")
+ or headers.get("anthropic-version")
+ or headers.get("Authorization", "").startswith("Bearer sk-ant-")
+ )
diff --git a/headroom/proxy/models.py b/headroom/proxy/models.py
index 559784d2e..3092d7f68 100644
--- a/headroom/proxy/models.py
+++ b/headroom/proxy/models.py
@@ -1,250 +1,262 @@
-"""Data models for the Headroom proxy.
-
-Contains configuration and data classes used across the proxy modules.
-Extracted from server.py to keep the codebase maintainable.
-"""
-
-from __future__ import annotations
-
-from dataclasses import dataclass, field
-from datetime import datetime
-from typing import Any, Literal
-
-# =============================================================================
-# Data Models
-# =============================================================================
-
-
-@dataclass
-class RequestLog:
- """Complete log of a single request."""
-
- request_id: str
- timestamp: str
- provider: str
- model: str
-
- # Tokens
- input_tokens_original: int
- input_tokens_optimized: int
- output_tokens: int | None
- tokens_saved: int
- savings_percent: float
-
- # Performance
- optimization_latency_ms: float
- total_latency_ms: float | None
-
- # Metadata
- tags: dict[str, str]
- cache_hit: bool
- transforms_applied: list[str]
-
- # Waste signals detected in original messages
- waste_signals: dict[str, int] | None = None
-
- # Request/Response (optional, for debugging)
- request_messages: list[dict] | None = None
- response_content: str | None = None
- error: str | None = None
-
- # NOTE (Unit 2 follow-up): stage timings and session_id were briefly
- # added here but are now emitted exclusively through
- # ``emit_stage_timings_log`` (structured log line) and Prometheus.
- # They were never populated on ``RequestLog`` instances, so the
- # fields were removed to avoid confusing readers who expect
- # them to be set. If a JSONL consumer needs them, have the consumer
- # merge ``stage_timings`` log lines by ``request_id``.
-
-
-@dataclass
-class CacheEntry:
- """Cached response entry."""
-
- response_body: bytes
- response_headers: dict[str, str]
- created_at: datetime
- ttl_seconds: int
- hit_count: int = 0
- tokens_saved_per_hit: int = 0
-
-
-@dataclass
-class RateLimitState:
- """Token bucket rate limiter state."""
-
- tokens: float
- last_update: float
-
-
-@dataclass
-class ProxyConfig:
- """Proxy configuration."""
-
- # Server
- host: str = "127.0.0.1"
- port: int = 8787
- anthropic_api_url: str | None = None # Custom Anthropic API URL override
- openai_api_url: str | None = None # Custom OpenAI API URL override
- gemini_api_url: str | None = None # Custom Gemini API URL override
- cloudcode_api_url: str | None = None # Custom Cloud Code Assist API URL override
-
- # Backend: "anthropic" (direct API), "litellm-*" (via LiteLLM), or "anyllm" (via any-llm)
- backend: str = "anthropic"
- bedrock_region: str = "us-west-2"
- bedrock_profile: str | None = None
- anyllm_provider: str = "openai"
-
- # Optimization mode: "token" (rewrite for max compression) or
- # "cache" (freeze prior turns for prefix-cache stability).
- mode: str = "token"
-
- # Optimization
- optimize: bool = True
- image_optimize: bool = True
- min_tokens_to_crush: int = 500
- max_items_after_crush: int = 50
- keep_last_turns: int = 4
-
- # CCR Tool Injection
- ccr_inject_tool: bool = True
- ccr_inject_system_instructions: bool = False
-
- # CCR Response Handling
- ccr_handle_responses: bool = True
- ccr_max_retrieval_rounds: int = 3
-
- # CCR Context Tracking
- ccr_context_tracking: bool = True
- ccr_proactive_expansion: bool = True
- ccr_max_proactive_expansions: int = 2
-
- # Code-aware compression (disabled by default — use code graph tools instead)
- code_aware_enabled: bool = False
-
- # Code graph live watcher (triggers incremental reindex on file changes)
- code_graph_watcher: bool = False
-
- # Per-tool compression profiles
- tool_profiles: dict[str, Any] | None = None
-
- # Read lifecycle management
- read_lifecycle: bool = True
-
- # Smart content routing
- smart_routing: bool = True
-
- # Intelligent context management
- intelligent_context: bool = True
- intelligent_context_scoring: bool = True
- intelligent_context_compress_first: bool = True
-
- # Caching
- cache_enabled: bool = True
- cache_ttl_seconds: int = 3600
- cache_max_entries: int = 1000
-
- # Rate limiting
- rate_limit_enabled: bool = True
- rate_limit_requests_per_minute: int = 60
- rate_limit_tokens_per_minute: int = 100000
-
- # Retry
- retry_enabled: bool = True
- retry_max_attempts: int = 3
- retry_base_delay_ms: int = 1000
- retry_max_delay_ms: int = 30000
-
- # Prefix freeze
- prefix_freeze_enabled: bool = True
- prefix_freeze_session_ttl: int = 600
-
- # Cost tracking
- cost_tracking_enabled: bool = True
- budget_limit_usd: float | None = None
- budget_period: Literal["hourly", "daily", "monthly"] = "daily"
-
- # Logging
- log_requests: bool = True
- log_file: str | None = None
- log_full_messages: bool = False
-
- # Fallback
- fallback_enabled: bool = False
- fallback_provider: str | None = None
-
- # Timeouts
- request_timeout_seconds: int = 300
- connect_timeout_seconds: int = 10
-
- # Connection pool
- max_connections: int = 500
- max_keepalive_connections: int = 100
- http2: bool = True
-
- # Memory System
- memory_enabled: bool = False
- memory_backend: Literal["local", "qdrant-neo4j"] = "local"
- memory_db_path: str = "" # Empty = auto: {cwd}/.headroom/memory.db
- memory_inject_tools: bool = True
- traffic_learning_enabled: bool = False
- traffic_learning_agent_type: str = "unknown" # Which agent is being wrapped
- memory_use_native_tool: bool = False
- memory_inject_context: bool = True
- memory_top_k: int = 10
- memory_min_similarity: float = 0.3
- memory_qdrant_host: str = "localhost"
- memory_qdrant_port: int = 6333
- memory_neo4j_uri: str = "neo4j://localhost:7687"
- memory_neo4j_user: str = "neo4j"
- memory_neo4j_password: str = "password"
- memory_bridge_enabled: bool = False
- memory_bridge_md_paths: list[str] = field(default_factory=list)
- memory_bridge_md_format: str = "auto"
- memory_bridge_auto_import: bool = False
- memory_bridge_export_path: str = ""
-
- # License / Usage Reporting
- license_key: str | None = None
- license_cloud_url: str = "https://app.headroomlabs.ai"
- license_report_interval: int = 300
-
- # Compression Hooks
- hooks: Any = None
- pipeline_extensions: list[Any] = field(default_factory=list)
- discover_pipeline_extensions: bool = True
-
- # Subscription Window Tracking (Anthropic OAuth accounts)
- subscription_tracking_enabled: bool = True
- subscription_poll_interval_s: int = 10
- subscription_active_window_s: int = 60
-
- # Stateless mode — disable all filesystem writes for read-only / container deployments
- stateless: bool = False
-
- # Unit 4: Bounded pre-upstream concurrency for Anthropic replay storms.
- #
- # Caps the number of simultaneous requests allowed to run the
- # pre-upstream phase of ``handle_anthropic_messages`` (request JSON
- # read → deep-copy → first compression stage → memory-context lookup
- # → first upstream connect). Prevents cold-start replay storms from
- # monopolising the event loop / thread pool and starving ``/livez``,
- # ``/readyz``, and new Codex WS opens. Compression stays on.
- #
- # ``None`` (default) -> auto-compute ``max(2, min(8, os.cpu_count() or 4))``.
- # ``0`` or negative -> disables the semaphore (unbounded); useful for
- # the Unit 6 counter-factual and for deliberately reproducing the
- # original starvation. Any positive integer is honored verbatim.
- #
- # CLI: ``--anthropic-pre-upstream-concurrency``.
- # Env: ``HEADROOM_ANTHROPIC_PRE_UPSTREAM_CONCURRENCY``.
- # Precedence: CLI > env > auto-compute.
- anthropic_pre_upstream_concurrency: int | None = None
- # Upper bound for waiting on the Anthropic pre-upstream semaphore
- # before failing fast with a 503 + Retry-After. Keeps the queue bounded
- # when all pre-upstream slots are occupied by slow/hung work.
- anthropic_pre_upstream_acquire_timeout_seconds: float = 15.0
- # Fail-open timeout for Anthropic memory-context lookup while the request
- # is still holding a pre-upstream slot. Compression already has its own
- # COMPRESSION_TIMEOUT_SECONDS guard; this bounds the memory leg too.
- anthropic_pre_upstream_memory_context_timeout_seconds: float = 2.0
+"""Data models for the Headroom proxy.
+
+Contains configuration and data classes used across the proxy modules.
+Extracted from server.py to keep the codebase maintainable.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from datetime import datetime
+from typing import Any, Literal
+
+from headroom.providers.registry import ProviderApiOverrides
+
+# =============================================================================
+# Data Models
+# =============================================================================
+
+
+@dataclass
+class RequestLog:
+ """Complete log of a single request."""
+
+ request_id: str
+ timestamp: str
+ provider: str
+ model: str
+
+ # Tokens
+ input_tokens_original: int
+ input_tokens_optimized: int
+ output_tokens: int | None
+ tokens_saved: int
+ savings_percent: float
+
+ # Performance
+ optimization_latency_ms: float
+ total_latency_ms: float | None
+
+ # Metadata
+ tags: dict[str, str]
+ cache_hit: bool
+ transforms_applied: list[str]
+
+ # Waste signals detected in original messages
+ waste_signals: dict[str, int] | None = None
+
+ # Request/Response (optional, for debugging)
+ request_messages: list[dict] | None = None
+ response_content: str | None = None
+ error: str | None = None
+
+ # NOTE (Unit 2 follow-up): stage timings and session_id were briefly
+ # added here but are now emitted exclusively through
+ # ``emit_stage_timings_log`` (structured log line) and Prometheus.
+ # They were never populated on ``RequestLog`` instances, so the
+ # fields were removed to avoid confusing readers who expect
+ # them to be set. If a JSONL consumer needs them, have the consumer
+ # merge ``stage_timings`` log lines by ``request_id``.
+
+
+@dataclass
+class CacheEntry:
+ """Cached response entry."""
+
+ response_body: bytes
+ response_headers: dict[str, str]
+ created_at: datetime
+ ttl_seconds: int
+ hit_count: int = 0
+ tokens_saved_per_hit: int = 0
+
+
+@dataclass
+class RateLimitState:
+ """Token bucket rate limiter state."""
+
+ tokens: float
+ last_update: float
+
+
+@dataclass
+class ProxyConfig:
+ """Proxy configuration."""
+
+ # Server
+ host: str = "127.0.0.1"
+ port: int = 8787
+ anthropic_api_url: str | None = None # Custom Anthropic API URL override
+ openai_api_url: str | None = None # Custom OpenAI API URL override
+ gemini_api_url: str | None = None # Custom Gemini API URL override
+ cloudcode_api_url: str | None = None # Custom Cloud Code Assist API URL override
+
+ # Backend: "anthropic" (direct API), "litellm-*" (via LiteLLM), or "anyllm" (via any-llm)
+ backend: str = "anthropic"
+ bedrock_region: str = "us-west-2"
+ bedrock_profile: str | None = None
+ anyllm_provider: str = "openai"
+
+ # Optimization mode: "token" (rewrite for max compression) or
+ # "cache" (freeze prior turns for prefix-cache stability).
+ mode: str = "token"
+
+ # Optimization
+ optimize: bool = True
+ image_optimize: bool = True
+ min_tokens_to_crush: int = 500
+ max_items_after_crush: int = 50
+ keep_last_turns: int = 4
+
+ # CCR Tool Injection
+ ccr_inject_tool: bool = True
+ ccr_inject_system_instructions: bool = False
+
+ # CCR Response Handling
+ ccr_handle_responses: bool = True
+ ccr_max_retrieval_rounds: int = 3
+
+ # CCR Context Tracking
+ ccr_context_tracking: bool = True
+ ccr_proactive_expansion: bool = True
+ ccr_max_proactive_expansions: int = 2
+
+ # Code-aware compression (disabled by default — use code graph tools instead)
+ code_aware_enabled: bool = False
+
+ # Code graph live watcher (triggers incremental reindex on file changes)
+ code_graph_watcher: bool = False
+
+ # Per-tool compression profiles
+ tool_profiles: dict[str, Any] | None = None
+
+ # Read lifecycle management
+ read_lifecycle: bool = True
+
+ # Smart content routing
+ smart_routing: bool = True
+
+ # Intelligent context management
+ intelligent_context: bool = True
+ intelligent_context_scoring: bool = True
+ intelligent_context_compress_first: bool = True
+
+ # Caching
+ cache_enabled: bool = True
+ cache_ttl_seconds: int = 3600
+ cache_max_entries: int = 1000
+
+ # Rate limiting
+ rate_limit_enabled: bool = True
+ rate_limit_requests_per_minute: int = 60
+ rate_limit_tokens_per_minute: int = 100000
+
+ # Retry
+ retry_enabled: bool = True
+ retry_max_attempts: int = 3
+ retry_base_delay_ms: int = 1000
+ retry_max_delay_ms: int = 30000
+
+ # Prefix freeze
+ prefix_freeze_enabled: bool = True
+ prefix_freeze_session_ttl: int = 600
+
+ # Cost tracking
+ cost_tracking_enabled: bool = True
+ budget_limit_usd: float | None = None
+ budget_period: Literal["hourly", "daily", "monthly"] = "daily"
+
+ # Logging
+ log_requests: bool = True
+ log_file: str | None = None
+ log_full_messages: bool = False
+
+ # Fallback
+ fallback_enabled: bool = False
+ fallback_provider: str | None = None
+
+ # Timeouts
+ request_timeout_seconds: int = 300
+ connect_timeout_seconds: int = 10
+
+ # Connection pool
+ max_connections: int = 500
+ max_keepalive_connections: int = 100
+ http2: bool = True
+
+ # Memory System
+ memory_enabled: bool = False
+ memory_backend: Literal["local", "qdrant-neo4j"] = "local"
+ memory_db_path: str = "" # Empty = auto: {cwd}/.headroom/memory.db
+ memory_inject_tools: bool = True
+ traffic_learning_enabled: bool = False
+ traffic_learning_agent_type: str = "unknown" # Which agent is being wrapped
+ memory_use_native_tool: bool = False
+ memory_inject_context: bool = True
+ memory_top_k: int = 10
+ memory_min_similarity: float = 0.3
+ memory_qdrant_host: str = "localhost"
+ memory_qdrant_port: int = 6333
+ memory_neo4j_uri: str = "neo4j://localhost:7687"
+ memory_neo4j_user: str = "neo4j"
+ memory_neo4j_password: str = "password"
+ memory_bridge_enabled: bool = False
+ memory_bridge_md_paths: list[str] = field(default_factory=list)
+ memory_bridge_md_format: str = "auto"
+ memory_bridge_auto_import: bool = False
+ memory_bridge_export_path: str = ""
+
+ # License / Usage Reporting
+ license_key: str | None = None
+ license_cloud_url: str = "https://app.headroomlabs.ai"
+ license_report_interval: int = 300
+
+ # Compression Hooks
+ hooks: Any = None
+ pipeline_extensions: list[Any] = field(default_factory=list)
+ discover_pipeline_extensions: bool = True
+
+ # Subscription Window Tracking (Anthropic OAuth accounts)
+ subscription_tracking_enabled: bool = True
+ subscription_poll_interval_s: int = 10
+ subscription_active_window_s: int = 60
+
+ # Stateless mode — disable all filesystem writes for read-only / container deployments
+ stateless: bool = False
+
+ # Unit 4: Bounded pre-upstream concurrency for Anthropic replay storms.
+ #
+ # Caps the number of simultaneous requests allowed to run the
+ # pre-upstream phase of ``handle_anthropic_messages`` (request JSON
+ # read → deep-copy → first compression stage → memory-context lookup
+ # → first upstream connect). Prevents cold-start replay storms from
+ # monopolising the event loop / thread pool and starving ``/livez``,
+ # ``/readyz``, and new Codex WS opens. Compression stays on.
+ #
+ # ``None`` (default) -> auto-compute ``max(2, min(8, os.cpu_count() or 4))``.
+ # ``0`` or negative -> disables the semaphore (unbounded); useful for
+ # the Unit 6 counter-factual and for deliberately reproducing the
+ # original starvation. Any positive integer is honored verbatim.
+ #
+ # CLI: ``--anthropic-pre-upstream-concurrency``.
+ # Env: ``HEADROOM_ANTHROPIC_PRE_UPSTREAM_CONCURRENCY``.
+ # Precedence: CLI > env > auto-compute.
+ anthropic_pre_upstream_concurrency: int | None = None
+ # Upper bound for waiting on the Anthropic pre-upstream semaphore
+ # before failing fast with a 503 + Retry-After. Keeps the queue bounded
+ # when all pre-upstream slots are occupied by slow/hung work.
+ anthropic_pre_upstream_acquire_timeout_seconds: float = 15.0
+ # Fail-open timeout for Anthropic memory-context lookup while the request
+ # is still holding a pre-upstream slot. Compression already has its own
+ # COMPRESSION_TIMEOUT_SECONDS guard; this bounds the memory leg too.
+ anthropic_pre_upstream_memory_context_timeout_seconds: float = 2.0
+
+ @property
+ def provider_api_overrides(self) -> ProviderApiOverrides:
+ """Return provider API URL overrides as a dedicated provider config object."""
+ return ProviderApiOverrides(
+ anthropic=self.anthropic_api_url,
+ openai=self.openai_api_url,
+ gemini=self.gemini_api_url,
+ cloudcode=self.cloudcode_api_url,
+ )
diff --git a/headroom/proxy/server.py b/headroom/proxy/server.py
index 7e6229119..31b117d7d 100644
--- a/headroom/proxy/server.py
+++ b/headroom/proxy/server.py
@@ -1,3029 +1,2555 @@
-"""Headroom Proxy Server - Production Ready.
-
-A full-featured LLM proxy with optimization, caching, rate limiting,
-and observability.
-
-Features:
-- Context optimization (SmartCrusher, CacheAligner, RollingWindow)
-- Semantic caching (save costs on repeated queries)
-- Rate limiting (token bucket)
-- Retry with exponential backoff
-- Cost tracking and budgets
-- Request tagging and metadata
-- Provider fallback
-- Prometheus metrics
-- Full request/response logging
-
-Usage:
- python -m headroom.proxy.server --port 8787
-
- # With Claude Code:
- ANTHROPIC_BASE_URL=http://localhost:8787 claude
-"""
-
-from __future__ import annotations
-
-import argparse
-import asyncio
-import json
-import logging
-import os
-import sys
-import time
-from datetime import datetime, timezone
-from pathlib import Path
-from typing import TYPE_CHECKING, Any, Literal
-
-if TYPE_CHECKING:
- from ..backends.base import Backend
- from ..cache.compression_cache import CompressionCache
- from ..memory.tracker import MemoryTracker
-
-
-import httpx
-
-try:
- import uvicorn
- from fastapi import Depends, FastAPI, HTTPException, Request, Response, WebSocket
- from fastapi.middleware.cors import CORSMiddleware
- from fastapi.responses import HTMLResponse, JSONResponse, PlainTextResponse
-
- FASTAPI_AVAILABLE = True
-except ImportError:
- FASTAPI_AVAILABLE = False
-
-# Add parent to path for imports
-sys.path.insert(0, str(Path(__file__).parent.parent.parent))
-
-from headroom._version import __version__
-from headroom.cache.compression_feedback import get_compression_feedback
-from headroom.cache.compression_store import get_compression_store
-from headroom.ccr import (
- CCR_TOOL_NAME,
- # Batch processing
- CCRResponseHandler,
- CCRToolInjector,
- ContextTracker,
- ContextTrackerConfig,
- ResponseHandlerConfig,
- parse_tool_call,
-)
-from headroom.config import (
- CacheAlignerConfig,
- CCRConfig,
- IntelligentContextConfig,
- ReadLifecycleConfig,
- RollingWindowConfig,
- SmartCrusherConfig,
-)
-from headroom.dashboard import get_dashboard_html
-from headroom.observability import (
- LangfuseTracingConfig,
- OTelMetricsConfig,
- configure_langfuse_tracing,
- configure_otel_metrics,
- get_langfuse_tracing_status,
- get_otel_metrics_status,
- shutdown_headroom_tracing,
- shutdown_otel_metrics,
-)
-from headroom.pipeline import PipelineExtensionManager, PipelineStage
-from headroom.providers.anthropic import AnthropicProvider
-from headroom.providers.openai import OpenAIProvider
-
-# =============================================================================
-# Extracted modules (re-exported for backward compatibility)
-# =============================================================================
-from headroom.proxy.cost import (
- _CACHE_ECONOMICS, # noqa: F401
- CostTracker, # noqa: F401
- _summarize_transforms, # noqa: F401
- build_prefix_cache_stats, # noqa: F401
- build_session_summary, # noqa: F401
- merge_cost_stats, # noqa: F401
-)
-from headroom.proxy.helpers import (
- COMPRESSION_TIMEOUT_SECONDS, # noqa: F401
- MAX_COMPRESSION_CACHE_SESSIONS, # noqa: F401
- MAX_MESSAGE_ARRAY_LENGTH, # noqa: F401
- MAX_REQUEST_BODY_SIZE, # noqa: F401
- MAX_SSE_BUFFER_SIZE, # noqa: F401
- _get_image_compressor, # noqa: F401
- _get_rtk_stats, # noqa: F401
- _read_request_json, # noqa: F401
- _setup_file_logging, # noqa: F401
- is_anthropic_auth, # noqa: F401
- jitter_delay_ms,
-)
-from headroom.proxy.memory_handler import MemoryConfig, MemoryHandler
-
-# Data models (extracted to headroom/proxy/models.py for maintainability)
-from headroom.proxy.models import CacheEntry, ProxyConfig, RateLimitState, RequestLog # noqa: F401
-from headroom.proxy.modes import (
- PROXY_MODE_CACHE,
- PROXY_MODE_TOKEN,
- is_token_mode,
- normalize_proxy_mode,
-)
-from headroom.proxy.prometheus_metrics import PrometheusMetrics # noqa: F401
-from headroom.proxy.rate_limiter import TokenBucketRateLimiter # noqa: F401
-from headroom.proxy.request_logger import RequestLogger # noqa: F401
-from headroom.proxy.semantic_cache import SemanticCache # noqa: F401
-from headroom.proxy.warmup import WarmupRegistry
-from headroom.proxy.ws_session_registry import WebSocketSessionRegistry
-from headroom.subscription.base import get_quota_registry, reset_quota_registry
-from headroom.subscription.codex_rate_limits import get_codex_rate_limit_state
-from headroom.subscription.copilot_quota import get_copilot_quota_tracker
-from headroom.subscription.tracker import (
- configure_subscription_tracker,
- get_subscription_tracker,
-)
-from headroom.telemetry import get_telemetry_collector
-from headroom.telemetry.beacon import is_telemetry_enabled
-from headroom.telemetry.toin import get_toin
-from headroom.transforms import (
- CacheAligner,
- CodeAwareCompressor,
- CodeCompressorConfig,
- ContentRouter,
- ContentRouterConfig,
- IntelligentContextManager,
- RollingWindow,
- SmartCrusher,
- Transform,
- TransformPipeline,
- is_tree_sitter_available,
-)
-
-fcntl: Any = None
-try:
- import fcntl as _fcntl
-
- fcntl = _fcntl
- HAS_FCNTL = True
-except ImportError:
- HAS_FCNTL = False
-
-_build_prefix_cache_stats = build_prefix_cache_stats
-_build_session_summary = build_session_summary
-_merge_cost_stats = merge_cost_stats
-
-
-logging.basicConfig(
- level=logging.INFO, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
-)
-logger = logging.getLogger("headroom.proxy")
-
-# Preserve module-level backend symbols for tests and patch-based integrations
-# while still importing the actual backend implementations lazily at runtime.
-AnyLLMBackend = None
-LiteLLMBackend = None
-
-# Always-on file logging to ~/.headroom/logs/ for `headroom perf` analysis
-_setup_file_logging()
-
-
-# Compression pipeline timeout in seconds
-
-
-from headroom.proxy.handlers import ( # noqa: E402
- AnthropicHandlerMixin,
- BatchHandlerMixin,
- GeminiHandlerMixin,
- OpenAIHandlerMixin,
- StreamingMixin,
-)
-
-
-class HeadroomProxy(
- StreamingMixin,
- AnthropicHandlerMixin,
- OpenAIHandlerMixin,
- GeminiHandlerMixin,
- BatchHandlerMixin,
-):
- """Production-ready Headroom optimization proxy."""
-
- ANTHROPIC_API_URL = "https://api.anthropic.com"
- OPENAI_API_URL = "https://api.openai.com"
- GEMINI_API_URL = "https://generativelanguage.googleapis.com"
- CLOUDCODE_API_URL = "https://cloudcode-pa.googleapis.com"
-
- def __init__(self, config: ProxyConfig):
- self.config = config
- self.config.mode = normalize_proxy_mode(self.config.mode)
- self.pipeline_extensions = PipelineExtensionManager(
- hooks=config.hooks,
- extensions=config.pipeline_extensions,
- discover=config.discover_pipeline_extensions,
- )
-
- # Reset per-instance API targets first so test runs and multiple app instances
- # do not leak class-level overrides across each other.
- HeadroomProxy.ANTHROPIC_API_URL = "https://api.anthropic.com"
- HeadroomProxy.OPENAI_API_URL = "https://api.openai.com"
- HeadroomProxy.GEMINI_API_URL = "https://generativelanguage.googleapis.com"
- HeadroomProxy.CLOUDCODE_API_URL = "https://cloudcode-pa.googleapis.com"
-
- # Override ANTHROPIC_API_URL with config if set.
- # Strip trailing /v1 or /v1/ to avoid double-path (e.g., .../v1/v1/models).
- if config.anthropic_api_url:
- url = config.anthropic_api_url.rstrip("/")
- if url.endswith("/v1"):
- url = url[:-3]
- HeadroomProxy.ANTHROPIC_API_URL = url
-
- # Override OPENAI_API_URL with config if set.
- # Strip trailing /v1 or /v1/ to avoid double-path (e.g., .../v1/v1/models).
- if config.openai_api_url:
- url = config.openai_api_url.rstrip("/")
- if url.endswith("/v1"):
- url = url[:-3]
- HeadroomProxy.OPENAI_API_URL = url
-
- # Override GEMINI_API_URL with config if set.
- if config.gemini_api_url:
- gurl = config.gemini_api_url.rstrip("/")
- if gurl.endswith("/v1"):
- gurl = gurl[:-3]
- HeadroomProxy.GEMINI_API_URL = gurl
-
- # Override CLOUDCODE_API_URL with config if set.
- if config.cloudcode_api_url:
- curl = config.cloudcode_api_url.rstrip("/")
- if curl.endswith("/v1"):
- curl = curl[:-3]
- HeadroomProxy.CLOUDCODE_API_URL = curl
-
- # Initialize providers
- self.anthropic_provider = AnthropicProvider()
- self.openai_provider = OpenAIProvider()
-
- # Initialize transforms based on routing mode
- # Choose context manager: IntelligentContextManager (smart) or RollingWindow (legacy)
- context_manager: Transform # Can be either IntelligentContextManager or RollingWindow
- if config.intelligent_context:
- # Get TOIN instance for learned pattern integration
- toin = get_toin() if config.intelligent_context_scoring else None
- context_manager = IntelligentContextManager(
- config=IntelligentContextConfig(
- enabled=True,
- keep_system=True,
- keep_last_turns=config.keep_last_turns,
- use_importance_scoring=config.intelligent_context_scoring,
- toin_integration=config.intelligent_context_scoring,
- compress_threshold=0.10 if config.intelligent_context_compress_first else 0.0,
- ),
- toin=toin,
- )
- self._context_manager_status = "intelligent"
- else:
- context_manager = RollingWindow(
- RollingWindowConfig(
- enabled=True,
- keep_system=True,
- keep_last_turns=config.keep_last_turns,
- )
- )
- self._context_manager_status = "rolling_window"
-
- if config.smart_routing:
- # Smart routing: ContentRouter handles all content types intelligently
- # It lazy-loads compressors only when needed
- router_config = ContentRouterConfig(
- enable_code_aware=config.code_aware_enabled,
- tool_profiles=config.tool_profiles,
- read_lifecycle=ReadLifecycleConfig(enabled=config.read_lifecycle),
- )
- # Token mode: allow compression of older excluded-tool results
- if is_token_mode(config.mode):
- router_config.protect_recent_reads_fraction = 0.3
- transforms = [
- CacheAligner(CacheAlignerConfig(enabled=False)),
- ContentRouter(router_config),
- context_manager,
- ]
- self._code_aware_status = "lazy" if config.code_aware_enabled else "disabled"
- else:
- # Legacy mode: sequential pipeline
- transforms = [
- CacheAligner(CacheAlignerConfig(enabled=False)),
- SmartCrusher(
- SmartCrusherConfig( # type: ignore[arg-type]
- enabled=True,
- min_tokens_to_crush=config.min_tokens_to_crush,
- max_items_after_crush=config.max_items_after_crush,
- ),
- ccr_config=CCRConfig(
- enabled=config.ccr_inject_tool,
- inject_retrieval_marker=config.ccr_inject_tool, # Add CCR markers
- ),
- ),
- context_manager,
- ]
- # Add CodeAware if enabled and available
- self._code_aware_status = self._setup_code_aware(config, transforms)
-
- self.anthropic_pipeline = TransformPipeline(
- transforms=transforms,
- provider=self.anthropic_provider,
- )
- self.openai_pipeline = TransformPipeline(
- transforms=transforms,
- provider=self.openai_provider,
- )
-
- # Initialize components
- self.cache = (
- SemanticCache(
- max_entries=config.cache_max_entries,
- ttl_seconds=config.cache_ttl_seconds,
- )
- if config.cache_enabled
- else None
- )
-
- self.rate_limiter = (
- TokenBucketRateLimiter(
- requests_per_minute=config.rate_limit_requests_per_minute,
- tokens_per_minute=config.rate_limit_tokens_per_minute,
- )
- if config.rate_limit_enabled
- else None
- )
-
- self.cost_tracker = (
- CostTracker(
- budget_limit_usd=config.budget_limit_usd,
- budget_period=config.budget_period,
- )
- if config.cost_tracking_enabled
- else None
- )
-
- self.metrics = PrometheusMetrics(cost_tracker=self.cost_tracker)
-
- # Prefix cache tracking: freeze already-cached messages to avoid
- # invalidating the provider's prefix cache with our transforms
- from headroom.cache.prefix_tracker import PrefixFreezeConfig, SessionTrackerStore
-
- self.session_tracker_store = SessionTrackerStore(
- default_config=PrefixFreezeConfig(
- enabled=config.prefix_freeze_enabled,
- session_ttl_seconds=config.prefix_freeze_session_ttl,
- )
- )
-
- # Compression cache store for token mode (session-scoped)
- self._compression_caches: dict[str, CompressionCache] = {}
-
- self.logger = (
- RequestLogger(
- log_file=config.log_file,
- log_full_messages=config.log_full_messages,
- )
- if config.log_requests
- else None
- )
-
- # Enterprise security plugin (loaded dynamically if available + licensed)
- self.security = None
-
- # HTTP client
- self.http_client: httpx.AsyncClient | None = None
-
- # Shared cold-start warmup registry (populated by startup()).
- # Holds typed slots with loaded / loading / null / error status for
- # each preloaded heavy asset. Exposed as ``proxy.warmup`` and
- # serialized by the /debug/warmup route (Unit 5).
- self.warmup: WarmupRegistry = WarmupRegistry()
- # Unit 3: live registry of Codex WS sessions. Populated by
- # ``handle_openai_responses_ws`` on accept; drained in its
- # outermost ``finally``. Consumed by ``/debug/ws-sessions``.
- self.ws_sessions: WebSocketSessionRegistry = WebSocketSessionRegistry()
-
- # Unit 4: bounded pre-upstream concurrency for the Anthropic HTTP
- # path. Caps how many ``handle_anthropic_messages`` calls may be
- # running deep-copy / first-stage compression / memory-context
- # lookup / upstream connect concurrently. ``/livez``, ``/readyz``,
- # ``/health``, ``/metrics``, ``/stats``, and the Codex WS path are
- # intentionally NOT gated by this semaphore.
- #
- # A value of ``0`` or negative disables the semaphore (unbounded
- # mode); this is useful for the Unit 6 counter-factual where we
- # deliberately reproduce the original starvation. The default is
- # ``max(2, min(8, os.cpu_count() or 4))``.
- _pre_upstream_cfg = config.anthropic_pre_upstream_concurrency
- if _pre_upstream_cfg is None:
- _pre_upstream_resolved = max(2, min(8, os.cpu_count() or 4))
- else:
- _pre_upstream_resolved = _pre_upstream_cfg
- self.anthropic_pre_upstream_concurrency: int = _pre_upstream_resolved
- self.anthropic_pre_upstream_acquire_timeout_seconds = float(
- config.anthropic_pre_upstream_acquire_timeout_seconds
- )
- self.anthropic_pre_upstream_memory_context_timeout_seconds = float(
- config.anthropic_pre_upstream_memory_context_timeout_seconds
- )
- if _pre_upstream_resolved > 0:
- self.anthropic_pre_upstream_sem: asyncio.Semaphore | None = asyncio.Semaphore(
- _pre_upstream_resolved
- )
- else:
- self.anthropic_pre_upstream_sem = None
-
- # Backend for Anthropic API (direct, LiteLLM, or any-llm)
- # Supports: "anthropic" (direct), "bedrock", "vertex", "litellm-<provider>", or "anyllm"
- self.anthropic_backend: Backend | None = None
- if config.backend != "anthropic":
- backend = config.backend
-
- # Handle any-llm backend
- if backend == "anyllm" or backend.startswith("anyllm-"):
- provider = config.anyllm_provider
- try:
- global AnyLLMBackend
- if AnyLLMBackend is None:
- from headroom.backends.anyllm import AnyLLMBackend as ImportedAnyLLMBackend
-
- AnyLLMBackend = ImportedAnyLLMBackend
-
- self.anthropic_backend = AnyLLMBackend(provider=provider)
- logger.info(f"any-llm backend enabled (provider={provider})")
- except ImportError as e:
- logger.warning(f"any-llm backend not available: {e}")
- except Exception as e:
- logger.error(f"Failed to initialize any-llm backend: {e}")
- else:
- # Handle LiteLLM backend
- # Normalize backend name: "bedrock" -> "litellm-bedrock"
- if not backend.startswith("litellm-"):
- backend = f"litellm-{backend}"
- provider = backend.replace("litellm-", "")
-
- try:
- global LiteLLMBackend
- if LiteLLMBackend is None:
- from headroom.backends.litellm import (
- LiteLLMBackend as ImportedLiteLLMBackend,
- )
-
- LiteLLMBackend = ImportedLiteLLMBackend
-
- self.anthropic_backend = LiteLLMBackend(
- provider=provider,
- region=config.bedrock_region,
- )
- logger.info(
- f"LiteLLM backend enabled (provider={provider}, region={config.bedrock_region})"
- )
- except ImportError as e:
- logger.warning(f"LiteLLM backend not available: {e}")
- except Exception as e:
- logger.error(f"Failed to initialize LiteLLM backend: {e}")
-
- # Request counter for IDs
- self._request_counter = 0
- self._request_counter_lock = asyncio.Lock()
-
- # CCR tool injectors (one per provider)
- self.anthropic_tool_injector = CCRToolInjector(
- provider="anthropic",
- inject_tool=config.ccr_inject_tool,
- inject_system_instructions=config.ccr_inject_system_instructions,
- )
- self.openai_tool_injector = CCRToolInjector(
- provider="openai",
- inject_tool=config.ccr_inject_tool,
- inject_system_instructions=config.ccr_inject_system_instructions,
- )
-
- # CCR Response Handler (handles CCR tool calls automatically)
- self.ccr_response_handler = (
- CCRResponseHandler(
- ResponseHandlerConfig(
- enabled=True,
- max_retrieval_rounds=config.ccr_max_retrieval_rounds,
- )
- )
- if config.ccr_handle_responses
- else None
- )
-
- # CCR Context Tracker (tracks compressed content across turns)
- self.ccr_context_tracker = (
- ContextTracker(
- ContextTrackerConfig(
- enabled=True,
- proactive_expansion=config.ccr_proactive_expansion,
- max_proactive_expansions=config.ccr_max_proactive_expansions,
- )
- )
- if config.ccr_context_tracking
- else None
- )
-
- # Turn counter for context tracking
- self._turn_counter = 0
-
- # Memory Handler (persistent user memory)
- self.memory_handler: MemoryHandler | None = None
- if config.memory_enabled:
- # Resolve memory DB path: empty → project-scoped default
- _mem_db_path = config.memory_db_path
- if not _mem_db_path:
- _mem_dir = Path.cwd() / ".headroom"
- _mem_dir.mkdir(parents=True, exist_ok=True)
- _mem_db_path = str(_mem_dir / "memory.db")
- logger.info(f"Memory: Project-scoped DB at {_mem_db_path}")
-
- memory_config = MemoryConfig(
- enabled=True,
- backend=config.memory_backend,
- db_path=_mem_db_path,
- inject_tools=config.memory_inject_tools,
- use_native_tool=config.memory_use_native_tool,
- inject_context=config.memory_inject_context,
- top_k=config.memory_top_k,
- min_similarity=config.memory_min_similarity,
- qdrant_host=config.memory_qdrant_host,
- qdrant_port=config.memory_qdrant_port,
- neo4j_uri=config.memory_neo4j_uri,
- neo4j_user=config.memory_neo4j_user,
- neo4j_password=config.memory_neo4j_password,
- bridge_enabled=config.memory_bridge_enabled,
- bridge_md_paths=config.memory_bridge_md_paths,
- bridge_md_format=config.memory_bridge_md_format,
- bridge_auto_import=config.memory_bridge_auto_import,
- bridge_export_path=config.memory_bridge_export_path,
- )
- self.memory_handler = MemoryHandler(
- memory_config,
- agent_type=config.traffic_learning_agent_type,
- )
-
- # Usage Reporter (license validation + phone-home for managed/enterprise)
- self.usage_reporter: UsageReporter | None = None
- if config.license_key:
- from headroom.telemetry.reporter import UsageReporter
-
- self.usage_reporter = UsageReporter(
- license_key=config.license_key,
- cloud_url=config.license_cloud_url,
- report_interval=config.license_report_interval,
- )
-
- # Traffic Learner (live pattern extraction from proxy traffic)
- # Only activates with --learn flag; requires --memory for backend
- self.traffic_learner: TrafficLearner | None = None
- self.traffic_learning_agent_type: str = config.traffic_learning_agent_type
- if config.traffic_learning_enabled:
- from headroom.memory.traffic_learner import TrafficLearner
-
- self.traffic_learner = TrafficLearner(
- user_id=os.environ.get("HEADROOM_USER_ID", os.environ.get("USER", "default")),
- agent_type=config.traffic_learning_agent_type,
- )
-
- # Code graph file watcher (live reindex on file changes)
- self.code_graph_watcher: CodeGraphWatcher | None = None # type: ignore[annotation-unchecked]
- if config.code_graph_watcher:
- from headroom.graph.watcher import CodeGraphWatcher
-
- self.code_graph_watcher = CodeGraphWatcher(project_dir=Path.cwd())
- if self.code_graph_watcher.start():
- logger.info("Code graph: file watcher started")
- else:
- self.code_graph_watcher = None
-
- self.pipeline_extensions.emit(
- PipelineStage.SETUP,
- operation="proxy.setup",
- metadata={
- "mode": self.config.mode,
- "optimize": self.config.optimize,
- "backend": self.config.backend,
- "memory_enabled": self.config.memory_enabled,
- },
- )
-
- def _get_compression_cache(self, session_id: str) -> CompressionCache:
- """Get or create a CompressionCache for a session."""
- if session_id not in self._compression_caches:
- from headroom.cache.compression_cache import CompressionCache
-
- # Evict oldest caches if at capacity
- if len(self._compression_caches) >= MAX_COMPRESSION_CACHE_SESSIONS:
- # Remove oldest quarter to amortize cleanup cost
- oldest_keys = list(self._compression_caches.keys())[
- : MAX_COMPRESSION_CACHE_SESSIONS // 4
- ]
- for key in oldest_keys:
- del self._compression_caches[key]
- logger.info(
- "Evicted %d compression caches (exceeded %d max sessions)",
- len(oldest_keys),
- MAX_COMPRESSION_CACHE_SESSIONS,
- )
-
- self._compression_caches[session_id] = CompressionCache()
- return self._compression_caches[session_id]
-
- def _setup_code_aware(self, config: ProxyConfig, transforms: list) -> str:
- """Set up code-aware compression if enabled.
-
- Args:
- config: Proxy configuration
- transforms: Transform list to append to
-
- Returns:
- Status string for logging: 'enabled', 'disabled', 'available', 'unavailable'
- """
- if config.code_aware_enabled:
- if is_tree_sitter_available():
- code_config = CodeCompressorConfig(
- preserve_imports=True,
- preserve_signatures=True,
- preserve_type_annotations=True,
- )
- # Insert before RollingWindow (which should be last)
- transforms.insert(-1, CodeAwareCompressor(code_config))
- return "enabled"
- else:
- logger.warning(
- "Code-aware compression requested but tree-sitter not installed. "
- "Install with: pip install headroom-ai[code]"
- )
- return "unavailable"
- else:
- if is_tree_sitter_available():
- return "available" # Available but not enabled
- return "disabled"
-
- async def startup(self):
- """Initialize async resources."""
- self.pipeline_extensions.emit(
- PipelineStage.PRE_START,
- operation="proxy.startup",
- metadata={"port": self.config.port, "host": self.config.host},
- )
- self.http_client = httpx.AsyncClient(
- timeout=httpx.Timeout(
- connect=self.config.connect_timeout_seconds,
- read=self.config.request_timeout_seconds,
- write=self.config.request_timeout_seconds,
- pool=self.config.connect_timeout_seconds,
- ),
- limits=httpx.Limits(
- max_connections=self.config.max_connections,
- max_keepalive_connections=self.config.max_keepalive_connections,
- ),
- http2=self.config.http2,
- )
- logger.info("Headroom Proxy started")
- logger.info(f"Optimization: {'ENABLED' if self.config.optimize else 'DISABLED'}")
- self.config.mode = normalize_proxy_mode(self.config.mode)
- logger.info(f"Mode: {self.config.mode}")
- if self.config.mode == PROXY_MODE_TOKEN:
- logger.info(" Prefix freeze: re-freeze after compression")
- logger.info(" Read protection window: 30%% of excluded-tool messages")
- logger.info(" CCR TTL: extended for session lifetime")
- logger.info(" Compression cache: active")
- if self.config.mode == PROXY_MODE_CACHE:
- logger.info(" Prefix freeze: strict (all prior turns immutable)")
- logger.info(" Mutations: latest turn only")
- logger.info(f"Caching: {'ENABLED' if self.config.cache_enabled else 'DISABLED'}")
- logger.info(f"Rate Limiting: {'ENABLED' if self.config.rate_limit_enabled else 'DISABLED'}")
- logger.info(
- f"Connection Pool: max_connections={self.config.max_connections}, "
- f"max_keepalive={self.config.max_keepalive_connections}, "
- f"http2={'ENABLED' if self.config.http2 else 'DISABLED'}"
- )
-
- # Unit 4 pre-upstream concurrency announcement. Report the resolved
- # value (auto-detected vs. explicit) so operators can correlate
- # ``pre_upstream_wait_ms`` log lines with the configured cap.
- if self.anthropic_pre_upstream_sem is None:
- logger.info("Anthropic pre-upstream concurrency: unbounded (explicitly disabled)")
- else:
- _explicit = self.config.anthropic_pre_upstream_concurrency
- _origin = "auto-detected" if _explicit is None else "explicit"
- logger.info(
- "Anthropic pre-upstream concurrency: %d (%s)",
- self.anthropic_pre_upstream_concurrency,
- _origin,
- )
- logger.info(
- "Anthropic pre-upstream timeouts: acquire=%.1fs compression=%.1fs memory_context=%.1fs",
- self.anthropic_pre_upstream_acquire_timeout_seconds,
- float(COMPRESSION_TIMEOUT_SECONDS),
- self.anthropic_pre_upstream_memory_context_timeout_seconds,
- )
-
- # Smart routing status
- if self.config.smart_routing:
- logger.info("Smart Routing: ENABLED (intelligent content detection)")
- else:
- logger.info("Smart Routing: DISABLED (legacy sequential mode)")
-
- # Eagerly load ALL compressors, parsers, and detectors at startup
- # This eliminates cold-start latency spikes on first requests.
- # Iterate BOTH pipelines (Anthropic + OpenAI) and dedupe transforms
- # by id() so shared-transform instances never load twice. The
- # resulting status dict is merged into ``self.warmup`` so /debug/warmup
- # (Unit 5) and /readyz have a single source of truth.
- self._kompress_status = "not installed"
- eager_status: dict[str, str] = {}
-
- if self.config.optimize:
- logger.info("Pre-loading compressors and parsers...")
- seen_transform_ids: set[int] = set()
- pipelines = (self.anthropic_pipeline, self.openai_pipeline)
- for pipeline in pipelines:
- for transform in pipeline.transforms:
- if id(transform) in seen_transform_ids:
- continue
- seen_transform_ids.add(id(transform))
- if not hasattr(transform, "eager_load_compressors"):
- continue
- try:
- transform_status = transform.eager_load_compressors()
- except Exception as exc:
- logger.warning(
- "Eager preload failed for %s: %s",
- type(transform).__name__,
- exc,
- )
- continue
- if not isinstance(transform_status, dict):
- continue
- # Merge: later writers win only if the key wasn't set.
- # Preload a transform ONCE — if another pipeline also has
- # ``eager_load_compressors`` it contributes only new keys.
- for key, value in transform_status.items():
- eager_status.setdefault(key, value)
- self.warmup.merge_transform_status(transform_status)
-
- # Update internal status from eager loading results
- if eager_status.get("kompress") == "enabled":
- self._kompress_status = "enabled"
- if eager_status.get("code_aware") == "enabled":
- self._code_aware_status = "enabled"
-
- # Log component status
- if self._kompress_status == "enabled":
- logger.info("Kompress: ENABLED (ModernBERT token compressor)")
- elif self.config.optimize:
- logger.info("Kompress: not installed (pip install headroom-ai[ml] for ML compression)")
-
- if self._code_aware_status == "enabled":
- logger.info("Code-Aware: ENABLED (AST-based compression)")
- if "tree_sitter" in eager_status:
- logger.info(f"Tree-Sitter: {eager_status['tree_sitter']}")
- elif self._code_aware_status == "lazy":
- logger.info("Code-Aware: LAZY (will load when code content detected)")
- elif self._code_aware_status == "available":
- logger.info("Code-Aware: available but disabled (use --code-aware)")
- elif self._code_aware_status == "unavailable":
- logger.info("Code-Aware: not installed (pip install headroom-ai[code])")
- elif self._code_aware_status == "disabled":
- logger.info("Code-Aware: DISABLED")
-
- if eager_status.get("magika") == "enabled":
- logger.info("Magika: ENABLED (ML content detection)")
-
- if self.memory_handler:
- self.warmup.memory_backend.mark_loading()
- try:
- await self.memory_handler.ensure_initialized()
- except Exception as exc: # pragma: no cover - defensive
- self.warmup.memory_backend.mark_error(str(exc))
- logger.warning("Memory: backend initialization failed (startup continues): %s", exc)
- memory_status = self.memory_handler.health_status()
- if memory_status.get("initialized"):
- self.warmup.memory_backend.mark_loaded(
- handle=self.memory_handler,
- backend=memory_status.get("backend"),
- )
- # Force one embed call so the ONNX graph is compiled now,
- # not lazily during the first request. Best-effort — any
- # failure is swallowed inside warmup_embedder.
- self.warmup.memory_embedder.mark_loading()
- warmed = await self.memory_handler.warmup_embedder()
- if warmed:
- self.warmup.memory_embedder.mark_loaded()
- else:
- # Not an error — e.g. qdrant-neo4j has no embedder slot
- # we can reach, or the backend simply exposes no handle.
- self.warmup.memory_embedder.mark_null()
- else:
- if self.warmup.memory_backend.status != "error":
- self.warmup.memory_backend.mark_null()
- self.warmup.memory_embedder.mark_null()
- logger.info(
- "Memory: ENABLED "
- f"(backend={memory_status['backend']}, initialized={memory_status['initialized']})"
- )
- else:
- logger.info("Memory: DISABLED")
-
- # CCR status
- ccr_features = []
- if self.config.ccr_inject_tool:
- ccr_features.append("tool_injection")
- if self.config.ccr_handle_responses:
- ccr_features.append("response_handling")
- if self.config.ccr_context_tracking:
- ccr_features.append("context_tracking")
- if self.config.ccr_proactive_expansion:
- ccr_features.append("proactive_expansion")
- if ccr_features:
- logger.info(f"CCR (Compress-Cache-Retrieve): ENABLED ({', '.join(ccr_features)})")
- else:
- logger.info("CCR: DISABLED")
- logger.info(f"Savings history: {self.metrics.savings_tracker.storage_path}")
-
- # Reset and rebuild the quota tracker registry for this server instance.
- # reset_quota_registry() ensures a clean slate when the proxy is restarted
- # (e.g. in tests that spin up multiple app instances in the same process).
- reset_quota_registry()
- registry = get_quota_registry()
- tracker = configure_subscription_tracker(
- poll_interval_s=self.config.subscription_poll_interval_s,
- active_window_s=self.config.subscription_active_window_s,
- enabled=self.config.subscription_tracking_enabled,
- )
- registry.register(tracker)
- registry.register(get_codex_rate_limit_state())
- registry.register(get_copilot_quota_tracker())
- await registry.start_all()
-
- if self.config.subscription_tracking_enabled:
- logger.info(
- "Subscription tracking: ENABLED "
- f"(poll_interval={self.config.subscription_poll_interval_s}s, "
- f"active_window={self.config.subscription_active_window_s}s)"
- )
- else:
- logger.info("Subscription tracking: DISABLED")
-
- copilot_tracker = get_copilot_quota_tracker()
- if copilot_tracker.is_available():
- logger.info("GitHub Copilot quota tracking: ENABLED")
- else:
- logger.info(
- "GitHub Copilot quota tracking: DISABLED "
- "(set GITHUB_TOKEN or GITHUB_COPILOT_GITHUB_TOKEN to enable)"
- )
-
- # Log anonymous telemetry status so operators can see it in the log stream
- if is_telemetry_enabled():
- logger.info(
- "Anonymous telemetry: ENABLED (aggregate stats only — no prompts or content). "
- "Opt out: HEADROOM_TELEMETRY=off or --no-telemetry"
- )
- else:
- logger.info("Anonymous telemetry: DISABLED")
-
- self.pipeline_extensions.emit(
- PipelineStage.POST_START,
- operation="proxy.startup",
- metadata={
- "port": self.config.port,
- "host": self.config.host,
- "warmup": self.warmup.to_dict(),
- },
- )
-
- async def shutdown(self):
- """Cleanup async resources."""
- if self.http_client:
- await self.http_client.aclose()
- self.http_client = None
-
- if self.memory_handler and hasattr(self.memory_handler, "close"):
- await self.memory_handler.close()
-
- # Stop all quota trackers via the registry
- await get_quota_registry().stop_all()
-
- # Print final stats
- self._print_summary()
-
- def _print_summary(self):
- """Print session summary."""
- m = self.metrics
- logger.info("=" * 70)
- logger.info("HEADROOM PROXY SESSION SUMMARY")
- logger.info("=" * 70)
- logger.info(f"Total requests: {m.requests_total}")
- logger.info(f"Cached responses: {m.requests_cached}")
- logger.info(f"Rate limited: {m.requests_rate_limited}")
- logger.info(f"Failed: {m.requests_failed}")
- logger.info(f"Input tokens: {m.tokens_input_total:,}")
- logger.info(f"Output tokens: {m.tokens_output_total:,}")
- logger.info(f"Tokens saved: {m.tokens_saved_total:,}")
- if m.tokens_input_total > 0:
- savings_pct = (
- m.tokens_saved_total / (m.tokens_input_total + m.tokens_saved_total)
- ) * 100
- logger.info(f"Token savings: {savings_pct:.1f}%")
- if m.latency_count > 0:
- avg_latency = m.latency_sum_ms / m.latency_count
- logger.info(f"Avg latency: {avg_latency:.0f}ms")
- logger.info("=" * 70)
-
- async def _next_request_id(self) -> str:
- """Generate unique request ID."""
- async with self._request_counter_lock:
- self._request_counter += 1
- return f"hr_{int(time.time())}_{self._request_counter:06d}"
-
- def _extract_tags(self, headers: dict) -> dict[str, str]:
- """Extract Headroom tags from headers."""
- tags = {}
- for key, value in headers.items():
- if key.lower().startswith("x-headroom-"):
- tag_name = key.lower().replace("x-headroom-", "")
- tags[tag_name] = value
- return tags
-
- def _inject_system_context(
- self,
- messages: list[dict[str, Any]],
- context: str,
- body: dict[str, Any] | None = None,
- ) -> list[dict[str, Any]]:
- """Inject context into the system message/parameter.
-
- For Anthropic API: Uses top-level 'system' parameter (not messages array).
- For OpenAI API: Uses system role in messages array.
-
- Args:
- messages: The messages list.
- context: Context to inject.
- body: Optional request body to update system parameter (for Anthropic).
-
- Returns:
- Updated messages list.
- """
- messages = list(messages) # Copy to avoid mutation
-
- # For Anthropic API: use top-level 'system' parameter
- if body is not None:
- existing_system = body.get("system", "")
- if isinstance(existing_system, str):
- body["system"] = (existing_system + "\n\n" + context).strip()
- elif isinstance(existing_system, list):
- # system is a list of content blocks (e.g., with cache_control).
- # Append memory context as a new text block — never overwrite.
- body["system"] = existing_system + [{"type": "text", "text": context}]
- else:
- # No existing system prompt — set as string
- body["system"] = context
- return messages
-
- # For OpenAI API: use system role in messages
- for i, msg in enumerate(messages):
- if msg.get("role") == "system":
- content = msg.get("content", "")
- if isinstance(content, str):
- messages[i] = {**msg, "content": content + "\n\n" + context}
- return messages
-
- # No system message found - prepend one
- messages.insert(0, {"role": "system", "content": context})
- return messages
-
- async def _retry_request(
- self,
- method: str,
- url: str,
- headers: dict,
- body: dict,
- stream: bool = False,
- ) -> httpx.Response:
- """Make request with retry and exponential backoff."""
- last_error = None
-
- for attempt in range(self.config.retry_max_attempts):
- try:
- if stream:
- # For streaming, we return early - retry happens at higher level
- return await self.http_client.post(url, json=body, headers=headers) # type: ignore[union-attr]
- else:
- response = await self.http_client.post(url, json=body, headers=headers) # type: ignore[union-attr]
-
- # Don't retry client errors (4xx)
- if 400 <= response.status_code < 500:
- return response
-
- # Retry server errors (5xx)
- if response.status_code >= 500:
- raise httpx.HTTPStatusError(
- f"Server error: {response.status_code}",
- request=response.request,
- response=response,
- )
-
- return response
-
- except (httpx.ConnectError, httpx.ReadTimeout, httpx.HTTPStatusError) as e:
- last_error = e
-
- if not self.config.retry_enabled or attempt >= self.config.retry_max_attempts - 1:
- raise
-
- # Exponential backoff with jitter
- delay_with_jitter = jitter_delay_ms(
- self.config.retry_base_delay_ms,
- self.config.retry_max_delay_ms,
- attempt,
- )
-
- logger.warning(
- f"Request failed (attempt {attempt + 1}), retrying in {delay_with_jitter:.0f}ms: {e}"
- )
- await asyncio.sleep(delay_with_jitter / 1000)
-
- raise last_error # type: ignore[misc]
-
-
-async def _log_toin_stats_periodically(interval_seconds: int = 300) -> None:
- """Background task that logs TOIN stats periodically.
-
- Args:
- interval_seconds: How often to log stats (default: 5 minutes).
- """
- while True:
- await asyncio.sleep(interval_seconds)
- try:
- toin = get_toin()
- stats = toin.get_stats()
- total_compressions = stats.get("total_compressions", 0)
- if total_compressions > 0:
- patterns = stats.get("patterns_tracked", 0)
- retrievals = stats.get("total_retrievals", 0)
- retrieval_rate = stats.get("global_retrieval_rate", 0.0)
- logger.info(
- "TOIN: %d patterns, %d compressions, %d retrievals, %.1f%% retrieval rate",
- patterns,
- total_compressions,
- retrievals,
- retrieval_rate * 100,
- )
- except Exception as e:
- logger.debug("Failed to log TOIN stats: %s", e)
-
-
-def _register_memory_components(proxy: HeadroomProxy, tracker: MemoryTracker) -> None:
- """Register all memory-tracked components with the tracker.
-
- This function is idempotent - it checks if components are already registered.
-
- Args:
- proxy: The HeadroomProxy instance.
- tracker: The MemoryTracker instance.
- """
- # Register compression store (global singleton)
- if "compression_store" not in tracker.registered_components:
- store = get_compression_store()
- tracker.register("compression_store", store.get_memory_stats)
-
- # Register semantic cache (instance on proxy)
- if proxy.cache and "semantic_cache" not in tracker.registered_components:
- tracker.register("semantic_cache", proxy.cache.get_memory_stats)
-
- # Register request logger (instance on proxy)
- if proxy.logger and "request_logger" not in tracker.registered_components:
- tracker.register("request_logger", proxy.logger.get_memory_stats)
-
- # Register batch context store (global singleton)
- if "batch_context_store" not in tracker.registered_components:
- try:
- from ..ccr.batch_store import get_batch_context_store
-
- batch_store = get_batch_context_store()
- if hasattr(batch_store, "get_memory_stats"):
- tracker.register("batch_context_store", batch_store.get_memory_stats)
- except ImportError:
- pass
-
- # Note: graph_store and vector_index are created per-user within the
- # LocalMemoryBackend, not as global singletons. They would need to be
- # registered when the memory system is initialized with specific backends.
-
-
-def create_app(config: ProxyConfig | None = None) -> FastAPI:
- """Create FastAPI application."""
- if not FASTAPI_AVAILABLE:
- raise ImportError("FastAPI required. Install: pip install fastapi uvicorn httpx")
-
- from contextlib import asynccontextmanager
-
- config = config or ProxyConfig()
- proxy = HeadroomProxy(config)
-
- # Telemetry beacon (anonymous aggregate stats).
- # With uvicorn workers > 1, each worker runs the lifespan independently.
- # We must ensure only ONE beacon runs across all workers — otherwise each
- # worker creates its own beacon, spamming the telemetry table with N rows
- # per cycle instead of 1 (all reading the same /stats from the same port).
- #
- # Strategy: use a file lock to ensure only the first worker starts the
- # beacon. Other workers see the lock and skip.
- from headroom.telemetry.beacon import TelemetryBeacon
-
- _beacon = TelemetryBeacon(
- port=config.port if hasattr(config, "port") else 8787,
- sdk=os.environ.get("HEADROOM_SDK", "proxy").strip() or "proxy",
- backend=config.backend if hasattr(config, "backend") else "anthropic",
- )
- from headroom import paths as _hr_paths
-
- _beacon_lock_path = _hr_paths.beacon_lock_path(config.port)
- _beacon_lock_fd: list = [None] # mutable holder for the lock file descriptor
- _beacon_is_owner: list = [False]
-
- def _try_acquire_beacon_lock() -> bool:
- """Try to acquire the beacon file lock (non-blocking).
-
- Returns True if this process is the beacon owner.
- """
- if not HAS_FCNTL:
- return True
-
- fd = None
- try:
- _beacon_lock_path.parent.mkdir(parents=True, exist_ok=True)
- fd = open(_beacon_lock_path, "w") # noqa: SIM115
- fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
- fd.write(str(os.getpid()))
- fd.flush()
- _beacon_lock_fd[0] = fd
- return True
- except OSError:
- if fd is not None:
- fd.close()
- return False
-
- def _release_beacon_lock() -> None:
- """Release the beacon file lock."""
- fd = _beacon_lock_fd[0]
- if fd:
- try:
- if HAS_FCNTL:
- fcntl.flock(fd, fcntl.LOCK_UN)
- fd.close()
- except Exception:
- pass
- _beacon_lock_fd[0] = None
- try:
- _beacon_lock_path.unlink(missing_ok=True)
- except Exception:
- pass
-
- @asynccontextmanager
- async def lifespan(app: FastAPI): # type: ignore[no-untyped-def]
- configure_otel_metrics(OTelMetricsConfig.from_env(default_service_name="headroom-proxy"))
- configure_langfuse_tracing(
- LangfuseTracingConfig.from_env(default_service_name="headroom-proxy")
- )
-
- app.state.started_at = time.time()
- app.state.ready = False
- app.state.startup_error = None
-
- try:
- try:
- # Startup
- await proxy.startup()
- asyncio.create_task(_log_toin_stats_periodically())
- if proxy.usage_reporter:
- await proxy.usage_reporter.start(proxy)
- if proxy.traffic_learner:
- await proxy.traffic_learner.start()
-
- # Only start beacon if we acquire the lock (first worker wins)
- _beacon_is_owner[0] = _try_acquire_beacon_lock()
- if _beacon_is_owner[0]:
- await _beacon.start()
- else:
- logger.debug("Beacon: skipping (another worker owns the lock)")
-
- app.state.ready = True
- yield
- except Exception as exc:
- app.state.startup_error = str(exc)
- raise
- finally:
- app.state.ready = False
- # Shutdown
- if _beacon_is_owner[0]:
- await _beacon.stop()
- _release_beacon_lock()
- if proxy.usage_reporter:
- await proxy.usage_reporter.stop()
- if proxy.traffic_learner:
- await proxy.traffic_learner.stop()
- if proxy.code_graph_watcher:
- proxy.code_graph_watcher.stop()
- await proxy.shutdown()
- shutdown_headroom_tracing()
- shutdown_otel_metrics()
-
- app = FastAPI(
- title="Headroom Proxy",
- description="Production-ready LLM optimization proxy",
- version=__version__,
- lifespan=lifespan,
- )
- app.state.proxy = proxy
- app.state.started_at = None
- app.state.ready = False
- app.state.startup_error = None
-
- def _iso_utc_now() -> str:
- return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
-
- def _uptime_seconds() -> float:
- started_at = getattr(app.state, "started_at", None)
- if not isinstance(started_at, int | float):
- return 0.0
- return round(max(0.0, time.time() - float(started_at)), 3)
-
- def _component_health(
- *,
- enabled: bool,
- ready: bool,
- **details: Any,
- ) -> dict[str, Any]:
- status = "disabled" if not enabled else ("healthy" if ready else "unhealthy")
- return {
- "enabled": enabled,
- "ready": (ready if enabled else True),
- "status": status,
- **details,
- }
-
- def _health_checks() -> dict[str, dict[str, Any]]:
- memory_status = (
- proxy.memory_handler.health_status()
- if proxy.memory_handler
- else {
- "enabled": False,
- "backend": None,
- "initialized": False,
- "native_tool": False,
- "bridge_enabled": False,
- }
- )
- memory_enabled = bool(memory_status.get("enabled", False))
- memory_initialized = bool(memory_status.get("initialized", False))
- return {
- "startup": _component_health(
- enabled=True,
- ready=bool(getattr(app.state, "ready", False)),
- error=getattr(app.state, "startup_error", None),
- ),
- "http_client": _component_health(
- enabled=True,
- ready=proxy.http_client is not None,
- ),
- "cache": _component_health(
- enabled=config.cache_enabled,
- ready=(proxy.cache is not None),
- ),
- "rate_limiter": _component_health(
- enabled=config.rate_limit_enabled,
- ready=(proxy.rate_limiter is not None),
- ),
- "memory": _component_health(
- enabled=memory_enabled,
- ready=memory_initialized,
- backend=memory_status["backend"],
- initialized=memory_initialized,
- native_tool=bool(memory_status.get("native_tool", False)),
- bridge_enabled=bool(memory_status.get("bridge_enabled", False)),
- ),
- }
-
- def _runtime_payload() -> dict[str, Any]:
- ws_registry = getattr(proxy, "ws_sessions", None)
- ws_active_sessions = ws_registry.active_count() if ws_registry is not None else 0
- ws_active_relay_tasks = (
- ws_registry.active_relay_task_count() if ws_registry is not None else 0
- )
- return {
- "anthropic_pre_upstream": {
- "enabled": proxy.anthropic_pre_upstream_sem is not None,
- "resolved_concurrency": proxy.anthropic_pre_upstream_concurrency,
- "source": (
- "auto" if config.anthropic_pre_upstream_concurrency is None else "explicit"
- ),
- "acquire_timeout_seconds": proxy.anthropic_pre_upstream_acquire_timeout_seconds,
- "compression_timeout_seconds": float(COMPRESSION_TIMEOUT_SECONDS),
- "memory_context_timeout_seconds": (
- proxy.anthropic_pre_upstream_memory_context_timeout_seconds
- ),
- "codex_ws_gated": False,
- },
- "websocket_sessions": {
- "active_sessions": ws_active_sessions,
- "active_relay_tasks": ws_active_relay_tasks,
- },
- }
-
- def _health_payload(*, include_config: bool) -> dict[str, Any]:
- checks = _health_checks()
- ready = all(check["ready"] for check in checks.values())
- payload: dict[str, Any] = {
- "service": "headroom-proxy",
- "status": "healthy" if ready else "unhealthy",
- "ready": ready,
- "version": __version__,
- "timestamp": _iso_utc_now(),
- "uptime_seconds": _uptime_seconds(),
- "checks": checks,
- "runtime": _runtime_payload(),
- }
- deployment_profile = os.environ.get("HEADROOM_DEPLOYMENT_PROFILE")
- if deployment_profile:
- payload["deployment"] = {
- "profile": deployment_profile,
- "preset": os.environ.get("HEADROOM_DEPLOYMENT_PRESET"),
- "runtime": os.environ.get("HEADROOM_DEPLOYMENT_RUNTIME"),
- "supervisor": os.environ.get("HEADROOM_DEPLOYMENT_SUPERVISOR"),
- "scope": os.environ.get("HEADROOM_DEPLOYMENT_SCOPE"),
- }
- if include_config:
- payload["config"] = {
- "backend": config.backend,
- "optimize": config.optimize,
- "cache": config.cache_enabled,
- "rate_limit": config.rate_limit_enabled,
- "memory": config.memory_enabled,
- "learn": config.traffic_learning_enabled,
- "code_graph": config.code_graph_watcher,
- "pid": os.getpid(),
- }
- return payload
-
- # CORS
- app.add_middleware(
- CORSMiddleware,
- allow_origins=["*"],
- allow_credentials=True,
- allow_methods=["*"],
- allow_headers=["*"],
- )
-
- # X-Headroom-Stack: SDK adapters (TS openai/anthropic/etc.) tag their
- # requests so telemetry can segment by integration surface. Registered
- # before extension middleware so any extension-level auth/guards run
- # outermost and we don't count requests they reject.
- @app.middleware("http")
- async def _record_headroom_stack(request, call_next):
- if request.url.path.startswith("/v1/"):
- stack = request.headers.get("x-headroom-stack")
- if stack:
- try:
- proxy.metrics.record_stack(stack)
- except Exception:
- logger.debug("record_stack failed", exc_info=True)
- return await call_next(request)
-
- # Third-party proxy extensions (Enterprise, custom plugins). Discovered via
- # the `headroom.proxy_extension` entry-point group. An extension that raises
- # from its install() is a deliberate fail-closed signal and aborts startup.
- from headroom.proxy.extensions import install_all as _install_extensions
-
- _install_extensions(app, config)
-
- # Health & Metrics
- @app.get("/livez")
- async def livez():
- return JSONResponse(
- status_code=200,
- content={
- "service": "headroom-proxy",
- "status": "healthy",
- "alive": True,
- "version": __version__,
- "timestamp": _iso_utc_now(),
- "uptime_seconds": _uptime_seconds(),
- },
- )
-
- @app.get("/readyz")
- async def readyz():
- payload = _health_payload(include_config=False)
- return JSONResponse(status_code=200 if payload["ready"] else 503, content=payload)
-
- @app.get("/health")
- async def health():
- payload = _health_payload(include_config=True)
- return JSONResponse(status_code=200, content=payload)
-
- # Loopback-only debug introspection (Unit 5). A remote IP gets 404 —
- # debug endpoints are invisible to external scanners.
- from headroom.proxy.debug_introspection import (
- collect_tasks as _collect_tasks,
- )
- from headroom.proxy.loopback_guard import require_loopback as _require_loopback
-
- @app.get("/debug/tasks", dependencies=[Depends(_require_loopback)])
- async def debug_tasks(stack: bool = False):
- """Enumerate running asyncio tasks.
-
- Default is cheap — ``stack_depth`` is ``null`` in every entry so
- a storm snapshot does not walk 50+ coroutine frames synchronously.
- Pass ``?stack=true`` to compute ``stack_depth`` for each task
- (useful for single-shot human debugging).
- """
- ws_registry = getattr(proxy, "ws_sessions", None)
- return JSONResponse(
- status_code=200,
- content=_collect_tasks(ws_registry, with_stack_depth=stack),
- )
-
- @app.get("/debug/ws-sessions", dependencies=[Depends(_require_loopback)])
- async def debug_ws_sessions():
- ws_registry = getattr(proxy, "ws_sessions", None)
- snapshot = ws_registry.snapshot() if ws_registry is not None else []
- return JSONResponse(status_code=200, content=snapshot)
-
- @app.get("/debug/warmup", dependencies=[Depends(_require_loopback)])
- async def debug_warmup():
- warmup_registry = getattr(proxy, "warmup", None)
- payload = warmup_registry.to_dict() if warmup_registry is not None else {}
- payload["runtime"] = _runtime_payload()
- return JSONResponse(status_code=200, content=payload)
-
- @app.get("/dashboard", response_class=HTMLResponse)
- async def dashboard():
- """Serve the Headroom dashboard UI."""
- return get_dashboard_html()
-
- @app.get("/stats")
- async def stats():
- """Get comprehensive proxy statistics.
-
- This is the main stats endpoint - it aggregates data from all subsystems:
- - Request metrics (total, cached, failed, by model/provider)
- - Token usage and savings
- - Cost tracking
- - Canonical persisted display_session metrics for downstream dashboards
- - Compression (CCR) statistics
- - Telemetry/TOIN (data flywheel) statistics
- - Cache and rate limiter stats
- """
- m = proxy.metrics
-
- # Calculate average latency
- avg_latency_ms = round(m.latency_sum_ms / m.latency_count, 2) if m.latency_count > 0 else 0
- min_latency_ms = (
- round(m.latency_min_ms, 2)
- if m.latency_count > 0 and m.latency_min_ms != float("inf")
- else 0
- )
- max_latency_ms = round(m.latency_max_ms, 2) if m.latency_count > 0 else 0
-
- # Calculate Headroom overhead (optimization time only, excludes pass-through requests)
- avg_overhead_ms = (
- round(m.overhead_sum_ms / m.overhead_count, 2) if m.overhead_count > 0 else 0
- )
- min_overhead_ms = (
- round(m.overhead_min_ms, 2)
- if m.overhead_count > 0 and m.overhead_min_ms != float("inf")
- else 0
- )
- max_overhead_ms = round(m.overhead_max_ms, 2) if m.overhead_count > 0 else 0
-
- # Calculate TTFB (time to first byte)
- avg_ttfb_ms = round(m.ttfb_sum_ms / m.ttfb_count, 2) if m.ttfb_count > 0 else 0
- min_ttfb_ms = (
- round(m.ttfb_min_ms, 2) if m.ttfb_count > 0 and m.ttfb_min_ms != float("inf") else 0
- )
- max_ttfb_ms = round(m.ttfb_max_ms, 2) if m.ttfb_count > 0 else 0
-
- # Get compression store stats
- store = get_compression_store()
- compression_stats = store.get_stats()
-
- # Get telemetry/TOIN stats
- telemetry = get_telemetry_collector()
- telemetry_stats = telemetry.get_stats()
-
- # Get feedback loop stats
- feedback = get_compression_feedback()
- feedback_stats = feedback.get_stats()
-
- # Build prefix cache stats once (used in both prefix_cache and cost)
- prefix_cache_stats = _build_prefix_cache_stats(m, proxy.cost_tracker)
-
- # Fetch CLI filtering savings (rtk — tokens avoided before reaching context)
- cli_filtering_stats = _get_rtk_stats()
- cli_tokens_avoided = (
- cli_filtering_stats.get("tokens_saved", 0) if cli_filtering_stats else 0
- )
-
- # Calculate total tokens before compression
- total_tokens_before = m.tokens_input_total + m.tokens_saved_total
-
- # Build human-readable summary
- summary = _build_session_summary(
- proxy, m, prefix_cache_stats, cli_tokens_avoided, total_tokens_before
- )
- # DEBUG: log the summary payload for external upsert consumers
- try:
- logger.debug("/stats summary data: %r", summary)
- except Exception:
- logger.warning("Failed to log /stats summary payload")
-
- # Compression cache stats (token mode)
- compression_cache_stats: dict = {}
- if proxy.config.mode == PROXY_MODE_TOKEN and proxy._compression_caches:
- total_entries = 0
- total_hits = 0
- total_misses = 0
- total_tokens_saved = 0
- for cache in proxy._compression_caches.values():
- s = cache.get_stats()
- total_entries += s.get("entries", 0)
- total_hits += s.get("hits", 0)
- total_misses += s.get("misses", 0)
- total_tokens_saved += s.get("total_tokens_saved", 0)
- compression_cache_stats = {
- "mode": PROXY_MODE_TOKEN,
- "active_sessions": len(proxy._compression_caches),
- "total_entries": total_entries,
- "total_hits": total_hits,
- "total_misses": total_misses,
- "hit_rate": round(total_hits / max(1, total_hits + total_misses) * 100, 1),
- "total_tokens_saved": total_tokens_saved,
- }
- else:
- compression_cache_stats = {"mode": proxy.config.mode}
-
- # Build unified savings summary (all layers)
- compression_tokens = m.tokens_saved_total
- cache_net_usd = prefix_cache_stats.get("totals", {}).get("net_savings_usd", 0.0)
- total_tokens_all_layers = compression_tokens + cli_tokens_avoided
- persistent_savings = m.savings_tracker.stats_preview()
- display_session = persistent_savings.get("display_session", {})
-
- return {
- "summary": summary,
- "savings": {
- "total_tokens": total_tokens_all_layers,
- "by_layer": {
- "cli_filtering": {
- "tokens": cli_tokens_avoided,
- "description": "Tokens avoided by CLI output filtering (rtk) before reaching context",
- },
- "compression": {
- "tokens": compression_tokens,
- "description": "Tokens removed by proxy compression (SmartCrusher, ContentRouter, etc.)",
- },
- "prefix_cache": {
- "discount_usd": round(cache_net_usd, 4),
- "description": (
- "Cost discount from provider prefix caching. "
- "Headroom's CacheAligner improves hit rates; "
- "baseline caching is provider-native."
- ),
- },
- },
- },
- "requests": {
- "total": m.requests_total,
- "cached": m.requests_cached,
- "rate_limited": m.requests_rate_limited,
- "failed": m.requests_failed,
- "by_provider": dict(m.requests_by_provider),
- "by_model": dict(m.requests_by_model),
- "by_stack": dict(m.requests_by_stack),
- },
- "tokens": {
- "input": m.tokens_input_total,
- "output": m.tokens_output_total,
- "saved": m.tokens_saved_total,
- "cli_tokens_avoided": cli_tokens_avoided,
- "total_before_compression": total_tokens_before,
- "savings_percent": round(
- (m.tokens_saved_total / total_tokens_before * 100)
- if total_tokens_before > 0
- else 0,
- 2,
- ),
- },
- "latency": {
- "average_ms": avg_latency_ms,
- "min_ms": min_latency_ms,
- "max_ms": max_latency_ms,
- "total_requests": m.latency_count,
- },
- "overhead": {
- "average_ms": avg_overhead_ms,
- "min_ms": min_overhead_ms,
- "max_ms": max_overhead_ms,
- },
- "ttfb": {
- "average_ms": avg_ttfb_ms,
- "min_ms": min_ttfb_ms,
- "max_ms": max_ttfb_ms,
- },
- "pipeline_timing": {
- name: {
- "average_ms": round(
- m.transform_timing_sum[name] / m.transform_timing_count[name], 2
- ),
- "max_ms": round(m.transform_timing_max[name], 2),
- "count": m.transform_timing_count[name],
- }
- for name in sorted(m.transform_timing_sum.keys())
- }
- if m.transform_timing_sum
- else {},
- "waste_signals": dict(m.waste_signals_total) if m.waste_signals_total else {},
- "savings_history": m.savings_history[-100:], # Last 100 data points
- "display_session": display_session,
- "persistent_savings": persistent_savings,
- "prefix_cache": prefix_cache_stats,
- "cost": _merge_cost_stats(
- proxy.cost_tracker.stats() if proxy.cost_tracker else None,
- prefix_cache_stats,
- cli_tokens_avoided=cli_tokens_avoided,
- ),
- "compression": {
- "ccr_entries": compression_stats.get("entry_count", 0),
- "ccr_max_entries": compression_stats.get("max_entries", 0),
- "original_tokens_cached": compression_stats.get("total_original_tokens", 0),
- "compressed_tokens_cached": compression_stats.get("total_compressed_tokens", 0),
- "ccr_retrievals": compression_stats.get("total_retrievals", 0),
- },
- "compression_cache": compression_cache_stats,
- "anon_telemetry_shipping": is_telemetry_enabled(),
- "telemetry": {
- "enabled": telemetry_stats.get("enabled", False),
- "total_compressions": telemetry_stats.get("total_compressions", 0),
- "total_retrievals": telemetry_stats.get("total_retrievals", 0),
- "global_retrieval_rate": round(telemetry_stats.get("global_retrieval_rate", 0), 4),
- "tool_signatures_tracked": telemetry_stats.get("tool_signatures_tracked", 0),
- "avg_compression_ratio": round(telemetry_stats.get("avg_compression_ratio", 0), 4),
- "avg_token_reduction": round(telemetry_stats.get("avg_token_reduction", 0), 4),
- },
- "otel": get_otel_metrics_status(),
- "langfuse": get_langfuse_tracing_status(),
- "feedback_loop": {
- "tools_tracked": feedback_stats.get("tools_tracked", 0),
- "total_compressions": feedback_stats.get("total_compressions", 0),
- "total_retrievals": feedback_stats.get("total_retrievals", 0),
- "global_retrieval_rate": round(feedback_stats.get("global_retrieval_rate", 0), 4),
- "tools_with_high_retrieval": sum(
- 1
- for p in feedback_stats.get("tool_patterns", {}).values()
- if p.get("retrieval_rate", 0) > 0.3
- ),
- },
- "toin": get_toin().get_stats(),
- "cli_filtering": cli_filtering_stats,
- "cache": await proxy.cache.stats() if proxy.cache else None,
- "rate_limiter": await proxy.rate_limiter.stats() if proxy.rate_limiter else None,
- "recent_requests": proxy.logger.get_recent(10) if proxy.logger else [],
- "log_full_messages": proxy.config.log_full_messages if proxy else False,
- **get_quota_registry().get_all_stats(),
- }
-
- @app.get("/stats-history")
- async def stats_history(
- format: Literal["json", "csv"] = "json",
- series: Literal["history", "hourly", "daily", "weekly", "monthly"] = "history",
- history_mode: Literal["compact", "full", "none"] = "compact",
- ):
- """Get durable proxy compression history plus display-session state."""
- if format == "csv":
- filename = f"headroom-stats-history-{series}.csv"
- return Response(
- content=proxy.metrics.savings_tracker.export_csv(series=series),
- media_type="text/csv; charset=utf-8",
- headers={"Content-Disposition": f'attachment; filename="{filename}"'},
- )
-
- return proxy.metrics.savings_tracker.history_response(history_mode=history_mode)
-
- @app.get("/transformations/feed")
- async def transformations_feed(limit: int = 20):
- """Get recent message transformations for the live feed.
-
- Returns empty list if log_full_messages is disabled (messages are not stored).
- """
- if limit > 100:
- limit = 100
-
- transformations = []
- log_full_messages = proxy.config.log_full_messages if proxy else False
-
- if proxy and proxy.logger:
- logs = proxy.logger.get_recent_with_messages(limit)
- for log in logs:
- transformations.append(
- {
- "request_id": log.get("request_id"),
- "timestamp": log.get("timestamp"),
- "provider": log.get("provider"),
- "model": log.get("model"),
- "input_tokens_original": log.get("input_tokens_original"),
- "input_tokens_optimized": log.get("input_tokens_optimized"),
- "tokens_saved": log.get("tokens_saved"),
- "savings_percent": log.get("savings_percent"),
- "transforms_applied": log.get("transforms_applied", []),
- "request_messages": log.get("request_messages"),
- "response_content": log.get("response_content"),
- }
- )
-
- return {"transformations": transformations, "log_full_messages": log_full_messages}
-
- @app.get("/subscription-window")
- async def subscription_window():
- """Current Anthropic subscription window utilisation and Headroom contribution."""
- tracker = get_subscription_tracker()
- if tracker is None:
- return JSONResponse(
- status_code=503,
- content={"error": "Subscription tracking is not enabled"},
- )
- return JSONResponse(content=tracker.state)
-
- @app.get("/quota")
- async def quota():
- """Unified quota/rate-limit stats for all registered providers (Anthropic, Codex, Copilot)."""
- return JSONResponse(content=get_quota_registry().get_all_stats())
-
- @app.get("/metrics")
- async def metrics():
- """Prometheus metrics endpoint."""
- return PlainTextResponse(
- await proxy.metrics.export(),
- media_type="text/plain; version=0.0.4",
- )
-
- # Debug endpoints
- @app.get("/debug/memory")
- async def debug_memory():
- """Get detailed memory usage statistics.
-
- Returns memory usage for all tracked components including:
- - Process-level memory (RSS, VMS, percent)
- - Per-component memory usage and budgets
- - Cache hit/miss statistics
- - Total tracked vs target budget
-
- This endpoint is useful for debugging memory issues and
- monitoring memory budgets.
- """
- from ..memory.tracker import MemoryTracker
-
- tracker = MemoryTracker.get()
-
- # Register components if not already registered
- _register_memory_components(proxy, tracker)
-
- report = tracker.get_report()
- return report.to_dict()
-
- @app.post("/cache/clear")
- async def clear_cache():
- """Clear the response cache."""
- if proxy.cache:
- await proxy.cache.clear()
- return {"status": "cleared"}
- return {"status": "cache disabled"}
-
- # CCR (Compress-Cache-Retrieve) endpoints
- @app.post("/v1/retrieve")
- async def ccr_retrieve(request: Request):
- """Retrieve original content from CCR compression cache.
-
- This is the "Retrieve" part of CCR (Compress-Cache-Retrieve).
- When SmartCrusher compresses tool outputs, the original data is cached.
- LLMs can call this endpoint to get more data if needed.
-
- Request body:
- hash (str): Hash key from compression marker (required)
- query (str): Optional search query to filter results
-
- Response:
- Full retrieval: {"hash": "...", "original_content": "...", ...}
- Search: {"hash": "...", "query": "...", "results": [...], "count": N}
- """
- data = await request.json()
- hash_key = data.get("hash")
- query = data.get("query")
-
- if not hash_key:
- raise HTTPException(status_code=400, detail="hash required")
-
- store = get_compression_store()
-
- if query:
- # Search within cached content
- results = store.search(hash_key, query)
- return {
- "hash": hash_key,
- "query": query,
- "results": results,
- "count": len(results),
- }
- else:
- # Return full original content
- entry = store.retrieve(hash_key)
- if entry:
- return {
- "hash": hash_key,
- "original_content": entry.original_content,
- "original_tokens": entry.original_tokens,
- "original_item_count": entry.original_item_count,
- "compressed_item_count": entry.compressed_item_count,
- "tool_name": entry.tool_name,
- "retrieval_count": entry.retrieval_count,
- }
- raise HTTPException(
- status_code=404, detail="Entry not found or expired (TTL: 5 minutes)"
- )
-
- @app.get("/v1/retrieve/stats")
- async def ccr_stats():
- """Get CCR compression store statistics."""
- store = get_compression_store()
- stats = store.get_stats()
- events = store.get_retrieval_events(limit=20)
- return {
- "store": stats,
- "recent_retrievals": [
- {
- "hash": e.hash,
- "query": e.query,
- "items_retrieved": e.items_retrieved,
- "total_items": e.total_items,
- "tool_name": e.tool_name,
- "retrieval_type": e.retrieval_type,
- }
- for e in events
- ],
- }
-
- @app.get("/v1/feedback")
- async def ccr_feedback():
- """Get CCR feedback loop statistics and learned patterns.
-
- This endpoint exposes the feedback loop's learned patterns for monitoring
- and debugging. It shows:
- - Per-tool retrieval rates (high = compress less aggressively)
- - Common search queries per tool
- - Queried fields (suggest what to preserve)
-
- Use this to understand how well compression is working and whether
- the feedback loop is adjusting appropriately.
- """
- feedback = get_compression_feedback()
- stats = feedback.get_stats()
- return {
- "feedback": stats,
- "hints_example": {
- tool_name: {
- "hints": {
- "max_items": hints.max_items
- if (hints := feedback.get_compression_hints(tool_name))
- else 15,
- "suggested_items": hints.suggested_items if hints else None,
- "skip_compression": hints.skip_compression if hints else False,
- "preserve_fields": hints.preserve_fields if hints else [],
- "reason": hints.reason if hints else "",
- }
- }
- for tool_name in list(stats.get("tool_patterns", {}).keys())[:5]
- },
- }
-
- @app.get("/v1/feedback/{tool_name}")
- async def ccr_feedback_for_tool(tool_name: str):
- """Get compression hints for a specific tool.
-
- Returns feedback-based hints that would be used for compressing
- this tool's output.
- """
- feedback = get_compression_feedback()
- hints = feedback.get_compression_hints(tool_name)
- patterns = feedback.get_all_patterns().get(tool_name)
-
- return {
- "tool_name": tool_name,
- "hints": {
- "max_items": hints.max_items,
- "min_items": hints.min_items,
- "suggested_items": hints.suggested_items,
- "aggressiveness": hints.aggressiveness,
- "skip_compression": hints.skip_compression,
- "preserve_fields": hints.preserve_fields,
- "reason": hints.reason,
- },
- "pattern": {
- "total_compressions": patterns.total_compressions if patterns else 0,
- "total_retrievals": patterns.total_retrievals if patterns else 0,
- "retrieval_rate": patterns.retrieval_rate if patterns else 0.0,
- "full_retrieval_rate": patterns.full_retrieval_rate if patterns else 0.0,
- "search_rate": patterns.search_rate if patterns else 0.0,
- "common_queries": list(patterns.common_queries.keys())[:10] if patterns else [],
- "queried_fields": list(patterns.queried_fields.keys())[:10] if patterns else [],
- }
- if patterns
- else None,
- }
-
- # Telemetry endpoints (Data Flywheel)
- @app.get("/v1/telemetry")
- async def telemetry_stats():
- """Get telemetry statistics for the data flywheel.
-
- This endpoint exposes privacy-preserving telemetry data that powers
- the data flywheel - learning optimal compression strategies across
- tool types based on usage patterns.
-
- What's collected (anonymized):
- - Tool output structure patterns (field types, not values)
- - Compression decisions and ratios
- - Retrieval patterns (rate, type, not content)
- - Strategy effectiveness
-
- What's NOT collected:
- - Actual data values
- - User identifiers
- - Queries or search terms
- - File paths or tool names (hashed by default)
- """
- telemetry = get_telemetry_collector()
- return telemetry.get_stats()
-
- @app.get("/v1/telemetry/export")
- async def telemetry_export():
- """Export full telemetry data for aggregation.
-
- This endpoint exports all telemetry data in a format suitable for
- cross-user aggregation. The data is privacy-preserving - no actual
- values are included, only structural patterns and statistics.
-
- Use this for:
- - Building a central learning service
- - Sharing learned patterns across instances
- - Analysis and debugging
- """
- telemetry = get_telemetry_collector()
- return telemetry.export_stats()
-
- @app.post("/v1/telemetry/import")
- async def telemetry_import(request: Request):
- """Import telemetry data from another source.
-
- This allows merging telemetry from multiple sources for cross-user
- learning. The imported data is merged with existing statistics.
-
- Request body: Telemetry export data from /v1/telemetry/export
- """
- telemetry = get_telemetry_collector()
- data = await request.json()
- telemetry.import_stats(data)
- return {"status": "imported", "current_stats": telemetry.get_stats()}
-
- @app.get("/v1/telemetry/tools")
- async def telemetry_tools():
- """Get telemetry statistics for all tracked tool signatures.
-
- Returns statistics per tool signature (anonymized), including:
- - Compression ratios and strategy usage
- - Retrieval rates (high = compression too aggressive)
- - Learned recommendations
- """
- telemetry = get_telemetry_collector()
- all_stats = telemetry.get_all_tool_stats()
- return {
- "tool_count": len(all_stats),
- "tools": {sig_hash: stats.to_dict() for sig_hash, stats in all_stats.items()},
- }
-
- @app.get("/v1/telemetry/tools/{signature_hash}")
- async def telemetry_tool_detail(signature_hash: str):
- """Get detailed telemetry for a specific tool signature.
-
- Includes learned recommendations if enough data has been collected.
- """
- telemetry = get_telemetry_collector()
- stats = telemetry.get_tool_stats(signature_hash)
- recommendations = telemetry.get_recommendations(signature_hash)
-
- if stats is None:
- raise HTTPException(
- status_code=404, detail=f"No telemetry found for signature: {signature_hash}"
- )
-
- return {
- "signature_hash": signature_hash,
- "stats": stats.to_dict(),
- "recommendations": recommendations,
- }
-
- # TOIN (Tool Output Intelligence Network) endpoints
- @app.get("/v1/toin/stats")
- async def toin_stats():
- """Get overall TOIN statistics.
-
- Returns aggregated statistics from the Tool Output Intelligence Network,
- which learns optimal compression strategies across all tool types.
-
- Response includes:
- - enabled: Whether TOIN is enabled
- - patterns_tracked: Number of unique tool patterns being tracked
- - total_compressions: Total compression events recorded
- - total_retrievals: Total retrieval events recorded
- - global_retrieval_rate: Overall retrieval rate (high = compression too aggressive)
- - patterns_with_recommendations: Patterns with enough data for recommendations
- """
- toin = get_toin()
- return toin.get_stats()
-
- @app.get("/v1/toin/patterns")
- async def toin_patterns(limit: int = 20):
- """List TOIN patterns with most samples.
-
- Returns patterns sorted by sample_size descending. Use this to see
- which tool types have the most data and their learned behaviors.
-
- Query params:
- limit: Maximum number of patterns to return (default 20)
-
- Response includes for each pattern:
- - hash: Truncated tool signature hash (12 chars)
- - compressions: Total compression events
- - retrievals: Total retrieval events
- - retrieval_rate: Percentage of compressions that triggered retrieval
- - confidence: Confidence level in recommendations (0.0-1.0)
- - skip_recommended: Whether TOIN recommends skipping compression
- - optimal_max_items: Learned optimal max_items setting
- """
- toin = get_toin()
- exported = toin.export_patterns()
- patterns_data = exported.get("patterns", {})
-
- # Convert to list and sort by sample_size
- patterns_list = []
- for sig_hash, pattern_dict in patterns_data.items():
- sample_size = pattern_dict.get("sample_size", 0)
- total_compressions = pattern_dict.get("total_compressions", 0)
- total_retrievals = pattern_dict.get("total_retrievals", 0)
- retrieval_rate = (
- total_retrievals / total_compressions if total_compressions > 0 else 0.0
- )
-
- patterns_list.append(
- {
- "hash": sig_hash[:12],
- "compressions": total_compressions,
- "retrievals": total_retrievals,
- "retrieval_rate": f"{retrieval_rate:.1%}",
- "confidence": round(pattern_dict.get("confidence", 0.0), 3),
- "skip_recommended": pattern_dict.get("skip_compression_recommended", False),
- "optimal_max_items": pattern_dict.get("optimal_max_items", 20),
- "sample_size": sample_size,
- }
- )
-
- # Sort by sample_size descending
- patterns_list.sort(key=lambda p: p["sample_size"], reverse=True)
-
- # Remove sample_size from output (used only for sorting)
- for p in patterns_list:
- del p["sample_size"]
-
- return patterns_list[:limit]
-
- @app.get("/v1/toin/pattern/{hash_prefix}")
- async def toin_pattern_detail(hash_prefix: str):
- """Get detailed TOIN pattern info by hash prefix.
-
- Searches for a pattern where the tool signature hash starts with
- the provided prefix. Returns full pattern details if found.
-
- Path params:
- hash_prefix: Beginning of the tool signature hash (min 4 chars recommended)
-
- Response: Full pattern.to_dict() with all learned statistics and recommendations.
- """
- toin = get_toin()
- exported = toin.export_patterns()
- patterns_data = exported.get("patterns", {})
-
- # Search for pattern with matching hash prefix
- for sig_hash, pattern_dict in patterns_data.items():
- if sig_hash.startswith(hash_prefix):
- return pattern_dict
-
- raise HTTPException(
- status_code=404, detail=f"No TOIN pattern found with hash starting with: {hash_prefix}"
- )
-
- @app.get("/v1/retrieve/{hash_key}")
- async def ccr_retrieve_get(hash_key: str, query: str | None = None):
- """GET version of CCR retrieve for easier testing."""
- store = get_compression_store()
-
- if query:
- results = store.search(hash_key, query)
- return {
- "hash": hash_key,
- "query": query,
- "results": results,
- "count": len(results),
- }
- else:
- entry = store.retrieve(hash_key)
- if entry:
- return {
- "hash": hash_key,
- "original_content": entry.original_content,
- "original_tokens": entry.original_tokens,
- "original_item_count": entry.original_item_count,
- "compressed_item_count": entry.compressed_item_count,
- "tool_name": entry.tool_name,
- "retrieval_count": entry.retrieval_count,
- }
- raise HTTPException(status_code=404, detail="Entry not found or expired")
-
- # CCR Tool Call Handler - for agent frameworks to call when LLM uses headroom_retrieve
- @app.post("/v1/retrieve/tool_call")
- async def ccr_handle_tool_call(request: Request):
- """Handle a CCR tool call from an LLM response.
-
- This endpoint accepts tool call formats from various providers and returns
- a properly formatted tool result. Agent frameworks can use this to handle
- CCR tool calls without implementing the retrieval logic themselves.
-
- Request body (Anthropic format):
- {
- "tool_call": {
- "id": "toolu_123",
- "name": "headroom_retrieve",
- "input": {"hash": "abc123", "query": "optional search"}
- },
- "provider": "anthropic"
- }
-
- Request body (OpenAI format):
- {
- "tool_call": {
- "id": "call_123",
- "function": {
- "name": "headroom_retrieve",
- "arguments": "{\"hash\": \"abc123\"}"
- }
- },
- "provider": "openai"
- }
-
- Response:
- {
- "tool_result": {...}, # Formatted for the provider
- "success": true,
- "data": {...} # Raw retrieval data
- }
- """
- data = await request.json()
- tool_call = data.get("tool_call", {})
- provider = data.get("provider", "anthropic")
-
- # Parse the tool call
- hash_key, query = parse_tool_call(tool_call, provider)
-
- if hash_key is None:
- raise HTTPException(
- status_code=400, detail=f"Invalid tool call or not a {CCR_TOOL_NAME} call"
- )
-
- # Perform retrieval
- store = get_compression_store()
-
- if query:
- results = store.search(hash_key, query)
- retrieval_data = {
- "hash": hash_key,
- "query": query,
- "results": results,
- "count": len(results),
- }
- else:
- entry = store.retrieve(hash_key)
- if entry:
- retrieval_data = {
- "hash": hash_key,
- "original_content": entry.original_content,
- "original_item_count": entry.original_item_count,
- "compressed_item_count": entry.compressed_item_count,
- }
- else:
- retrieval_data = {
- "error": "Entry not found or expired (TTL: 5 minutes)",
- "hash": hash_key,
- }
-
- # Format tool result for provider
- tool_call_id = tool_call.get("id", "")
- result_content = json.dumps(retrieval_data, indent=2)
-
- if provider == "anthropic":
- tool_result = {
- "type": "tool_result",
- "tool_use_id": tool_call_id,
- "content": result_content,
- }
- elif provider == "openai":
- tool_result = {
- "role": "tool",
- "tool_call_id": tool_call_id,
- "content": result_content,
- }
- else:
- tool_result = {
- "tool_call_id": tool_call_id,
- "content": result_content,
- }
-
- return {
- "tool_result": tool_result,
- "success": "error" not in retrieval_data,
- "data": retrieval_data,
- }
-
- # Compression-only endpoint (for TypeScript SDK and other HTTP clients)
- @app.post("/v1/compress")
- async def compress_messages(request: Request):
- return await proxy.handle_compress(request)
-
- # Anthropic endpoints
- @app.post("/v1/messages")
- async def anthropic_messages(request: Request):
- return await proxy.handle_anthropic_messages(request)
-
- @app.post("/v1/messages/count_tokens")
- async def anthropic_count_tokens(request: Request):
- return await proxy.handle_passthrough(
- request, proxy.ANTHROPIC_API_URL, "count_tokens", "anthropic"
- )
-
- # Anthropic Message Batches API endpoints
- @app.post("/v1/messages/batches")
- async def anthropic_batch_create(request: Request):
- """Create a message batch with compression applied to all requests."""
- return await proxy.handle_anthropic_batch_create(request)
-
- @app.get("/v1/messages/batches")
- async def anthropic_batch_list(request: Request):
- """List message batches (passthrough)."""
- return await proxy.handle_anthropic_batch_passthrough(request)
-
- @app.get("/v1/messages/batches/{batch_id}")
- async def anthropic_batch_get(request: Request, batch_id: str):
- """Get a specific message batch (passthrough)."""
- return await proxy.handle_anthropic_batch_passthrough(request, batch_id)
-
- @app.get("/v1/messages/batches/{batch_id}/results")
- async def anthropic_batch_results(request: Request, batch_id: str):
- """Get results for a message batch with CCR post-processing."""
- return await proxy.handle_anthropic_batch_results(request, batch_id)
-
- @app.post("/v1/messages/batches/{batch_id}/cancel")
- async def anthropic_batch_cancel(request: Request, batch_id: str):
- """Cancel a message batch (passthrough)."""
- return await proxy.handle_anthropic_batch_passthrough(request, batch_id)
-
- # OpenAI endpoints
- @app.post("/v1/chat/completions")
- async def openai_chat(request: Request):
- return await proxy.handle_openai_chat(request)
-
- @app.post("/v1/responses")
- async def openai_responses(request: Request):
- """OpenAI Responses API (new API introduced March 2025)."""
- return await proxy.handle_openai_responses(request)
-
- @app.post("/v1/codex/responses")
- async def openai_v1_codex_responses(request: Request):
- """Pi/OpenAI Codex compatibility path for OpenAI-style /v1 base URLs."""
- return await proxy.handle_openai_responses(request)
-
- @app.post("/backend-api/responses")
- async def openai_codex_responses(request: Request):
- """OpenAI Codex Responses API path preserved from ChatGPT backend."""
- return await proxy.handle_openai_responses(request)
-
- @app.post("/backend-api/codex/responses")
- async def openai_codex_nested_responses(request: Request):
- """OpenAI Codex Responses API path for codex-shaped proxy base URLs."""
- return await proxy.handle_openai_responses(request)
-
- @app.websocket("/v1/responses")
- async def openai_responses_ws(websocket: WebSocket):
- """OpenAI Responses API via WebSocket (Codex gpt-5.4+)."""
- await proxy.handle_openai_responses_ws(websocket)
-
- @app.websocket("/v1/codex/responses")
- async def openai_v1_codex_responses_ws(websocket: WebSocket):
- """Pi/OpenAI Codex compatibility WebSocket path for /v1 base URLs."""
- await proxy.handle_openai_responses_ws(websocket)
-
- # OpenAI Responses API sub-endpoints (passthrough).
- # Codex sub-agents use /v1/responses/compact and other sub-paths
- # that we don't need to compress — just forward with correct auth routing.
- @app.api_route("/v1/responses/{sub_path:path}", methods=["GET", "POST", "DELETE"])
- async def openai_responses_sub(request: Request, sub_path: str):
- """Passthrough for /v1/responses/* sub-endpoints (compact, cancel, etc.)."""
- from fastapi.responses import Response
-
- from headroom.proxy.handlers.openai import _resolve_codex_routing_headers
-
- headers = dict(request.headers.items())
- headers.pop("host", None)
- headers, is_chatgpt_auth = _resolve_codex_routing_headers(headers)
-
- # Route to correct endpoint based on auth mode.
- # ChatGPT session auth (codex login) uses chatgpt.com with /responses/...
- # path (no /v1/ prefix). API key auth uses api.openai.com/v1/responses/...
- if is_chatgpt_auth:
- url = f"https://chatgpt.com/backend-api/codex/responses/{sub_path}"
- else:
- url = f"{proxy.OPENAI_API_URL}/v1/responses/{sub_path}"
-
- if request.url.query:
- url = f"{url}?{request.url.query}"
-
- body = await request.body()
- try:
- assert proxy.http_client is not None
- resp = await proxy.http_client.request(
- request.method,
- url,
- headers=headers,
- content=body,
- timeout=120.0,
- )
- return Response(
- content=resp.content,
- status_code=resp.status_code,
- headers=dict(resp.headers),
- )
- except Exception as e:
- logger.error(f"Passthrough /v1/responses/{sub_path} failed: {e}")
- return Response(content=str(e), status_code=502)
-
- @app.api_route("/v1/codex/responses/{sub_path:path}", methods=["GET", "POST", "DELETE"])
- async def openai_v1_codex_responses_sub(request: Request, sub_path: str):
- """Passthrough for Pi/OpenAI Codex /v1/codex/responses/* sub-endpoints."""
- return await openai_responses_sub(request, sub_path)
-
- @app.websocket("/backend-api/responses")
- async def openai_codex_responses_ws(websocket: WebSocket):
- """OpenAI Codex Responses WebSocket path preserved from ChatGPT backend."""
- await proxy.handle_openai_responses_ws(websocket)
-
- @app.websocket("/backend-api/codex/responses")
- async def openai_codex_nested_responses_ws(websocket: WebSocket):
- """OpenAI Codex Responses WebSocket path for codex-shaped proxy base URLs."""
- await proxy.handle_openai_responses_ws(websocket)
-
- @app.api_route("/backend-api/responses/{sub_path:path}", methods=["GET", "POST", "DELETE"])
- async def openai_codex_responses_sub(request: Request, sub_path: str):
- """Passthrough for /backend-api/responses/* sub-endpoints."""
- return await openai_responses_sub(request, sub_path)
-
- @app.api_route(
- "/backend-api/codex/responses/{sub_path:path}", methods=["GET", "POST", "DELETE"]
- )
- async def openai_codex_nested_responses_sub(request: Request, sub_path: str):
- """Passthrough for /backend-api/codex/responses/* sub-endpoints."""
- return await openai_responses_sub(request, sub_path)
-
- # OpenAI Batch API endpoints (with compression!)
- @app.post("/v1/batches")
- async def create_batch(request: Request):
- """Create a batch with automatic compression of messages."""
- return await proxy.handle_batch_create(request)
-
- @app.get("/v1/batches")
- async def list_batches(request: Request):
- """List batches (passthrough to OpenAI)."""
- return await proxy.handle_batch_list(request)
-
- @app.get("/v1/batches/{batch_id}")
- async def get_batch(request: Request, batch_id: str):
- """Get batch details (passthrough to OpenAI)."""
- return await proxy.handle_batch_get(request, batch_id)
-
- @app.post("/v1/batches/{batch_id}/cancel")
- async def cancel_batch(request: Request, batch_id: str):
- """Cancel a batch (passthrough to OpenAI)."""
- return await proxy.handle_batch_cancel(request, batch_id)
-
- # Gemini native endpoints
- @app.post("/v1beta/models/{model}:generateContent")
- async def gemini_generate_content(request: Request, model: str):
- """Gemini native generateContent API."""
- return await proxy.handle_gemini_generate_content(request, model)
-
- @app.post("/v1beta/models/{model}:streamGenerateContent")
- async def gemini_stream_generate_content(request: Request, model: str):
- """Gemini native streaming generateContent API."""
- return await proxy.handle_gemini_stream_generate_content(request, model)
-
- @app.post("/v1beta/models/{model}:countTokens")
- async def gemini_count_tokens(request: Request, model: str):
- """Gemini countTokens API with compression applied."""
- return await proxy.handle_gemini_count_tokens(request, model)
-
- @app.post("/v1internal:streamGenerateContent")
- async def google_cloudcode_stream_generate_content(request: Request):
- """Google Cloud Code Assist / Antigravity compatibility streaming endpoint."""
- return await proxy.handle_google_cloudcode_stream(request)
-
- @app.post("/v1/v1internal:streamGenerateContent")
- async def google_cloudcode_stream_generate_content_v1(request: Request):
- """Compatibility endpoint for clients configured with a /v1 proxy base URL."""
- return await proxy.handle_google_cloudcode_stream(request)
-
- # =========================================================================
- # Databricks Native Endpoints
- # =========================================================================
-
- @app.post("/serving-endpoints/{model}/invocations")
- async def databricks_invocations(request: Request, model: str):
- """Databricks native serving endpoint - compatible with Databricks CLI.
-
- This allows using the Databricks CLI directly with Headroom proxy:
- databricks serving-endpoints query <model> --profile HEADROOM --json '{"messages": [...]}'
-
- The request format is identical to OpenAI chat completions.
- """
- return await proxy.handle_databricks_invocations(request, model)
-
- # =========================================================================
- # Passthrough Endpoints (no compression needed)
- # =========================================================================
-
- # --- OpenAI Passthrough Endpoints ---
-
- @app.get("/v1/models")
- async def list_models(request: Request):
- """List models - route based on auth header.
-
- - x-api-key / anthropic-version / Bearer sk-ant-* -> Anthropic
- - Otherwise -> OpenAI
- """
- if is_anthropic_auth(dict(request.headers)):
- return await proxy.handle_passthrough(
- request, proxy.ANTHROPIC_API_URL, "models", "anthropic"
- )
- return await proxy.handle_passthrough(request, proxy.OPENAI_API_URL, "models", "openai")
-
- @app.get("/v1/models/{model_id}")
- async def get_model(request: Request, model_id: str):
- """Get model details - route based on auth header.
-
- - x-api-key / anthropic-version / Bearer sk-ant-* -> Anthropic
- - Otherwise -> OpenAI
- """
- if is_anthropic_auth(dict(request.headers)):
- return await proxy.handle_passthrough(
- request, proxy.ANTHROPIC_API_URL, "models", "anthropic"
- )
- return await proxy.handle_passthrough(request, proxy.OPENAI_API_URL, "models", "openai")
-
- @app.post("/v1/embeddings")
- async def openai_embeddings(request: Request):
- """OpenAI embeddings API - passthrough."""
- return await proxy.handle_passthrough(request, proxy.OPENAI_API_URL, "embeddings", "openai")
-
- @app.post("/v1/moderations")
- async def openai_moderations(request: Request):
- """OpenAI moderations API - passthrough."""
- return await proxy.handle_passthrough(
- request, proxy.OPENAI_API_URL, "moderations", "openai"
- )
-
- @app.post("/v1/images/generations")
- async def openai_images_generations(request: Request):
- """OpenAI image generation API - passthrough."""
- return await proxy.handle_passthrough(
- request, proxy.OPENAI_API_URL, "images/generations", "openai"
- )
-
- @app.post("/v1/audio/transcriptions")
- async def openai_audio_transcriptions(request: Request):
- """OpenAI audio transcription API (multipart/form-data) - passthrough."""
- return await proxy.handle_passthrough(
- request, proxy.OPENAI_API_URL, "audio/transcriptions", "openai"
- )
-
- @app.post("/v1/audio/speech")
- async def openai_audio_speech(request: Request):
- """OpenAI text-to-speech API - passthrough."""
- return await proxy.handle_passthrough(
- request, proxy.OPENAI_API_URL, "audio/speech", "openai"
- )
-
- # --- Gemini Passthrough Endpoints ---
-
- @app.get("/v1beta/models")
- async def gemini_list_models(request: Request):
- """Gemini list models API - passthrough."""
- return await proxy.handle_passthrough(request, proxy.GEMINI_API_URL, "models", "gemini")
-
- @app.get("/v1beta/models/{model_name}")
- async def gemini_get_model(request: Request, model_name: str):
- """Gemini get model API - passthrough.
-
- Note: This handles GET /v1beta/models/{model_name} but NOT :countTokens
- which is handled by a separate POST route above.
- """
- return await proxy.handle_passthrough(request, proxy.GEMINI_API_URL, "models", "gemini")
-
- @app.post("/v1beta/models/{model}:embedContent")
- async def gemini_embed_content(request: Request, model: str):
- """Gemini embedding API - passthrough."""
- return await proxy.handle_passthrough(
- request, proxy.GEMINI_API_URL, "embedContent", "gemini"
- )
-
- @app.post("/v1beta/models/{model}:batchEmbedContents")
- async def gemini_batch_embed_contents(request: Request, model: str):
- """Gemini batch embeddings API - passthrough."""
- return await proxy.handle_passthrough(
- request, proxy.GEMINI_API_URL, "batchEmbedContents", "gemini"
- )
-
- # Google/Gemini Batch API endpoints (with compression!)
- @app.post("/v1beta/models/{model}:batchGenerateContent")
- async def gemini_batch_create(request: Request, model: str):
- """Create a Gemini batch with compression applied to all requests."""
- return await proxy.handle_google_batch_create(request, model)
-
- @app.get("/v1beta/batches/{batch_name}")
- async def gemini_batch_get(request: Request, batch_name: str):
- """Get a specific Gemini batch with CCR post-processing."""
- return await proxy.handle_google_batch_results(request, batch_name)
-
- @app.post("/v1beta/batches/{batch_name}:cancel")
- async def gemini_batch_cancel(request: Request, batch_name: str):
- """Cancel a Gemini batch (passthrough)."""
- return await proxy.handle_google_batch_passthrough(request, batch_name)
-
- @app.delete("/v1beta/batches/{batch_name}")
- async def gemini_batch_delete(request: Request, batch_name: str):
- """Delete a Gemini batch (passthrough)."""
- return await proxy.handle_google_batch_passthrough(request, batch_name)
-
- @app.post("/v1beta/cachedContents")
- async def gemini_create_cached_content(request: Request):
- """Gemini create cached content API - passthrough."""
- return await proxy.handle_passthrough(
- request, proxy.GEMINI_API_URL, "cachedContents", "gemini"
- )
-
- @app.get("/v1beta/cachedContents")
- async def gemini_list_cached_contents(request: Request):
- """Gemini list cached contents API - passthrough."""
- return await proxy.handle_passthrough(
- request, proxy.GEMINI_API_URL, "cachedContents", "gemini"
- )
-
- @app.get("/v1beta/cachedContents/{cache_id}")
- async def gemini_get_cached_content(request: Request, cache_id: str):
- """Gemini get cached content API - passthrough."""
- return await proxy.handle_passthrough(
- request, proxy.GEMINI_API_URL, "cachedContents", "gemini"
- )
-
- @app.delete("/v1beta/cachedContents/{cache_id}")
- async def gemini_delete_cached_content(request: Request, cache_id: str):
- """Gemini delete cached content API - passthrough."""
- return await proxy.handle_passthrough(
- request, proxy.GEMINI_API_URL, "cachedContents", "gemini"
- )
-
- # =========================================================================
- # Catch-all Passthrough
- # =========================================================================
-
- # Passthrough - route to correct backend based on headers
- @app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE"])
- async def passthrough(request: Request, path: str):
- # Allow explicit base URL override (for Azure, custom endpoints, etc.)
- custom_base = request.headers.get("x-headroom-base-url")
- if custom_base:
- return await proxy.handle_passthrough(request, custom_base.rstrip("/"))
-
- # Anthropic: x-api-key, anthropic-version, or Bearer sk-ant-* token
- if is_anthropic_auth(dict(request.headers)):
- base_url = proxy.ANTHROPIC_API_URL
- # Gemini: sends x-goog-api-key
- elif request.headers.get("x-goog-api-key"):
- base_url = proxy.GEMINI_API_URL
- # Azure OpenAI: sends api-key header (not x-api-key)
- elif request.headers.get("api-key"):
- # Azure requires explicit base URL (varies per deployment)
- azure_base = request.headers.get("x-headroom-base-url", "")
- if azure_base:
- base_url = azure_base.rstrip("/")
- else:
- base_url = proxy.OPENAI_API_URL # Fallback
- else:
- # Default: OpenAI
- base_url = proxy.OPENAI_API_URL
- return await proxy.handle_passthrough(request, base_url)
-
- return app
-
-
-def _get_code_aware_banner_status(config: ProxyConfig) -> str:
- """Get code-aware compression status line for banner."""
- if config.code_aware_enabled:
- if is_tree_sitter_available():
- return "ENABLED (AST-based)"
- else:
- return "NOT INSTALLED (pip install headroom-ai[code])"
- else:
- if is_tree_sitter_available():
- return "DISABLED (remove --no-code-aware to enable)"
- return "DISABLED"
-
-
-def run_server(
- config: ProxyConfig | None = None,
- workers: int = 1,
- limit_concurrency: int = 1000,
-):
- """Run the proxy server.
-
- Args:
- config: Proxy configuration
- workers: Number of worker processes (use N for multi-core scaling)
- limit_concurrency: Max concurrent connections before 503 response
- """
- if not FASTAPI_AVAILABLE:
- print("ERROR: FastAPI required. Install: pip install fastapi uvicorn httpx")
- sys.exit(1)
-
- config = config or ProxyConfig()
- app = create_app(config)
-
- code_aware_status = _get_code_aware_banner_status(config)
-
- # Format connection pool info
- pool_info = f"max={config.max_connections}, keepalive={config.max_keepalive_connections}"
- http2_status = "ENABLED" if config.http2 else "DISABLED"
-
- # Backend status - use provider registry for display info
- if config.backend == "anthropic":
- backend_status = "ANTHROPIC (direct API)"
- elif config.backend == "anyllm" or config.backend.startswith("anyllm-"):
- backend_status = f"{config.anyllm_provider.title()} via any-llm"
- else:
- from headroom.backends.litellm import get_provider_config
-
- provider = config.backend.replace("litellm-", "")
- provider_config = get_provider_config(provider)
- if provider_config.uses_region:
- backend_status = (
- f"{provider_config.display_name} via LiteLLM (region={config.bedrock_region})"
- )
- else:
- backend_status = f"{provider_config.display_name} via LiteLLM"
-
- print(f"""
-╔══════════════════════════════════════════════════════════════════════╗
-║ HEADROOM PROXY SERVER ║
-╠══════════════════════════════════════════════════════════════════════╣
-║ Version: 1.0.0 ║
-║ Listening: http://{config.host}:{config.port:<5} ║
-║ Workers: {workers:<3} Concurrency Limit: {limit_concurrency:<5} ║
-║ Backend: {backend_status:<59}║
-╠══════════════════════════════════════════════════════════════════════╣
-║ FEATURES: ║
-║ Optimization: {"ENABLED " if config.optimize else "DISABLED"} ║
-║ Caching: {"ENABLED " if config.cache_enabled else "DISABLED"} (TTL: {config.cache_ttl_seconds}s) ║
-║ Rate Limiting: {"ENABLED " if config.rate_limit_enabled else "DISABLED"} ({config.rate_limit_requests_per_minute} req/min, {config.rate_limit_tokens_per_minute:,} tok/min) ║
-║ Retry: {"ENABLED " if config.retry_enabled else "DISABLED"} (max {config.retry_max_attempts} attempts) ║
-║ Cost Tracking: {"ENABLED " if config.cost_tracking_enabled else "DISABLED"} (budget: {"$" + str(config.budget_limit_usd) + "/" + config.budget_period if config.budget_limit_usd else "unlimited"}) ║
-║ Code-Aware: {code_aware_status:<52}║
-║ HTTP/2: {http2_status:<52}║
-║ Conn Pool: {pool_info:<52}║
-╠══════════════════════════════════════════════════════════════════════╣
-║ USAGE: ║
-║ Claude Code: ANTHROPIC_BASE_URL=http://{config.host}:{config.port} claude ║
-║ Cursor: Set base URL in settings ║
-╠══════════════════════════════════════════════════════════════════════╣
-║ ENDPOINTS: ║
-║ /livez Process liveness ║
-║ /readyz Traffic readiness ║
-║ /health Aggregate health ║
-║ /stats Detailed statistics ║
-║ /metrics Prometheus metrics ║
-║ /cache/clear Clear response cache ║
-║ /v1/retrieve CCR: Retrieve compressed content ║
-║ /v1/retrieve/stats CCR: Compression store stats ║
-║ /v1/retrieve/tool_call CCR: Handle LLM tool calls ║
-║ /v1/feedback CCR: Feedback loop stats & patterns ║
-║ /v1/feedback/{{tool}} CCR: Compression hints for a tool ║
-║ /v1/telemetry Data flywheel: Telemetry stats ║
-║ /v1/telemetry/export Data flywheel: Export for aggregation ║
-║ /v1/telemetry/tools Data flywheel: Per-tool stats ║
-║ /v1/toin/stats TOIN: Overall intelligence stats ║
-║ /v1/toin/patterns TOIN: List learned patterns ║
-║ /v1/toin/pattern/{{hash}} TOIN: Pattern details by hash ║
-╚══════════════════════════════════════════════════════════════════════╝
-""")
-
- uvicorn.run(
- app,
- host=config.host,
- port=config.port,
- log_level="warning",
- workers=workers if workers > 1 else None, # None = single process (default)
- limit_concurrency=limit_concurrency,
- # Defense-in-depth: the loopback guard for /debug/* endpoints trusts
- # request.client.host. uvicorn's ProxyHeadersMiddleware rewrites that
- # from X-Forwarded-For when FORWARDED_ALLOW_IPS is broader than the
- # default. Disabling proxy_headers here guarantees the guard sees the
- # real peer address regardless of env.
- proxy_headers=False,
- )
-
-
-def _get_env_bool(name: str, default: bool) -> bool:
- """Get boolean from environment variable."""
- val = os.environ.get(name)
- if val is None:
- return default
- return val.lower() in ("true", "1", "yes", "on")
-
-
-def _get_env_int(name: str, default: int) -> int:
- """Get integer from environment variable."""
- val = os.environ.get(name)
- if val is None:
- return default
- try:
- return int(val)
- except ValueError:
- return default
-
-
-def _get_env_float(name: str, default: float) -> float:
- """Get float from environment variable."""
- val = os.environ.get(name)
- if val is None:
- return default
- try:
- return float(val)
- except ValueError:
- return default
-
-
-def _get_env_str(name: str, default: str) -> str:
- """Get string from environment variable."""
- return os.environ.get(name, default)
-
-
-def _parse_tool_profiles(cli_profiles: list[str]) -> dict[str, Any]:
- """Parse tool profiles from CLI args and HEADROOM_TOOL_PROFILES env var.
-
- Format: ToolName:level (e.g., Grep:conservative, Bash:moderate)
- Env var format: comma-separated (e.g., "Grep:conservative,Bash:moderate")
-
- Returns:
- Dict mapping tool names to CompressionProfile instances.
- """
- from headroom.config import PROFILE_PRESETS, CompressionProfile
-
- profiles: dict[str, CompressionProfile] = {}
- raw_entries: list[str] = list(cli_profiles)
-
- # Also check env var
- env_val = os.environ.get("HEADROOM_TOOL_PROFILES", "")
- if env_val:
- raw_entries.extend(e.strip() for e in env_val.split(",") if e.strip())
-
- for entry in raw_entries:
- if ":" not in entry:
- logger.warning("Invalid tool profile format (expected ToolName:level): %s", entry)
- continue
- tool_name, level = entry.split(":", 1)
- tool_name = tool_name.strip()
- level = level.strip().lower()
-
- if level in PROFILE_PRESETS:
- profiles[tool_name] = PROFILE_PRESETS[level]
- else:
- logger.warning(
- "Unknown profile level '%s' for tool '%s'. Use: conservative, moderate, aggressive",
- level,
- tool_name,
- )
-
- return profiles
-
-
-if __name__ == "__main__":
- parser = argparse.ArgumentParser(description="Headroom Proxy Server")
-
- # Server
- parser.add_argument("--host", default="127.0.0.1")
- parser.add_argument("--port", type=int, default=8787)
- parser.add_argument(
- "--openai-api-url", help=f"Custom OpenAI API URL (default: {HeadroomProxy.OPENAI_API_URL})"
- )
- parser.add_argument(
- "--anthropic-api-url",
- help=f"Custom Anthropic API URL (default: {HeadroomProxy.ANTHROPIC_API_URL})",
- )
-
- # Backend (anthropic direct, bedrock, openrouter, anyllm, or litellm-<provider>)
- parser.add_argument(
- "--backend",
- default="anthropic",
- help=(
- "Backend: 'anthropic' (direct), 'bedrock' (AWS), 'openrouter', "
- "'anyllm' (any-llm), or 'litellm-<provider>' (e.g., litellm-hosted_vllm, litellm-vertex)"
- ),
- )
- parser.add_argument(
- "--bedrock-region",
- default="us-west-2",
- help="AWS region for Bedrock backend (default: us-west-2)",
- )
- parser.add_argument(
- "--bedrock-profile",
- help="AWS profile for Bedrock backend (default: use default credentials)",
- )
- parser.add_argument(
- "--openrouter-api-key",
- help="OpenRouter API key (or set OPENROUTER_API_KEY env var)",
- )
- parser.add_argument(
- "--anyllm-provider",
- default="openai",
- help="any-llm provider: openai, anthropic, mistral, groq, ollama, bedrock, etc. (default: openai)",
- )
-
- # Connection pool (scalability)
- parser.add_argument(
- "--max-connections",
- type=int,
- default=500,
- help="Max connections to upstream APIs (default: 500)",
- )
- parser.add_argument(
- "--max-keepalive", type=int, default=100, help="Max keepalive connections (default: 100)"
- )
- parser.add_argument(
- "--no-http2",
- action="store_true",
- help="Disable HTTP/2 (enabled by default for better throughput)",
- )
- parser.add_argument(
- "--workers",
- type=int,
- default=1,
- help="Number of worker processes (default: 1, use N for multi-core)",
- )
- parser.add_argument(
- "--limit-concurrency",
- type=int,
- default=1000,
- help="Max concurrent connections before 503 (default: 1000)",
- )
-
- # Optimization
- parser.add_argument("--no-optimize", action="store_true", help="Disable optimization")
- parser.add_argument("--min-tokens", type=int, default=500, help="Min tokens to crush")
- parser.add_argument("--max-items", type=int, default=50, help="Max items after crush")
- parser.add_argument(
- "--tool-profile",
- action="append",
- default=[],
- help="Per-tool compression profile: ToolName:level (e.g., Grep:conservative, Bash:moderate, WebFetch:aggressive). "
- "Can be specified multiple times. Also settable via HEADROOM_TOOL_PROFILES env var.",
- )
-
- # Caching
- parser.add_argument("--no-cache", action="store_true", help="Disable caching")
- parser.add_argument("--cache-ttl", type=int, default=3600, help="Cache TTL seconds")
-
- # Rate limiting
- parser.add_argument("--no-rate-limit", action="store_true", help="Disable rate limiting")
- parser.add_argument("--rpm", type=int, default=60, help="Requests per minute")
- parser.add_argument("--tpm", type=int, default=100000, help="Tokens per minute")
-
- # Cost
- parser.add_argument("--budget", type=float, help="Budget limit in USD")
- parser.add_argument("--budget-period", choices=["hourly", "daily", "monthly"], default="daily")
-
- # Logging
- parser.add_argument("--log-file", help="Log file path")
- parser.add_argument("--log-messages", action="store_true", help="Log full messages")
-
- # Smart routing (content-aware compression)
- parser.add_argument(
- "--no-smart-routing",
- action="store_true",
- help="Disable smart routing (use legacy sequential pipeline)",
- )
-
- # Code-aware compression
- parser.add_argument(
- "--code-aware",
- action="store_true",
- help="Enable AST-based code compression (requires: pip install headroom-ai[code])",
- )
- parser.add_argument(
- "--no-code-aware",
- action="store_true",
- help="Disable code-aware compression",
- )
-
- args = parser.parse_args()
-
- # Environment variable defaults (HEADROOM_* prefix)
- # CLI args override env vars, env vars override ProxyConfig defaults
- env_smart_routing = _get_env_bool("HEADROOM_SMART_ROUTING", True)
- env_code_aware = _get_env_bool("HEADROOM_CODE_AWARE_ENABLED", True)
- env_optimize = _get_env_bool("HEADROOM_OPTIMIZE", True)
- env_cache = _get_env_bool("HEADROOM_CACHE_ENABLED", True)
- env_rate_limit = _get_env_bool("HEADROOM_RATE_LIMIT_ENABLED", True)
-
- # Determine settings: CLI flags override env vars
- # --no-X explicitly disables, --X explicitly enables, neither uses env var
- smart_routing = env_smart_routing if not args.no_smart_routing else False
- code_aware_enabled = (
- env_code_aware
- if not (args.code_aware or args.no_code_aware)
- else (args.code_aware or not args.no_code_aware)
- )
- optimize = env_optimize if not args.no_optimize else False
- cache_enabled = env_cache if not args.no_cache else False
- rate_limit_enabled = env_rate_limit if not args.no_rate_limit else False
-
- # Set OpenRouter API key from CLI if provided
- if hasattr(args, "openrouter_api_key") and args.openrouter_api_key:
- os.environ["OPENROUTER_API_KEY"] = args.openrouter_api_key
-
- # Parse per-tool compression profiles from CLI and env var
- tool_profiles = _parse_tool_profiles(args.tool_profile)
-
- config = ProxyConfig(
- host=_get_env_str("HEADROOM_HOST", args.host),
- port=_get_env_int("HEADROOM_PORT", args.port),
- openai_api_url=_get_env_str("OPENAI_TARGET_API_URL", args.openai_api_url),
- anthropic_api_url=_get_env_str("ANTHROPIC_TARGET_API_URL", args.anthropic_api_url),
- # Backend settings
- backend=_get_env_str("HEADROOM_BACKEND", args.backend), # type: ignore[arg-type]
- bedrock_region=_get_env_str("HEADROOM_BEDROCK_REGION", args.bedrock_region),
- bedrock_profile=args.bedrock_profile or os.environ.get("AWS_PROFILE"),
- anyllm_provider=_get_env_str("HEADROOM_ANYLLM_PROVIDER", args.anyllm_provider),
- optimize=optimize,
- min_tokens_to_crush=_get_env_int("HEADROOM_MIN_TOKENS", args.min_tokens),
- max_items_after_crush=_get_env_int("HEADROOM_MAX_ITEMS", args.max_items),
- cache_enabled=cache_enabled,
- cache_ttl_seconds=_get_env_int("HEADROOM_CACHE_TTL", args.cache_ttl),
- rate_limit_enabled=rate_limit_enabled,
- rate_limit_requests_per_minute=_get_env_int("HEADROOM_RPM", args.rpm),
- rate_limit_tokens_per_minute=_get_env_int("HEADROOM_TPM", args.tpm),
- budget_limit_usd=args.budget,
- budget_period=args.budget_period,
- log_file=_get_env_str("HEADROOM_LOG_FILE", args.log_file)
- if args.log_file
- else os.environ.get("HEADROOM_LOG_FILE"),
- log_full_messages=args.log_messages or _get_env_bool("HEADROOM_LOG_MESSAGES", False),
- smart_routing=smart_routing,
- code_aware_enabled=code_aware_enabled,
- # Connection pool settings
- max_connections=_get_env_int("HEADROOM_MAX_CONNECTIONS", args.max_connections),
- max_keepalive_connections=_get_env_int("HEADROOM_MAX_KEEPALIVE", args.max_keepalive),
- http2=not args.no_http2 and _get_env_bool("HEADROOM_HTTP2", True),
- tool_profiles=tool_profiles if tool_profiles else None,
- mode=normalize_proxy_mode(_get_env_str("HEADROOM_MODE", PROXY_MODE_TOKEN)),
- )
-
- # Get worker and concurrency settings
- workers = _get_env_int("HEADROOM_WORKERS", args.workers)
- limit_concurrency = _get_env_int("HEADROOM_LIMIT_CONCURRENCY", args.limit_concurrency)
-
- run_server(config, workers=workers, limit_concurrency=limit_concurrency)
+"""Headroom Proxy Server - Production Ready.
+
+A full-featured LLM proxy with optimization, caching, rate limiting,
+and observability.
+
+Features:
+- Context optimization (SmartCrusher, CacheAligner, RollingWindow)
+- Semantic caching (save costs on repeated queries)
+- Rate limiting (token bucket)
+- Retry with exponential backoff
+- Cost tracking and budgets
+- Request tagging and metadata
+- Provider fallback
+- Prometheus metrics
+- Full request/response logging
+
+Usage:
+ python -m headroom.proxy.server --port 8787
+
+ # With Claude Code:
+ ANTHROPIC_BASE_URL=http://localhost:8787 claude
+"""
+
+from __future__ import annotations
+
+import argparse
+import asyncio
+import json
+import logging
+import os
+import sys
+import time
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import TYPE_CHECKING, Any, Literal
+
+if TYPE_CHECKING:
+ from ..backends.base import Backend
+ from ..cache.compression_cache import CompressionCache
+ from ..memory.tracker import MemoryTracker
+
+
+import httpx
+
+try:
+ import uvicorn
+ from fastapi import Depends, FastAPI, HTTPException, Request, Response
+ from fastapi.middleware.cors import CORSMiddleware
+ from fastapi.responses import HTMLResponse, JSONResponse, PlainTextResponse
+
+ FASTAPI_AVAILABLE = True
+except ImportError:
+ FASTAPI_AVAILABLE = False
+
+# Add parent to path for imports
+sys.path.insert(0, str(Path(__file__).parent.parent.parent))
+
+from headroom._version import __version__
+from headroom.cache.compression_feedback import get_compression_feedback
+from headroom.cache.compression_store import get_compression_store
+from headroom.ccr import (
+ CCR_TOOL_NAME,
+ # Batch processing
+ CCRResponseHandler,
+ CCRToolInjector,
+ ContextTracker,
+ ContextTrackerConfig,
+ ResponseHandlerConfig,
+ parse_tool_call,
+)
+from headroom.config import (
+ CacheAlignerConfig,
+ CCRConfig,
+ IntelligentContextConfig,
+ ReadLifecycleConfig,
+ RollingWindowConfig,
+ SmartCrusherConfig,
+)
+from headroom.dashboard import get_dashboard_html
+from headroom.observability import (
+ LangfuseTracingConfig,
+ OTelMetricsConfig,
+ configure_langfuse_tracing,
+ configure_otel_metrics,
+ get_langfuse_tracing_status,
+ get_otel_metrics_status,
+ shutdown_headroom_tracing,
+ shutdown_otel_metrics,
+)
+from headroom.pipeline import PipelineExtensionManager, PipelineStage
+from headroom.providers.proxy_routes import register_provider_routes
+from headroom.providers.registry import (
+ DEFAULT_ANTHROPIC_API_URL,
+ DEFAULT_OPENAI_API_URL,
+ build_proxy_provider_runtime,
+ create_proxy_backend,
+ format_backend_status,
+)
+
+# =============================================================================
+# Extracted modules (re-exported for backward compatibility)
+# =============================================================================
+from headroom.proxy.cost import (
+ _CACHE_ECONOMICS, # noqa: F401
+ CostTracker, # noqa: F401
+ _summarize_transforms, # noqa: F401
+ build_prefix_cache_stats, # noqa: F401
+ build_session_summary, # noqa: F401
+ merge_cost_stats, # noqa: F401
+)
+from headroom.proxy.helpers import (
+ COMPRESSION_TIMEOUT_SECONDS, # noqa: F401
+ MAX_COMPRESSION_CACHE_SESSIONS, # noqa: F401
+ MAX_MESSAGE_ARRAY_LENGTH, # noqa: F401
+ MAX_REQUEST_BODY_SIZE, # noqa: F401
+ MAX_SSE_BUFFER_SIZE, # noqa: F401
+ _get_image_compressor, # noqa: F401
+ _get_rtk_stats, # noqa: F401
+ _read_request_json, # noqa: F401
+ _setup_file_logging, # noqa: F401
+ is_anthropic_auth, # noqa: F401
+ jitter_delay_ms,
+)
+from headroom.proxy.memory_handler import MemoryConfig, MemoryHandler
+
+# Data models (extracted to headroom/proxy/models.py for maintainability)
+from headroom.proxy.models import CacheEntry, ProxyConfig, RateLimitState, RequestLog # noqa: F401
+from headroom.proxy.modes import (
+ PROXY_MODE_CACHE,
+ PROXY_MODE_TOKEN,
+ is_token_mode,
+ normalize_proxy_mode,
+)
+from headroom.proxy.prometheus_metrics import PrometheusMetrics # noqa: F401
+from headroom.proxy.rate_limiter import TokenBucketRateLimiter # noqa: F401
+from headroom.proxy.request_logger import RequestLogger # noqa: F401
+from headroom.proxy.semantic_cache import SemanticCache # noqa: F401
+from headroom.proxy.warmup import WarmupRegistry
+from headroom.proxy.ws_session_registry import WebSocketSessionRegistry
+from headroom.subscription.base import get_quota_registry, reset_quota_registry
+from headroom.subscription.codex_rate_limits import get_codex_rate_limit_state
+from headroom.subscription.copilot_quota import get_copilot_quota_tracker
+from headroom.subscription.tracker import (
+ configure_subscription_tracker,
+ get_subscription_tracker,
+)
+from headroom.telemetry import get_telemetry_collector
+from headroom.telemetry.beacon import is_telemetry_enabled
+from headroom.telemetry.toin import get_toin
+from headroom.transforms import (
+ CacheAligner,
+ CodeAwareCompressor,
+ CodeCompressorConfig,
+ ContentRouter,
+ ContentRouterConfig,
+ IntelligentContextManager,
+ RollingWindow,
+ SmartCrusher,
+ Transform,
+ TransformPipeline,
+ is_tree_sitter_available,
+)
+
+fcntl: Any = None
+try:
+ import fcntl as _fcntl
+
+ fcntl = _fcntl
+ HAS_FCNTL = True
+except ImportError:
+ HAS_FCNTL = False
+
+_build_prefix_cache_stats = build_prefix_cache_stats
+_build_session_summary = build_session_summary
+_merge_cost_stats = merge_cost_stats
+
+
+logging.basicConfig(
+ level=logging.INFO, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
+)
+logger = logging.getLogger("headroom.proxy")
+
+# Always-on file logging to ~/.headroom/logs/ for `headroom perf` analysis
+_setup_file_logging()
+
+
+# Compression pipeline timeout in seconds
+
+
+from headroom.proxy.handlers import ( # noqa: E402
+ AnthropicHandlerMixin,
+ BatchHandlerMixin,
+ GeminiHandlerMixin,
+ OpenAIHandlerMixin,
+ StreamingMixin,
+)
+
+
+class HeadroomProxy(
+ StreamingMixin,
+ AnthropicHandlerMixin,
+ OpenAIHandlerMixin,
+ GeminiHandlerMixin,
+ BatchHandlerMixin,
+):
+ """Production-ready Headroom optimization proxy."""
+
+ def __init__(self, config: ProxyConfig):
+ self.config = config
+ self.config.mode = normalize_proxy_mode(self.config.mode)
+ self.pipeline_extensions = PipelineExtensionManager(
+ hooks=config.hooks,
+ extensions=config.pipeline_extensions,
+ discover=config.discover_pipeline_extensions,
+ )
+
+ self.provider_runtime = build_proxy_provider_runtime(config)
+
+ # Initialize transforms based on routing mode
+ # Choose context manager: IntelligentContextManager (smart) or RollingWindow (legacy)
+ context_manager: Transform # Can be either IntelligentContextManager or RollingWindow
+ if config.intelligent_context:
+ # Get TOIN instance for learned pattern integration
+ toin = get_toin() if config.intelligent_context_scoring else None
+ context_manager = IntelligentContextManager(
+ config=IntelligentContextConfig(
+ enabled=True,
+ keep_system=True,
+ keep_last_turns=config.keep_last_turns,
+ use_importance_scoring=config.intelligent_context_scoring,
+ toin_integration=config.intelligent_context_scoring,
+ compress_threshold=0.10 if config.intelligent_context_compress_first else 0.0,
+ ),
+ toin=toin,
+ )
+ self._context_manager_status = "intelligent"
+ else:
+ context_manager = RollingWindow(
+ RollingWindowConfig(
+ enabled=True,
+ keep_system=True,
+ keep_last_turns=config.keep_last_turns,
+ )
+ )
+ self._context_manager_status = "rolling_window"
+
+ if config.smart_routing:
+ # Smart routing: ContentRouter handles all content types intelligently
+ # It lazy-loads compressors only when needed
+ router_config = ContentRouterConfig(
+ enable_code_aware=config.code_aware_enabled,
+ tool_profiles=config.tool_profiles,
+ read_lifecycle=ReadLifecycleConfig(enabled=config.read_lifecycle),
+ )
+ # Token mode: allow compression of older excluded-tool results
+ if is_token_mode(config.mode):
+ router_config.protect_recent_reads_fraction = 0.3
+ transforms = [
+ CacheAligner(CacheAlignerConfig(enabled=False)),
+ ContentRouter(router_config),
+ context_manager,
+ ]
+ self._code_aware_status = "lazy" if config.code_aware_enabled else "disabled"
+ else:
+ # Legacy mode: sequential pipeline
+ transforms = [
+ CacheAligner(CacheAlignerConfig(enabled=False)),
+ SmartCrusher(
+ SmartCrusherConfig( # type: ignore[arg-type]
+ enabled=True,
+ min_tokens_to_crush=config.min_tokens_to_crush,
+ max_items_after_crush=config.max_items_after_crush,
+ ),
+ ccr_config=CCRConfig(
+ enabled=config.ccr_inject_tool,
+ inject_retrieval_marker=config.ccr_inject_tool, # Add CCR markers
+ ),
+ ),
+ context_manager,
+ ]
+ # Add CodeAware if enabled and available
+ self._code_aware_status = self._setup_code_aware(config, transforms)
+
+ self.anthropic_pipeline = TransformPipeline(
+ transforms=transforms,
+ provider=self.provider_runtime.pipeline_provider("anthropic"),
+ )
+ self.openai_pipeline = TransformPipeline(
+ transforms=transforms,
+ provider=self.provider_runtime.pipeline_provider("openai"),
+ )
+
+ # Initialize components
+ self.cache = (
+ SemanticCache(
+ max_entries=config.cache_max_entries,
+ ttl_seconds=config.cache_ttl_seconds,
+ )
+ if config.cache_enabled
+ else None
+ )
+
+ self.rate_limiter = (
+ TokenBucketRateLimiter(
+ requests_per_minute=config.rate_limit_requests_per_minute,
+ tokens_per_minute=config.rate_limit_tokens_per_minute,
+ )
+ if config.rate_limit_enabled
+ else None
+ )
+
+ self.cost_tracker = (
+ CostTracker(
+ budget_limit_usd=config.budget_limit_usd,
+ budget_period=config.budget_period,
+ )
+ if config.cost_tracking_enabled
+ else None
+ )
+
+ self.metrics = PrometheusMetrics(cost_tracker=self.cost_tracker)
+
+ # Prefix cache tracking: freeze already-cached messages to avoid
+ # invalidating the provider's prefix cache with our transforms
+ from headroom.cache.prefix_tracker import PrefixFreezeConfig, SessionTrackerStore
+
+ self.session_tracker_store = SessionTrackerStore(
+ default_config=PrefixFreezeConfig(
+ enabled=config.prefix_freeze_enabled,
+ session_ttl_seconds=config.prefix_freeze_session_ttl,
+ )
+ )
+
+ # Compression cache store for token mode (session-scoped)
+ self._compression_caches: dict[str, CompressionCache] = {}
+
+ self.logger = (
+ RequestLogger(
+ log_file=config.log_file,
+ log_full_messages=config.log_full_messages,
+ )
+ if config.log_requests
+ else None
+ )
+
+ # Enterprise security plugin (loaded dynamically if available + licensed)
+ self.security = None
+
+ # HTTP client
+ self.http_client: httpx.AsyncClient | None = None
+
+ # Shared cold-start warmup registry (populated by startup()).
+ # Holds typed slots with loaded / loading / null / error status for
+ # each preloaded heavy asset. Exposed as ``proxy.warmup`` and
+ # serialized by the /debug/warmup route (Unit 5).
+ self.warmup: WarmupRegistry = WarmupRegistry()
+ # Unit 3: live registry of Codex WS sessions. Populated by
+ # ``handle_openai_responses_ws`` on accept; drained in its
+ # outermost ``finally``. Consumed by ``/debug/ws-sessions``.
+ self.ws_sessions: WebSocketSessionRegistry = WebSocketSessionRegistry()
+
+ # Unit 4: bounded pre-upstream concurrency for the Anthropic HTTP
+ # path. Caps how many ``handle_anthropic_messages`` calls may be
+ # running deep-copy / first-stage compression / memory-context
+ # lookup / upstream connect concurrently. ``/livez``, ``/readyz``,
+ # ``/health``, ``/metrics``, ``/stats``, and the Codex WS path are
+ # intentionally NOT gated by this semaphore.
+ #
+ # A value of ``0`` or negative disables the semaphore (unbounded
+ # mode); this is useful for the Unit 6 counter-factual where we
+ # deliberately reproduce the original starvation. The default is
+ # ``max(2, min(8, os.cpu_count() or 4))``.
+ _pre_upstream_cfg = config.anthropic_pre_upstream_concurrency
+ if _pre_upstream_cfg is None:
+ _pre_upstream_resolved = max(2, min(8, os.cpu_count() or 4))
+ else:
+ _pre_upstream_resolved = _pre_upstream_cfg
+ self.anthropic_pre_upstream_concurrency: int = _pre_upstream_resolved
+ self.anthropic_pre_upstream_acquire_timeout_seconds = float(
+ config.anthropic_pre_upstream_acquire_timeout_seconds
+ )
+ self.anthropic_pre_upstream_memory_context_timeout_seconds = float(
+ config.anthropic_pre_upstream_memory_context_timeout_seconds
+ )
+ if _pre_upstream_resolved > 0:
+ self.anthropic_pre_upstream_sem: asyncio.Semaphore | None = asyncio.Semaphore(
+ _pre_upstream_resolved
+ )
+ else:
+ self.anthropic_pre_upstream_sem = None
+
+ # Backend for Anthropic API (direct, LiteLLM, or any-llm)
+ # Supports: "anthropic" (direct), "bedrock", "vertex", "litellm-<provider>", or "anyllm"
+ self.anthropic_backend: Backend | None = create_proxy_backend(
+ backend=config.backend,
+ anyllm_provider=config.anyllm_provider,
+ bedrock_region=config.bedrock_region,
+ logger=logger,
+ )
+
+ # Request counter for IDs
+ self._request_counter = 0
+ self._request_counter_lock = asyncio.Lock()
+
+ # CCR tool injectors (one per provider)
+ self.anthropic_tool_injector = CCRToolInjector(
+ provider="anthropic",
+ inject_tool=config.ccr_inject_tool,
+ inject_system_instructions=config.ccr_inject_system_instructions,
+ )
+ self.openai_tool_injector = CCRToolInjector(
+ provider="openai",
+ inject_tool=config.ccr_inject_tool,
+ inject_system_instructions=config.ccr_inject_system_instructions,
+ )
+
+ # CCR Response Handler (handles CCR tool calls automatically)
+ self.ccr_response_handler = (
+ CCRResponseHandler(
+ ResponseHandlerConfig(
+ enabled=True,
+ max_retrieval_rounds=config.ccr_max_retrieval_rounds,
+ )
+ )
+ if config.ccr_handle_responses
+ else None
+ )
+
+ # CCR Context Tracker (tracks compressed content across turns)
+ self.ccr_context_tracker = (
+ ContextTracker(
+ ContextTrackerConfig(
+ enabled=True,
+ proactive_expansion=config.ccr_proactive_expansion,
+ max_proactive_expansions=config.ccr_max_proactive_expansions,
+ )
+ )
+ if config.ccr_context_tracking
+ else None
+ )
+
+ # Turn counter for context tracking
+ self._turn_counter = 0
+
+ # Memory Handler (persistent user memory)
+ self.memory_handler: MemoryHandler | None = None
+ if config.memory_enabled:
+ # Resolve memory DB path: empty → project-scoped default
+ _mem_db_path = config.memory_db_path
+ if not _mem_db_path:
+ _mem_dir = Path.cwd() / ".headroom"
+ _mem_dir.mkdir(parents=True, exist_ok=True)
+ _mem_db_path = str(_mem_dir / "memory.db")
+ logger.info(f"Memory: Project-scoped DB at {_mem_db_path}")
+
+ memory_config = MemoryConfig(
+ enabled=True,
+ backend=config.memory_backend,
+ db_path=_mem_db_path,
+ inject_tools=config.memory_inject_tools,
+ use_native_tool=config.memory_use_native_tool,
+ inject_context=config.memory_inject_context,
+ top_k=config.memory_top_k,
+ min_similarity=config.memory_min_similarity,
+ qdrant_host=config.memory_qdrant_host,
+ qdrant_port=config.memory_qdrant_port,
+ neo4j_uri=config.memory_neo4j_uri,
+ neo4j_user=config.memory_neo4j_user,
+ neo4j_password=config.memory_neo4j_password,
+ bridge_enabled=config.memory_bridge_enabled,
+ bridge_md_paths=config.memory_bridge_md_paths,
+ bridge_md_format=config.memory_bridge_md_format,
+ bridge_auto_import=config.memory_bridge_auto_import,
+ bridge_export_path=config.memory_bridge_export_path,
+ )
+ self.memory_handler = MemoryHandler(
+ memory_config,
+ agent_type=config.traffic_learning_agent_type,
+ )
+
+ # Usage Reporter (license validation + phone-home for managed/enterprise)
+ self.usage_reporter: UsageReporter | None = None
+ if config.license_key:
+ from headroom.telemetry.reporter import UsageReporter
+
+ self.usage_reporter = UsageReporter(
+ license_key=config.license_key,
+ cloud_url=config.license_cloud_url,
+ report_interval=config.license_report_interval,
+ )
+
+ # Traffic Learner (live pattern extraction from proxy traffic)
+ # Only activates with --learn flag; requires --memory for backend
+ self.traffic_learner: TrafficLearner | None = None
+ self.traffic_learning_agent_type: str = config.traffic_learning_agent_type
+ if config.traffic_learning_enabled:
+ from headroom.memory.traffic_learner import TrafficLearner
+
+ self.traffic_learner = TrafficLearner(
+ user_id=os.environ.get("HEADROOM_USER_ID", os.environ.get("USER", "default")),
+ agent_type=config.traffic_learning_agent_type,
+ )
+
+ # Code graph file watcher (live reindex on file changes)
+ self.code_graph_watcher: CodeGraphWatcher | None = None # type: ignore[annotation-unchecked]
+ if config.code_graph_watcher:
+ from headroom.graph.watcher import CodeGraphWatcher
+
+ self.code_graph_watcher = CodeGraphWatcher(project_dir=Path.cwd())
+ if self.code_graph_watcher.start():
+ logger.info("Code graph: file watcher started")
+ else:
+ self.code_graph_watcher = None
+
+ self.pipeline_extensions.emit(
+ PipelineStage.SETUP,
+ operation="proxy.setup",
+ metadata={
+ "mode": self.config.mode,
+ "optimize": self.config.optimize,
+ "backend": self.config.backend,
+ "memory_enabled": self.config.memory_enabled,
+ },
+ )
+
+ def _get_compression_cache(self, session_id: str) -> CompressionCache:
+ """Get or create a CompressionCache for a session."""
+ if session_id not in self._compression_caches:
+ from headroom.cache.compression_cache import CompressionCache
+
+ # Evict oldest caches if at capacity
+ if len(self._compression_caches) >= MAX_COMPRESSION_CACHE_SESSIONS:
+ # Remove oldest quarter to amortize cleanup cost
+ oldest_keys = list(self._compression_caches.keys())[
+ : MAX_COMPRESSION_CACHE_SESSIONS // 4
+ ]
+ for key in oldest_keys:
+ del self._compression_caches[key]
+ logger.info(
+ "Evicted %d compression caches (exceeded %d max sessions)",
+ len(oldest_keys),
+ MAX_COMPRESSION_CACHE_SESSIONS,
+ )
+
+ self._compression_caches[session_id] = CompressionCache()
+ return self._compression_caches[session_id]
+
+ def _setup_code_aware(self, config: ProxyConfig, transforms: list) -> str:
+ """Set up code-aware compression if enabled.
+
+ Args:
+ config: Proxy configuration
+ transforms: Transform list to append to
+
+ Returns:
+ Status string for logging: 'enabled', 'disabled', 'available', 'unavailable'
+ """
+ if config.code_aware_enabled:
+ if is_tree_sitter_available():
+ code_config = CodeCompressorConfig(
+ preserve_imports=True,
+ preserve_signatures=True,
+ preserve_type_annotations=True,
+ )
+ # Insert before RollingWindow (which should be last)
+ transforms.insert(-1, CodeAwareCompressor(code_config))
+ return "enabled"
+ else:
+ logger.warning(
+ "Code-aware compression requested but tree-sitter not installed. "
+ "Install with: pip install headroom-ai[code]"
+ )
+ return "unavailable"
+ else:
+ if is_tree_sitter_available():
+ return "available" # Available but not enabled
+ return "disabled"
+
+ async def startup(self):
+ """Initialize async resources."""
+ self.pipeline_extensions.emit(
+ PipelineStage.PRE_START,
+ operation="proxy.startup",
+ metadata={"port": self.config.port, "host": self.config.host},
+ )
+ self.http_client = httpx.AsyncClient(
+ timeout=httpx.Timeout(
+ connect=self.config.connect_timeout_seconds,
+ read=self.config.request_timeout_seconds,
+ write=self.config.request_timeout_seconds,
+ pool=self.config.connect_timeout_seconds,
+ ),
+ limits=httpx.Limits(
+ max_connections=self.config.max_connections,
+ max_keepalive_connections=self.config.max_keepalive_connections,
+ ),
+ http2=self.config.http2,
+ )
+ logger.info("Headroom Proxy started")
+ logger.info(f"Optimization: {'ENABLED' if self.config.optimize else 'DISABLED'}")
+ self.config.mode = normalize_proxy_mode(self.config.mode)
+ logger.info(f"Mode: {self.config.mode}")
+ if self.config.mode == PROXY_MODE_TOKEN:
+ logger.info(" Prefix freeze: re-freeze after compression")
+ logger.info(" Read protection window: 30%% of excluded-tool messages")
+ logger.info(" CCR TTL: extended for session lifetime")
+ logger.info(" Compression cache: active")
+ if self.config.mode == PROXY_MODE_CACHE:
+ logger.info(" Prefix freeze: strict (all prior turns immutable)")
+ logger.info(" Mutations: latest turn only")
+ logger.info(f"Caching: {'ENABLED' if self.config.cache_enabled else 'DISABLED'}")
+ logger.info(f"Rate Limiting: {'ENABLED' if self.config.rate_limit_enabled else 'DISABLED'}")
+ logger.info(
+ f"Connection Pool: max_connections={self.config.max_connections}, "
+ f"max_keepalive={self.config.max_keepalive_connections}, "
+ f"http2={'ENABLED' if self.config.http2 else 'DISABLED'}"
+ )
+
+ # Unit 4 pre-upstream concurrency announcement. Report the resolved
+ # value (auto-detected vs. explicit) so operators can correlate
+ # ``pre_upstream_wait_ms`` log lines with the configured cap.
+ if self.anthropic_pre_upstream_sem is None:
+ logger.info("Anthropic pre-upstream concurrency: unbounded (explicitly disabled)")
+ else:
+ _explicit = self.config.anthropic_pre_upstream_concurrency
+ _origin = "auto-detected" if _explicit is None else "explicit"
+ logger.info(
+ "Anthropic pre-upstream concurrency: %d (%s)",
+ self.anthropic_pre_upstream_concurrency,
+ _origin,
+ )
+ logger.info(
+ "Anthropic pre-upstream timeouts: acquire=%.1fs compression=%.1fs memory_context=%.1fs",
+ self.anthropic_pre_upstream_acquire_timeout_seconds,
+ float(COMPRESSION_TIMEOUT_SECONDS),
+ self.anthropic_pre_upstream_memory_context_timeout_seconds,
+ )
+
+ # Smart routing status
+ if self.config.smart_routing:
+ logger.info("Smart Routing: ENABLED (intelligent content detection)")
+ else:
+ logger.info("Smart Routing: DISABLED (legacy sequential mode)")
+
+ # Eagerly load ALL compressors, parsers, and detectors at startup
+ # This eliminates cold-start latency spikes on first requests.
+ # Iterate BOTH pipelines (Anthropic + OpenAI) and dedupe transforms
+ # by id() so shared-transform instances never load twice. The
+ # resulting status dict is merged into ``self.warmup`` so /debug/warmup
+ # (Unit 5) and /readyz have a single source of truth.
+ self._kompress_status = "not installed"
+ eager_status: dict[str, str] = {}
+
+ if self.config.optimize:
+ logger.info("Pre-loading compressors and parsers...")
+ seen_transform_ids: set[int] = set()
+ pipelines = (self.anthropic_pipeline, self.openai_pipeline)
+ for pipeline in pipelines:
+ for transform in pipeline.transforms:
+ if id(transform) in seen_transform_ids:
+ continue
+ seen_transform_ids.add(id(transform))
+ if not hasattr(transform, "eager_load_compressors"):
+ continue
+ try:
+ transform_status = transform.eager_load_compressors()
+ except Exception as exc:
+ logger.warning(
+ "Eager preload failed for %s: %s",
+ type(transform).__name__,
+ exc,
+ )
+ continue
+ if not isinstance(transform_status, dict):
+ continue
+ # Merge: later writers win only if the key wasn't set.
+ # Preload a transform ONCE — if another pipeline also has
+ # ``eager_load_compressors`` it contributes only new keys.
+ for key, value in transform_status.items():
+ eager_status.setdefault(key, value)
+ self.warmup.merge_transform_status(transform_status)
+
+ # Update internal status from eager loading results
+ if eager_status.get("kompress") == "enabled":
+ self._kompress_status = "enabled"
+ if eager_status.get("code_aware") == "enabled":
+ self._code_aware_status = "enabled"
+
+ # Log component status
+ if self._kompress_status == "enabled":
+ logger.info("Kompress: ENABLED (ModernBERT token compressor)")
+ elif self.config.optimize:
+ logger.info("Kompress: not installed (pip install headroom-ai[ml] for ML compression)")
+
+ if self._code_aware_status == "enabled":
+ logger.info("Code-Aware: ENABLED (AST-based compression)")
+ if "tree_sitter" in eager_status:
+ logger.info(f"Tree-Sitter: {eager_status['tree_sitter']}")
+ elif self._code_aware_status == "lazy":
+ logger.info("Code-Aware: LAZY (will load when code content detected)")
+ elif self._code_aware_status == "available":
+ logger.info("Code-Aware: available but disabled (use --code-aware)")
+ elif self._code_aware_status == "unavailable":
+ logger.info("Code-Aware: not installed (pip install headroom-ai[code])")
+ elif self._code_aware_status == "disabled":
+ logger.info("Code-Aware: DISABLED")
+
+ if eager_status.get("magika") == "enabled":
+ logger.info("Magika: ENABLED (ML content detection)")
+
+ if self.memory_handler:
+ self.warmup.memory_backend.mark_loading()
+ try:
+ await self.memory_handler.ensure_initialized()
+ except Exception as exc: # pragma: no cover - defensive
+ self.warmup.memory_backend.mark_error(str(exc))
+ logger.warning("Memory: backend initialization failed (startup continues): %s", exc)
+ memory_status = self.memory_handler.health_status()
+ if memory_status.get("initialized"):
+ self.warmup.memory_backend.mark_loaded(
+ handle=self.memory_handler,
+ backend=memory_status.get("backend"),
+ )
+ # Force one embed call so the ONNX graph is compiled now,
+ # not lazily during the first request. Best-effort — any
+ # failure is swallowed inside warmup_embedder.
+ self.warmup.memory_embedder.mark_loading()
+ warmed = await self.memory_handler.warmup_embedder()
+ if warmed:
+ self.warmup.memory_embedder.mark_loaded()
+ else:
+ # Not an error — e.g. qdrant-neo4j has no embedder slot
+ # we can reach, or the backend simply exposes no handle.
+ self.warmup.memory_embedder.mark_null()
+ else:
+ if self.warmup.memory_backend.status != "error":
+ self.warmup.memory_backend.mark_null()
+ self.warmup.memory_embedder.mark_null()
+ logger.info(
+ "Memory: ENABLED "
+ f"(backend={memory_status['backend']}, initialized={memory_status['initialized']})"
+ )
+ else:
+ logger.info("Memory: DISABLED")
+
+ # CCR status
+ ccr_features = []
+ if self.config.ccr_inject_tool:
+ ccr_features.append("tool_injection")
+ if self.config.ccr_handle_responses:
+ ccr_features.append("response_handling")
+ if self.config.ccr_context_tracking:
+ ccr_features.append("context_tracking")
+ if self.config.ccr_proactive_expansion:
+ ccr_features.append("proactive_expansion")
+ if ccr_features:
+ logger.info(f"CCR (Compress-Cache-Retrieve): ENABLED ({', '.join(ccr_features)})")
+ else:
+ logger.info("CCR: DISABLED")
+ logger.info(f"Savings history: {self.metrics.savings_tracker.storage_path}")
+
+ # Reset and rebuild the quota tracker registry for this server instance.
+ # reset_quota_registry() ensures a clean slate when the proxy is restarted
+ # (e.g. in tests that spin up multiple app instances in the same process).
+ reset_quota_registry()
+ registry = get_quota_registry()
+ tracker = configure_subscription_tracker(
+ poll_interval_s=self.config.subscription_poll_interval_s,
+ active_window_s=self.config.subscription_active_window_s,
+ enabled=self.config.subscription_tracking_enabled,
+ )
+ registry.register(tracker)
+ registry.register(get_codex_rate_limit_state())
+ registry.register(get_copilot_quota_tracker())
+ await registry.start_all()
+
+ if self.config.subscription_tracking_enabled:
+ logger.info(
+ "Subscription tracking: ENABLED "
+ f"(poll_interval={self.config.subscription_poll_interval_s}s, "
+ f"active_window={self.config.subscription_active_window_s}s)"
+ )
+ else:
+ logger.info("Subscription tracking: DISABLED")
+
+ copilot_tracker = get_copilot_quota_tracker()
+ if copilot_tracker.is_available():
+ logger.info("GitHub Copilot quota tracking: ENABLED")
+ else:
+ logger.info(
+ "GitHub Copilot quota tracking: DISABLED "
+ "(set GITHUB_TOKEN or GITHUB_COPILOT_GITHUB_TOKEN to enable)"
+ )
+
+ # Log anonymous telemetry status so operators can see it in the log stream
+ if is_telemetry_enabled():
+ logger.info(
+ "Anonymous telemetry: ENABLED (aggregate stats only — no prompts or content). "
+ "Opt out: HEADROOM_TELEMETRY=off or --no-telemetry"
+ )
+ else:
+ logger.info("Anonymous telemetry: DISABLED")
+
+ self.pipeline_extensions.emit(
+ PipelineStage.POST_START,
+ operation="proxy.startup",
+ metadata={
+ "port": self.config.port,
+ "host": self.config.host,
+ "warmup": self.warmup.to_dict(),
+ },
+ )
+
+ async def shutdown(self):
+ """Cleanup async resources."""
+ if self.http_client:
+ await self.http_client.aclose()
+ self.http_client = None
+
+ if self.memory_handler and hasattr(self.memory_handler, "close"):
+ await self.memory_handler.close()
+
+ # Stop all quota trackers via the registry
+ await get_quota_registry().stop_all()
+
+ # Print final stats
+ self._print_summary()
+
+ def _print_summary(self):
+ """Print session summary."""
+ m = self.metrics
+ logger.info("=" * 70)
+ logger.info("HEADROOM PROXY SESSION SUMMARY")
+ logger.info("=" * 70)
+ logger.info(f"Total requests: {m.requests_total}")
+ logger.info(f"Cached responses: {m.requests_cached}")
+ logger.info(f"Rate limited: {m.requests_rate_limited}")
+ logger.info(f"Failed: {m.requests_failed}")
+ logger.info(f"Input tokens: {m.tokens_input_total:,}")
+ logger.info(f"Output tokens: {m.tokens_output_total:,}")
+ logger.info(f"Tokens saved: {m.tokens_saved_total:,}")
+ if m.tokens_input_total > 0:
+ savings_pct = (
+ m.tokens_saved_total / (m.tokens_input_total + m.tokens_saved_total)
+ ) * 100
+ logger.info(f"Token savings: {savings_pct:.1f}%")
+ if m.latency_count > 0:
+ avg_latency = m.latency_sum_ms / m.latency_count
+ logger.info(f"Avg latency: {avg_latency:.0f}ms")
+ logger.info("=" * 70)
+
+ async def _next_request_id(self) -> str:
+ """Generate unique request ID."""
+ async with self._request_counter_lock:
+ self._request_counter += 1
+ return f"hr_{int(time.time())}_{self._request_counter:06d}"
+
+ def _extract_tags(self, headers: dict) -> dict[str, str]:
+ """Extract Headroom tags from headers."""
+ tags = {}
+ for key, value in headers.items():
+ if key.lower().startswith("x-headroom-"):
+ tag_name = key.lower().replace("x-headroom-", "")
+ tags[tag_name] = value
+ return tags
+
+ def _inject_system_context(
+ self,
+ messages: list[dict[str, Any]],
+ context: str,
+ body: dict[str, Any] | None = None,
+ ) -> list[dict[str, Any]]:
+ """Inject context into the system message/parameter.
+
+ For Anthropic API: Uses top-level 'system' parameter (not messages array).
+ For OpenAI API: Uses system role in messages array.
+
+ Args:
+ messages: The messages list.
+ context: Context to inject.
+ body: Optional request body to update system parameter (for Anthropic).
+
+ Returns:
+ Updated messages list.
+ """
+ messages = list(messages) # Copy to avoid mutation
+
+ # For Anthropic API: use top-level 'system' parameter
+ if body is not None:
+ existing_system = body.get("system", "")
+ if isinstance(existing_system, str):
+ body["system"] = (existing_system + "\n\n" + context).strip()
+ elif isinstance(existing_system, list):
+ # system is a list of content blocks (e.g., with cache_control).
+ # Append memory context as a new text block — never overwrite.
+ body["system"] = existing_system + [{"type": "text", "text": context}]
+ else:
+ # No existing system prompt — set as string
+ body["system"] = context
+ return messages
+
+ # For OpenAI API: use system role in messages
+ for i, msg in enumerate(messages):
+ if msg.get("role") == "system":
+ content = msg.get("content", "")
+ if isinstance(content, str):
+ messages[i] = {**msg, "content": content + "\n\n" + context}
+ return messages
+
+ # No system message found - prepend one
+ messages.insert(0, {"role": "system", "content": context})
+ return messages
+
+ async def _retry_request(
+ self,
+ method: str,
+ url: str,
+ headers: dict,
+ body: dict,
+ stream: bool = False,
+ ) -> httpx.Response:
+ """Make request with retry and exponential backoff."""
+ last_error = None
+
+ for attempt in range(self.config.retry_max_attempts):
+ try:
+ if stream:
+ # For streaming, we return early - retry happens at higher level
+ return await self.http_client.post(url, json=body, headers=headers) # type: ignore[union-attr]
+ else:
+ response = await self.http_client.post(url, json=body, headers=headers) # type: ignore[union-attr]
+
+ # Don't retry client errors (4xx)
+ if 400 <= response.status_code < 500:
+ return response
+
+ # Retry server errors (5xx)
+ if response.status_code >= 500:
+ raise httpx.HTTPStatusError(
+ f"Server error: {response.status_code}",
+ request=response.request,
+ response=response,
+ )
+
+ return response
+
+ except (httpx.ConnectError, httpx.ReadTimeout, httpx.HTTPStatusError) as e:
+ last_error = e
+
+ if not self.config.retry_enabled or attempt >= self.config.retry_max_attempts - 1:
+ raise
+
+ # Exponential backoff with jitter
+ delay_with_jitter = jitter_delay_ms(
+ self.config.retry_base_delay_ms,
+ self.config.retry_max_delay_ms,
+ attempt,
+ )
+
+ logger.warning(
+ f"Request failed (attempt {attempt + 1}), retrying in {delay_with_jitter:.0f}ms: {e}"
+ )
+ await asyncio.sleep(delay_with_jitter / 1000)
+
+ raise last_error # type: ignore[misc]
+
+
+async def _log_toin_stats_periodically(interval_seconds: int = 300) -> None:
+ """Background task that logs TOIN stats periodically.
+
+ Args:
+ interval_seconds: How often to log stats (default: 5 minutes).
+ """
+ while True:
+ await asyncio.sleep(interval_seconds)
+ try:
+ toin = get_toin()
+ stats = toin.get_stats()
+ total_compressions = stats.get("total_compressions", 0)
+ if total_compressions > 0:
+ patterns = stats.get("patterns_tracked", 0)
+ retrievals = stats.get("total_retrievals", 0)
+ retrieval_rate = stats.get("global_retrieval_rate", 0.0)
+ logger.info(
+ "TOIN: %d patterns, %d compressions, %d retrievals, %.1f%% retrieval rate",
+ patterns,
+ total_compressions,
+ retrievals,
+ retrieval_rate * 100,
+ )
+ except Exception as e:
+ logger.debug("Failed to log TOIN stats: %s", e)
+
+
+def _register_memory_components(proxy: HeadroomProxy, tracker: MemoryTracker) -> None:
+ """Register all memory-tracked components with the tracker.
+
+ This function is idempotent - it checks if components are already registered.
+
+ Args:
+ proxy: The HeadroomProxy instance.
+ tracker: The MemoryTracker instance.
+ """
+ # Register compression store (global singleton)
+ if "compression_store" not in tracker.registered_components:
+ store = get_compression_store()
+ tracker.register("compression_store", store.get_memory_stats)
+
+ # Register semantic cache (instance on proxy)
+ if proxy.cache and "semantic_cache" not in tracker.registered_components:
+ tracker.register("semantic_cache", proxy.cache.get_memory_stats)
+
+ # Register request logger (instance on proxy)
+ if proxy.logger and "request_logger" not in tracker.registered_components:
+ tracker.register("request_logger", proxy.logger.get_memory_stats)
+
+ # Register batch context store (global singleton)
+ if "batch_context_store" not in tracker.registered_components:
+ try:
+ from ..ccr.batch_store import get_batch_context_store
+
+ batch_store = get_batch_context_store()
+ if hasattr(batch_store, "get_memory_stats"):
+ tracker.register("batch_context_store", batch_store.get_memory_stats)
+ except ImportError:
+ pass
+
+ # Note: graph_store and vector_index are created per-user within the
+ # LocalMemoryBackend, not as global singletons. They would need to be
+ # registered when the memory system is initialized with specific backends.
+
+
+def create_app(config: ProxyConfig | None = None) -> FastAPI:
+ """Create FastAPI application."""
+ if not FASTAPI_AVAILABLE:
+ raise ImportError("FastAPI required. Install: pip install fastapi uvicorn httpx")
+
+ from contextlib import asynccontextmanager
+
+ config = config or ProxyConfig()
+ proxy = HeadroomProxy(config)
+
+ # Telemetry beacon (anonymous aggregate stats).
+ # With uvicorn workers > 1, each worker runs the lifespan independently.
+ # We must ensure only ONE beacon runs across all workers — otherwise each
+ # worker creates its own beacon, spamming the telemetry table with N rows
+ # per cycle instead of 1 (all reading the same /stats from the same port).
+ #
+ # Strategy: use a file lock to ensure only the first worker starts the
+ # beacon. Other workers see the lock and skip.
+ from headroom.telemetry.beacon import TelemetryBeacon
+
+ _beacon = TelemetryBeacon(
+ port=config.port if hasattr(config, "port") else 8787,
+ sdk=os.environ.get("HEADROOM_SDK", "proxy").strip() or "proxy",
+ backend=config.backend if hasattr(config, "backend") else "anthropic",
+ )
+ from headroom import paths as _hr_paths
+
+ _beacon_lock_path = _hr_paths.beacon_lock_path(config.port)
+ _beacon_lock_fd: list = [None] # mutable holder for the lock file descriptor
+ _beacon_is_owner: list = [False]
+
+ def _try_acquire_beacon_lock() -> bool:
+ """Try to acquire the beacon file lock (non-blocking).
+
+ Returns True if this process is the beacon owner.
+ """
+ if not HAS_FCNTL:
+ return True
+
+ fd = None
+ try:
+ _beacon_lock_path.parent.mkdir(parents=True, exist_ok=True)
+ fd = open(_beacon_lock_path, "w") # noqa: SIM115
+ fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
+ fd.write(str(os.getpid()))
+ fd.flush()
+ _beacon_lock_fd[0] = fd
+ return True
+ except OSError:
+ if fd is not None:
+ fd.close()
+ return False
+
+ def _release_beacon_lock() -> None:
+ """Release the beacon file lock."""
+ fd = _beacon_lock_fd[0]
+ if fd:
+ try:
+ if HAS_FCNTL:
+ fcntl.flock(fd, fcntl.LOCK_UN)
+ fd.close()
+ except Exception:
+ pass
+ _beacon_lock_fd[0] = None
+ try:
+ _beacon_lock_path.unlink(missing_ok=True)
+ except Exception:
+ pass
+
+ @asynccontextmanager
+ async def lifespan(app: FastAPI): # type: ignore[no-untyped-def]
+ configure_otel_metrics(OTelMetricsConfig.from_env(default_service_name="headroom-proxy"))
+ configure_langfuse_tracing(
+ LangfuseTracingConfig.from_env(default_service_name="headroom-proxy")
+ )
+
+ app.state.started_at = time.time()
+ app.state.ready = False
+ app.state.startup_error = None
+
+ try:
+ try:
+ # Startup
+ await proxy.startup()
+ asyncio.create_task(_log_toin_stats_periodically())
+ if proxy.usage_reporter:
+ await proxy.usage_reporter.start(proxy)
+ if proxy.traffic_learner:
+ await proxy.traffic_learner.start()
+
+ # Only start beacon if we acquire the lock (first worker wins)
+ _beacon_is_owner[0] = _try_acquire_beacon_lock()
+ if _beacon_is_owner[0]:
+ await _beacon.start()
+ else:
+ logger.debug("Beacon: skipping (another worker owns the lock)")
+
+ app.state.ready = True
+ yield
+ except Exception as exc:
+ app.state.startup_error = str(exc)
+ raise
+ finally:
+ app.state.ready = False
+ # Shutdown
+ if _beacon_is_owner[0]:
+ await _beacon.stop()
+ _release_beacon_lock()
+ if proxy.usage_reporter:
+ await proxy.usage_reporter.stop()
+ if proxy.traffic_learner:
+ await proxy.traffic_learner.stop()
+ if proxy.code_graph_watcher:
+ proxy.code_graph_watcher.stop()
+ await proxy.shutdown()
+ shutdown_headroom_tracing()
+ shutdown_otel_metrics()
+
+ app = FastAPI(
+ title="Headroom Proxy",
+ description="Production-ready LLM optimization proxy",
+ version=__version__,
+ lifespan=lifespan,
+ )
+ app.state.proxy = proxy
+ app.state.started_at = None
+ app.state.ready = False
+ app.state.startup_error = None
+
+ def _iso_utc_now() -> str:
+ return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
+
+ def _uptime_seconds() -> float:
+ started_at = getattr(app.state, "started_at", None)
+ if not isinstance(started_at, int | float):
+ return 0.0
+ return round(max(0.0, time.time() - float(started_at)), 3)
+
+ def _component_health(
+ *,
+ enabled: bool,
+ ready: bool,
+ **details: Any,
+ ) -> dict[str, Any]:
+ status = "disabled" if not enabled else ("healthy" if ready else "unhealthy")
+ return {
+ "enabled": enabled,
+ "ready": (ready if enabled else True),
+ "status": status,
+ **details,
+ }
+
+ def _health_checks() -> dict[str, dict[str, Any]]:
+ memory_status = (
+ proxy.memory_handler.health_status()
+ if proxy.memory_handler
+ else {
+ "enabled": False,
+ "backend": None,
+ "initialized": False,
+ "native_tool": False,
+ "bridge_enabled": False,
+ }
+ )
+ memory_enabled = bool(memory_status.get("enabled", False))
+ memory_initialized = bool(memory_status.get("initialized", False))
+ return {
+ "startup": _component_health(
+ enabled=True,
+ ready=bool(getattr(app.state, "ready", False)),
+ error=getattr(app.state, "startup_error", None),
+ ),
+ "http_client": _component_health(
+ enabled=True,
+ ready=proxy.http_client is not None,
+ ),
+ "cache": _component_health(
+ enabled=config.cache_enabled,
+ ready=(proxy.cache is not None),
+ ),
+ "rate_limiter": _component_health(
+ enabled=config.rate_limit_enabled,
+ ready=(proxy.rate_limiter is not None),
+ ),
+ "memory": _component_health(
+ enabled=memory_enabled,
+ ready=memory_initialized,
+ backend=memory_status["backend"],
+ initialized=memory_initialized,
+ native_tool=bool(memory_status.get("native_tool", False)),
+ bridge_enabled=bool(memory_status.get("bridge_enabled", False)),
+ ),
+ }
+
+ def _runtime_payload() -> dict[str, Any]:
+ ws_registry = getattr(proxy, "ws_sessions", None)
+ ws_active_sessions = ws_registry.active_count() if ws_registry is not None else 0
+ ws_active_relay_tasks = (
+ ws_registry.active_relay_task_count() if ws_registry is not None else 0
+ )
+ return {
+ "anthropic_pre_upstream": {
+ "enabled": proxy.anthropic_pre_upstream_sem is not None,
+ "resolved_concurrency": proxy.anthropic_pre_upstream_concurrency,
+ "source": (
+ "auto" if config.anthropic_pre_upstream_concurrency is None else "explicit"
+ ),
+ "acquire_timeout_seconds": proxy.anthropic_pre_upstream_acquire_timeout_seconds,
+ "compression_timeout_seconds": float(COMPRESSION_TIMEOUT_SECONDS),
+ "memory_context_timeout_seconds": (
+ proxy.anthropic_pre_upstream_memory_context_timeout_seconds
+ ),
+ "codex_ws_gated": False,
+ },
+ "websocket_sessions": {
+ "active_sessions": ws_active_sessions,
+ "active_relay_tasks": ws_active_relay_tasks,
+ },
+ }
+
+ def _health_payload(*, include_config: bool) -> dict[str, Any]:
+ checks = _health_checks()
+ ready = all(check["ready"] for check in checks.values())
+ payload: dict[str, Any] = {
+ "service": "headroom-proxy",
+ "status": "healthy" if ready else "unhealthy",
+ "ready": ready,
+ "version": __version__,
+ "timestamp": _iso_utc_now(),
+ "uptime_seconds": _uptime_seconds(),
+ "checks": checks,
+ "runtime": _runtime_payload(),
+ }
+ deployment_profile = os.environ.get("HEADROOM_DEPLOYMENT_PROFILE")
+ if deployment_profile:
+ payload["deployment"] = {
+ "profile": deployment_profile,
+ "preset": os.environ.get("HEADROOM_DEPLOYMENT_PRESET"),
+ "runtime": os.environ.get("HEADROOM_DEPLOYMENT_RUNTIME"),
+ "supervisor": os.environ.get("HEADROOM_DEPLOYMENT_SUPERVISOR"),
+ "scope": os.environ.get("HEADROOM_DEPLOYMENT_SCOPE"),
+ }
+ if include_config:
+ payload["config"] = {
+ "backend": config.backend,
+ "optimize": config.optimize,
+ "cache": config.cache_enabled,
+ "rate_limit": config.rate_limit_enabled,
+ "memory": config.memory_enabled,
+ "learn": config.traffic_learning_enabled,
+ "code_graph": config.code_graph_watcher,
+ "pid": os.getpid(),
+ }
+ return payload
+
+ # CORS
+ app.add_middleware(
+ CORSMiddleware,
+ allow_origins=["*"],
+ allow_credentials=True,
+ allow_methods=["*"],
+ allow_headers=["*"],
+ )
+
+ # X-Headroom-Stack: SDK adapters (TS openai/anthropic/etc.) tag their
+ # requests so telemetry can segment by integration surface. Registered
+ # before extension middleware so any extension-level auth/guards run
+ # outermost and we don't count requests they reject.
+ @app.middleware("http")
+ async def _record_headroom_stack(request, call_next):
+ if request.url.path.startswith("/v1/"):
+ stack = request.headers.get("x-headroom-stack")
+ if stack:
+ try:
+ proxy.metrics.record_stack(stack)
+ except Exception:
+ logger.debug("record_stack failed", exc_info=True)
+ return await call_next(request)
+
+ # Third-party proxy extensions (Enterprise, custom plugins). Discovered via
+ # the `headroom.proxy_extension` entry-point group. An extension that raises
+ # from its install() is a deliberate fail-closed signal and aborts startup.
+ from headroom.proxy.extensions import install_all as _install_extensions
+
+ _install_extensions(app, config)
+
+ # Health & Metrics
+ @app.get("/livez")
+ async def livez():
+ return JSONResponse(
+ status_code=200,
+ content={
+ "service": "headroom-proxy",
+ "status": "healthy",
+ "alive": True,
+ "version": __version__,
+ "timestamp": _iso_utc_now(),
+ "uptime_seconds": _uptime_seconds(),
+ },
+ )
+
+ @app.get("/readyz")
+ async def readyz():
+ payload = _health_payload(include_config=False)
+ return JSONResponse(status_code=200 if payload["ready"] else 503, content=payload)
+
+ @app.get("/health")
+ async def health():
+ payload = _health_payload(include_config=True)
+ return JSONResponse(status_code=200, content=payload)
+
+ # Loopback-only debug introspection (Unit 5). A remote IP gets 404 —
+ # debug endpoints are invisible to external scanners.
+ from headroom.proxy.debug_introspection import (
+ collect_tasks as _collect_tasks,
+ )
+ from headroom.proxy.loopback_guard import require_loopback as _require_loopback
+
+ @app.get("/debug/tasks", dependencies=[Depends(_require_loopback)])
+ async def debug_tasks(stack: bool = False):
+ """Enumerate running asyncio tasks.
+
+ Default is cheap — ``stack_depth`` is ``null`` in every entry so
+ a storm snapshot does not walk 50+ coroutine frames synchronously.
+ Pass ``?stack=true`` to compute ``stack_depth`` for each task
+ (useful for single-shot human debugging).
+ """
+ ws_registry = getattr(proxy, "ws_sessions", None)
+ return JSONResponse(
+ status_code=200,
+ content=_collect_tasks(ws_registry, with_stack_depth=stack),
+ )
+
+ @app.get("/debug/ws-sessions", dependencies=[Depends(_require_loopback)])
+ async def debug_ws_sessions():
+ ws_registry = getattr(proxy, "ws_sessions", None)
+ snapshot = ws_registry.snapshot() if ws_registry is not None else []
+ return JSONResponse(status_code=200, content=snapshot)
+
+ @app.get("/debug/warmup", dependencies=[Depends(_require_loopback)])
+ async def debug_warmup():
+ warmup_registry = getattr(proxy, "warmup", None)
+ payload = warmup_registry.to_dict() if warmup_registry is not None else {}
+ payload["runtime"] = _runtime_payload()
+ return JSONResponse(status_code=200, content=payload)
+
+ @app.get("/dashboard", response_class=HTMLResponse)
+ async def dashboard():
+ """Serve the Headroom dashboard UI."""
+ return get_dashboard_html()
+
+ @app.get("/stats")
+ async def stats():
+ """Get comprehensive proxy statistics.
+
+ This is the main stats endpoint - it aggregates data from all subsystems:
+ - Request metrics (total, cached, failed, by model/provider)
+ - Token usage and savings
+ - Cost tracking
+ - Canonical persisted display_session metrics for downstream dashboards
+ - Compression (CCR) statistics
+ - Telemetry/TOIN (data flywheel) statistics
+ - Cache and rate limiter stats
+ """
+ m = proxy.metrics
+
+ # Calculate average latency
+ avg_latency_ms = round(m.latency_sum_ms / m.latency_count, 2) if m.latency_count > 0 else 0
+ min_latency_ms = (
+ round(m.latency_min_ms, 2)
+ if m.latency_count > 0 and m.latency_min_ms != float("inf")
+ else 0
+ )
+ max_latency_ms = round(m.latency_max_ms, 2) if m.latency_count > 0 else 0
+
+ # Calculate Headroom overhead (optimization time only, excludes pass-through requests)
+ avg_overhead_ms = (
+ round(m.overhead_sum_ms / m.overhead_count, 2) if m.overhead_count > 0 else 0
+ )
+ min_overhead_ms = (
+ round(m.overhead_min_ms, 2)
+ if m.overhead_count > 0 and m.overhead_min_ms != float("inf")
+ else 0
+ )
+ max_overhead_ms = round(m.overhead_max_ms, 2) if m.overhead_count > 0 else 0
+
+ # Calculate TTFB (time to first byte)
+ avg_ttfb_ms = round(m.ttfb_sum_ms / m.ttfb_count, 2) if m.ttfb_count > 0 else 0
+ min_ttfb_ms = (
+ round(m.ttfb_min_ms, 2) if m.ttfb_count > 0 and m.ttfb_min_ms != float("inf") else 0
+ )
+ max_ttfb_ms = round(m.ttfb_max_ms, 2) if m.ttfb_count > 0 else 0
+
+ # Get compression store stats
+ store = get_compression_store()
+ compression_stats = store.get_stats()
+
+ # Get telemetry/TOIN stats
+ telemetry = get_telemetry_collector()
+ telemetry_stats = telemetry.get_stats()
+
+ # Get feedback loop stats
+ feedback = get_compression_feedback()
+ feedback_stats = feedback.get_stats()
+
+ # Build prefix cache stats once (used in both prefix_cache and cost)
+ prefix_cache_stats = _build_prefix_cache_stats(m, proxy.cost_tracker)
+
+ # Fetch CLI filtering savings (rtk — tokens avoided before reaching context)
+ cli_filtering_stats = _get_rtk_stats()
+ cli_tokens_avoided = (
+ cli_filtering_stats.get("tokens_saved", 0) if cli_filtering_stats else 0
+ )
+
+ # Calculate total tokens before compression
+ total_tokens_before = m.tokens_input_total + m.tokens_saved_total
+
+ # Build human-readable summary
+ summary = _build_session_summary(
+ proxy, m, prefix_cache_stats, cli_tokens_avoided, total_tokens_before
+ )
+ # DEBUG: log the summary payload for external upsert consumers
+ try:
+ logger.debug("/stats summary data: %r", summary)
+ except Exception:
+ logger.warning("Failed to log /stats summary payload")
+
+ # Compression cache stats (token mode)
+ compression_cache_stats: dict = {}
+ if proxy.config.mode == PROXY_MODE_TOKEN and proxy._compression_caches:
+ total_entries = 0
+ total_hits = 0
+ total_misses = 0
+ total_tokens_saved = 0
+ for cache in proxy._compression_caches.values():
+ s = cache.get_stats()
+ total_entries += s.get("entries", 0)
+ total_hits += s.get("hits", 0)
+ total_misses += s.get("misses", 0)
+ total_tokens_saved += s.get("total_tokens_saved", 0)
+ compression_cache_stats = {
+ "mode": PROXY_MODE_TOKEN,
+ "active_sessions": len(proxy._compression_caches),
+ "total_entries": total_entries,
+ "total_hits": total_hits,
+ "total_misses": total_misses,
+ "hit_rate": round(total_hits / max(1, total_hits + total_misses) * 100, 1),
+ "total_tokens_saved": total_tokens_saved,
+ }
+ else:
+ compression_cache_stats = {"mode": proxy.config.mode}
+
+ # Build unified savings summary (all layers)
+ compression_tokens = m.tokens_saved_total
+ cache_net_usd = prefix_cache_stats.get("totals", {}).get("net_savings_usd", 0.0)
+ total_tokens_all_layers = compression_tokens + cli_tokens_avoided
+ persistent_savings = m.savings_tracker.stats_preview()
+ display_session = persistent_savings.get("display_session", {})
+
+ return {
+ "summary": summary,
+ "savings": {
+ "total_tokens": total_tokens_all_layers,
+ "by_layer": {
+ "cli_filtering": {
+ "tokens": cli_tokens_avoided,
+ "description": "Tokens avoided by CLI output filtering (rtk) before reaching context",
+ },
+ "compression": {
+ "tokens": compression_tokens,
+ "description": "Tokens removed by proxy compression (SmartCrusher, ContentRouter, etc.)",
+ },
+ "prefix_cache": {
+ "discount_usd": round(cache_net_usd, 4),
+ "description": (
+ "Cost discount from provider prefix caching. "
+ "Headroom's CacheAligner improves hit rates; "
+ "baseline caching is provider-native."
+ ),
+ },
+ },
+ },
+ "requests": {
+ "total": m.requests_total,
+ "cached": m.requests_cached,
+ "rate_limited": m.requests_rate_limited,
+ "failed": m.requests_failed,
+ "by_provider": dict(m.requests_by_provider),
+ "by_model": dict(m.requests_by_model),
+ "by_stack": dict(m.requests_by_stack),
+ },
+ "tokens": {
+ "input": m.tokens_input_total,
+ "output": m.tokens_output_total,
+ "saved": m.tokens_saved_total,
+ "cli_tokens_avoided": cli_tokens_avoided,
+ "total_before_compression": total_tokens_before,
+ "savings_percent": round(
+ (m.tokens_saved_total / total_tokens_before * 100)
+ if total_tokens_before > 0
+ else 0,
+ 2,
+ ),
+ },
+ "latency": {
+ "average_ms": avg_latency_ms,
+ "min_ms": min_latency_ms,
+ "max_ms": max_latency_ms,
+ "total_requests": m.latency_count,
+ },
+ "overhead": {
+ "average_ms": avg_overhead_ms,
+ "min_ms": min_overhead_ms,
+ "max_ms": max_overhead_ms,
+ },
+ "ttfb": {
+ "average_ms": avg_ttfb_ms,
+ "min_ms": min_ttfb_ms,
+ "max_ms": max_ttfb_ms,
+ },
+ "pipeline_timing": {
+ name: {
+ "average_ms": round(
+ m.transform_timing_sum[name] / m.transform_timing_count[name], 2
+ ),
+ "max_ms": round(m.transform_timing_max[name], 2),
+ "count": m.transform_timing_count[name],
+ }
+ for name in sorted(m.transform_timing_sum.keys())
+ }
+ if m.transform_timing_sum
+ else {},
+ "waste_signals": dict(m.waste_signals_total) if m.waste_signals_total else {},
+ "savings_history": m.savings_history[-100:], # Last 100 data points
+ "display_session": display_session,
+ "persistent_savings": persistent_savings,
+ "prefix_cache": prefix_cache_stats,
+ "cost": _merge_cost_stats(
+ proxy.cost_tracker.stats() if proxy.cost_tracker else None,
+ prefix_cache_stats,
+ cli_tokens_avoided=cli_tokens_avoided,
+ ),
+ "compression": {
+ "ccr_entries": compression_stats.get("entry_count", 0),
+ "ccr_max_entries": compression_stats.get("max_entries", 0),
+ "original_tokens_cached": compression_stats.get("total_original_tokens", 0),
+ "compressed_tokens_cached": compression_stats.get("total_compressed_tokens", 0),
+ "ccr_retrievals": compression_stats.get("total_retrievals", 0),
+ },
+ "compression_cache": compression_cache_stats,
+ "anon_telemetry_shipping": is_telemetry_enabled(),
+ "telemetry": {
+ "enabled": telemetry_stats.get("enabled", False),
+ "total_compressions": telemetry_stats.get("total_compressions", 0),
+ "total_retrievals": telemetry_stats.get("total_retrievals", 0),
+ "global_retrieval_rate": round(telemetry_stats.get("global_retrieval_rate", 0), 4),
+ "tool_signatures_tracked": telemetry_stats.get("tool_signatures_tracked", 0),
+ "avg_compression_ratio": round(telemetry_stats.get("avg_compression_ratio", 0), 4),
+ "avg_token_reduction": round(telemetry_stats.get("avg_token_reduction", 0), 4),
+ },
+ "otel": get_otel_metrics_status(),
+ "langfuse": get_langfuse_tracing_status(),
+ "feedback_loop": {
+ "tools_tracked": feedback_stats.get("tools_tracked", 0),
+ "total_compressions": feedback_stats.get("total_compressions", 0),
+ "total_retrievals": feedback_stats.get("total_retrievals", 0),
+ "global_retrieval_rate": round(feedback_stats.get("global_retrieval_rate", 0), 4),
+ "tools_with_high_retrieval": sum(
+ 1
+ for p in feedback_stats.get("tool_patterns", {}).values()
+ if p.get("retrieval_rate", 0) > 0.3
+ ),
+ },
+ "toin": get_toin().get_stats(),
+ "cli_filtering": cli_filtering_stats,
+ "cache": await proxy.cache.stats() if proxy.cache else None,
+ "rate_limiter": await proxy.rate_limiter.stats() if proxy.rate_limiter else None,
+ "recent_requests": proxy.logger.get_recent(10) if proxy.logger else [],
+ "log_full_messages": proxy.config.log_full_messages if proxy else False,
+ **get_quota_registry().get_all_stats(),
+ }
+
+ @app.get("/stats-history")
+ async def stats_history(
+ format: Literal["json", "csv"] = "json",
+ series: Literal["history", "hourly", "daily", "weekly", "monthly"] = "history",
+ ):
+ """Get durable proxy compression history plus display-session state."""
+ if format == "csv":
+ filename = f"headroom-stats-history-{series}.csv"
+ return Response(
+ content=proxy.metrics.savings_tracker.export_csv(series=series),
+ media_type="text/csv; charset=utf-8",
+ headers={"Content-Disposition": f'attachment; filename="{filename}"'},
+ )
+
+ return proxy.metrics.savings_tracker.history_response()
+
+ @app.get("/transformations/feed")
+ async def transformations_feed(limit: int = 20):
+ """Get recent message transformations for the live feed.
+
+ Returns empty list if log_full_messages is disabled (messages are not stored).
+ """
+ if limit > 100:
+ limit = 100
+
+ transformations = []
+ log_full_messages = proxy.config.log_full_messages if proxy else False
+
+ if proxy and proxy.logger:
+ logs = proxy.logger.get_recent_with_messages(limit)
+ for log in logs:
+ transformations.append(
+ {
+ "request_id": log.get("request_id"),
+ "timestamp": log.get("timestamp"),
+ "provider": log.get("provider"),
+ "model": log.get("model"),
+ "input_tokens_original": log.get("input_tokens_original"),
+ "input_tokens_optimized": log.get("input_tokens_optimized"),
+ "tokens_saved": log.get("tokens_saved"),
+ "savings_percent": log.get("savings_percent"),
+ "transforms_applied": log.get("transforms_applied", []),
+ "request_messages": log.get("request_messages"),
+ "response_content": log.get("response_content"),
+ }
+ )
+
+ return {"transformations": transformations, "log_full_messages": log_full_messages}
+
+ @app.get("/subscription-window")
+ async def subscription_window():
+ """Current Anthropic subscription window utilisation and Headroom contribution."""
+ tracker = get_subscription_tracker()
+ if tracker is None:
+ return JSONResponse(
+ status_code=503,
+ content={"error": "Subscription tracking is not enabled"},
+ )
+ return JSONResponse(content=tracker.state)
+
+ @app.get("/quota")
+ async def quota():
+ """Unified quota/rate-limit stats for all registered providers (Anthropic, Codex, Copilot)."""
+ return JSONResponse(content=get_quota_registry().get_all_stats())
+
+ @app.get("/metrics")
+ async def metrics():
+ """Prometheus metrics endpoint."""
+ return PlainTextResponse(
+ await proxy.metrics.export(),
+ media_type="text/plain; version=0.0.4",
+ )
+
+ # Debug endpoints
+ @app.get("/debug/memory")
+ async def debug_memory():
+ """Get detailed memory usage statistics.
+
+ Returns memory usage for all tracked components including:
+ - Process-level memory (RSS, VMS, percent)
+ - Per-component memory usage and budgets
+ - Cache hit/miss statistics
+ - Total tracked vs target budget
+
+ This endpoint is useful for debugging memory issues and
+ monitoring memory budgets.
+ """
+ from ..memory.tracker import MemoryTracker
+
+ tracker = MemoryTracker.get()
+
+ # Register components if not already registered
+ _register_memory_components(proxy, tracker)
+
+ report = tracker.get_report()
+ return report.to_dict()
+
+ @app.post("/cache/clear")
+ async def clear_cache():
+ """Clear the response cache."""
+ if proxy.cache:
+ await proxy.cache.clear()
+ return {"status": "cleared"}
+ return {"status": "cache disabled"}
+
+ # CCR (Compress-Cache-Retrieve) endpoints
+ @app.post("/v1/retrieve")
+ async def ccr_retrieve(request: Request):
+ """Retrieve original content from CCR compression cache.
+
+ This is the "Retrieve" part of CCR (Compress-Cache-Retrieve).
+ When SmartCrusher compresses tool outputs, the original data is cached.
+ LLMs can call this endpoint to get more data if needed.
+
+ Request body:
+ hash (str): Hash key from compression marker (required)
+ query (str): Optional search query to filter results
+
+ Response:
+ Full retrieval: {"hash": "...", "original_content": "...", ...}
+ Search: {"hash": "...", "query": "...", "results": [...], "count": N}
+ """
+ data = await request.json()
+ hash_key = data.get("hash")
+ query = data.get("query")
+
+ if not hash_key:
+ raise HTTPException(status_code=400, detail="hash required")
+
+ store = get_compression_store()
+
+ if query:
+ # Search within cached content
+ results = store.search(hash_key, query)
+ return {
+ "hash": hash_key,
+ "query": query,
+ "results": results,
+ "count": len(results),
+ }
+ else:
+ # Return full original content
+ entry = store.retrieve(hash_key)
+ if entry:
+ return {
+ "hash": hash_key,
+ "original_content": entry.original_content,
+ "original_tokens": entry.original_tokens,
+ "original_item_count": entry.original_item_count,
+ "compressed_item_count": entry.compressed_item_count,
+ "tool_name": entry.tool_name,
+ "retrieval_count": entry.retrieval_count,
+ }
+ raise HTTPException(
+ status_code=404, detail="Entry not found or expired (TTL: 5 minutes)"
+ )
+
+ @app.get("/v1/retrieve/stats")
+ async def ccr_stats():
+ """Get CCR compression store statistics."""
+ store = get_compression_store()
+ stats = store.get_stats()
+ events = store.get_retrieval_events(limit=20)
+ return {
+ "store": stats,
+ "recent_retrievals": [
+ {
+ "hash": e.hash,
+ "query": e.query,
+ "items_retrieved": e.items_retrieved,
+ "total_items": e.total_items,
+ "tool_name": e.tool_name,
+ "retrieval_type": e.retrieval_type,
+ }
+ for e in events
+ ],
+ }
+
+ @app.get("/v1/feedback")
+ async def ccr_feedback():
+ """Get CCR feedback loop statistics and learned patterns.
+
+ This endpoint exposes the feedback loop's learned patterns for monitoring
+ and debugging. It shows:
+ - Per-tool retrieval rates (high = compress less aggressively)
+ - Common search queries per tool
+ - Queried fields (suggest what to preserve)
+
+ Use this to understand how well compression is working and whether
+ the feedback loop is adjusting appropriately.
+ """
+ feedback = get_compression_feedback()
+ stats = feedback.get_stats()
+ return {
+ "feedback": stats,
+ "hints_example": {
+ tool_name: {
+ "hints": {
+ "max_items": hints.max_items
+ if (hints := feedback.get_compression_hints(tool_name))
+ else 15,
+ "suggested_items": hints.suggested_items if hints else None,
+ "skip_compression": hints.skip_compression if hints else False,
+ "preserve_fields": hints.preserve_fields if hints else [],
+ "reason": hints.reason if hints else "",
+ }
+ }
+ for tool_name in list(stats.get("tool_patterns", {}).keys())[:5]
+ },
+ }
+
+ @app.get("/v1/feedback/{tool_name}")
+ async def ccr_feedback_for_tool(tool_name: str):
+ """Get compression hints for a specific tool.
+
+ Returns feedback-based hints that would be used for compressing
+ this tool's output.
+ """
+ feedback = get_compression_feedback()
+ hints = feedback.get_compression_hints(tool_name)
+ patterns = feedback.get_all_patterns().get(tool_name)
+
+ return {
+ "tool_name": tool_name,
+ "hints": {
+ "max_items": hints.max_items,
+ "min_items": hints.min_items,
+ "suggested_items": hints.suggested_items,
+ "aggressiveness": hints.aggressiveness,
+ "skip_compression": hints.skip_compression,
+ "preserve_fields": hints.preserve_fields,
+ "reason": hints.reason,
+ },
+ "pattern": {
+ "total_compressions": patterns.total_compressions if patterns else 0,
+ "total_retrievals": patterns.total_retrievals if patterns else 0,
+ "retrieval_rate": patterns.retrieval_rate if patterns else 0.0,
+ "full_retrieval_rate": patterns.full_retrieval_rate if patterns else 0.0,
+ "search_rate": patterns.search_rate if patterns else 0.0,
+ "common_queries": list(patterns.common_queries.keys())[:10] if patterns else [],
+ "queried_fields": list(patterns.queried_fields.keys())[:10] if patterns else [],
+ }
+ if patterns
+ else None,
+ }
+
+ # Telemetry endpoints (Data Flywheel)
+ @app.get("/v1/telemetry")
+ async def telemetry_stats():
+ """Get telemetry statistics for the data flywheel.
+
+ This endpoint exposes privacy-preserving telemetry data that powers
+ the data flywheel - learning optimal compression strategies across
+ tool types based on usage patterns.
+
+ What's collected (anonymized):
+ - Tool output structure patterns (field types, not values)
+ - Compression decisions and ratios
+ - Retrieval patterns (rate, type, not content)
+ - Strategy effectiveness
+
+ What's NOT collected:
+ - Actual data values
+ - User identifiers
+ - Queries or search terms
+ - File paths or tool names (hashed by default)
+ """
+ telemetry = get_telemetry_collector()
+ return telemetry.get_stats()
+
+ @app.get("/v1/telemetry/export")
+ async def telemetry_export():
+ """Export full telemetry data for aggregation.
+
+ This endpoint exports all telemetry data in a format suitable for
+ cross-user aggregation. The data is privacy-preserving - no actual
+ values are included, only structural patterns and statistics.
+
+ Use this for:
+ - Building a central learning service
+ - Sharing learned patterns across instances
+ - Analysis and debugging
+ """
+ telemetry = get_telemetry_collector()
+ return telemetry.export_stats()
+
+ @app.post("/v1/telemetry/import")
+ async def telemetry_import(request: Request):
+ """Import telemetry data from another source.
+
+ This allows merging telemetry from multiple sources for cross-user
+ learning. The imported data is merged with existing statistics.
+
+ Request body: Telemetry export data from /v1/telemetry/export
+ """
+ telemetry = get_telemetry_collector()
+ data = await request.json()
+ telemetry.import_stats(data)
+ return {"status": "imported", "current_stats": telemetry.get_stats()}
+
+ @app.get("/v1/telemetry/tools")
+ async def telemetry_tools():
+ """Get telemetry statistics for all tracked tool signatures.
+
+ Returns statistics per tool signature (anonymized), including:
+ - Compression ratios and strategy usage
+ - Retrieval rates (high = compression too aggressive)
+ - Learned recommendations
+ """
+ telemetry = get_telemetry_collector()
+ all_stats = telemetry.get_all_tool_stats()
+ return {
+ "tool_count": len(all_stats),
+ "tools": {sig_hash: stats.to_dict() for sig_hash, stats in all_stats.items()},
+ }
+
+ @app.get("/v1/telemetry/tools/{signature_hash}")
+ async def telemetry_tool_detail(signature_hash: str):
+ """Get detailed telemetry for a specific tool signature.
+
+ Includes learned recommendations if enough data has been collected.
+ """
+ telemetry = get_telemetry_collector()
+ stats = telemetry.get_tool_stats(signature_hash)
+ recommendations = telemetry.get_recommendations(signature_hash)
+
+ if stats is None:
+ raise HTTPException(
+ status_code=404, detail=f"No telemetry found for signature: {signature_hash}"
+ )
+
+ return {
+ "signature_hash": signature_hash,
+ "stats": stats.to_dict(),
+ "recommendations": recommendations,
+ }
+
+ # TOIN (Tool Output Intelligence Network) endpoints
+ @app.get("/v1/toin/stats")
+ async def toin_stats():
+ """Get overall TOIN statistics.
+
+ Returns aggregated statistics from the Tool Output Intelligence Network,
+ which learns optimal compression strategies across all tool types.
+
+ Response includes:
+ - enabled: Whether TOIN is enabled
+ - patterns_tracked: Number of unique tool patterns being tracked
+ - total_compressions: Total compression events recorded
+ - total_retrievals: Total retrieval events recorded
+ - global_retrieval_rate: Overall retrieval rate (high = compression too aggressive)
+ - patterns_with_recommendations: Patterns with enough data for recommendations
+ """
+ toin = get_toin()
+ return toin.get_stats()
+
+ @app.get("/v1/toin/patterns")
+ async def toin_patterns(limit: int = 20):
+ """List TOIN patterns with most samples.
+
+ Returns patterns sorted by sample_size descending. Use this to see
+ which tool types have the most data and their learned behaviors.
+
+ Query params:
+ limit: Maximum number of patterns to return (default 20)
+
+ Response includes for each pattern:
+ - hash: Truncated tool signature hash (12 chars)
+ - compressions: Total compression events
+ - retrievals: Total retrieval events
+ - retrieval_rate: Percentage of compressions that triggered retrieval
+ - confidence: Confidence level in recommendations (0.0-1.0)
+ - skip_recommended: Whether TOIN recommends skipping compression
+ - optimal_max_items: Learned optimal max_items setting
+ """
+ toin = get_toin()
+ exported = toin.export_patterns()
+ patterns_data = exported.get("patterns", {})
+
+ # Convert to list and sort by sample_size
+ patterns_list = []
+ for sig_hash, pattern_dict in patterns_data.items():
+ sample_size = pattern_dict.get("sample_size", 0)
+ total_compressions = pattern_dict.get("total_compressions", 0)
+ total_retrievals = pattern_dict.get("total_retrievals", 0)
+ retrieval_rate = (
+ total_retrievals / total_compressions if total_compressions > 0 else 0.0
+ )
+
+ patterns_list.append(
+ {
+ "hash": sig_hash[:12],
+ "compressions": total_compressions,
+ "retrievals": total_retrievals,
+ "retrieval_rate": f"{retrieval_rate:.1%}",
+ "confidence": round(pattern_dict.get("confidence", 0.0), 3),
+ "skip_recommended": pattern_dict.get("skip_compression_recommended", False),
+ "optimal_max_items": pattern_dict.get("optimal_max_items", 20),
+ "sample_size": sample_size,
+ }
+ )
+
+ # Sort by sample_size descending
+ patterns_list.sort(key=lambda p: p["sample_size"], reverse=True)
+
+ # Remove sample_size from output (used only for sorting)
+ for p in patterns_list:
+ del p["sample_size"]
+
+ return patterns_list[:limit]
+
+ @app.get("/v1/toin/pattern/{hash_prefix}")
+ async def toin_pattern_detail(hash_prefix: str):
+ """Get detailed TOIN pattern info by hash prefix.
+
+ Searches for a pattern where the tool signature hash starts with
+ the provided prefix. Returns full pattern details if found.
+
+ Path params:
+ hash_prefix: Beginning of the tool signature hash (min 4 chars recommended)
+
+ Response: Full pattern.to_dict() with all learned statistics and recommendations.
+ """
+ toin = get_toin()
+ exported = toin.export_patterns()
+ patterns_data = exported.get("patterns", {})
+
+ # Search for pattern with matching hash prefix
+ for sig_hash, pattern_dict in patterns_data.items():
+ if sig_hash.startswith(hash_prefix):
+ return pattern_dict
+
+ raise HTTPException(
+ status_code=404, detail=f"No TOIN pattern found with hash starting with: {hash_prefix}"
+ )
+
+ @app.get("/v1/retrieve/{hash_key}")
+ async def ccr_retrieve_get(hash_key: str, query: str | None = None):
+ """GET version of CCR retrieve for easier testing."""
+ store = get_compression_store()
+
+ if query:
+ results = store.search(hash_key, query)
+ return {
+ "hash": hash_key,
+ "query": query,
+ "results": results,
+ "count": len(results),
+ }
+ else:
+ entry = store.retrieve(hash_key)
+ if entry:
+ return {
+ "hash": hash_key,
+ "original_content": entry.original_content,
+ "original_tokens": entry.original_tokens,
+ "original_item_count": entry.original_item_count,
+ "compressed_item_count": entry.compressed_item_count,
+ "tool_name": entry.tool_name,
+ "retrieval_count": entry.retrieval_count,
+ }
+ raise HTTPException(status_code=404, detail="Entry not found or expired")
+
+ # CCR Tool Call Handler - for agent frameworks to call when LLM uses headroom_retrieve
+ @app.post("/v1/retrieve/tool_call")
+ async def ccr_handle_tool_call(request: Request):
+ """Handle a CCR tool call from an LLM response.
+
+ This endpoint accepts tool call formats from various providers and returns
+ a properly formatted tool result. Agent frameworks can use this to handle
+ CCR tool calls without implementing the retrieval logic themselves.
+
+ Request body (Anthropic format):
+ {
+ "tool_call": {
+ "id": "toolu_123",
+ "name": "headroom_retrieve",
+ "input": {"hash": "abc123", "query": "optional search"}
+ },
+ "provider": "anthropic"
+ }
+
+ Request body (OpenAI format):
+ {
+ "tool_call": {
+ "id": "call_123",
+ "function": {
+ "name": "headroom_retrieve",
+ "arguments": "{\"hash\": \"abc123\"}"
+ }
+ },
+ "provider": "openai"
+ }
+
+ Response:
+ {
+ "tool_result": {...}, # Formatted for the provider
+ "success": true,
+ "data": {...} # Raw retrieval data
+ }
+ """
+ data = await request.json()
+ tool_call = data.get("tool_call", {})
+ provider = data.get("provider", "anthropic")
+
+ # Parse the tool call
+ hash_key, query = parse_tool_call(tool_call, provider)
+
+ if hash_key is None:
+ raise HTTPException(
+ status_code=400, detail=f"Invalid tool call or not a {CCR_TOOL_NAME} call"
+ )
+
+ # Perform retrieval
+ store = get_compression_store()
+
+ if query:
+ results = store.search(hash_key, query)
+ retrieval_data = {
+ "hash": hash_key,
+ "query": query,
+ "results": results,
+ "count": len(results),
+ }
+ else:
+ entry = store.retrieve(hash_key)
+ if entry:
+ retrieval_data = {
+ "hash": hash_key,
+ "original_content": entry.original_content,
+ "original_item_count": entry.original_item_count,
+ "compressed_item_count": entry.compressed_item_count,
+ }
+ else:
+ retrieval_data = {
+ "error": "Entry not found or expired (TTL: 5 minutes)",
+ "hash": hash_key,
+ }
+
+ # Format tool result for provider
+ tool_call_id = tool_call.get("id", "")
+ result_content = json.dumps(retrieval_data, indent=2)
+
+ if provider == "anthropic":
+ tool_result = {
+ "type": "tool_result",
+ "tool_use_id": tool_call_id,
+ "content": result_content,
+ }
+ elif provider == "openai":
+ tool_result = {
+ "role": "tool",
+ "tool_call_id": tool_call_id,
+ "content": result_content,
+ }
+ else:
+ tool_result = {
+ "tool_call_id": tool_call_id,
+ "content": result_content,
+ }
+
+ return {
+ "tool_result": tool_result,
+ "success": "error" not in retrieval_data,
+ "data": retrieval_data,
+ }
+
+ # Compression-only endpoint (for TypeScript SDK and other HTTP clients)
+ @app.post("/v1/compress")
+ async def compress_messages(request: Request):
+ return await proxy.handle_compress(request)
+
+ register_provider_routes(app, proxy)
+
+ return app
+
+
+def _get_code_aware_banner_status(config: ProxyConfig) -> str:
+ """Get code-aware compression status line for banner."""
+ if config.code_aware_enabled:
+ if is_tree_sitter_available():
+ return "ENABLED (AST-based)"
+ else:
+ return "NOT INSTALLED (pip install headroom-ai[code])"
+ else:
+ if is_tree_sitter_available():
+ return "DISABLED (remove --no-code-aware to enable)"
+ return "DISABLED"
+
+
+def run_server(
+ config: ProxyConfig | None = None,
+ workers: int = 1,
+ limit_concurrency: int = 1000,
+):
+ """Run the proxy server.
+
+ Args:
+ config: Proxy configuration
+ workers: Number of worker processes (use N for multi-core scaling)
+ limit_concurrency: Max concurrent connections before 503 response
+ """
+ if not FASTAPI_AVAILABLE:
+ print("ERROR: FastAPI required. Install: pip install fastapi uvicorn httpx")
+ sys.exit(1)
+
+ config = config or ProxyConfig()
+ app = create_app(config)
+
+ code_aware_status = _get_code_aware_banner_status(config)
+
+ # Format connection pool info
+ pool_info = f"max={config.max_connections}, keepalive={config.max_keepalive_connections}"
+ http2_status = "ENABLED" if config.http2 else "DISABLED"
+
+ backend_status = format_backend_status(
+ backend=config.backend,
+ anyllm_provider=config.anyllm_provider,
+ bedrock_region=config.bedrock_region,
+ )
+
+ print(f"""
+╔══════════════════════════════════════════════════════════════════════╗
+║ HEADROOM PROXY SERVER ║
+╠══════════════════════════════════════════════════════════════════════╣
+║ Version: 1.0.0 ║
+║ Listening: http://{config.host}:{config.port:<5} ║
+║ Workers: {workers:<3} Concurrency Limit: {limit_concurrency:<5} ║
+║ Backend: {backend_status:<59}║
+╠══════════════════════════════════════════════════════════════════════╣
+║ FEATURES: ║
+║ Optimization: {"ENABLED " if config.optimize else "DISABLED"} ║
+║ Caching: {"ENABLED " if config.cache_enabled else "DISABLED"} (TTL: {config.cache_ttl_seconds}s) ║
+║ Rate Limiting: {"ENABLED " if config.rate_limit_enabled else "DISABLED"} ({config.rate_limit_requests_per_minute} req/min, {config.rate_limit_tokens_per_minute:,} tok/min) ║
+║ Retry: {"ENABLED " if config.retry_enabled else "DISABLED"} (max {config.retry_max_attempts} attempts) ║
+║ Cost Tracking: {"ENABLED " if config.cost_tracking_enabled else "DISABLED"} (budget: {"$" + str(config.budget_limit_usd) + "/" + config.budget_period if config.budget_limit_usd else "unlimited"}) ║
+║ Code-Aware: {code_aware_status:<52}║
+║ HTTP/2: {http2_status:<52}║
+║ Conn Pool: {pool_info:<52}║
+╠══════════════════════════════════════════════════════════════════════╣
+║ USAGE: ║
+║ Claude Code: ANTHROPIC_BASE_URL=http://{config.host}:{config.port} claude ║
+║ Cursor: Set base URL in settings ║
+╠══════════════════════════════════════════════════════════════════════╣
+║ ENDPOINTS: ║
+║ /livez Process liveness ║
+║ /readyz Traffic readiness ║
+║ /health Aggregate health ║
+║ /stats Detailed statistics ║
+║ /metrics Prometheus metrics ║
+║ /cache/clear Clear response cache ║
+║ /v1/retrieve CCR: Retrieve compressed content ║
+║ /v1/retrieve/stats CCR: Compression store stats ║
+║ /v1/retrieve/tool_call CCR: Handle LLM tool calls ║
+║ /v1/feedback CCR: Feedback loop stats & patterns ║
+║ /v1/feedback/{{tool}} CCR: Compression hints for a tool ║
+║ /v1/telemetry Data flywheel: Telemetry stats ║
+║ /v1/telemetry/export Data flywheel: Export for aggregation ║
+║ /v1/telemetry/tools Data flywheel: Per-tool stats ║
+║ /v1/toin/stats TOIN: Overall intelligence stats ║
+║ /v1/toin/patterns TOIN: List learned patterns ║
+║ /v1/toin/pattern/{{hash}} TOIN: Pattern details by hash ║
+╚══════════════════════════════════════════════════════════════════════╝
+""")
+
+ uvicorn.run(
+ app,
+ host=config.host,
+ port=config.port,
+ log_level="warning",
+ workers=workers if workers > 1 else None, # None = single process (default)
+ limit_concurrency=limit_concurrency,
+ # Defense-in-depth: the loopback guard for /debug/* endpoints trusts
+ # request.client.host. uvicorn's ProxyHeadersMiddleware rewrites that
+ # from X-Forwarded-For when FORWARDED_ALLOW_IPS is broader than the
+ # default. Disabling proxy_headers here guarantees the guard sees the
+ # real peer address regardless of env.
+ proxy_headers=False,
+ )
+
+
+def _get_env_bool(name: str, default: bool) -> bool:
+ """Get boolean from environment variable."""
+ val = os.environ.get(name)
+ if val is None:
+ return default
+ return val.lower() in ("true", "1", "yes", "on")
+
+
+def _get_env_int(name: str, default: int) -> int:
+ """Get integer from environment variable."""
+ val = os.environ.get(name)
+ if val is None:
+ return default
+ try:
+ return int(val)
+ except ValueError:
+ return default
+
+
+def _get_env_float(name: str, default: float) -> float:
+ """Get float from environment variable."""
+ val = os.environ.get(name)
+ if val is None:
+ return default
+ try:
+ return float(val)
+ except ValueError:
+ return default
+
+
+def _get_env_str(name: str, default: str) -> str:
+ """Get string from environment variable."""
+ return os.environ.get(name, default)
+
+
+def _parse_tool_profiles(cli_profiles: list[str]) -> dict[str, Any]:
+ """Parse tool profiles from CLI args and HEADROOM_TOOL_PROFILES env var.
+
+ Format: ToolName:level (e.g., Grep:conservative, Bash:moderate)
+ Env var format: comma-separated (e.g., "Grep:conservative,Bash:moderate")
+
+ Returns:
+ Dict mapping tool names to CompressionProfile instances.
+ """
+ from headroom.config import PROFILE_PRESETS, CompressionProfile
+
+ profiles: dict[str, CompressionProfile] = {}
+ raw_entries: list[str] = list(cli_profiles)
+
+ # Also check env var
+ env_val = os.environ.get("HEADROOM_TOOL_PROFILES", "")
+ if env_val:
+ raw_entries.extend(e.strip() for e in env_val.split(",") if e.strip())
+
+ for entry in raw_entries:
+ if ":" not in entry:
+ logger.warning("Invalid tool profile format (expected ToolName:level): %s", entry)
+ continue
+ tool_name, level = entry.split(":", 1)
+ tool_name = tool_name.strip()
+ level = level.strip().lower()
+
+ if level in PROFILE_PRESETS:
+ profiles[tool_name] = PROFILE_PRESETS[level]
+ else:
+ logger.warning(
+ "Unknown profile level '%s' for tool '%s'. Use: conservative, moderate, aggressive",
+ level,
+ tool_name,
+ )
+
+ return profiles
+
+
+if __name__ == "__main__":
+ parser = argparse.ArgumentParser(description="Headroom Proxy Server")
+
+ # Server
+ parser.add_argument("--host", default="127.0.0.1")
+ parser.add_argument("--port", type=int, default=8787)
+ parser.add_argument(
+ "--openai-api-url", help=f"Custom OpenAI API URL (default: {DEFAULT_OPENAI_API_URL})"
+ )
+ parser.add_argument(
+ "--anthropic-api-url",
+ help=f"Custom Anthropic API URL (default: {DEFAULT_ANTHROPIC_API_URL})",
+ )
+
+ # Backend (anthropic direct, bedrock, openrouter, anyllm, or litellm-<provider>)
+ parser.add_argument(
+ "--backend",
+ default="anthropic",
+ help=(
+ "Backend: 'anthropic' (direct), 'bedrock' (AWS), 'openrouter', "
+ "'anyllm' (any-llm), or 'litellm-<provider>' (e.g., litellm-hosted_vllm, litellm-vertex)"
+ ),
+ )
+ parser.add_argument(
+ "--bedrock-region",
+ default="us-west-2",
+ help="AWS region for Bedrock backend (default: us-west-2)",
+ )
+ parser.add_argument(
+ "--bedrock-profile",
+ help="AWS profile for Bedrock backend (default: use default credentials)",
+ )
+ parser.add_argument(
+ "--openrouter-api-key",
+ help="OpenRouter API key (or set OPENROUTER_API_KEY env var)",
+ )
+ parser.add_argument(
+ "--anyllm-provider",
+ default="openai",
+ help="any-llm provider: openai, anthropic, mistral, groq, ollama, bedrock, etc. (default: openai)",
+ )
+
+ # Connection pool (scalability)
+ parser.add_argument(
+ "--max-connections",
+ type=int,
+ default=500,
+ help="Max connections to upstream APIs (default: 500)",
+ )
+ parser.add_argument(
+ "--max-keepalive", type=int, default=100, help="Max keepalive connections (default: 100)"
+ )
+ parser.add_argument(
+ "--no-http2",
+ action="store_true",
+ help="Disable HTTP/2 (enabled by default for better throughput)",
+ )
+ parser.add_argument(
+ "--workers",
+ type=int,
+ default=1,
+ help="Number of worker processes (default: 1, use N for multi-core)",
+ )
+ parser.add_argument(
+ "--limit-concurrency",
+ type=int,
+ default=1000,
+ help="Max concurrent connections before 503 (default: 1000)",
+ )
+
+ # Optimization
+ parser.add_argument("--no-optimize", action="store_true", help="Disable optimization")
+ parser.add_argument("--min-tokens", type=int, default=500, help="Min tokens to crush")
+ parser.add_argument("--max-items", type=int, default=50, help="Max items after crush")
+ parser.add_argument(
+ "--tool-profile",
+ action="append",
+ default=[],
+ help="Per-tool compression profile: ToolName:level (e.g., Grep:conservative, Bash:moderate, WebFetch:aggressive). "
+ "Can be specified multiple times. Also settable via HEADROOM_TOOL_PROFILES env var.",
+ )
+
+ # Caching
+ parser.add_argument("--no-cache", action="store_true", help="Disable caching")
+ parser.add_argument("--cache-ttl", type=int, default=3600, help="Cache TTL seconds")
+
+ # Rate limiting
+ parser.add_argument("--no-rate-limit", action="store_true", help="Disable rate limiting")
+ parser.add_argument("--rpm", type=int, default=60, help="Requests per minute")
+ parser.add_argument("--tpm", type=int, default=100000, help="Tokens per minute")
+
+ # Cost
+ parser.add_argument("--budget", type=float, help="Budget limit in USD")
+ parser.add_argument("--budget-period", choices=["hourly", "daily", "monthly"], default="daily")
+
+ # Logging
+ parser.add_argument("--log-file", help="Log file path")
+ parser.add_argument("--log-messages", action="store_true", help="Log full messages")
+
+ # Smart routing (content-aware compression)
+ parser.add_argument(
+ "--no-smart-routing",
+ action="store_true",
+ help="Disable smart routing (use legacy sequential pipeline)",
+ )
+
+ # Code-aware compression
+ parser.add_argument(
+ "--code-aware",
+ action="store_true",
+ help="Enable AST-based code compression (requires: pip install headroom-ai[code])",
+ )
+ parser.add_argument(
+ "--no-code-aware",
+ action="store_true",
+ help="Disable code-aware compression",
+ )
+
+ args = parser.parse_args()
+
+ # Environment variable defaults (HEADROOM_* prefix)
+ # CLI args override env vars, env vars override ProxyConfig defaults
+ env_smart_routing = _get_env_bool("HEADROOM_SMART_ROUTING", True)
+ env_code_aware = _get_env_bool("HEADROOM_CODE_AWARE_ENABLED", True)
+ env_optimize = _get_env_bool("HEADROOM_OPTIMIZE", True)
+ env_cache = _get_env_bool("HEADROOM_CACHE_ENABLED", True)
+ env_rate_limit = _get_env_bool("HEADROOM_RATE_LIMIT_ENABLED", True)
+
+ # Determine settings: CLI flags override env vars
+ # --no-X explicitly disables, --X explicitly enables, neither uses env var
+ smart_routing = env_smart_routing if not args.no_smart_routing else False
+ code_aware_enabled = (
+ env_code_aware
+ if not (args.code_aware or args.no_code_aware)
+ else (args.code_aware or not args.no_code_aware)
+ )
+ optimize = env_optimize if not args.no_optimize else False
+ cache_enabled = env_cache if not args.no_cache else False
+ rate_limit_enabled = env_rate_limit if not args.no_rate_limit else False
+
+ # Set OpenRouter API key from CLI if provided
+ if hasattr(args, "openrouter_api_key") and args.openrouter_api_key:
+ os.environ["OPENROUTER_API_KEY"] = args.openrouter_api_key
+
+ # Parse per-tool compression profiles from CLI and env var
+ tool_profiles = _parse_tool_profiles(args.tool_profile)
+
+ config = ProxyConfig(
+ host=_get_env_str("HEADROOM_HOST", args.host),
+ port=_get_env_int("HEADROOM_PORT", args.port),
+ openai_api_url=_get_env_str("OPENAI_TARGET_API_URL", args.openai_api_url),
+ anthropic_api_url=_get_env_str("ANTHROPIC_TARGET_API_URL", args.anthropic_api_url),
+ # Backend settings
+ backend=_get_env_str("HEADROOM_BACKEND", args.backend), # type: ignore[arg-type]
+ bedrock_region=_get_env_str("HEADROOM_BEDROCK_REGION", args.bedrock_region),
+ bedrock_profile=args.bedrock_profile or os.environ.get("AWS_PROFILE"),
+ anyllm_provider=_get_env_str("HEADROOM_ANYLLM_PROVIDER", args.anyllm_provider),
+ optimize=optimize,
+ min_tokens_to_crush=_get_env_int("HEADROOM_MIN_TOKENS", args.min_tokens),
+ max_items_after_crush=_get_env_int("HEADROOM_MAX_ITEMS", args.max_items),
+ cache_enabled=cache_enabled,
+ cache_ttl_seconds=_get_env_int("HEADROOM_CACHE_TTL", args.cache_ttl),
+ rate_limit_enabled=rate_limit_enabled,
+ rate_limit_requests_per_minute=_get_env_int("HEADROOM_RPM", args.rpm),
+ rate_limit_tokens_per_minute=_get_env_int("HEADROOM_TPM", args.tpm),
+ budget_limit_usd=args.budget,
+ budget_period=args.budget_period,
+ log_file=_get_env_str("HEADROOM_LOG_FILE", args.log_file)
+ if args.log_file
+ else os.environ.get("HEADROOM_LOG_FILE"),
+ log_full_messages=args.log_messages or _get_env_bool("HEADROOM_LOG_MESSAGES", False),
+ smart_routing=smart_routing,
+ code_aware_enabled=code_aware_enabled,
+ # Connection pool settings
+ max_connections=_get_env_int("HEADROOM_MAX_CONNECTIONS", args.max_connections),
+ max_keepalive_connections=_get_env_int("HEADROOM_MAX_KEEPALIVE", args.max_keepalive),
+ http2=not args.no_http2 and _get_env_bool("HEADROOM_HTTP2", True),
+ tool_profiles=tool_profiles if tool_profiles else None,
+ mode=normalize_proxy_mode(_get_env_str("HEADROOM_MODE", PROXY_MODE_TOKEN)),
+ )
+
+ # Get worker and concurrency settings
+ workers = _get_env_int("HEADROOM_WORKERS", args.workers)
+ limit_concurrency = _get_env_int("HEADROOM_LIMIT_CONCURRENCY", args.limit_concurrency)
+
+ run_server(config, workers=workers, limit_concurrency=limit_concurrency)
diff --git a/tests/test_cli/test_wrap_aider.py b/tests/test_cli/test_wrap_aider.py
new file mode 100644
index 000000000..8abfd9bfb
--- /dev/null
+++ b/tests/test_cli/test_wrap_aider.py
@@ -0,0 +1,39 @@
+"""Tests for `headroom wrap aider` command."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from unittest.mock import patch
+
+import pytest
+from click.testing import CliRunner
+
+from headroom.cli.main import main
+
+
+def runner() -> CliRunner:
+ return CliRunner()
+
+
+def test_wrap_aider_sets_provider_envs(
+ runner: CliRunner, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ monkeypatch.chdir(tmp_path)
+ captured: dict[str, object] = {}
+
+ def fake_launch_tool(**kwargs): # noqa: ANN003
+ captured.update(kwargs)
+
+ with patch("headroom.cli.wrap.shutil.which", return_value="aider"):
+ with patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch_tool):
+ result = runner.invoke(main, ["wrap", "aider", "--no-rtk", "--", "--model", "gpt-4o"])
+
+ assert result.exit_code == 0, result.output
+ env = captured["env"]
+ assert isinstance(env, dict)
+ assert env["OPENAI_API_BASE"] == "http://127.0.0.1:8787/v1"
+ assert env["ANTHROPIC_BASE_URL"] == "http://127.0.0.1:8787"
+ assert captured["tool_label"] == "AIDER"
+ assert captured["agent_type"] == "aider"
+ assert captured["args"] == ("--model", "gpt-4o")
diff --git a/tests/test_cli/test_wrap_copilot.py b/tests/test_cli/test_wrap_copilot.py
index dc8bafbab..098a0cbab 100644
--- a/tests/test_cli/test_wrap_copilot.py
+++ b/tests/test_cli/test_wrap_copilot.py
@@ -1,264 +1,198 @@
-"""Tests for `headroom wrap copilot` command."""
-
-from __future__ import annotations
-
-import importlib
-import sys
-import types
-from pathlib import Path
-from unittest.mock import patch
-
-import click
-import pytest
-from click.testing import CliRunner
-
-from headroom.copilot_auth import DEFAULT_API_URL
-
-fake_main_module = types.ModuleType("headroom.cli.main")
-fake_main_module.main = click.Group()
-sys.modules["headroom.cli.main"] = fake_main_module
-sys.modules.pop("headroom.cli", None)
-sys.modules.pop("headroom.cli.wrap", None)
-
-wrap_cli = importlib.import_module("headroom.cli.wrap")
-main = fake_main_module.main
-
-
-def runner() -> CliRunner:
- return CliRunner()
-
-
[email protected](autouse=True)
-def no_running_proxy(monkeypatch: pytest.MonkeyPatch) -> None:
- monkeypatch.setattr(wrap_cli, "_check_proxy", lambda _port: False)
-
-
-def test_wrap_copilot_auto_anthropic_injects_instructions(
- runner: CliRunner, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
-) -> None:
- monkeypatch.chdir(tmp_path)
- monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-test-dummy")
- captured: dict[str, object] = {}
-
- def fake_launch_tool(**kwargs): # noqa: ANN003
- captured.update(kwargs)
-
- with patch("headroom.cli.wrap.shutil.which", return_value="copilot"):
- with patch("headroom.cli.wrap._ensure_rtk_binary", return_value=Path("/tmp/rtk")):
- with patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch_tool):
- result = runner.invoke(
- main,
- ["wrap", "copilot", "--", "--model", "claude-sonnet-4-20250514"],
- )
-
- assert result.exit_code == 0, result.output
- instructions = tmp_path / ".github" / "copilot-instructions.md"
- assert instructions.exists()
- content = instructions.read_text()
- assert wrap_cli._RTK_MARKER in content
- assert "RTK (Rust Token Killer)" in content
-
- env = captured["env"]
- assert isinstance(env, dict)
- assert env["COPILOT_PROVIDER_TYPE"] == "anthropic"
- assert env["COPILOT_PROVIDER_BASE_URL"] == "http://127.0.0.1:8787"
- assert "COPILOT_PROVIDER_WIRE_API" not in env
- assert captured["agent_type"] == "copilot"
- assert captured["tool_label"] == "COPILOT"
- assert captured["args"] == ("--model", "claude-sonnet-4-20250514")
-
-
-def test_wrap_copilot_openai_backend_sets_completions_env(
- runner: CliRunner, monkeypatch: pytest.MonkeyPatch
-) -> None:
- monkeypatch.setenv("OPENAI_API_KEY", "sk-test-dummy")
- captured: dict[str, object] = {}
-
- def fake_launch_tool(**kwargs): # noqa: ANN003
- captured.update(kwargs)
-
- with patch("headroom.cli.wrap.shutil.which", return_value="copilot"):
- with patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch_tool):
- result = runner.invoke(
- main,
- [
- "wrap",
- "copilot",
- "--no-rtk",
- "--backend",
- "anyllm",
- "--anyllm-provider",
- "groq",
- "--region",
- "us-central1",
- "--",
- "--model",
- "gpt-4o",
- ],
- )
-
- assert result.exit_code == 0, result.output
-
- env = captured["env"]
- assert isinstance(env, dict)
- assert env["COPILOT_PROVIDER_TYPE"] == "openai"
- assert env["COPILOT_PROVIDER_BASE_URL"] == "http://127.0.0.1:8787/v1"
- assert env["COPILOT_PROVIDER_WIRE_API"] == "completions"
- assert captured["backend"] == "anyllm"
- assert captured["anyllm_provider"] == "groq"
- assert captured["region"] == "us-central1"
- assert captured["args"] == ("--model", "gpt-4o")
-
-
-def test_wrap_copilot_auto_detects_running_proxy_backend(
- runner: CliRunner, monkeypatch: pytest.MonkeyPatch
-) -> None:
- monkeypatch.setenv("OPENAI_API_KEY", "sk-test-dummy")
- captured: dict[str, object] = {}
-
- def fake_launch_tool(**kwargs): # noqa: ANN003
- captured.update(kwargs)
-
- with patch("headroom.cli.wrap.shutil.which", return_value="copilot"):
- with patch("headroom.cli.wrap._check_proxy", return_value=True):
- with patch("headroom.cli.wrap._detect_running_proxy_backend", return_value="anyllm"):
- with patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch_tool):
- result = runner.invoke(
- main,
- ["wrap", "copilot", "--no-rtk", "--", "--model", "gpt-4o"],
- )
-
- assert result.exit_code == 0, result.output
- env = captured["env"]
- assert isinstance(env, dict)
- assert env["COPILOT_PROVIDER_TYPE"] == "openai"
- assert env["COPILOT_PROVIDER_BASE_URL"] == "http://127.0.0.1:8787/v1"
- assert env["COPILOT_PROVIDER_WIRE_API"] == "completions"
-
-
-def test_wrap_copilot_prefers_existing_oauth_session(
- runner: CliRunner, monkeypatch: pytest.MonkeyPatch
-) -> None:
- monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-test-dummy")
- captured: dict[str, object] = {}
-
- def fake_launch_tool(**kwargs): # noqa: ANN003
- captured.update(kwargs)
-
- with patch("headroom.cli.wrap.shutil.which", return_value="copilot"):
- with patch("headroom.cli.wrap.resolve_client_bearer_token", return_value="gho-existing"):
- with patch("headroom.cli.wrap.has_oauth_auth", return_value=True):
- with patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch_tool):
- result = runner.invoke(
- main,
- ["wrap", "copilot", "--no-rtk", "--", "--model", "claude-sonnet-4.6"],
- )
-
- assert result.exit_code == 0, result.output
- env = captured["env"]
- assert isinstance(env, dict)
- assert env["COPILOT_PROVIDER_TYPE"] == "openai"
- assert env["COPILOT_PROVIDER_BASE_URL"] == "http://127.0.0.1:8787/v1"
- assert env["COPILOT_PROVIDER_WIRE_API"] == "completions"
- assert env["COPILOT_PROVIDER_BEARER_TOKEN"] == "gho-existing"
- assert "COPILOT_PROVIDER_API_KEY" not in env
- assert captured["openai_api_url"] == DEFAULT_API_URL
-
-
-def test_wrap_copilot_translated_backend_still_requires_byok(
- runner: CliRunner,
-) -> None:
- with patch("headroom.cli.wrap.shutil.which", return_value="copilot"):
- with patch("headroom.cli.wrap.has_oauth_auth", return_value=True):
- result = runner.invoke(
- main,
- [
- "wrap",
- "copilot",
- "--backend",
- "anyllm",
- "--",
- "--model",
- "gpt-4o",
- ],
- )
-
- assert result.exit_code == 1
- assert "Copilot BYOK mode requires a provider API key" in result.output
-
-
-def test_wrap_copilot_rejects_wire_api_for_anthropic_provider(runner: CliRunner) -> None:
- with patch("headroom.cli.wrap.shutil.which", return_value="copilot"):
- result = runner.invoke(
- main,
- [
- "wrap",
- "copilot",
- "--wire-api",
- "responses",
- "--",
- "--model",
- "claude-sonnet-4-20250514",
- ],
- )
-
- assert result.exit_code != 0
- assert "--wire-api is only valid" in result.output
-
-
-def test_wrap_copilot_rejects_responses_for_translated_backends(runner: CliRunner) -> None:
- with patch("headroom.cli.wrap.shutil.which", return_value="copilot"):
- result = runner.invoke(
- main,
- [
- "wrap",
- "copilot",
- "--backend",
- "anyllm",
- "--wire-api",
- "responses",
- "--",
- "--model",
- "gpt-4o",
- ],
- )
-
- assert result.exit_code != 0
- assert "not supported with translated backends" in result.output
-
-
-def test_wrap_copilot_clears_stale_wire_api_in_anthropic_mode(
- runner: CliRunner, monkeypatch: pytest.MonkeyPatch
-) -> None:
- monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-test-dummy")
- captured: dict[str, object] = {}
-
- def fake_launch_tool(**kwargs): # noqa: ANN003
- captured.update(kwargs)
-
- with patch("headroom.cli.wrap.shutil.which", return_value="copilot"):
- with patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch_tool):
- result = runner.invoke(
- main,
- ["wrap", "copilot", "--no-rtk", "--", "--model", "claude-sonnet-4-20250514"],
- env={
- "COPILOT_PROVIDER_WIRE_API": "responses",
- "ANTHROPIC_API_KEY": "sk-test-dummy",
- },
- )
-
- assert result.exit_code == 0, result.output
- env = captured["env"]
- assert isinstance(env, dict)
- assert env["COPILOT_PROVIDER_TYPE"] == "anthropic"
- assert "COPILOT_PROVIDER_WIRE_API" not in env
-
-
-def test_wrap_copilot_fails_when_binary_missing(runner: CliRunner) -> None:
- with patch("headroom.cli.wrap.shutil.which", return_value=None):
- result = runner.invoke(main, ["wrap", "copilot", "--", "--model", "gpt-4o"])
-
- assert result.exit_code == 1
- assert "'copilot' not found in PATH" in result.output
- assert "Install GitHub Copilot CLI" in result.output
+"""Tests for `headroom wrap copilot` command."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from unittest.mock import patch
+
+import pytest
+from click.testing import CliRunner
+
+from headroom.cli import wrap as wrap_cli
+from headroom.cli.main import main
+
+
+def runner() -> CliRunner:
+ return CliRunner()
+
+
+def test_wrap_copilot_auto_anthropic_injects_instructions(
+ runner: CliRunner, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ monkeypatch.chdir(tmp_path)
+ monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-test-dummy")
+ captured: dict[str, object] = {}
+
+ def fake_launch_tool(**kwargs): # noqa: ANN003
+ captured.update(kwargs)
+
+ with patch("headroom.cli.wrap.shutil.which", return_value="copilot"):
+ with patch("headroom.cli.wrap._ensure_rtk_binary", return_value=Path("/tmp/rtk")):
+ with patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch_tool):
+ result = runner.invoke(
+ main,
+ ["wrap", "copilot", "--", "--model", "claude-sonnet-4-20250514"],
+ )
+
+ assert result.exit_code == 0, result.output
+ instructions = tmp_path / ".github" / "copilot-instructions.md"
+ assert instructions.exists()
+ content = instructions.read_text()
+ assert wrap_cli._RTK_MARKER in content
+ assert "RTK (Rust Token Killer)" in content
+
+ env = captured["env"]
+ assert isinstance(env, dict)
+ assert env["COPILOT_PROVIDER_TYPE"] == "anthropic"
+ assert env["COPILOT_PROVIDER_BASE_URL"] == "http://127.0.0.1:8787"
+ assert "COPILOT_PROVIDER_WIRE_API" not in env
+ assert captured["agent_type"] == "copilot"
+ assert captured["tool_label"] == "COPILOT"
+ assert captured["args"] == ("--model", "claude-sonnet-4-20250514")
+
+
+def test_wrap_copilot_openai_backend_sets_completions_env(
+ runner: CliRunner, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ monkeypatch.setenv("OPENAI_API_KEY", "sk-test-dummy")
+ captured: dict[str, object] = {}
+
+ def fake_launch_tool(**kwargs): # noqa: ANN003
+ captured.update(kwargs)
+
+ with patch("headroom.cli.wrap.shutil.which", return_value="copilot"):
+ with patch("headroom.cli.wrap._check_proxy", return_value=False):
+ with patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch_tool):
+ result = runner.invoke(
+ main,
+ [
+ "wrap",
+ "copilot",
+ "--no-rtk",
+ "--backend",
+ "anyllm",
+ "--anyllm-provider",
+ "groq",
+ "--region",
+ "us-central1",
+ "--",
+ "--model",
+ "gpt-4o",
+ ],
+ )
+
+ assert result.exit_code == 0, result.output
+
+ env = captured["env"]
+ assert isinstance(env, dict)
+ assert env["COPILOT_PROVIDER_TYPE"] == "openai"
+ assert env["COPILOT_PROVIDER_BASE_URL"] == "http://127.0.0.1:8787/v1"
+ assert env["COPILOT_PROVIDER_WIRE_API"] == "completions"
+ assert captured["backend"] == "anyllm"
+ assert captured["anyllm_provider"] == "groq"
+ assert captured["region"] == "us-central1"
+ assert captured["args"] == ("--model", "gpt-4o")
+
+
+def test_wrap_copilot_auto_detects_running_proxy_backend(
+ runner: CliRunner, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ monkeypatch.setenv("OPENAI_API_KEY", "sk-test-dummy")
+ captured: dict[str, object] = {}
+
+ def fake_launch_tool(**kwargs): # noqa: ANN003
+ captured.update(kwargs)
+
+ with patch("headroom.cli.wrap.shutil.which", return_value="copilot"):
+ with patch("headroom.cli.wrap._check_proxy", return_value=True):
+ with patch("headroom.cli.wrap._detect_running_proxy_backend", return_value="anyllm"):
+ with patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch_tool):
+ result = runner.invoke(
+ main,
+ ["wrap", "copilot", "--no-rtk", "--", "--model", "gpt-4o"],
+ )
+
+ assert result.exit_code == 0, result.output
+ env = captured["env"]
+ assert isinstance(env, dict)
+ assert env["COPILOT_PROVIDER_TYPE"] == "openai"
+ assert env["COPILOT_PROVIDER_BASE_URL"] == "http://127.0.0.1:8787/v1"
+ assert env["COPILOT_PROVIDER_WIRE_API"] == "completions"
+
+
+def test_wrap_copilot_rejects_wire_api_for_anthropic_provider(runner: CliRunner) -> None:
+ with patch("headroom.cli.wrap.shutil.which", return_value="copilot"):
+ result = runner.invoke(
+ main,
+ [
+ "wrap",
+ "copilot",
+ "--wire-api",
+ "responses",
+ "--",
+ "--model",
+ "claude-sonnet-4-20250514",
+ ],
+ )
+
+ assert result.exit_code != 0
+ assert "--wire-api is only valid" in result.output
+
+
+def test_wrap_copilot_rejects_responses_for_translated_backends(runner: CliRunner) -> None:
+ with patch("headroom.cli.wrap.shutil.which", return_value="copilot"):
+ with patch("headroom.cli.wrap._check_proxy", return_value=False):
+ result = runner.invoke(
+ main,
+ [
+ "wrap",
+ "copilot",
+ "--backend",
+ "anyllm",
+ "--wire-api",
+ "responses",
+ "--",
+ "--model",
+ "gpt-4o",
+ ],
+ )
+
+ assert result.exit_code != 0
+ assert "not supported with translated backends" in result.output
+
+
+def test_wrap_copilot_clears_stale_wire_api_in_anthropic_mode(
+ runner: CliRunner, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-test-dummy")
+ captured: dict[str, object] = {}
+
+ def fake_launch_tool(**kwargs): # noqa: ANN003
+ captured.update(kwargs)
+
+ with patch("headroom.cli.wrap.shutil.which", return_value="copilot"):
+ with patch("headroom.cli.wrap._launch_tool", side_effect=fake_launch_tool):
+ result = runner.invoke(
+ main,
+ ["wrap", "copilot", "--no-rtk", "--", "--model", "claude-sonnet-4-20250514"],
+ env={
+ "COPILOT_PROVIDER_WIRE_API": "responses",
+ "ANTHROPIC_API_KEY": "sk-test-dummy",
+ },
+ )
+
+ assert result.exit_code == 0, result.output
+ env = captured["env"]
+ assert isinstance(env, dict)
+ assert env["COPILOT_PROVIDER_TYPE"] == "anthropic"
+ assert "COPILOT_PROVIDER_WIRE_API" not in env
+
+
+def test_wrap_copilot_fails_when_binary_missing(runner: CliRunner) -> None:
+ with patch("headroom.cli.wrap.shutil.which", return_value=None):
+ result = runner.invoke(main, ["wrap", "copilot", "--", "--model", "gpt-4o"])
+
+ assert result.exit_code == 1
+ assert "'copilot' not found in PATH" in result.output
+ assert "Install GitHub Copilot CLI" in result.output
diff --git a/tests/test_provider_cursor.py b/tests/test_provider_cursor.py
new file mode 100644
index 000000000..ab2dcee0d
--- /dev/null
+++ b/tests/test_provider_cursor.py
@@ -0,0 +1,18 @@
+from __future__ import annotations
+
+from headroom.providers.cursor import build_proxy_targets, render_setup_lines
+
+
+def test_cursor_proxy_targets_use_local_headroom_proxy() -> None:
+ targets = build_proxy_targets(9999)
+
+ assert targets.openai_base_url == "http://127.0.0.1:9999/v1"
+ assert targets.anthropic_base_url == "http://127.0.0.1:9999"
+
+
+def test_cursor_setup_lines_include_both_provider_urls() -> None:
+ lines = render_setup_lines(8787)
+ joined = "\n".join(lines)
+
+ assert "http://127.0.0.1:8787/v1" in joined
+ assert "http://127.0.0.1:8787" in joined
diff --git a/tests/test_provider_registry.py b/tests/test_provider_registry.py
new file mode 100644
index 000000000..9b2e274a4
--- /dev/null
+++ b/tests/test_provider_registry.py
@@ -0,0 +1,83 @@
+from __future__ import annotations
+
+from headroom.providers.registry import (
+ ProviderApiOverrides,
+ build_proxy_provider_runtime,
+ format_backend_status,
+ resolve_api_overrides,
+ resolve_api_targets,
+)
+from headroom.proxy.models import ProxyConfig
+
+
+def test_resolve_api_overrides_prefers_explicit_values_over_environment(monkeypatch) -> None:
+ monkeypatch.setenv("ANTHROPIC_TARGET_API_URL", "https://env.anthropic.example/v1")
+ monkeypatch.setenv("OPENAI_TARGET_API_URL", "https://env.openai.example/v1")
+
+ overrides = resolve_api_overrides(
+ anthropic_api_url="https://cli.anthropic.example/v1",
+ openai_api_url=None,
+ gemini_api_url=None,
+ cloudcode_api_url=None,
+ )
+
+ assert overrides == ProviderApiOverrides(
+ anthropic="https://cli.anthropic.example/v1",
+ openai="https://env.openai.example/v1",
+ gemini=None,
+ cloudcode=None,
+ )
+
+
+def test_resolve_api_targets_normalizes_trailing_v1() -> None:
+ targets = resolve_api_targets(
+ ProviderApiOverrides(
+ anthropic="https://anthropic.example/v1/",
+ openai="https://openai.example/v1",
+ gemini="https://gemini.example/v1",
+ cloudcode="https://cloudcode.example/v1/",
+ )
+ )
+
+ assert targets.anthropic == "https://anthropic.example"
+ assert targets.openai == "https://openai.example"
+ assert targets.gemini == "https://gemini.example"
+ assert targets.cloudcode == "https://cloudcode.example"
+
+
+def test_proxy_config_exposes_provider_api_overrides() -> None:
+ config = ProxyConfig(
+ anthropic_api_url="https://anthropic.example",
+ openai_api_url="https://openai.example",
+ gemini_api_url=None,
+ cloudcode_api_url="https://cloudcode.example",
+ )
+
+ assert config.provider_api_overrides == ProviderApiOverrides(
+ anthropic="https://anthropic.example",
+ openai="https://openai.example",
+ gemini=None,
+ cloudcode="https://cloudcode.example",
+ )
+
+
+def test_format_backend_status_for_anyllm() -> None:
+ assert (
+ format_backend_status(
+ backend="anyllm",
+ anyllm_provider="groq",
+ bedrock_region="us-central1",
+ )
+ == "Groq via any-llm"
+ )
+
+
+def test_proxy_provider_runtime_routes_model_metadata_and_passthrough() -> None:
+ runtime = build_proxy_provider_runtime(ProxyConfig())
+
+ assert runtime.model_metadata_provider({"x-api-key": "test"}) == "anthropic"
+ assert runtime.model_metadata_provider({}) == "openai"
+ assert (
+ runtime.select_passthrough_base_url({"x-goog-api-key": "test"})
+ == runtime.api_targets.gemini
+ )