Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

Add seamless LangChain integration

chopratejasbb041047c8e2
diff --git a/README.md b/README.md
index 5e5272567..bffaaf093 100644
--- a/README.md
+++ b/README.md
@@ -27,45 +27,89 @@
## What It Does
-Headroom is a **smart compression proxy** for LLM applications:
+Headroom is a **smart compression layer** for LLM applications:
- **Compresses tool outputs** — 1000 search results → 15 items (keeps errors, anomalies, relevant items)
- **Enables provider caching** — Stabilizes prefixes so cache hits actually happen
- **Manages context windows** — Prevents token limit failures without breaking tool calls
- **Reversible compression** — LLM can retrieve original data if needed ([CCR architecture](docs/ccr.md))
-**Zero code changes required** — point your existing tools at the proxy.
+Works as a **proxy** (zero code changes) or **SDK** (fine-grained control).
---
## 30-Second Quickstart
+### Option 1: Proxy (Zero Code Changes)
+
```bash
-# Install
pip install "headroom-ai[proxy]"
-
-# Start proxy
headroom proxy --port 8787
-
-# Verify
-curl http://localhost:8787/health
```
-**Use with your tools:**
+Point your tools at the proxy:
```bash
# Claude Code
ANTHROPIC_BASE_URL=http://localhost:8787 claude
-# Cursor / Continue / any OpenAI client
+# Any OpenAI-compatible client
OPENAI_BASE_URL=http://localhost:8787/v1 cursor
+```
+
+### Option 2: LangChain Integration
-# Python scripts
-export OPENAI_BASE_URL=http://localhost:8787/v1
-python your_script.py
+```bash
+pip install "headroom-ai[langchain]"
```
-That's it. You're saving tokens.
+```python
+from langchain_openai import ChatOpenAI
+from headroom.integrations import HeadroomChatModel
+
+# Wrap your model - that's it!
+llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))
+
+# Use exactly like before
+response = llm.invoke("Hello!")
+```
+
+See the full [LangChain Integration Guide](docs/langchain.md) for memory, retrievers, agents, and more.
+
+---
+
+## Framework Integrations
+
+| Framework | Integration | Docs |
+|-----------|-------------|------|
+| **LangChain** | `HeadroomChatModel`, memory, retrievers, agents | [Guide](docs/langchain.md) |
+| **MCP** | Tool output compression for Claude | [Guide](docs/ccr.md) |
+| **Any OpenAI Client** | Proxy server | [Guide](docs/proxy.md) |
+
+### LangChain Highlights
+
+```python
+from headroom.integrations import (
+ HeadroomChatModel, # Wrap any chat model
+ HeadroomChatMessageHistory, # Auto-compress conversation history
+ HeadroomDocumentCompressor, # Filter retrieved documents
+ wrap_tools_with_headroom, # Compress agent tool outputs
+)
+
+# Memory that auto-compresses when over 4K tokens
+memory = ConversationBufferMemory(
+ chat_memory=HeadroomChatMessageHistory(base_history)
+)
+
+# Retriever that keeps only relevant docs
+retriever = ContextualCompressionRetriever(
+ base_compressor=HeadroomDocumentCompressor(max_documents=10),
+ base_retriever=vectorstore.as_retriever(search_kwargs={"k": 50}),
+)
+
+# Agent tools with automatic output compression
+tools = wrap_tools_with_headroom([search_tool, database_tool])
+```
---
@@ -82,13 +126,21 @@ curl http://localhost:8787/stats
}
```
+Or in Python:
+
+```python
+print(llm.get_metrics())
+# {'tokens_saved': 12500, 'savings_percent': 45.2}
+```
+
---
## Installation
```bash
-pip install "headroom-ai[proxy]" # Proxy server (recommended)
pip install headroom-ai # SDK only
+pip install "headroom-ai[proxy]" # Proxy server
+pip install "headroom-ai[langchain]" # LangChain integration
pip install "headroom-ai[code]" # AST-based code compression
pip install "headroom-ai[llmlingua]" # ML-based compression
pip install "headroom-ai[all]" # Everything
@@ -106,10 +158,10 @@ pip install "headroom-ai[all]" # Everything
| **CacheAligner** | Stabilizes prefixes for provider caching | [Transforms](docs/transforms.md) |
| **RollingWindow** | Manages context limits without breaking tools | [Transforms](docs/transforms.md) |
| **CCR** | Reversible compression with automatic retrieval | [CCR Guide](docs/ccr.md) |
+| **LangChain** | Memory, retrievers, agents, streaming | [LangChain](docs/langchain.md) |
| **Text Utilities** | Opt-in compression for search/logs | [Text Compression](docs/text-compression.md) |
| **LLMLingua-2** | ML-based 20x compression (opt-in) | [LLMLingua](docs/llmlingua.md) |
| **Code-Aware** | AST-based code compression (tree-sitter) | [Transforms](docs/transforms.md) |
-| **ContentRouter** | Auto-routes content to optimal compressor | [Transforms](docs/transforms.md) |
---
@@ -123,7 +175,7 @@ pip install "headroom-ai[all]" # Everything
| Cohere | Official API | - |
| Mistral | Official tokenizer | - |
-**New models auto-supported** — Unknown models get sensible defaults based on naming patterns (e.g., `claude-opus-*` gets Opus pricing). Custom limits via `~/.headroom/models.json` or `HEADROOM_MODEL_LIMITS` env var.
+**New models auto-supported** — Unknown models get sensible defaults based on naming patterns.
---
@@ -134,6 +186,7 @@ pip install "headroom-ai[all]" # Everything
| Search results (1000 items) | 45,000 tokens | 4,500 tokens | 90% |
| Log analysis (500 entries) | 22,000 tokens | 3,300 tokens | 85% |
| Long conversation (50 turns) | 80,000 tokens | 32,000 tokens | 60% |
+| Agent with tools (10 calls) | 100,000 tokens | 15,000 tokens | 85% |
Overhead: ~1-5ms per request.
@@ -152,13 +205,13 @@ Overhead: ~1-5ms per request.
| Guide | Description |
|-------|-------------|
+| [LangChain Integration](docs/langchain.md) | Full LangChain support |
| [SDK Guide](docs/sdk.md) | Wrap your client for fine-grained control |
| [Proxy Guide](docs/proxy.md) | Production deployment |
| [Configuration](docs/configuration.md) | All configuration options |
| [CCR Guide](docs/ccr.md) | Reversible compression architecture |
| [Metrics](docs/metrics.md) | Monitoring and observability |
| [Troubleshooting](docs/troubleshooting.md) | Common issues |
-| [Architecture](docs/ARCHITECTURE.md) | How it works internally |
---
@@ -168,6 +221,8 @@ See [`examples/`](examples/) for runnable code:
- `basic_usage.py` — Simple SDK usage
- `proxy_integration.py` — Using with different clients
+- `langchain_agent.py` — LangChain ReAct agent with Headroom
+- `rag_pipeline.py` — RAG with document compression
- `ccr_demo.py` — CCR architecture demonstration
---
diff --git a/docs/README.md b/docs/README.md
index be80126b3..1e1658423 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -10,6 +10,13 @@ Welcome to the Headroom documentation.
| [SDK Guide](sdk.md) | Python SDK usage |
| [Proxy Guide](proxy.md) | Proxy server deployment |
+## Framework Integrations
+
+| Framework | Description |
+|-----------|-------------|
+| [LangChain](langchain.md) | Chat models, memory, retrievers, agents, streaming |
+| MCP | See [CCR Guide](ccr.md) for tool compression |
+
## Core Concepts
| Topic | Description |
diff --git a/docs/langchain.md b/docs/langchain.md
new file mode 100644
index 000000000..4629adb30
--- /dev/null
+++ b/docs/langchain.md
@@ -0,0 +1,622 @@
+# LangChain Integration
+
+Headroom provides seamless integration with LangChain, enabling automatic context optimization across all LangChain patterns: chat models, memory, retrievers, agents, and observability.
+
+## Installation
+
+```bash
+pip install "headroom-ai[langchain]"
+```
+
+This installs Headroom with LangChain dependencies (`langchain-core`).
+
+## Quick Start
+
+### Wrap Any Chat Model (1 Line)
+
+```python
+from langchain_openai import ChatOpenAI
+from headroom.integrations import HeadroomChatModel
+
+# Wrap your model - that's it!
+llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))
+
+# Use exactly like before
+response = llm.invoke("Hello!")
+```
+
+Headroom automatically:
+- Detects the provider (OpenAI, Anthropic, Google)
+- Compresses tool outputs in conversation history
+- Optimizes for provider caching
+- Tracks token savings
+
+### Check Your Savings
+
+```python
+# After some usage
+print(llm.get_metrics())
+# {'tokens_saved': 12500, 'savings_percent': 45.2, 'requests': 50}
+```
+
+---
+
+## Integration Patterns
+
+### 1. Chat Model Wrapper
+
+The `HeadroomChatModel` wraps any LangChain `BaseChatModel`:
+
+```python
+from langchain_openai import ChatOpenAI
+from langchain_anthropic import ChatAnthropic
+from headroom.integrations import HeadroomChatModel
+
+# OpenAI
+llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))
+
+# Anthropic (auto-detected)
+llm = HeadroomChatModel(ChatAnthropic(model="claude-3-5-sonnet-20241022"))
+
+# Custom configuration
+from headroom import HeadroomConfig, HeadroomMode
+
+config = HeadroomConfig(
+ default_mode=HeadroomMode.OPTIMIZE,
+ smart_crusher_target_ratio=0.3, # Target 70% compression
+)
+llm = HeadroomChatModel(
+ ChatOpenAI(model="gpt-4o"),
+ headroom_config=config,
+)
+```
+
+#### Async Support
+
+Full async support for `ainvoke` and `astream`:
+
+```python
+# Async invoke
+response = await llm.ainvoke("Hello!")
+
+# Async streaming
+async for chunk in llm.astream("Tell me a story"):
+ print(chunk.content, end="", flush=True)
+```
+
+#### Tool Calling
+
+Works seamlessly with LangChain tool calling:
+
+```python
+from langchain_core.tools import tool
+
+@tool
+def search(query: str) -> str:
+ """Search the web."""
+ return {"results": [...]} # Large JSON response
+
+llm_with_tools = llm.bind_tools([search])
+response = llm_with_tools.invoke("Search for Python tutorials")
+# Tool outputs are automatically compressed in subsequent turns
+```
+
+---
+
+### 2. Memory Integration
+
+`HeadroomChatMessageHistory` wraps any chat history with automatic compression:
+
+```python
+from langchain.memory import ConversationBufferMemory
+from langchain_community.chat_message_histories import ChatMessageHistory
+from headroom.integrations import HeadroomChatMessageHistory
+
+# Wrap any history
+base_history = ChatMessageHistory()
+compressed_history = HeadroomChatMessageHistory(
+ base_history,
+ compress_threshold_tokens=4000, # Compress when over 4K tokens
+ keep_recent_turns=5, # Always keep last 5 turns
+)
+
+# Use with any memory class
+memory = ConversationBufferMemory(chat_memory=compressed_history)
+
+# Zero changes to your chain!
+chain = ConversationChain(llm=llm, memory=memory)
+```
+
+**Why this matters**: Long conversations can blow up to 50K+ tokens. HeadroomChatMessageHistory automatically compresses older turns while preserving recent context.
+
+```python
+# Check compression stats
+print(compressed_history.get_compression_stats())
+# {'compression_count': 12, 'total_tokens_saved': 28000}
+```
+
+---
+
+### 3. Retriever Integration
+
+`HeadroomDocumentCompressor` filters retrieved documents by relevance:
+
+```python
+from langchain.retrievers import ContextualCompressionRetriever
+from langchain_community.vectorstores import FAISS
+from headroom.integrations import HeadroomDocumentCompressor
+
+# Create vector store retriever (retrieve many for recall)
+vectorstore = FAISS.from_documents(documents, embeddings)
+base_retriever = vectorstore.as_retriever(search_kwargs={"k": 50})
+
+# Wrap with Headroom compression (keep best for precision)
+compressor = HeadroomDocumentCompressor(
+ max_documents=10, # Keep top 10
+ min_relevance=0.3, # Minimum relevance score
+ prefer_diverse=True, # MMR-style diversity
+)
+
+retriever = ContextualCompressionRetriever(
+ base_compressor=compressor,
+ base_retriever=base_retriever,
+)
+
+# Retrieves 50 docs, returns best 10
+docs = retriever.invoke("What is Python?")
+```
+
+**Why this matters**: Vector search often returns many marginally-relevant documents. HeadroomDocumentCompressor uses BM25-style scoring to keep only the most relevant ones, reducing context size while improving answer quality.
+
+---
+
+### 4. Agent Tool Wrapping
+
+`wrap_tools_with_headroom` compresses tool outputs for agents:
+
+```python
+from langchain.agents import create_openai_tools_agent, AgentExecutor
+from langchain_core.tools import tool
+from headroom.integrations import wrap_tools_with_headroom
+
+@tool
+def search_database(query: str) -> str:
+ """Search the database."""
+ # Returns 1000 results as JSON
+ return json.dumps({"results": [...], "total": 1000})
+
+@tool
+def fetch_logs(service: str) -> str:
+ """Fetch service logs."""
+ # Returns 500 log entries
+ return json.dumps({"logs": [...]})
+
+# Wrap tools with compression
+tools = [search_database, fetch_logs]
+wrapped_tools = wrap_tools_with_headroom(
+ tools,
+ min_chars_to_compress=1000, # Only compress large outputs
+)
+
+# Create agent with wrapped tools
+agent = create_openai_tools_agent(llm, wrapped_tools, prompt)
+executor = AgentExecutor(agent=agent, tools=wrapped_tools)
+
+# Tool outputs are automatically compressed
+result = executor.invoke({"input": "Find users who logged in yesterday"})
+```
+
+**Per-tool metrics:**
+
+```python
+from headroom.integrations import get_tool_metrics
+
+metrics = get_tool_metrics()
+print(metrics.get_summary())
+# {
+# 'total_invocations': 25,
+# 'total_compressions': 18,
+# 'total_chars_saved': 450000,
+# 'by_tool': {
+# 'search_database': {'invocations': 15, 'chars_saved': 320000},
+# 'fetch_logs': {'invocations': 10, 'chars_saved': 130000},
+# }
+# }
+```
+
+---
+
+### 5. Streaming Metrics
+
+Track output tokens during streaming:
+
+```python
+from headroom.integrations import StreamingMetricsTracker
+
+tracker = StreamingMetricsTracker(model="gpt-4o")
+
+for chunk in llm.stream("Write a poem about coding"):
+ tracker.add_chunk(chunk)
+ print(chunk.content, end="", flush=True)
+
+metrics = tracker.finish()
+print(f"\nOutput tokens: {metrics.output_tokens}")
+print(f"Duration: {metrics.duration_ms:.0f}ms")
+```
+
+**Context manager style:**
+
+```python
+from headroom.integrations import StreamingMetricsCallback
+
+with StreamingMetricsCallback(model="gpt-4o") as tracker:
+ for chunk in llm.stream(messages):
+ tracker.add_chunk(chunk)
+ print(chunk.content, end="")
+
+print(f"Metrics: {tracker.metrics}")
+```
+
+---
+
+### 6. LangSmith Integration
+
+Add Headroom metrics to LangSmith traces:
+
+```python
+from headroom.integrations import HeadroomLangSmithCallbackHandler
+
+# Create callback handler
+langsmith_handler = HeadroomLangSmithCallbackHandler()
+
+# Use with your LLM
+llm = HeadroomChatModel(
+ ChatOpenAI(model="gpt-4o"),
+ callbacks=[langsmith_handler],
+)
+
+# After calls, metrics appear in LangSmith traces:
+# - headroom.tokens_before
+# - headroom.tokens_after
+# - headroom.tokens_saved
+# - headroom.compression_ratio
+```
+
+---
+
+## Real-World Examples
+
+### Example 1: LangGraph ReAct Agent
+
+The ReAct pattern is the most common agent architecture. Here's how to optimize it:
+
+```python
+from langchain_openai import ChatOpenAI
+from langchain_core.tools import tool
+from langgraph.prebuilt import create_react_agent
+from headroom.integrations import HeadroomChatModel, wrap_tools_with_headroom
+
+# Define tools that return large outputs
+@tool
+def search_web(query: str) -> str:
+ """Search the web for information."""
+ # Simulating large search results
+ return json.dumps({
+ "results": [
+ {"title": f"Result {i}", "snippet": "..." * 100, "url": f"https://..."}
+ for i in range(100)
+ ],
+ "total": 1000,
+ })
+
+@tool
+def query_database(sql: str) -> str:
+ """Execute SQL query."""
+ return json.dumps({
+ "rows": [{"id": i, "data": "..." * 50} for i in range(500)],
+ "total": 500,
+ })
+
+# Wrap model with Headroom
+llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))
+
+# Wrap tools with compression
+tools = wrap_tools_with_headroom([search_web, query_database])
+
+# Create ReAct agent
+agent = create_react_agent(llm, tools)
+
+# Run - tool outputs are automatically compressed between iterations
+result = agent.invoke({
+ "messages": [("user", "Find all users who signed up last week and their activity")]
+})
+
+# Check savings
+print(f"Tokens saved: {llm.get_metrics()['tokens_saved']}")
+```
+
+**Without Headroom**: Each tool call adds 10-50K tokens to context.
+**With Headroom**: Tool outputs compressed to 1-2K tokens, agent runs faster and cheaper.
+
+---
+
+### Example 2: RAG Pipeline with Document Filtering
+
+```python
+from langchain_openai import ChatOpenAI, OpenAIEmbeddings
+from langchain_community.vectorstores import Chroma
+from langchain.chains import RetrievalQA
+from langchain.retrievers import ContextualCompressionRetriever
+from headroom.integrations import HeadroomChatModel, HeadroomDocumentCompressor
+
+# Setup vector store
+embeddings = OpenAIEmbeddings()
+vectorstore = Chroma.from_documents(documents, embeddings)
+
+# High-recall retriever (get many candidates)
+base_retriever = vectorstore.as_retriever(search_kwargs={"k": 50})
+
+# Headroom compressor for precision
+compressor = HeadroomDocumentCompressor(
+ max_documents=5, # Keep only top 5
+ min_relevance=0.4, # Must be 40%+ relevant
+ prefer_diverse=True, # Avoid redundant docs
+)
+
+# Combine into compression retriever
+retriever = ContextualCompressionRetriever(
+ base_compressor=compressor,
+ base_retriever=base_retriever,
+)
+
+# Wrap LLM
+llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))
+
+# Create QA chain
+qa_chain = RetrievalQA.from_chain_type(
+ llm=llm,
+ retriever=retriever,
+ return_source_documents=True,
+)
+
+# Query - retrieves 50 docs, uses best 5
+result = qa_chain.invoke({"query": "How do I configure authentication?"})
+print(f"Answer: {result['result']}")
+print(f"Sources: {len(result['source_documents'])} docs")
+```
+
+**Impact**:
+- Without filtering: 50 docs × ~500 tokens = 25K context tokens
+- With Headroom: 5 docs × ~500 tokens = 2.5K context tokens (90% reduction)
+
+---
+
+### Example 3: Conversational Agent with Memory
+
+```python
+from langchain_openai import ChatOpenAI
+from langchain.memory import ConversationBufferMemory
+from langchain_community.chat_message_histories import ChatMessageHistory
+from langchain.chains import ConversationChain
+from headroom.integrations import HeadroomChatModel, HeadroomChatMessageHistory
+
+# Wrap LLM
+llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))
+
+# Wrap memory with auto-compression
+base_history = ChatMessageHistory()
+compressed_history = HeadroomChatMessageHistory(
+ base_history,
+ compress_threshold_tokens=8000, # Compress when over 8K
+ keep_recent_turns=10, # Always keep last 10 turns
+)
+
+memory = ConversationBufferMemory(
+ chat_memory=compressed_history,
+ return_messages=True,
+)
+
+# Create conversation chain
+chain = ConversationChain(llm=llm, memory=memory)
+
+# Long conversation - memory auto-compresses
+for i in range(100):
+ response = chain.invoke({"input": f"Tell me about topic {i}"})
+ print(f"Turn {i}: {len(response['response'])} chars")
+
+# Check memory stats
+print(compressed_history.get_compression_stats())
+# {'compression_count': 8, 'total_tokens_saved': 45000}
+```
+
+**Impact**: Without compression, 100-turn conversation = 100K+ tokens. With HeadroomChatMessageHistory, it stays under 8K tokens while preserving recent context.
+
+---
+
+### Example 4: Multi-Tool Research Agent
+
+```python
+from langchain_openai import ChatOpenAI
+from langchain.agents import AgentExecutor, create_openai_tools_agent
+from langchain_core.prompts import ChatPromptTemplate
+from langchain_core.tools import tool
+from headroom.integrations import (
+ HeadroomChatModel,
+ wrap_tools_with_headroom,
+ get_tool_metrics,
+ reset_tool_metrics,
+)
+
+@tool
+def search_arxiv(query: str) -> str:
+ """Search arXiv for papers."""
+ return json.dumps({"papers": [{"title": f"Paper {i}", "abstract": "..." * 200} for i in range(50)]})
+
+@tool
+def search_github(query: str) -> str:
+ """Search GitHub repositories."""
+ return json.dumps({"repos": [{"name": f"repo-{i}", "description": "..." * 100, "stars": i * 100} for i in range(100)]})
+
+@tool
+def fetch_documentation(url: str) -> str:
+ """Fetch documentation from URL."""
+ return "..." * 5000 # Large doc content
+
+# Wrap everything
+llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))
+tools = wrap_tools_with_headroom([search_arxiv, search_github, fetch_documentation])
+
+prompt = ChatPromptTemplate.from_messages([
+ ("system", "You are a research assistant. Use tools to gather information."),
+ ("human", "{input}"),
+ ("placeholder", "{agent_scratchpad}"),
+])
+
+agent = create_openai_tools_agent(llm, tools, prompt)
+executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
+
+# Reset metrics for this session
+reset_tool_metrics()
+
+# Run complex research task
+result = executor.invoke({
+ "input": "Research the latest advances in LLM context compression and find relevant GitHub projects"
+})
+
+# Check per-tool metrics
+metrics = get_tool_metrics().get_summary()
+print(f"Total chars saved: {metrics['total_chars_saved']:,}")
+print(f"Per-tool breakdown: {metrics['by_tool']}")
+```
+
+---
+
+## Configuration Options
+
+### HeadroomChatModel
+
+```python
+HeadroomChatModel(
+ wrapped_model, # Any LangChain BaseChatModel
+ headroom_config=HeadroomConfig(), # Headroom configuration
+ auto_detect_provider=True, # Auto-detect from wrapped model
+)
+```
+
+### HeadroomChatMessageHistory
+
+```python
+HeadroomChatMessageHistory(
+ base_history, # Any BaseChatMessageHistory
+ compress_threshold_tokens=4000, # Token threshold for compression
+ keep_recent_turns=5, # Minimum turns to preserve
+ model="gpt-4o", # Model for token counting
+)
+```
+
+### HeadroomDocumentCompressor
+
+```python
+HeadroomDocumentCompressor(
+ max_documents=10, # Maximum docs to return
+ min_relevance=0.0, # Minimum relevance score (0-1)
+ prefer_diverse=False, # Use MMR for diversity
+)
+```
+
+### wrap_tools_with_headroom
+
+```python
+wrap_tools_with_headroom(
+ tools, # List of LangChain tools
+ min_chars_to_compress=1000, # Minimum output size
+ smart_crusher_config=None, # SmartCrusher configuration
+)
+```
+
+---
+
+## Import Reference
+
+```python
+from headroom.integrations import (
+ # Chat Model
+ HeadroomChatModel,
+
+ # Memory
+ HeadroomChatMessageHistory,
+
+ # Retrievers
+ HeadroomDocumentCompressor,
+
+ # Agents
+ HeadroomToolWrapper,
+ wrap_tools_with_headroom,
+ get_tool_metrics,
+ reset_tool_metrics,
+
+ # Streaming
+ StreamingMetricsTracker,
+ StreamingMetricsCallback,
+ track_streaming_response,
+
+ # LangSmith
+ HeadroomLangSmithCallbackHandler,
+
+ # Provider Detection
+ detect_provider,
+ get_headroom_provider,
+)
+
+# Or import from subpackage directly
+from headroom.integrations.langchain import HeadroomChatModel
+from headroom.integrations.langchain.memory import HeadroomChatMessageHistory
+```
+
+---
+
+## Troubleshooting
+
+### LangChain not detected
+
+```python
+from headroom.integrations import langchain_available
+
+if not langchain_available():
+ print("Install with: pip install headroom-ai[langchain]")
+```
+
+### Provider detection failing
+
+```python
+# Force a specific provider
+from headroom.providers import AnthropicProvider
+
+llm = HeadroomChatModel(
+ ChatAnthropic(model="claude-3-5-sonnet-20241022"),
+ auto_detect_provider=False,
+)
+llm._provider = AnthropicProvider()
+```
+
+### Memory not compressing
+
+Check that your message count exceeds the threshold:
+
+```python
+history = HeadroomChatMessageHistory(
+ base_history,
+ compress_threshold_tokens=1000, # Lower threshold
+ keep_recent_turns=2, # Fewer preserved turns
+)
+```
+
+---
+
+## Performance Tips
+
+1. **Use tool wrapping for agents** - Agents with tools benefit most from compression
+2. **Set appropriate thresholds** - Don't compress small conversations
+3. **Enable diversity for RAG** - `prefer_diverse=True` improves answer quality
+4. **Monitor with LangSmith** - Use the callback handler to track savings over time
+5. **Batch similar requests** - Provider caching works better with stable prefixes
diff --git a/headroom/cache/compression_store.py b/headroom/cache/compression_store.py
index 5515d0bb7..1001256e5 100644
--- a/headroom/cache/compression_store.py
+++ b/headroom/cache/compression_store.py
@@ -292,7 +292,8 @@ class CompressionStore:
tool_signature_hash=entry.tool_signature_hash,
)
- # CRITICAL: Make a deep copy to return (entry could be modified/evicted after lock release)
+ # CRITICAL: Make a deep copy to return
+ # (entry could be modified/evicted after lock release)
# The entry contains mutable fields (search_queries list) that must be copied
result_entry = replace(entry, search_queries=list(entry.search_queries))
diff --git a/headroom/cache/dynamic_detector.py b/headroom/cache/dynamic_detector.py
index 276ab6c3c..7e53ef122 100644
--- a/headroom/cache/dynamic_detector.py
+++ b/headroom/cache/dynamic_detector.py
@@ -588,13 +588,19 @@ class NERDetector:
self._load_error: str | None = None
if not _SPACY_AVAILABLE:
- self._load_error = "spaCy not installed. Install with: pip install spacy && python -m spacy download en_core_web_sm"
+ self._load_error = (
+ "spaCy not installed. Install with: "
+ "pip install spacy && python -m spacy download en_core_web_sm"
+ )
return
try:
self._nlp = spacy.load(config.spacy_model)
except OSError:
- self._load_error = f"spaCy model '{config.spacy_model}' not found. Install with: python -m spacy download {config.spacy_model}"
+ self._load_error = (
+ f"spaCy model '{config.spacy_model}' not found. "
+ f"Install with: python -m spacy download {config.spacy_model}"
+ )
@property
def is_available(self) -> bool:
@@ -704,7 +710,10 @@ class SemanticDetector:
self._load_error: str | None = None
if not _SENTENCE_TRANSFORMERS_AVAILABLE:
- self._load_error = "sentence-transformers not installed. Install with: pip install sentence-transformers"
+ self._load_error = (
+ "sentence-transformers not installed. "
+ "Install with: pip install sentence-transformers"
+ )
return
try:
diff --git a/headroom/ccr/mcp_server.py b/headroom/ccr/mcp_server.py
index fe21d0074..e3a7e4c3a 100644
--- a/headroom/ccr/mcp_server.py
+++ b/headroom/ccr/mcp_server.py
@@ -109,9 +109,10 @@ class CCRMCPServer:
Tool(
name=CCR_TOOL_NAME,
description=(
- "Retrieve original uncompressed content that was compressed to save tokens. "
- "Use this when you need more data than what's shown in compressed tool results. "
- "The hash is provided in compression markers like [N items compressed... hash=abc123]."
+ "Retrieve original uncompressed content that was compressed "
+ "to save tokens. Use this when you need more data than what's "
+ "shown in compressed tool results. The hash is provided in "
+ "compression markers like [N items compressed... hash=abc123]."
),
inputSchema={
"type": "object",
diff --git a/headroom/integrations/__init__.py b/headroom/integrations/__init__.py
index 99e50f7ca..f97530a6f 100644
--- a/headroom/integrations/__init__.py
+++ b/headroom/integrations/__init__.py
@@ -1,18 +1,69 @@
"""Headroom integrations with popular LLM frameworks.
Available integrations:
-- LangChain: HeadroomChatModel, HeadroomCallbackHandler, optimize_messages
-- MCP: HeadroomMCPCompressor, compress_tool_result, HeadroomMCPClientWrapper
-Install LangChain support: pip install headroom[langchain]
+LangChain (pip install headroom[langchain]):
+ - HeadroomChatModel: Drop-in wrapper for any LangChain chat model
+ - HeadroomChatMessageHistory: Automatic conversation compression
+ - HeadroomDocumentCompressor: Relevance-based document filtering
+ - HeadroomToolWrapper: Tool output compression for agents
+ - StreamingMetricsTracker: Token counting during streaming
+ - HeadroomLangSmithCallbackHandler: LangSmith trace enrichment
+
+MCP (Model Context Protocol):
+ - HeadroomMCPCompressor: Compress MCP tool results
+ - compress_tool_result: Simple function for tool compression
+
+Example:
+ # LangChain integration
+ from headroom.integrations import HeadroomChatModel
+ # or explicitly:
+ from headroom.integrations.langchain import HeadroomChatModel
+
+ # MCP integration
+ from headroom.integrations import compress_tool_result
+ # or explicitly:
+ from headroom.integrations.mcp import compress_tool_result
"""
+# Re-export from langchain subpackage for backwards compatibility
from .langchain import (
+ # Retrievers
+ CompressionMetrics,
+ # Core
HeadroomCallbackHandler,
+ # Memory
+ HeadroomChatMessageHistory,
HeadroomChatModel,
+ HeadroomDocumentCompressor,
+ # LangSmith
+ HeadroomLangSmithCallbackHandler,
HeadroomRunnable,
+ # Agents
+ HeadroomToolWrapper,
+ OptimizationMetrics,
+ # Streaming
+ StreamingMetrics,
+ StreamingMetricsCallback,
+ StreamingMetricsTracker,
+ ToolCompressionMetrics,
+ ToolMetricsCollector,
+ # Provider Detection
+ detect_provider,
+ get_headroom_provider,
+ get_model_name_from_langchain,
+ get_tool_metrics,
+ is_langsmith_available,
+ is_langsmith_tracing_enabled,
+ langchain_available,
optimize_messages,
+ reset_tool_metrics,
+ track_async_streaming_response,
+ track_streaming_response,
+ wrap_tools_with_headroom,
)
+
+# Re-export from mcp subpackage for backwards compatibility
from .mcp import (
DEFAULT_MCP_PROFILES,
HeadroomMCPClientWrapper,
@@ -25,11 +76,39 @@ from .mcp import (
)
__all__ = [
- # LangChain
+ # LangChain Core
"HeadroomChatModel",
"HeadroomCallbackHandler",
- "optimize_messages",
"HeadroomRunnable",
+ "OptimizationMetrics",
+ "optimize_messages",
+ "langchain_available",
+ # Provider Detection
+ "detect_provider",
+ "get_headroom_provider",
+ "get_model_name_from_langchain",
+ # Memory
+ "HeadroomChatMessageHistory",
+ # Retrievers
+ "HeadroomDocumentCompressor",
+ "CompressionMetrics",
+ # Agents
+ "HeadroomToolWrapper",
+ "ToolCompressionMetrics",
+ "ToolMetricsCollector",
+ "wrap_tools_with_headroom",
+ "get_tool_metrics",
+ "reset_tool_metrics",
+ # LangSmith
+ "HeadroomLangSmithCallbackHandler",
+ "is_langsmith_available",
+ "is_langsmith_tracing_enabled",
+ # Streaming
+ "StreamingMetricsTracker",
+ "StreamingMetricsCallback",
+ "StreamingMetrics",
+ "track_streaming_response",
+ "track_async_streaming_response",
# MCP
"HeadroomMCPCompressor",
"HeadroomMCPClientWrapper",
diff --git a/headroom/integrations/langchain/__init__.py b/headroom/integrations/langchain/__init__.py
new file mode 100644
index 000000000..03b3a8e75
--- /dev/null
+++ b/headroom/integrations/langchain/__init__.py
@@ -0,0 +1,106 @@
+"""LangChain integration for Headroom.
+
+This package provides seamless integration with LangChain, including:
+- HeadroomChatModel: Drop-in wrapper for any LangChain chat model
+- HeadroomChatMessageHistory: Automatic conversation compression
+- HeadroomDocumentCompressor: Relevance-based document filtering
+- HeadroomToolWrapper: Tool output compression for agents
+- StreamingMetricsTracker: Token counting during streaming
+- HeadroomLangSmithCallbackHandler: LangSmith trace enrichment
+
+Example:
+ from langchain_openai import ChatOpenAI
+ from headroom.integrations.langchain import HeadroomChatModel
+
+ # Wrap any LangChain model
+ llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))
+
+ # Use like normal - optimization happens automatically
+ response = llm.invoke("Hello!")
+
+Install: pip install headroom[langchain]
+"""
+
+# Core chat model wrapper
+# Agent tool wrapping
+from .agents import (
+ HeadroomToolWrapper,
+ ToolCompressionMetrics,
+ ToolMetricsCollector,
+ get_tool_metrics,
+ reset_tool_metrics,
+ wrap_tools_with_headroom,
+)
+from .chat_model import (
+ HeadroomCallbackHandler,
+ HeadroomChatModel,
+ HeadroomRunnable,
+ OptimizationMetrics,
+ langchain_available,
+ optimize_messages,
+)
+
+# LangSmith integration
+from .langsmith import (
+ HeadroomLangSmithCallbackHandler,
+ is_langsmith_available,
+ is_langsmith_tracing_enabled,
+)
+
+# Memory integration
+from .memory import HeadroomChatMessageHistory
+
+# Provider auto-detection
+from .providers import (
+ detect_provider,
+ get_headroom_provider,
+ get_model_name_from_langchain,
+)
+
+# Retriever integration
+from .retriever import CompressionMetrics, HeadroomDocumentCompressor
+
+# Streaming metrics
+from .streaming import (
+ StreamingMetrics,
+ StreamingMetricsCallback,
+ StreamingMetricsTracker,
+ track_async_streaming_response,
+ track_streaming_response,
+)
+
+__all__ = [
+ # Core
+ "HeadroomChatModel",
+ "HeadroomCallbackHandler",
+ "HeadroomRunnable",
+ "OptimizationMetrics",
+ "optimize_messages",
+ "langchain_available",
+ # Provider Detection
+ "detect_provider",
+ "get_headroom_provider",
+ "get_model_name_from_langchain",
+ # Memory
+ "HeadroomChatMessageHistory",
+ # Retrievers
+ "HeadroomDocumentCompressor",
+ "CompressionMetrics",
+ # Agents
+ "HeadroomToolWrapper",
+ "ToolCompressionMetrics",
+ "ToolMetricsCollector",
+ "wrap_tools_with_headroom",
+ "get_tool_metrics",
+ "reset_tool_metrics",
+ # LangSmith
+ "HeadroomLangSmithCallbackHandler",
+ "is_langsmith_available",
+ "is_langsmith_tracing_enabled",
+ # Streaming
+ "StreamingMetricsTracker",
+ "StreamingMetricsCallback",
+ "StreamingMetrics",
+ "track_streaming_response",
+ "track_async_streaming_response",
+]
diff --git a/headroom/integrations/langchain/agents.py b/headroom/integrations/langchain/agents.py
new file mode 100644
index 000000000..409e5999f
--- /dev/null
+++ b/headroom/integrations/langchain/agents.py
@@ -0,0 +1,326 @@
+"""Agent tool integration for LangChain with output compression.
+
+This module provides HeadroomToolWrapper and wrap_tools_with_headroom
+for wrapping LangChain tools to automatically compress their outputs
+and track per-tool compression metrics.
+
+Example:
+ from langchain.agents import create_openai_tools_agent
+ from langchain.tools import Tool
+ from headroom.integrations import wrap_tools_with_headroom
+
+ # Define tools
+ tools = [
+ Tool(name="search", func=search_func, description="Search"),
+ Tool(name="database", func=db_func, description="Query DB"),
+ ]
+
+ # Wrap with Headroom compression
+ wrapped_tools = wrap_tools_with_headroom(tools)
+
+ # Use in agent - outputs are automatically compressed
+ agent = create_openai_tools_agent(llm, wrapped_tools, prompt)
+"""
+
+from __future__ import annotations
+
+import logging
+from dataclasses import dataclass, field
+from datetime import datetime
+from typing import Any
+
+# LangChain imports - these are optional dependencies
+try:
+ from langchain_core.tools import BaseTool, StructuredTool, Tool
+
+ LANGCHAIN_AVAILABLE = True
+except ImportError:
+ LANGCHAIN_AVAILABLE = False
+ BaseTool = object # type: ignore[misc,assignment]
+ StructuredTool = object # type: ignore[misc,assignment]
+ Tool = object # type: ignore[misc,assignment]
+
+from headroom.integrations.mcp import compress_tool_result
+
+logger = logging.getLogger(__name__)
+
+
+def _check_langchain_available() -> None:
+ """Raise ImportError if LangChain is not installed."""
+ if not LANGCHAIN_AVAILABLE:
+ raise ImportError(
+ "LangChain is required for this integration. "
+ "Install with: pip install headroom[langchain] "
+ "or: pip install langchain-core"
+ )
+
+
+@dataclass
+class ToolCompressionMetrics:
+ """Metrics from a single tool compression."""
+
+ tool_name: str
+ timestamp: datetime
+ chars_before: int
+ chars_after: int
+ chars_saved: int
+ compression_ratio: float
+ was_compressed: bool
+
+
+@dataclass
+class ToolMetricsCollector:
+ """Collects compression metrics across all tool invocations."""
+
+ metrics: list[ToolCompressionMetrics] = field(default_factory=list)
+
+ def add(self, metric: ToolCompressionMetrics) -> None:
+ """Add a metric entry."""
+ self.metrics.append(metric)
+ # Keep only last 1000
+ if len(self.metrics) > 1000:
+ self.metrics = self.metrics[-1000:]
+
+ def get_summary(self) -> dict[str, Any]:
+ """Get summary statistics."""
+ if not self.metrics:
+ return {
+ "total_invocations": 0,
+ "total_compressions": 0,
+ "total_chars_saved": 0,
+ }
+
+ compressed = [m for m in self.metrics if m.was_compressed]
+ return {
+ "total_invocations": len(self.metrics),
+ "total_compressions": len(compressed),
+ "total_chars_saved": sum(m.chars_saved for m in self.metrics),
+ "average_compression_ratio": (
+ sum(m.compression_ratio for m in compressed) / len(compressed) if compressed else 0
+ ),
+ "by_tool": self._get_by_tool_stats(),
+ }
+
+ def _get_by_tool_stats(self) -> dict[str, dict[str, Any]]:
+ """Get per-tool statistics."""
+ by_tool: dict[str, list[ToolCompressionMetrics]] = {}
+ for m in self.metrics:
+ if m.tool_name not in by_tool:
+ by_tool[m.tool_name] = []
+ by_tool[m.tool_name].append(m)
+
+ result = {}
+ for name, tool_metrics in by_tool.items():
+ compressed = [m for m in tool_metrics if m.was_compressed]
+ result[name] = {
+ "invocations": len(tool_metrics),
+ "compressions": len(compressed),
+ "chars_saved": sum(m.chars_saved for m in tool_metrics),
+ }
+ return result
+
+
+# Global metrics collector
+_global_metrics = ToolMetricsCollector()
+
+
+def get_tool_metrics() -> ToolMetricsCollector:
+ """Get the global tool metrics collector."""
+ return _global_metrics
+
+
+def reset_tool_metrics() -> None:
+ """Reset global tool metrics."""
+ global _global_metrics
+ _global_metrics = ToolMetricsCollector()
+
+
+class HeadroomToolWrapper:
+ """Wraps a LangChain tool to compress its output.
+
+ Applies SmartCrusher compression to tool outputs, particularly
+ useful for tools that return large JSON arrays (search results,
+ database queries, etc.).
+
+ Example:
+ from langchain.tools import Tool
+ from headroom.integrations import HeadroomToolWrapper
+
+ def search(query: str) -> str:
+ # Returns large JSON with 1000 results
+ return json.dumps({"results": [...1000 items...]})
+
+ search_tool = Tool(name="search", func=search, description="Search")
+ wrapped = HeadroomToolWrapper(search_tool)
+
+ # Use wrapped tool - output automatically compressed
+ result = wrapped("python tutorials")
+
+ Attributes:
+ tool: The wrapped LangChain tool
+ min_chars_to_compress: Minimum output size to trigger compression
+ metrics_collector: Collector for compression metrics
+ """
+
+ def __init__(
+ self,
+ tool: BaseTool,
+ min_chars_to_compress: int = 1000,
+ metrics_collector: ToolMetricsCollector | None = None,
+ ):
+ """Initialize HeadroomToolWrapper.
+
+ Args:
+ tool: The LangChain BaseTool to wrap.
+ min_chars_to_compress: Minimum character count for output
+ before compression is applied. Default 1000.
+ metrics_collector: Collector for metrics. Uses global
+ collector if not specified.
+ """
+ _check_langchain_available()
+
+ self.tool = tool
+ self.min_chars_to_compress = min_chars_to_compress
+ self._metrics = metrics_collector or _global_metrics
+
+ # Copy tool metadata
+ self.name = tool.name
+ self.description = tool.description
+
+ def __call__(self, *args: Any, **kwargs: Any) -> str:
+ """Invoke the tool and compress output.
+
+ Args:
+ *args: Arguments to pass to the tool.
+ **kwargs: Keyword arguments to pass to the tool.
+
+ Returns:
+ Compressed tool output as string.
+ """
+ # Invoke underlying tool
+ result = self.tool.invoke(*args, **kwargs)
+
+ # Convert to string if needed
+ if not isinstance(result, str):
+ result = str(result)
+
+ # Check if compression is needed
+ if len(result) < self.min_chars_to_compress:
+ self._record_metrics(result, result, was_compressed=False)
+ return result
+
+ # Try to compress
+ compressed = self._compress_output(result)
+ self._record_metrics(result, compressed, was_compressed=True)
+
+ return compressed
+
+ def invoke(self, *args: Any, **kwargs: Any) -> str:
+ """Invoke the tool (alias for __call__)."""
+ return self(*args, **kwargs)
+
+ def _compress_output(self, output: str) -> str:
+ """Apply compression to tool output.
+
+ Args:
+ output: Tool output string.
+
+ Returns:
+ Compressed output.
+ """
+ try:
+ return compress_tool_result(
+ content=output,
+ tool_name=self.name,
+ )
+ except Exception as e:
+ logger.debug(f"Tool compression failed: {e}")
+ return output
+
+ def _record_metrics(self, original: str, compressed: str, was_compressed: bool) -> None:
+ """Record compression metrics.
+
+ Args:
+ original: Original output.
+ compressed: Compressed output.
+ was_compressed: Whether compression was applied.
+ """
+ chars_before = len(original)
+ chars_after = len(compressed)
+ chars_saved = chars_before - chars_after
+
+ metric = ToolCompressionMetrics(
+ tool_name=self.name,
+ timestamp=datetime.now(),
+ chars_before=chars_before,
+ chars_after=chars_after,
+ chars_saved=max(0, chars_saved),
+ compression_ratio=chars_after / chars_before if chars_before > 0 else 1.0,
+ was_compressed=was_compressed and chars_saved > 0,
+ )
+
+ self._metrics.add(metric)
+
+ if was_compressed and chars_saved > 0:
+ logger.info(
+ f"HeadroomToolWrapper[{self.name}]: {chars_before} -> {chars_after} chars "
+ f"({chars_saved} saved, {metric.compression_ratio:.1%} of original)"
+ )
+
+ def as_langchain_tool(self) -> StructuredTool:
+ """Convert wrapper back to a LangChain tool.
+
+ Useful when you need to pass the wrapped tool to APIs
+ that expect a LangChain tool type.
+
+ Returns:
+ StructuredTool that wraps this wrapper.
+ """
+ return StructuredTool.from_function(
+ func=self.__call__,
+ name=self.name,
+ description=self.description,
+ )
+
+
+def wrap_tools_with_headroom(
+ tools: list[BaseTool],
+ min_chars_to_compress: int = 1000,
+ metrics_collector: ToolMetricsCollector | None = None,
+) -> list[StructuredTool]:
+ """Wrap multiple LangChain tools with Headroom compression.
+
+ Convenience function to wrap all tools in a list at once.
+
+ Args:
+ tools: List of LangChain tools to wrap.
+ min_chars_to_compress: Minimum output size for compression.
+ metrics_collector: Shared metrics collector for all tools.
+
+ Returns:
+ List of wrapped tools as StructuredTools.
+
+ Example:
+ from langchain.tools import Tool
+ from headroom.integrations import wrap_tools_with_headroom
+
+ tools = [search_tool, database_tool, api_tool]
+ wrapped = wrap_tools_with_headroom(tools)
+
+ # Use wrapped tools in agent
+ agent = create_openai_tools_agent(llm, wrapped, prompt)
+ """
+ _check_langchain_available()
+
+ collector = metrics_collector or _global_metrics
+
+ wrapped = []
+ for tool in tools:
+ wrapper = HeadroomToolWrapper(
+ tool=tool,
+ min_chars_to_compress=min_chars_to_compress,
+ metrics_collector=collector,
+ )
+ wrapped.append(wrapper.as_langchain_tool())
+
+ return wrapped
diff --git a/headroom/integrations/langchain.py b/headroom/integrations/langchain/chat_model.py
similarity index 87%
rename from headroom/integrations/langchain.py
rename to headroom/integrations/langchain/chat_model.py
index 38afb9af3..d05a1c759 100644
--- a/headroom/integrations/langchain.py
+++ b/headroom/integrations/langchain/chat_model.py
@@ -27,9 +27,10 @@ Example:
from __future__ import annotations
+import asyncio
import json
import logging
-from collections.abc import Iterator, Sequence
+from collections.abc import AsyncIterator, Iterator, Sequence
from dataclasses import dataclass
from datetime import datetime
from typing import Any
@@ -48,13 +49,14 @@ try:
)
from langchain_core.outputs import ChatGeneration, ChatResult
from langchain_core.runnables import RunnableLambda
- from pydantic import Field, PrivateAttr
+ from pydantic import ConfigDict, Field, PrivateAttr
LANGCHAIN_AVAILABLE = True
except ImportError:
LANGCHAIN_AVAILABLE = False
BaseChatModel = object
BaseCallbackHandler = object
+ ConfigDict = lambda **kwargs: {} # type: ignore[assignment,misc] # noqa: E731
Field = lambda **kwargs: None # type: ignore[assignment] # noqa: E731
PrivateAttr = lambda **kwargs: None # type: ignore[assignment] # noqa: E731
@@ -62,10 +64,12 @@ from headroom import HeadroomConfig, HeadroomMode
from headroom.providers import OpenAIProvider
from headroom.transforms import TransformPipeline
+from .providers import get_headroom_provider, get_model_name_from_langchain
+
logger = logging.getLogger(__name__)
-def _check_langchain_available():
+def _check_langchain_available() -> None:
"""Raise ImportError if LangChain is not installed."""
if not LANGCHAIN_AVAILABLE:
raise ImportError(
@@ -133,6 +137,10 @@ class HeadroomChatModel(BaseChatModel):
wrapped_model: Any = Field(description="The wrapped LangChain chat model")
headroom_config: Any = Field(default=None, description="Headroom configuration")
mode: HeadroomMode = Field(default=HeadroomMode.OPTIMIZE, description="Headroom mode")
+ auto_detect_provider: bool = Field(
+ default=True,
+ description="Auto-detect provider from wrapped model (OpenAI, Anthropic, Google)",
+ )
# Private attributes (not serialized)
_metrics_history: list = PrivateAttr(default_factory=list)
@@ -140,24 +148,27 @@ class HeadroomChatModel(BaseChatModel):
_pipeline: Any = PrivateAttr(default=None)
_provider: Any = PrivateAttr(default=None)
- class Config:
- """Pydantic config for LangChain compatibility."""
-
- arbitrary_types_allowed = True
+ # Pydantic v2 config for LangChain compatibility
+ model_config = ConfigDict(arbitrary_types_allowed=True)
def __init__(
self,
wrapped_model: BaseChatModel,
config: HeadroomConfig | None = None,
mode: HeadroomMode = HeadroomMode.OPTIMIZE,
- **kwargs,
- ):
+ auto_detect_provider: bool = True,
+ **kwargs: Any,
+ ) -> None:
"""Initialize HeadroomChatModel.
Args:
wrapped_model: Any LangChain BaseChatModel to wrap
config: HeadroomConfig for optimization settings
mode: HeadroomMode (AUDIT, OPTIMIZE, or SIMULATE)
+ auto_detect_provider: Auto-detect provider from wrapped model.
+ When True (default), automatically detects if the wrapped model
+ is OpenAI, Anthropic, Google, etc. and uses the appropriate
+ Headroom provider for accurate token counting.
**kwargs: Additional arguments passed to BaseChatModel
"""
_check_langchain_available()
@@ -166,6 +177,7 @@ class HeadroomChatModel(BaseChatModel):
wrapped_model=wrapped_model,
headroom_config=config or HeadroomConfig(),
mode=mode,
+ auto_detect_provider=auto_detect_provider,
**kwargs,
)
self._metrics_history = []
@@ -188,9 +200,17 @@ class HeadroomChatModel(BaseChatModel):
@property
def pipeline(self) -> TransformPipeline:
- """Lazily initialize TransformPipeline."""
+ """Lazily initialize TransformPipeline.
+
+ When auto_detect_provider is True, automatically detects the provider
+ from the wrapped model's class path (e.g., ChatAnthropic -> AnthropicProvider).
+ """
if self._pipeline is None:
- self._provider = OpenAIProvider()
+ if self.auto_detect_provider:
+ self._provider = get_headroom_provider(self.wrapped_model)
+ logger.debug(f"Auto-detected provider: {self._provider.__class__.__name__}")
+ else:
+ self._provider = OpenAIProvider()
self._pipeline = TransformPipeline(
config=self.headroom_config,
provider=self._provider,
@@ -290,10 +310,11 @@ class HeadroomChatModel(BaseChatModel):
# Convert to OpenAI format
openai_messages = self._convert_messages_to_openai(messages)
- # Get model name and context limit
- model = getattr(self.wrapped_model, "model_name", None)
- if model is None:
- model = getattr(self.wrapped_model, "model", "gpt-4o")
+ # Get model name from wrapped model
+ model = get_model_name_from_langchain(self.wrapped_model)
+
+ # Ensure pipeline is initialized (this also sets up provider)
+ _ = self.pipeline
# Get model context limit from provider
model_limit = self._provider.get_context_limit(model) if self._provider else 128000
@@ -342,7 +363,7 @@ class HeadroomChatModel(BaseChatModel):
messages: list[BaseMessage],
stop: list[str] | None = None,
run_manager: Any = None,
- **kwargs,
+ **kwargs: Any,
) -> ChatResult:
"""Generate response with Headroom optimization.
@@ -371,14 +392,15 @@ class HeadroomChatModel(BaseChatModel):
messages: list[BaseMessage],
stop: list[str] | None = None,
run_manager: Any = None,
- **kwargs,
+ **kwargs: Any,
) -> Iterator[ChatGeneration]:
"""Stream response with Headroom optimization."""
# Optimize messages
optimized_messages, metrics = self._optimize_messages(messages)
logger.info(
- f"Headroom optimized (streaming): {metrics.tokens_before} -> {metrics.tokens_after} tokens"
+ f"Headroom optimized (streaming): {metrics.tokens_before} -> "
+ f"{metrics.tokens_after} tokens"
)
# Stream from wrapped model
@@ -389,13 +411,78 @@ class HeadroomChatModel(BaseChatModel):
**kwargs,
)
- def bind_tools(self, tools: Sequence[Any], **kwargs) -> HeadroomChatModel:
+ async def _agenerate(
+ self,
+ messages: list[BaseMessage],
+ stop: list[str] | None = None,
+ run_manager: Any = None,
+ **kwargs: Any,
+ ) -> ChatResult:
+ """Async generate response with Headroom optimization.
+
+ This enables `await model.ainvoke(messages)` to work correctly.
+ The optimization step runs in a thread executor since it's CPU-bound.
+ """
+ # Run optimization in executor (CPU-bound)
+ loop = asyncio.get_event_loop()
+ optimized_messages, metrics = await loop.run_in_executor(
+ None, self._optimize_messages, messages
+ )
+
+ logger.info(
+ f"Headroom optimized (async): {metrics.tokens_before} -> {metrics.tokens_after} tokens "
+ f"({metrics.savings_percent:.1f}% saved)"
+ )
+
+ # Call wrapped model's async generate
+ result = await self.wrapped_model._agenerate(
+ optimized_messages,
+ stop=stop,
+ run_manager=run_manager,
+ **kwargs,
+ )
+
+ return result
+
+ async def _astream(
+ self,
+ messages: list[BaseMessage],
+ stop: list[str] | None = None,
+ run_manager: Any = None,
+ **kwargs: Any,
+ ) -> AsyncIterator[ChatGeneration]:
+ """Async stream response with Headroom optimization.
+
+ This enables `async for chunk in model.astream(messages)` to work correctly.
+ """
+ # Run optimization in executor (CPU-bound)
+ loop = asyncio.get_event_loop()
+ optimized_messages, metrics = await loop.run_in_executor(
+ None, self._optimize_messages, messages
+ )
+
+ logger.info(
+ f"Headroom optimized (async streaming): {metrics.tokens_before} -> "
+ f"{metrics.tokens_after} tokens"
+ )
+
+ # Async stream from wrapped model
+ async for chunk in self.wrapped_model._astream(
+ optimized_messages,
+ stop=stop,
+ run_manager=run_manager,
+ **kwargs,
+ ):
+ yield chunk
+
+ def bind_tools(self, tools: Sequence[Any], **kwargs: Any) -> HeadroomChatModel:
"""Bind tools to the wrapped model."""
new_wrapped = self.wrapped_model.bind_tools(tools, **kwargs)
return HeadroomChatModel(
wrapped_model=new_wrapped,
config=self.headroom_config,
mode=self.mode,
+ auto_detect_provider=self.auto_detect_provider,
)
def get_savings_summary(self) -> dict[str, Any]:
@@ -494,7 +581,7 @@ class HeadroomCallbackHandler(BaseCallbackHandler):
self,
serialized: dict[str, Any],
prompts: list[str],
- **kwargs,
+ **kwargs: Any,
) -> None:
"""Called when LLM starts processing."""
self._current_request = {
@@ -511,7 +598,7 @@ class HeadroomCallbackHandler(BaseCallbackHandler):
self,
serialized: dict[str, Any],
messages: list[list[BaseMessage]],
- **kwargs,
+ **kwargs: Any,
) -> None:
"""Called when chat model starts processing."""
# Estimate tokens from messages
@@ -532,7 +619,10 @@ class HeadroomCallbackHandler(BaseCallbackHandler):
# Check token alert
if self.token_alert_threshold and estimated_tokens > self.token_alert_threshold:
- alert = f"Token alert: {estimated_tokens} tokens exceeds threshold {self.token_alert_threshold}"
+ alert = (
+ f"Token alert: {estimated_tokens} tokens exceeds "
+ f"threshold {self.token_alert_threshold}"
+ )
self._alerts.append(alert)
logger.warning(alert)
@@ -542,7 +632,7 @@ class HeadroomCallbackHandler(BaseCallbackHandler):
f"Chat model request: ~{estimated_tokens} input tokens",
)
- def on_llm_end(self, response: Any, **kwargs) -> None:
+ def on_llm_end(self, response: Any, **kwargs: Any) -> None:
"""Called when LLM finishes processing."""
if self._current_request is None:
return
@@ -579,7 +669,7 @@ class HeadroomCallbackHandler(BaseCallbackHandler):
self._current_request = None
- def on_llm_error(self, error: Exception, **kwargs) -> None:
+ def on_llm_error(self, error: Exception, **kwargs: Any) -> None:
"""Called when LLM encounters an error."""
if self._current_request:
self._current_request["error"] = str(error)
@@ -677,19 +767,19 @@ class HeadroomRunnable:
)
return self._pipeline
- def __or__(self, other):
+ def __or__(self, other: Any) -> Any:
"""Support pipe operator for LCEL composition."""
from langchain_core.runnables import RunnableSequence
return RunnableSequence(first=self.as_runnable(), last=other)
- def __ror__(self, other):
+ def __ror__(self, other: Any) -> Any:
"""Support reverse pipe operator."""
from langchain_core.runnables import RunnableSequence
return RunnableSequence(first=other, last=self.as_runnable())
- def as_runnable(self):
+ def as_runnable(self) -> RunnableLambda:
"""Convert to LangChain Runnable."""
return RunnableLambda(self._optimize)
diff --git a/headroom/integrations/langchain/langsmith.py b/headroom/integrations/langchain/langsmith.py
new file mode 100644
index 000000000..6dfb4930f
--- /dev/null
+++ b/headroom/integrations/langchain/langsmith.py
@@ -0,0 +1,324 @@
+"""LangSmith integration for Headroom compression metrics.
+
+This module provides HeadroomLangSmithCallbackHandler, a LangChain callback
+handler that adds Headroom compression metrics to LangSmith traces.
+
+When used with HeadroomChatModel, it automatically captures:
+- Tokens before/after optimization
+- Savings percentage
+- Transforms applied
+- Per-request compression details
+
+Example:
+ import os
+ from langchain_openai import ChatOpenAI
+ from headroom.integrations import (
+ HeadroomChatModel,
+ HeadroomLangSmithCallbackHandler,
+ )
+
+ # Enable LangSmith tracing
+ os.environ["LANGCHAIN_TRACING_V2"] = "true"
+ os.environ["LANGCHAIN_API_KEY"] = "..."
+
+ # Create handler
+ handler = HeadroomLangSmithCallbackHandler()
+
+ # Use with HeadroomChatModel
+ llm = HeadroomChatModel(
+ ChatOpenAI(model="gpt-4o"),
+ callbacks=[handler],
+ )
+
+ # Traces will include headroom.* metadata
+ response = llm.invoke("Hello!")
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+from dataclasses import dataclass, field
+from datetime import datetime
+from typing import Any
+from uuid import UUID
+
+# LangChain imports - these are optional dependencies
+try:
+ from langchain_core.callbacks import BaseCallbackHandler
+ from langchain_core.messages import BaseMessage
+ from langchain_core.outputs import LLMResult
+
+ LANGCHAIN_AVAILABLE = True
+except ImportError:
+ LANGCHAIN_AVAILABLE = False
+ BaseCallbackHandler = object # type: ignore[misc,assignment]
+ LLMResult = object # type: ignore[misc,assignment]
+
+# LangSmith imports - optional
+try:
+ from langsmith import Client as LangSmithClient
+
+ LANGSMITH_AVAILABLE = True
+except ImportError:
+ LANGSMITH_AVAILABLE = False
+ LangSmithClient = None # type: ignore[misc,assignment]
+
+logger = logging.getLogger(__name__)
+
+
+def _check_langchain_available() -> None:
+ """Raise ImportError if LangChain is not installed."""
+ if not LANGCHAIN_AVAILABLE:
+ raise ImportError(
+ "LangChain is required for this integration. "
+ "Install with: pip install headroom[langchain] "
+ "or: pip install langchain-core"
+ )
+
+
+@dataclass
+class PendingMetrics:
+ """Metrics pending attachment to a LangSmith run."""
+
+ tokens_before: int
+ tokens_after: int
+ tokens_saved: int
+ savings_percent: float
+ transforms_applied: list[str]
+ timestamp: datetime = field(default_factory=datetime.now)
+
+
+class HeadroomLangSmithCallbackHandler(BaseCallbackHandler):
+ """Callback handler that adds Headroom metrics to LangSmith traces.
+
+ Integrates with LangSmith to provide visibility into context
+ optimization within traces. Metrics appear as metadata with
+ the `headroom.` prefix.
+
+ Works automatically when:
+ 1. LANGCHAIN_TRACING_V2=true is set
+ 2. Used as a callback with HeadroomChatModel
+ 3. LangSmith API key is configured
+
+ Example:
+ from headroom.integrations import (
+ HeadroomChatModel,
+ HeadroomLangSmithCallbackHandler,
+ )
+
+ handler = HeadroomLangSmithCallbackHandler()
+ llm = HeadroomChatModel(
+ ChatOpenAI(model="gpt-4o"),
+ callbacks=[handler],
+ )
+
+ response = llm.invoke("Hello!")
+ # LangSmith trace now includes:
+ # - headroom.tokens_before
+ # - headroom.tokens_after
+ # - headroom.tokens_saved
+ # - headroom.savings_percent
+ # - headroom.transforms_applied
+
+ Attributes:
+ langsmith_client: LangSmith client for updating runs.
+ pending_metrics: Metrics waiting to be attached to runs.
+ """
+
+ def __init__(
+ self,
+ langsmith_client: Any = None,
+ auto_update_runs: bool = True,
+ ):
+ """Initialize HeadroomLangSmithCallbackHandler.
+
+ Args:
+ langsmith_client: LangSmith client instance. Auto-creates
+ one if not provided and LangSmith is available.
+ auto_update_runs: If True, automatically updates LangSmith
+ runs with Headroom metadata. Default True.
+ """
+ _check_langchain_available()
+
+ self._client = langsmith_client
+ self._auto_update = auto_update_runs
+ self._pending_metrics: dict[str, PendingMetrics] = {}
+ self._run_metrics: dict[str, dict[str, Any]] = {}
+
+ # Initialize LangSmith client if available and not provided
+ if self._client is None and LANGSMITH_AVAILABLE and auto_update_runs:
+ try:
+ if os.environ.get("LANGCHAIN_API_KEY"):
+ self._client = LangSmithClient()
+ except Exception as e:
+ logger.debug(f"Could not initialize LangSmith client: {e}")
+
+ def set_headroom_metrics(
+ self,
+ run_id: str | UUID,
+ tokens_before: int,
+ tokens_after: int,
+ transforms_applied: list[str] | None = None,
+ ) -> None:
+ """Set Headroom metrics for a run.
+
+ Call this from HeadroomChatModel after optimization to attach
+ metrics to the current run.
+
+ Args:
+ run_id: The LangSmith run ID.
+ tokens_before: Token count before optimization.
+ tokens_after: Token count after optimization.
+ transforms_applied: List of transforms that were applied.
+ """
+ run_id_str = str(run_id)
+ tokens_saved = tokens_before - tokens_after
+ savings_percent = (tokens_saved / tokens_before * 100) if tokens_before > 0 else 0.0
+
+ metrics = PendingMetrics(
+ tokens_before=tokens_before,
+ tokens_after=tokens_after,
+ tokens_saved=tokens_saved,
+ savings_percent=savings_percent,
+ transforms_applied=transforms_applied or [],
+ )
+
+ self._pending_metrics[run_id_str] = metrics
+
+ logger.debug(
+ f"Headroom metrics set for run {run_id_str}: "
+ f"{tokens_before} -> {tokens_after} tokens ({savings_percent:.1f}% saved)"
+ )
+
+ def on_chat_model_start(
+ self,
+ serialized: dict[str, Any],
+ messages: list[list[BaseMessage]],
+ *,
+ run_id: UUID,
+ **kwargs: Any,
+ ) -> None:
+ """Called when chat model starts.
+
+ Records the run ID for later metric attachment.
+ """
+ run_id_str = str(run_id)
+ # Initialize empty metrics for this run
+ self._run_metrics[run_id_str] = {}
+
+ def on_llm_end(
+ self,
+ response: LLMResult,
+ *,
+ run_id: UUID,
+ **kwargs: Any,
+ ) -> None:
+ """Called when LLM completes.
+
+ Attaches pending Headroom metrics to the LangSmith run.
+ """
+ run_id_str = str(run_id)
+
+ # Check for pending metrics
+ if run_id_str in self._pending_metrics:
+ metrics = self._pending_metrics.pop(run_id_str)
+ self._attach_metrics_to_run(run_id_str, metrics)
+
+ def _attach_metrics_to_run(self, run_id: str, metrics: PendingMetrics) -> None:
+ """Attach Headroom metrics to a LangSmith run.
+
+ Args:
+ run_id: The run ID.
+ metrics: Metrics to attach.
+ """
+ metadata = {
+ "headroom.tokens_before": metrics.tokens_before,
+ "headroom.tokens_after": metrics.tokens_after,
+ "headroom.tokens_saved": metrics.tokens_saved,
+ "headroom.savings_percent": round(metrics.savings_percent, 2),
+ "headroom.transforms_applied": metrics.transforms_applied,
+ "headroom.optimization_timestamp": metrics.timestamp.isoformat(),
+ }
+
+ # Store in run metrics
+ self._run_metrics[run_id] = metadata
+
+ # Update LangSmith run if client available
+ if self._client and self._auto_update:
+ try:
+ self._client.update_run(
+ run_id=run_id,
+ extra={"metadata": metadata},
+ )
+ logger.debug(f"Updated LangSmith run {run_id} with Headroom metrics")
+ except Exception as e:
+ logger.debug(f"Could not update LangSmith run: {e}")
+
+ def get_run_metrics(self, run_id: str | UUID) -> dict[str, Any]:
+ """Get Headroom metrics for a specific run.
+
+ Args:
+ run_id: The run ID.
+
+ Returns:
+ Dictionary of headroom.* metrics for the run.
+ """
+ return self._run_metrics.get(str(run_id), {})
+
+ def get_all_metrics(self) -> dict[str, dict[str, Any]]:
+ """Get all recorded run metrics.
+
+ Returns:
+ Dictionary mapping run IDs to their metrics.
+ """
+ return self._run_metrics.copy()
+
+ def get_summary(self) -> dict[str, Any]:
+ """Get summary statistics across all runs.
+
+ Returns:
+ Summary with total runs, tokens saved, etc.
+ """
+ if not self._run_metrics:
+ return {
+ "total_runs": 0,
+ "total_tokens_saved": 0,
+ "average_savings_percent": 0,
+ }
+
+ total_saved = sum(m.get("headroom.tokens_saved", 0) for m in self._run_metrics.values())
+ savings_percents = [
+ m.get("headroom.savings_percent", 0) for m in self._run_metrics.values()
+ ]
+
+ return {
+ "total_runs": len(self._run_metrics),
+ "total_tokens_saved": total_saved,
+ "average_savings_percent": (
+ sum(savings_percents) / len(savings_percents) if savings_percents else 0
+ ),
+ }
+
+ def reset(self) -> None:
+ """Clear all recorded metrics."""
+ self._pending_metrics.clear()
+ self._run_metrics.clear()
+
+
+def is_langsmith_available() -> bool:
+ """Check if LangSmith is available and configured.
+
+ Returns:
+ True if LangSmith is installed and API key is set.
+ """
+ return LANGSMITH_AVAILABLE and bool(os.environ.get("LANGCHAIN_API_KEY"))
+
+
+def is_langsmith_tracing_enabled() -> bool:
+ """Check if LangSmith tracing is enabled.
+
+ Returns:
+ True if LANGCHAIN_TRACING_V2 is set to "true".
+ """
+ return os.environ.get("LANGCHAIN_TRACING_V2", "").lower() == "true"
diff --git a/headroom/integrations/langchain/memory.py b/headroom/integrations/langchain/memory.py
new file mode 100644
index 000000000..bf2d80758
--- /dev/null
+++ b/headroom/integrations/langchain/memory.py
@@ -0,0 +1,319 @@
+"""Memory integration for LangChain with automatic compression.
+
+This module provides HeadroomChatMessageHistory, a wrapper for any LangChain
+chat message history that automatically compresses conversation history
+when it exceeds a token threshold.
+
+Example:
+ from langchain.memory import ConversationBufferMemory
+ from langchain_community.chat_message_histories import ChatMessageHistory
+ from headroom.integrations import HeadroomChatMessageHistory
+
+ # Wrap any chat message history
+ base_history = ChatMessageHistory()
+ compressed_history = HeadroomChatMessageHistory(base_history)
+
+ # Use with ConversationBufferMemory (zero code changes to chain)
+ memory = ConversationBufferMemory(chat_memory=compressed_history)
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import TYPE_CHECKING, Any
+
+if TYPE_CHECKING:
+ from headroom.providers.base import Provider
+
+# LangChain imports - these are optional dependencies
+try:
+ from langchain_core.chat_history import BaseChatMessageHistory
+ from langchain_core.messages import (
+ AIMessage,
+ BaseMessage,
+ HumanMessage,
+ SystemMessage,
+ ToolMessage,
+ )
+
+ LANGCHAIN_AVAILABLE = True
+except ImportError:
+ LANGCHAIN_AVAILABLE = False
+ BaseChatMessageHistory = object # type: ignore[misc,assignment]
+
+from headroom import HeadroomConfig
+from headroom.config import RollingWindowConfig
+from headroom.providers import OpenAIProvider
+from headroom.transforms import TransformPipeline
+
+logger = logging.getLogger(__name__)
+
+
+def _check_langchain_available() -> None:
+ """Raise ImportError if LangChain is not installed."""
+ if not LANGCHAIN_AVAILABLE:
+ raise ImportError(
+ "LangChain is required for this integration. "
+ "Install with: pip install headroom[langchain] "
+ "or: pip install langchain-core"
+ )
+
+
+class HeadroomChatMessageHistory(BaseChatMessageHistory):
+ """Wraps any LangChain chat message history with automatic compression.
+
+ When conversation history exceeds the token threshold, automatically
+ applies RollingWindow compression to keep recent turns while fitting
+ within the limit.
+
+ This works with ANY memory type because it wraps at the storage layer:
+ - ConversationBufferMemory
+ - ConversationSummaryMemory
+ - ConversationBufferWindowMemory
+ - Redis, PostgreSQL, or any custom history
+
+ Example:
+ from langchain.memory import ConversationBufferMemory
+ from langchain_community.chat_message_histories import ChatMessageHistory
+ from headroom.integrations import HeadroomChatMessageHistory
+
+ # Wrap base history
+ base = ChatMessageHistory()
+ compressed = HeadroomChatMessageHistory(
+ base,
+ compress_threshold_tokens=4000,
+ keep_recent_turns=5,
+ )
+
+ # Use with any memory class
+ memory = ConversationBufferMemory(chat_memory=compressed)
+
+ # Messages are compressed automatically when accessed
+ chain = ConversationChain(llm=llm, memory=memory)
+ chain.invoke({"input": "Hello!"})
+
+ Attributes:
+ base_history: The underlying chat message history
+ compress_threshold_tokens: Token count that triggers compression
+ keep_recent_turns: Minimum recent turns to always preserve
+ model: Model name for token counting (default: "gpt-4o")
+ """
+
+ def __init__(
+ self,
+ base_history: BaseChatMessageHistory,
+ compress_threshold_tokens: int = 4000,
+ keep_recent_turns: int = 5,
+ model: str = "gpt-4o",
+ provider: Provider | None = None,
+ ):
+ """Initialize HeadroomChatMessageHistory.
+
+ Args:
+ base_history: Any LangChain BaseChatMessageHistory to wrap
+ compress_threshold_tokens: Apply compression when history exceeds
+ this many tokens. Default 4000.
+ keep_recent_turns: Minimum number of recent user/assistant turns
+ to always preserve during compression. Default 5.
+ model: Model name for token counting. Default "gpt-4o".
+ provider: Headroom provider for token counting. Auto-uses
+ OpenAIProvider if not specified.
+ """
+ _check_langchain_available()
+
+ self._base = base_history
+ self._threshold = compress_threshold_tokens
+ self._keep_recent_turns = keep_recent_turns
+ self._model = model
+ self._provider: Provider = provider or OpenAIProvider()
+
+ # Track compression stats
+ self._compression_count = 0
+ self._total_tokens_saved = 0
+
+ @property
+ def messages(self) -> list[BaseMessage]:
+ """Get messages, applying compression if over threshold.
+
+ Returns:
+ List of messages, potentially compressed to fit within threshold.
+ """
+ raw_messages = self._base.messages
+
+ if not raw_messages:
+ return []
+
+ # Count tokens
+ token_count = self._count_tokens(raw_messages)
+
+ if token_count <= self._threshold:
+ return list(raw_messages)
+
+ # Apply compression
+ compressed = self._apply_rolling_window(raw_messages)
+ tokens_after = self._count_tokens(compressed)
+
+ self._compression_count += 1
+ self._total_tokens_saved += token_count - tokens_after
+
+ logger.info(
+ f"HeadroomChatMessageHistory compressed: {token_count} -> {tokens_after} tokens "
+ f"({len(raw_messages)} -> {len(compressed)} messages)"
+ )
+
+ return compressed
+
+ def add_message(self, message: BaseMessage) -> None:
+ """Add a message to the underlying history.
+
+ Args:
+ message: The message to add.
+ """
+ self._base.add_message(message)
+
+ def add_user_message(self, message: str) -> None:
+ """Add a user message to the history.
+
+ Args:
+ message: The user message content.
+ """
+ self._base.add_user_message(message)
+
+ def add_ai_message(self, message: str) -> None:
+ """Add an AI message to the history.
+
+ Args:
+ message: The AI message content.
+ """
+ self._base.add_ai_message(message)
+
+ def clear(self) -> None:
+ """Clear all messages from history."""
+ self._base.clear()
+
+ def _count_tokens(self, messages: list[BaseMessage]) -> int:
+ """Count tokens in messages using provider's tokenizer.
+
+ Args:
+ messages: List of messages to count.
+
+ Returns:
+ Total token count.
+ """
+ token_counter = self._provider.get_token_counter(self._model)
+ total = 0
+ for msg in messages:
+ content = msg.content if isinstance(msg.content, str) else str(msg.content)
+ total += token_counter.count_text(content)
+ return total
+
+ def _apply_rolling_window(self, messages: list[BaseMessage]) -> list[BaseMessage]:
+ """Apply RollingWindow compression to messages.
+
+ Args:
+ messages: Messages to compress.
+
+ Returns:
+ Compressed messages fitting within threshold.
+ """
+ # Convert to OpenAI format for Headroom transforms
+ openai_messages = self._convert_to_openai(messages)
+
+ # Use TransformPipeline which handles tokenizer setup
+ config = HeadroomConfig(
+ rolling_window=RollingWindowConfig(keep_last_turns=self._keep_recent_turns),
+ )
+ pipeline = TransformPipeline(config=config, provider=self._provider)
+
+ # Apply compression via pipeline
+ result = pipeline.apply(
+ messages=openai_messages,
+ model=self._model,
+ model_limit=self._threshold,
+ )
+
+ # Convert back to LangChain format
+ return self._convert_from_openai(result.messages)
+
+ def _convert_to_openai(self, messages: list[BaseMessage]) -> list[dict[str, Any]]:
+ """Convert LangChain messages to OpenAI format.
+
+ Args:
+ messages: LangChain messages.
+
+ Returns:
+ OpenAI format messages.
+ """
+ result = []
+ for msg in messages:
+ content = msg.content if isinstance(msg.content, str) else str(msg.content)
+
+ if isinstance(msg, SystemMessage):
+ result.append({"role": "system", "content": content})
+ elif isinstance(msg, HumanMessage):
+ result.append({"role": "user", "content": content})
+ elif isinstance(msg, AIMessage):
+ entry: dict[str, Any] = {"role": "assistant", "content": content}
+ if hasattr(msg, "tool_calls") and msg.tool_calls:
+ entry["tool_calls"] = msg.tool_calls
+ result.append(entry)
+ elif isinstance(msg, ToolMessage):
+ result.append(
+ {
+ "role": "tool",
+ "tool_call_id": getattr(msg, "tool_call_id", ""),
+ "content": content,
+ }
+ )
+ else:
+ # Generic fallback
+ result.append(
+ {
+ "role": getattr(msg, "type", "user"),
+ "content": content,
+ }
+ )
+ return result
+
+ def _convert_from_openai(self, messages: list[dict[str, Any]]) -> list[BaseMessage]:
+ """Convert OpenAI format back to LangChain messages.
+
+ Args:
+ messages: OpenAI format messages.
+
+ Returns:
+ LangChain messages.
+ """
+ result: list[BaseMessage] = []
+ for msg in messages:
+ role = msg.get("role", "user")
+ content = msg.get("content", "")
+
+ if role == "system":
+ result.append(SystemMessage(content=content))
+ elif role == "user":
+ result.append(HumanMessage(content=content))
+ elif role == "assistant":
+ tool_calls = msg.get("tool_calls", [])
+ result.append(AIMessage(content=content, tool_calls=tool_calls))
+ elif role == "tool":
+ result.append(
+ ToolMessage(
+ content=content,
+ tool_call_id=msg.get("tool_call_id", ""),
+ )
+ )
+ return result
+
+ def get_compression_stats(self) -> dict[str, Any]:
+ """Get statistics about compression operations.
+
+ Returns:
+ Dictionary with compression_count, total_tokens_saved.
+ """
+ return {
+ "compression_count": self._compression_count,
+ "total_tokens_saved": self._total_tokens_saved,
+ "threshold_tokens": self._threshold,
+ "keep_recent_turns": self._keep_recent_turns,
+ }
diff --git a/headroom/integrations/langchain/providers.py b/headroom/integrations/langchain/providers.py
new file mode 100644
index 000000000..a0bb149ee
--- /dev/null
+++ b/headroom/integrations/langchain/providers.py
@@ -0,0 +1,200 @@
+"""Provider detection for LangChain models.
+
+This module provides automatic provider detection from LangChain chat models
+without requiring explicit provider imports. It uses duck-typing based on
+class paths to identify the appropriate Headroom provider.
+
+Example:
+ from langchain_anthropic import ChatAnthropic
+ from headroom.integrations.langchain import get_headroom_provider
+
+ model = ChatAnthropic(model="claude-3-5-sonnet-20241022")
+ provider = get_headroom_provider(model) # Returns AnthropicProvider
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import TYPE_CHECKING, Any
+
+if TYPE_CHECKING:
+ from headroom.providers.base import Provider
+
+logger = logging.getLogger(__name__)
+
+# Provider detection patterns
+# Maps provider name to list of class path patterns to match
+PROVIDER_PATTERNS: dict[str, list[str]] = {
+ "openai": [
+ "langchain_openai.ChatOpenAI",
+ "langchain_openai.chat_models.ChatOpenAI",
+ "langchain_community.chat_models.ChatOpenAI",
+ "langchain.chat_models.ChatOpenAI",
+ "ChatOpenAI",
+ ],
+ "anthropic": [
+ "langchain_anthropic.ChatAnthropic",
+ "langchain_anthropic.chat_models.ChatAnthropic",
+ "langchain_community.chat_models.ChatAnthropic",
+ "langchain.chat_models.ChatAnthropic",
+ "ChatAnthropic",
+ ],
+ "google": [
+ "langchain_google_genai.ChatGoogleGenerativeAI",
+ "langchain_google_genai.chat_models.ChatGoogleGenerativeAI",
+ "langchain_community.chat_models.ChatGoogleGenerativeAI",
+ "ChatGoogleGenerativeAI",
+ # Also match Vertex AI
+ "langchain_google_vertexai.ChatVertexAI",
+ "ChatVertexAI",
+ ],
+ "cohere": [
+ "langchain_cohere.ChatCohere",
+ "langchain_community.chat_models.ChatCohere",
+ "ChatCohere",
+ ],
+ "mistral": [
+ "langchain_mistralai.ChatMistralAI",
+ "langchain_community.chat_models.ChatMistralAI",
+ "ChatMistralAI",
+ ],
+}
+
+# Model name patterns for fallback detection
+MODEL_NAME_PATTERNS: dict[str, list[str]] = {
+ "anthropic": ["claude", "anthropic"],
+ "openai": ["gpt", "o1", "o3", "davinci", "turbo"],
+ "google": ["gemini", "palm", "bison"],
+ "cohere": ["command", "cohere"],
+ "mistral": ["mistral", "mixtral"],
+}
+
+
+def detect_provider(model: Any) -> str:
+ """Detect provider name from a LangChain model using duck-typing.
+
+ Detection strategy:
+ 1. Check class module and name against known patterns
+ 2. Check model_name attribute against known model patterns
+ 3. Fall back to "openai" as safe default
+
+ Args:
+ model: Any LangChain chat model instance
+
+ Returns:
+ Provider name string: "openai", "anthropic", "google", "cohere", "mistral"
+
+ Example:
+ >>> from langchain_anthropic import ChatAnthropic
+ >>> model = ChatAnthropic(model="claude-3-5-sonnet-20241022")
+ >>> detect_provider(model)
+ 'anthropic'
+ """
+ # Strategy 1: Check class path
+ class_module = getattr(model.__class__, "__module__", "")
+ class_name = model.__class__.__name__
+ class_path = f"{class_module}.{class_name}"
+
+ for provider_name, patterns in PROVIDER_PATTERNS.items():
+ for pattern in patterns:
+ if pattern in class_path or class_name == pattern.split(".")[-1]:
+ logger.debug(f"Detected provider '{provider_name}' from class path: {class_path}")
+ return provider_name
+
+ # Strategy 2: Check model_name attribute
+ model_name = _get_model_name(model)
+ if model_name:
+ model_name_lower = model_name.lower()
+ for provider_name, name_patterns in MODEL_NAME_PATTERNS.items():
+ for pattern in name_patterns:
+ if pattern in model_name_lower:
+ logger.debug(
+ f"Detected provider '{provider_name}' from model name: {model_name}"
+ )
+ return provider_name
+
+ # Strategy 3: Fall back to OpenAI (most common, safe default)
+ logger.debug(f"Could not detect provider for {class_path}, falling back to 'openai'")
+ return "openai"
+
+
+def _get_model_name(model: Any) -> str | None:
+ """Extract model name from a LangChain model.
+
+ Tries common attribute names used by different LangChain models.
+ """
+ # Try common attribute names
+ for attr in ["model_name", "model", "model_id", "_model_name"]:
+ value = getattr(model, attr, None)
+ if isinstance(value, str):
+ return value
+
+ return None
+
+
+def get_headroom_provider(model: Any) -> Provider:
+ """Get appropriate Headroom Provider instance for a LangChain model.
+
+ This function automatically detects the provider from the model type
+ and returns a configured Headroom provider for accurate token counting
+ and context limit detection.
+
+ Args:
+ model: Any LangChain chat model instance
+
+ Returns:
+ Configured Headroom Provider instance
+
+ Example:
+ >>> from langchain_anthropic import ChatAnthropic
+ >>> model = ChatAnthropic(model="claude-3-5-sonnet-20241022")
+ >>> provider = get_headroom_provider(model)
+ >>> provider.name
+ 'anthropic'
+ """
+ # Import providers lazily to avoid circular imports
+ from headroom.providers import (
+ AnthropicProvider,
+ GoogleProvider,
+ OpenAIProvider,
+ )
+
+ provider_name = detect_provider(model)
+
+ if provider_name == "anthropic":
+ return AnthropicProvider()
+ elif provider_name == "google":
+ return GoogleProvider()
+ # Cohere and Mistral fall back to OpenAI-compatible for now
+ # TODO: Add dedicated providers when needed
+
+ # Default to OpenAI
+ return OpenAIProvider()
+
+
+def get_model_name_from_langchain(model: Any) -> str:
+ """Extract the model name string from a LangChain model.
+
+ Useful for getting the model identifier for token counting
+ and context limit lookup.
+
+ Args:
+ model: Any LangChain chat model instance
+
+ Returns:
+ Model name string (e.g., "gpt-4o", "claude-3-5-sonnet-20241022")
+ """
+ name = _get_model_name(model)
+ if name:
+ return name
+
+ # Try to infer from class name
+ class_name = model.__class__.__name__
+ if "GPT" in class_name or "OpenAI" in class_name:
+ return "gpt-4o" # Safe default for OpenAI
+ elif "Anthropic" in class_name or "Claude" in class_name:
+ return "claude-3-5-sonnet-20241022" # Safe default for Anthropic
+ elif "Google" in class_name or "Gemini" in class_name:
+ return "gemini-1.5-pro" # Safe default for Google
+
+ return "gpt-4o" # Ultimate fallback
diff --git a/headroom/integrations/langchain/retriever.py b/headroom/integrations/langchain/retriever.py
new file mode 100644
index 000000000..15d0c1023
--- /dev/null
+++ b/headroom/integrations/langchain/retriever.py
@@ -0,0 +1,371 @@
+"""Retriever integration for LangChain with intelligent document compression.
+
+This module provides HeadroomDocumentCompressor, a LangChain BaseDocumentCompressor
+that reduces retrieved documents based on relevance scoring while preserving
+the most important information.
+
+Example:
+ from langchain.retrievers import ContextualCompressionRetriever
+ from langchain_community.vectorstores import Chroma
+ from headroom.integrations import HeadroomDocumentCompressor
+
+ # Create vector store retriever
+ vectorstore = Chroma.from_documents(documents, embeddings)
+ base_retriever = vectorstore.as_retriever(search_kwargs={"k": 50})
+
+ # Wrap with Headroom compression
+ compressor = HeadroomDocumentCompressor(max_documents=10)
+ retriever = ContextualCompressionRetriever(
+ base_compressor=compressor,
+ base_retriever=base_retriever,
+ )
+
+ # Retrieve - automatically keeps most relevant documents
+ docs = retriever.invoke("What is the capital of France?")
+"""
+
+from __future__ import annotations
+
+import logging
+import re
+from collections.abc import Sequence
+from dataclasses import dataclass
+from typing import Any
+
+# LangChain imports - these are optional dependencies
+try:
+ from langchain_core.callbacks import Callbacks
+ from langchain_core.documents import Document
+
+ # BaseDocumentCompressor location varies by langchain version
+ try:
+ from langchain.retrievers.document_compressors import BaseDocumentCompressor
+ except ImportError:
+ try:
+ from langchain_core.documents.compressors import BaseDocumentCompressor
+ except ImportError:
+ # Fallback: create a minimal base class
+ class BaseDocumentCompressor: # type: ignore[no-redef]
+ """Minimal base class for document compression."""
+
+ def compress_documents(
+ self, documents: Sequence[Any], query: str, callbacks: Any = None
+ ) -> Sequence[Any]:
+ raise NotImplementedError
+
+ LANGCHAIN_AVAILABLE = True
+except ImportError:
+ LANGCHAIN_AVAILABLE = False
+ BaseDocumentCompressor = object # type: ignore[misc,assignment]
+ Document = object # type: ignore[misc,assignment]
+ Callbacks = None # type: ignore[misc,assignment]
+
+logger = logging.getLogger(__name__)
+
+
+def _check_langchain_available() -> None:
+ """Raise ImportError if LangChain is not installed."""
+ if not LANGCHAIN_AVAILABLE:
+ raise ImportError(
+ "LangChain is required for this integration. "
+ "Install with: pip install headroom[langchain] "
+ "or: pip install langchain-core"
+ )
+
+
+@dataclass
+class CompressionMetrics:
+ """Metrics from document compression."""
+
+ documents_before: int
+ documents_after: int
+ documents_removed: int
+ relevance_scores: list[float]
+
+
+class HeadroomDocumentCompressor(BaseDocumentCompressor):
+ """Compresses retrieved documents based on relevance to query.
+
+ Uses BM25-style relevance scoring to keep only the most relevant
+ documents from a larger retrieval set. This allows you to retrieve
+ many documents initially (for recall) and then compress down to
+ the most relevant ones (for precision).
+
+ Works with LangChain's ContextualCompressionRetriever pattern.
+
+ Example:
+ from langchain.retrievers import ContextualCompressionRetriever
+ from headroom.integrations import HeadroomDocumentCompressor
+
+ compressor = HeadroomDocumentCompressor(
+ max_documents=10,
+ min_relevance=0.3,
+ )
+
+ retriever = ContextualCompressionRetriever(
+ base_compressor=compressor,
+ base_retriever=base_retriever, # Any retriever
+ )
+
+ # Retrieves top 10 most relevant docs
+ docs = retriever.invoke("What is Python?")
+
+ Attributes:
+ max_documents: Maximum documents to return
+ min_relevance: Minimum relevance score (0-1) to include
+ prefer_diverse: Whether to prefer diverse results
+ """
+
+ max_documents: int = 10
+ min_relevance: float = 0.0
+ prefer_diverse: bool = False
+
+ def __init__(
+ self,
+ max_documents: int = 10,
+ min_relevance: float = 0.0,
+ prefer_diverse: bool = False,
+ **kwargs: Any,
+ ):
+ """Initialize HeadroomDocumentCompressor.
+
+ Args:
+ max_documents: Maximum number of documents to return. Default 10.
+ min_relevance: Minimum relevance score (0-1) for a document to
+ be included. Default 0.0 (no minimum).
+ prefer_diverse: If True, use MMR-style selection to prefer
+ diverse results over pure relevance. Default False.
+ **kwargs: Additional arguments for BaseDocumentCompressor.
+ """
+ _check_langchain_available()
+
+ super().__init__(**kwargs)
+ self.max_documents = max_documents
+ self.min_relevance = min_relevance
+ self.prefer_diverse = prefer_diverse
+ self._last_metrics: CompressionMetrics | None = None
+
+ def compress_documents(
+ self,
+ documents: Sequence[Document],
+ query: str,
+ callbacks: Callbacks = None,
+ ) -> Sequence[Document]:
+ """Compress documents based on relevance to query.
+
+ Args:
+ documents: Documents to compress.
+ query: Query to score relevance against.
+ callbacks: LangChain callbacks (unused).
+
+ Returns:
+ Compressed list of most relevant documents.
+ """
+ if not documents:
+ self._last_metrics = CompressionMetrics(
+ documents_before=0,
+ documents_after=0,
+ documents_removed=0,
+ relevance_scores=[],
+ )
+ return []
+
+ if len(documents) <= self.max_documents:
+ # No compression needed
+ scores = [self._score_document(doc, query) for doc in documents]
+ self._last_metrics = CompressionMetrics(
+ documents_before=len(documents),
+ documents_after=len(documents),
+ documents_removed=0,
+ relevance_scores=scores,
+ )
+ return list(documents)
+
+ # Score all documents
+ scored = [(doc, self._score_document(doc, query)) for doc in documents]
+
+ if self.prefer_diverse:
+ # Use MMR-style selection for diversity
+ selected = self._select_diverse(scored, query)
+ else:
+ # Sort by relevance score
+ scored.sort(key=lambda x: x[1], reverse=True)
+ selected = scored[: self.max_documents]
+
+ # Filter by minimum relevance
+ if self.min_relevance > 0:
+ selected = [(doc, score) for doc, score in selected if score >= self.min_relevance]
+
+ # Track metrics
+ final_docs = [doc for doc, _ in selected]
+ final_scores = [score for _, score in selected]
+
+ self._last_metrics = CompressionMetrics(
+ documents_before=len(documents),
+ documents_after=len(final_docs),
+ documents_removed=len(documents) - len(final_docs),
+ relevance_scores=final_scores,
+ )
+
+ logger.info(
+ f"HeadroomDocumentCompressor: {len(documents)} -> {len(final_docs)} documents "
+ f"(avg relevance: {sum(final_scores) / len(final_scores) if final_scores else 0:.2f})"
+ )
+
+ return final_docs
+
+ def _score_document(self, doc: Document, query: str) -> float:
+ """Score a document's relevance to the query using BM25-style scoring.
+
+ Args:
+ doc: Document to score.
+ query: Query to compare against.
+
+ Returns:
+ Relevance score between 0 and 1.
+ """
+ content = doc.page_content.lower()
+ query_lower = query.lower()
+
+ # Tokenize
+ query_terms = self._tokenize(query_lower)
+ doc_terms = self._tokenize(content)
+
+ if not query_terms or not doc_terms:
+ return 0.0
+
+ # BM25-style scoring
+ k1 = 1.5
+ b = 0.75
+ avg_dl = 100 # Assume average document length
+
+ doc_len = len(doc_terms)
+ term_freqs: dict[str, int] = {}
+ for term in doc_terms:
+ term_freqs[term] = term_freqs.get(term, 0) + 1
+
+ score = 0.0
+ for term in query_terms:
+ if term in term_freqs:
+ tf = term_freqs[term]
+ # Simplified BM25 (without IDF since we don't have corpus stats)
+ numerator = tf * (k1 + 1)
+ denominator = tf + k1 * (1 - b + b * (doc_len / avg_dl))
+ score += numerator / denominator
+
+ # Normalize to 0-1 range
+ max_possible = len(query_terms) * (k1 + 1)
+ normalized = score / max_possible if max_possible > 0 else 0.0
+
+ # Boost for exact phrase matches
+ if query_lower in content:
+ normalized = min(1.0, normalized + 0.3)
+
+ return min(1.0, normalized)
+
+ def _tokenize(self, text: str) -> list[str]:
+ """Tokenize text into terms.
+
+ Args:
+ text: Text to tokenize.
+
+ Returns:
+ List of tokens.
+ """
+ # Simple tokenization: split on non-alphanumeric, filter short terms
+ tokens = re.findall(r"\b\w+\b", text)
+ return [t for t in tokens if len(t) > 1]
+
+ def _select_diverse(
+ self, scored_docs: list[tuple[Document, float]], query: str
+ ) -> list[tuple[Document, float]]:
+ """Select diverse documents using MMR-style approach.
+
+ Balances relevance with diversity to avoid redundant results.
+
+ Args:
+ scored_docs: List of (document, relevance_score) tuples.
+ query: Original query.
+
+ Returns:
+ Selected documents with diversity considered.
+ """
+ if not scored_docs:
+ return []
+
+ # Sort by initial relevance
+ scored_docs = sorted(scored_docs, key=lambda x: x[1], reverse=True)
+
+ # Start with most relevant
+ selected = [scored_docs[0]]
+ remaining = scored_docs[1:]
+
+ lambda_param = 0.5 # Balance between relevance and diversity
+
+ while len(selected) < self.max_documents and remaining:
+ best_score = -1.0
+ best_idx = 0
+
+ for i, (doc, rel_score) in enumerate(remaining):
+ # Calculate max similarity to already selected docs
+ max_sim = max(self._document_similarity(doc, sel_doc) for sel_doc, _ in selected)
+
+ # MMR score: lambda * relevance - (1-lambda) * max_similarity
+ mmr_score = lambda_param * rel_score - (1 - lambda_param) * max_sim
+
+ if mmr_score > best_score:
+ best_score = mmr_score
+ best_idx = i
+
+ selected.append(remaining[best_idx])
+ remaining.pop(best_idx)
+
+ return selected
+
+ def _document_similarity(self, doc1: Document, doc2: Document) -> float:
+ """Calculate similarity between two documents.
+
+ Uses Jaccard similarity on terms for simplicity.
+
+ Args:
+ doc1: First document.
+ doc2: Second document.
+
+ Returns:
+ Similarity score between 0 and 1.
+ """
+ terms1 = set(self._tokenize(doc1.page_content.lower()))
+ terms2 = set(self._tokenize(doc2.page_content.lower()))
+
+ if not terms1 or not terms2:
+ return 0.0
+
+ intersection = len(terms1 & terms2)
+ union = len(terms1 | terms2)
+
+ return intersection / union if union > 0 else 0.0
+
+ @property
+ def last_metrics(self) -> CompressionMetrics | None:
+ """Get metrics from the last compression operation."""
+ return self._last_metrics
+
+ def get_compression_stats(self) -> dict[str, Any]:
+ """Get statistics from the last compression.
+
+ Returns:
+ Dictionary with compression metrics, or empty if no compression yet.
+ """
+ if self._last_metrics is None:
+ return {}
+
+ return {
+ "documents_before": self._last_metrics.documents_before,
+ "documents_after": self._last_metrics.documents_after,
+ "documents_removed": self._last_metrics.documents_removed,
+ "average_relevance": (
+ sum(self._last_metrics.relevance_scores) / len(self._last_metrics.relevance_scores)
+ if self._last_metrics.relevance_scores
+ else 0.0
+ ),
+ }
diff --git a/headroom/integrations/langchain/streaming.py b/headroom/integrations/langchain/streaming.py
new file mode 100644
index 000000000..24a92b09a
--- /dev/null
+++ b/headroom/integrations/langchain/streaming.py
@@ -0,0 +1,341 @@
+"""Streaming metrics tracking for LangChain.
+
+This module provides StreamingMetricsTracker for tracking output tokens
+during streaming responses from LangChain models.
+
+Example:
+ from langchain_openai import ChatOpenAI
+ from headroom.integrations import HeadroomChatModel, StreamingMetricsTracker
+
+ llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))
+ tracker = StreamingMetricsTracker(model="gpt-4o")
+
+ for chunk in llm.stream("Tell me a story"):
+ tracker.add_chunk(chunk)
+ print(chunk.content, end="", flush=True)
+
+ print(f"\\nOutput tokens: {tracker.output_tokens}")
+"""
+
+from __future__ import annotations
+
+import logging
+from dataclasses import dataclass
+from datetime import datetime
+from typing import Any
+
+# LangChain imports - these are optional dependencies
+try:
+ from langchain_core.messages import AIMessageChunk
+ from langchain_core.outputs import ChatGenerationChunk
+
+ LANGCHAIN_AVAILABLE = True
+except ImportError:
+ LANGCHAIN_AVAILABLE = False
+ AIMessageChunk = object # type: ignore[misc,assignment]
+ ChatGenerationChunk = object # type: ignore[misc,assignment]
+
+from headroom.providers import OpenAIProvider
+
+logger = logging.getLogger(__name__)
+
+
+def _check_langchain_available() -> None:
+ """Raise ImportError if LangChain is not installed."""
+ if not LANGCHAIN_AVAILABLE:
+ raise ImportError(
+ "LangChain is required for this integration. "
+ "Install with: pip install headroom[langchain] "
+ "or: pip install langchain-core"
+ )
+
+
+@dataclass
+class StreamingMetrics:
+ """Metrics from a streaming response."""
+
+ output_tokens: int
+ chunk_count: int
+ content_length: int
+ start_time: datetime
+ end_time: datetime | None
+ duration_ms: float | None
+
+ def to_dict(self) -> dict[str, Any]:
+ """Convert to dictionary."""
+ return {
+ "output_tokens": self.output_tokens,
+ "chunk_count": self.chunk_count,
+ "content_length": self.content_length,
+ "start_time": self.start_time.isoformat(),
+ "end_time": self.end_time.isoformat() if self.end_time else None,
+ "duration_ms": self.duration_ms,
+ }
+
+
+class StreamingMetricsTracker:
+ """Tracks output tokens and metrics during streaming.
+
+ Accumulates content from streaming chunks and provides accurate
+ token counting for the streamed output.
+
+ Example:
+ tracker = StreamingMetricsTracker(model="gpt-4o")
+
+ async for chunk in llm.astream(messages):
+ tracker.add_chunk(chunk)
+ print(chunk.content, end="")
+
+ print(f"\\nTokens: {tracker.output_tokens}")
+ print(f"Duration: {tracker.duration_ms}ms")
+
+ Attributes:
+ model: Model name for token counting
+ content: Accumulated content from all chunks
+ output_tokens: Estimated token count for output
+ chunk_count: Number of chunks received
+ """
+
+ def __init__(
+ self,
+ model: str = "gpt-4o",
+ provider: Any = None,
+ ):
+ """Initialize StreamingMetricsTracker.
+
+ Args:
+ model: Model name for token counting. Default "gpt-4o".
+ provider: Headroom provider for token counting. Uses
+ OpenAIProvider if not specified.
+ """
+ _check_langchain_available()
+
+ self._model = model
+ self._provider = provider or OpenAIProvider()
+ self._content = ""
+ self._chunk_count = 0
+ self._start_time: datetime | None = None
+ self._end_time: datetime | None = None
+
+ def add_chunk(self, chunk: Any) -> None:
+ """Add a streaming chunk to the tracker.
+
+ Extracts content from various chunk types:
+ - AIMessageChunk
+ - ChatGenerationChunk
+ - dict with 'content' key
+ - string
+
+ Args:
+ chunk: Streaming chunk from LangChain.
+ """
+ if self._start_time is None:
+ self._start_time = datetime.now()
+
+ self._chunk_count += 1
+
+ # Extract content from various chunk types
+ content = self._extract_content(chunk)
+ if content:
+ self._content += content
+
+ def _extract_content(self, chunk: Any) -> str:
+ """Extract string content from a chunk.
+
+ Args:
+ chunk: Streaming chunk of various types.
+
+ Returns:
+ Extracted content string.
+ """
+ # AIMessageChunk
+ if hasattr(chunk, "content"):
+ content = chunk.content
+ if isinstance(content, str):
+ return content
+ return str(content) if content else ""
+
+ # ChatGenerationChunk
+ if hasattr(chunk, "message") and hasattr(chunk.message, "content"):
+ content = chunk.message.content
+ if isinstance(content, str):
+ return content
+ return str(content) if content else ""
+
+ # dict
+ if isinstance(chunk, dict):
+ return str(chunk.get("content", ""))
+
+ # string
+ if isinstance(chunk, str):
+ return chunk
+
+ return ""
+
+ def finish(self) -> StreamingMetrics:
+ """Mark streaming as complete and return final metrics.
+
+ Returns:
+ StreamingMetrics with final values.
+ """
+ self._end_time = datetime.now()
+
+ duration_ms = None
+ if self._start_time:
+ duration_ms = (self._end_time - self._start_time).total_seconds() * 1000
+
+ return StreamingMetrics(
+ output_tokens=self.output_tokens,
+ chunk_count=self._chunk_count,
+ content_length=len(self._content),
+ start_time=self._start_time or self._end_time,
+ end_time=self._end_time,
+ duration_ms=duration_ms,
+ )
+
+ @property
+ def content(self) -> str:
+ """Get accumulated content."""
+ return self._content
+
+ @property
+ def output_tokens(self) -> int:
+ """Get estimated output token count."""
+ if not self._content:
+ return 0
+ token_counter = self._provider.get_token_counter(self._model)
+ return token_counter.count_text(self._content)
+
+ @property
+ def chunk_count(self) -> int:
+ """Get number of chunks received."""
+ return self._chunk_count
+
+ @property
+ def duration_ms(self) -> float | None:
+ """Get duration in milliseconds (after finish())."""
+ if self._start_time is None or self._end_time is None:
+ return None
+ return (self._end_time - self._start_time).total_seconds() * 1000
+
+ def reset(self) -> None:
+ """Reset tracker for reuse."""
+ self._content = ""
+ self._chunk_count = 0
+ self._start_time = None
+ self._end_time = None
+
+
+class StreamingMetricsCallback:
+ """Context manager for tracking streaming metrics.
+
+ Provides a clean interface for tracking a complete streaming
+ response with automatic timing.
+
+ Example:
+ with StreamingMetricsCallback(model="gpt-4o") as tracker:
+ for chunk in llm.stream(messages):
+ tracker.add_chunk(chunk)
+ print(chunk.content, end="")
+
+ print(f"\\nMetrics: {tracker.metrics}")
+
+ Attributes:
+ tracker: The underlying StreamingMetricsTracker
+ metrics: Final metrics after context exit
+ """
+
+ def __init__(self, model: str = "gpt-4o", provider: Any = None):
+ """Initialize StreamingMetricsCallback.
+
+ Args:
+ model: Model name for token counting.
+ provider: Headroom provider for token counting.
+ """
+ self._tracker = StreamingMetricsTracker(model=model, provider=provider)
+ self._metrics: StreamingMetrics | None = None
+
+ def __enter__(self) -> StreamingMetricsTracker:
+ """Enter context, return tracker."""
+ return self._tracker
+
+ def __exit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None:
+ """Exit context, finalize metrics."""
+ self._metrics = self._tracker.finish()
+
+ @property
+ def tracker(self) -> StreamingMetricsTracker:
+ """Get the tracker."""
+ return self._tracker
+
+ @property
+ def metrics(self) -> StreamingMetrics | None:
+ """Get final metrics (after context exit)."""
+ return self._metrics
+
+
+def track_streaming_response(
+ stream: Any,
+ model: str = "gpt-4o",
+ provider: Any = None,
+) -> tuple[str, StreamingMetrics]:
+ """Track a complete streaming response.
+
+ Convenience function that consumes a stream and returns the
+ accumulated content and metrics.
+
+ Args:
+ stream: Iterable of streaming chunks.
+ model: Model name for token counting.
+ provider: Headroom provider for token counting.
+
+ Returns:
+ Tuple of (accumulated_content, metrics).
+
+ Example:
+ content, metrics = track_streaming_response(
+ llm.stream(messages),
+ model="gpt-4o"
+ )
+ print(f"Content: {content}")
+ print(f"Tokens: {metrics.output_tokens}")
+ """
+ tracker = StreamingMetricsTracker(model=model, provider=provider)
+
+ for chunk in stream:
+ tracker.add_chunk(chunk)
+
+ metrics = tracker.finish()
+ return tracker.content, metrics
+
+
+async def track_async_streaming_response(
+ stream: Any,
+ model: str = "gpt-4o",
+ provider: Any = None,
+) -> tuple[str, StreamingMetrics]:
+ """Track a complete async streaming response.
+
+ Async version of track_streaming_response.
+
+ Args:
+ stream: Async iterable of streaming chunks.
+ model: Model name for token counting.
+ provider: Headroom provider for token counting.
+
+ Returns:
+ Tuple of (accumulated_content, metrics).
+
+ Example:
+ content, metrics = await track_async_streaming_response(
+ llm.astream(messages),
+ model="gpt-4o"
+ )
+ """
+ tracker = StreamingMetricsTracker(model=model, provider=provider)
+
+ async for chunk in stream:
+ tracker.add_chunk(chunk)
+
+ metrics = tracker.finish()
+ return tracker.content, metrics
diff --git a/headroom/integrations/mcp/__init__.py b/headroom/integrations/mcp/__init__.py
new file mode 100644
index 000000000..081416df3
--- /dev/null
+++ b/headroom/integrations/mcp/__init__.py
@@ -0,0 +1,37 @@
+"""MCP (Model Context Protocol) integration for Headroom.
+
+This package provides compression utilities for MCP tool results,
+helping reduce context usage when tools return large outputs.
+
+Example:
+ from headroom.integrations.mcp import compress_tool_result
+
+ # Compress large tool output
+ result = compress_tool_result(
+ tool_name="search",
+ result=large_json_result,
+ max_chars=5000,
+ )
+"""
+
+from .server import (
+ DEFAULT_MCP_PROFILES,
+ HeadroomMCPClientWrapper,
+ HeadroomMCPCompressor,
+ MCPCompressionResult,
+ MCPToolProfile,
+ compress_tool_result,
+ compress_tool_result_with_metrics,
+ create_headroom_mcp_proxy,
+)
+
+__all__ = [
+ "HeadroomMCPCompressor",
+ "HeadroomMCPClientWrapper",
+ "MCPCompressionResult",
+ "MCPToolProfile",
+ "compress_tool_result",
+ "compress_tool_result_with_metrics",
+ "create_headroom_mcp_proxy",
+ "DEFAULT_MCP_PROFILES",
+]
diff --git a/headroom/integrations/mcp.py b/headroom/integrations/mcp/server.py
similarity index 100%
rename from headroom/integrations/mcp.py
rename to headroom/integrations/mcp/server.py
diff --git a/headroom/transforms/llmlingua_compressor.py b/headroom/transforms/llmlingua_compressor.py
index 5e05d277f..e09e57b9a 100644
--- a/headroom/transforms/llmlingua_compressor.py
+++ b/headroom/transforms/llmlingua_compressor.py
@@ -88,7 +88,8 @@ def _get_llmlingua_compressor(model_name: str, device: str) -> Any:
from llmlingua import PromptCompressor
logger.info(
- "Loading LLMLingua-2 model: %s on device: %s (this may take 10-30s on first run)",
+ "Loading LLMLingua-2 model: %s on device: %s "
+ "(this may take 10-30s on first run)",
model_name,
device,
)
diff --git a/pyproject.toml b/pyproject.toml
index f9e198c32..7c4f6d1c7 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "headroom-ai"
-version = "0.2.2"
+version = "0.2.3"
description = "The Context Optimization Layer for LLM Applications - Cut costs by 50-90%"
readme = "README.md"
license = "Apache-2.0"
diff --git a/tests/test_integrations/langchain/__init__.py b/tests/test_integrations/langchain/__init__.py
new file mode 100644
index 000000000..e69de29bb
diff --git a/tests/test_integrations/test_langchain.py b/tests/test_integrations/langchain/test_chat_model.py
similarity index 98%
rename from tests/test_integrations/test_langchain.py
rename to tests/test_integrations/langchain/test_chat_model.py
index a0d127111..62421daff 100644
--- a/tests/test_integrations/test_langchain.py
+++ b/tests/test_integrations/langchain/test_chat_model.py
@@ -488,7 +488,7 @@ class TestOptimizeMessages:
"""Basic message optimization."""
from headroom.integrations import optimize_messages
- with patch("headroom.integrations.langchain.TransformPipeline") as MockPipeline:
+ with patch("headroom.integrations.langchain.chat_model.TransformPipeline") as MockPipeline:
mock_instance = MagicMock()
mock_result = MagicMock()
mock_result.messages = [
@@ -513,7 +513,7 @@ class TestOptimizeMessages:
config = HeadroomConfig(default_mode=HeadroomMode.AUDIT)
- with patch("headroom.integrations.langchain.TransformPipeline") as MockPipeline:
+ with patch("headroom.integrations.langchain.chat_model.TransformPipeline") as MockPipeline:
mock_instance = MagicMock()
mock_result = MagicMock()
mock_result.messages = []
@@ -547,7 +547,7 @@ class TestOptimizeMessages:
ToolMessage(content="Sunny", tool_call_id="1"),
]
- with patch("headroom.integrations.langchain.TransformPipeline") as MockPipeline:
+ with patch("headroom.integrations.langchain.chat_model.TransformPipeline") as MockPipeline:
mock_instance = MagicMock()
mock_result = MagicMock()
mock_result.messages = [
diff --git a/tests/test_integrations/test_langchain_evals.py b/tests/test_integrations/langchain/test_evals.py
similarity index 100%
rename from tests/test_integrations/test_langchain_evals.py
rename to tests/test_integrations/langchain/test_evals.py
diff --git a/tests/test_integrations/langchain/test_extended.py b/tests/test_integrations/langchain/test_extended.py
new file mode 100644
index 000000000..321f4a3e6
--- /dev/null
+++ b/tests/test_integrations/langchain/test_extended.py
@@ -0,0 +1,646 @@
+"""Tests for extended LangChain integration modules.
+
+Tests cover:
+1. langchain_providers - Provider auto-detection
+2. langchain_memory - HeadroomChatMessageHistory
+3. langchain_retriever - HeadroomDocumentCompressor
+4. langchain_agents - HeadroomToolWrapper
+5. langchain_langsmith - LangSmith integration
+6. langchain_streaming - Streaming metrics
+"""
+
+import json
+from unittest.mock import MagicMock
+
+import pytest
+
+# Check if LangChain is available
+try:
+ from langchain_core.documents import Document
+ from langchain_core.messages import AIMessage, HumanMessage
+ from langchain_core.tools import StructuredTool
+
+ LANGCHAIN_AVAILABLE = True
+except ImportError:
+ LANGCHAIN_AVAILABLE = False
+
+# Skip all tests if LangChain not installed
+pytestmark = pytest.mark.skipif(not LANGCHAIN_AVAILABLE, reason="LangChain not installed")
+
+
+class TestProviderDetection:
+ """Tests for langchain_providers module."""
+
+ def test_detect_openai_provider(self):
+ """Detect OpenAI from ChatOpenAI class."""
+ from headroom.integrations.langchain.providers import detect_provider
+
+ mock_model = MagicMock()
+ mock_model.__class__.__name__ = "ChatOpenAI"
+ mock_model.__class__.__module__ = "langchain_openai.chat_models"
+
+ provider = detect_provider(mock_model)
+ assert provider == "openai"
+
+ def test_detect_anthropic_provider(self):
+ """Detect Anthropic from ChatAnthropic class."""
+ from headroom.integrations.langchain.providers import detect_provider
+
+ mock_model = MagicMock()
+ mock_model.__class__.__name__ = "ChatAnthropic"
+ mock_model.__class__.__module__ = "langchain_anthropic.chat_models"
+
+ provider = detect_provider(mock_model)
+ assert provider == "anthropic"
+
+ def test_detect_google_provider(self):
+ """Detect Google from ChatGoogleGenerativeAI class."""
+ from headroom.integrations.langchain.providers import detect_provider
+
+ mock_model = MagicMock()
+ mock_model.__class__.__name__ = "ChatGoogleGenerativeAI"
+ mock_model.__class__.__module__ = "langchain_google_genai"
+
+ provider = detect_provider(mock_model)
+ assert provider == "google"
+
+ def test_detect_fallback_to_openai(self):
+ """Fall back to OpenAI for unknown models."""
+ from headroom.integrations.langchain.providers import detect_provider
+
+ mock_model = MagicMock()
+ mock_model.__class__.__name__ = "CustomChatModel"
+ mock_model.__class__.__module__ = "my_custom_module"
+
+ provider = detect_provider(mock_model)
+ assert provider == "openai"
+
+ def test_detect_from_model_name_claude(self):
+ """Detect Anthropic from model name containing 'claude'."""
+ from headroom.integrations.langchain.providers import detect_provider
+
+ mock_model = MagicMock()
+ mock_model.__class__.__name__ = "CustomModel"
+ mock_model.__class__.__module__ = "custom"
+ mock_model.model_name = "claude-3-5-sonnet-20241022"
+
+ provider = detect_provider(mock_model)
+ assert provider == "anthropic"
+
+ def test_get_headroom_provider_openai(self):
+ """Get OpenAIProvider for OpenAI model."""
+ from headroom.integrations.langchain.providers import get_headroom_provider
+ from headroom.providers import OpenAIProvider
+
+ mock_model = MagicMock()
+ mock_model.__class__.__name__ = "ChatOpenAI"
+ mock_model.__class__.__module__ = "langchain_openai"
+
+ provider = get_headroom_provider(mock_model)
+ assert isinstance(provider, OpenAIProvider)
+
+ def test_get_headroom_provider_anthropic(self):
+ """Get AnthropicProvider for Anthropic model."""
+ from headroom.integrations.langchain.providers import get_headroom_provider
+ from headroom.providers import AnthropicProvider
+
+ mock_model = MagicMock()
+ mock_model.__class__.__name__ = "ChatAnthropic"
+ mock_model.__class__.__module__ = "langchain_anthropic"
+
+ provider = get_headroom_provider(mock_model)
+ assert isinstance(provider, AnthropicProvider)
+
+ def test_get_model_name_from_langchain(self):
+ """Extract model name from LangChain model."""
+ from headroom.integrations.langchain.providers import get_model_name_from_langchain
+
+ mock_model = MagicMock()
+ mock_model.model_name = "gpt-4o"
+
+ name = get_model_name_from_langchain(mock_model)
+ assert name == "gpt-4o"
+
+ def test_get_model_name_fallback(self):
+ """Fall back when model name not available."""
+ from headroom.integrations.langchain.providers import get_model_name_from_langchain
+
+ mock_model = MagicMock(spec=[])
+ mock_model.__class__.__name__ = "ChatOpenAI"
+
+ name = get_model_name_from_langchain(mock_model)
+ assert name == "gpt-4o" # Default for OpenAI
+
+
+class TestHeadroomChatMessageHistory:
+ """Tests for HeadroomChatMessageHistory memory wrapper."""
+
+ def test_init(self):
+ """Initialize with base history."""
+ from headroom.integrations.langchain.memory import HeadroomChatMessageHistory
+
+ mock_history = MagicMock()
+ mock_history.messages = []
+
+ wrapper = HeadroomChatMessageHistory(
+ mock_history,
+ compress_threshold_tokens=4000,
+ keep_recent_turns=5,
+ )
+
+ assert wrapper._base is mock_history
+ assert wrapper._threshold == 4000
+ assert wrapper._keep_recent_turns == 5
+
+ def test_messages_passthrough_under_threshold(self):
+ """Messages pass through when under threshold."""
+ from headroom.integrations.langchain.memory import HeadroomChatMessageHistory
+
+ mock_history = MagicMock()
+ mock_history.messages = [
+ HumanMessage(content="Hello"),
+ AIMessage(content="Hi there!"),
+ ]
+
+ wrapper = HeadroomChatMessageHistory(
+ mock_history,
+ compress_threshold_tokens=10000, # High threshold
+ )
+
+ messages = wrapper.messages
+ assert len(messages) == 2
+ assert messages[0].content == "Hello"
+
+ def test_add_message_delegates(self):
+ """add_message delegates to base history."""
+ from headroom.integrations.langchain.memory import HeadroomChatMessageHistory
+
+ mock_history = MagicMock()
+ mock_history.messages = []
+
+ wrapper = HeadroomChatMessageHistory(mock_history)
+ message = HumanMessage(content="Test")
+ wrapper.add_message(message)
+
+ mock_history.add_message.assert_called_once_with(message)
+
+ def test_clear_delegates(self):
+ """clear delegates to base history."""
+ from headroom.integrations.langchain.memory import HeadroomChatMessageHistory
+
+ mock_history = MagicMock()
+ mock_history.messages = []
+
+ wrapper = HeadroomChatMessageHistory(mock_history)
+ wrapper.clear()
+
+ mock_history.clear.assert_called_once()
+
+ def test_get_compression_stats(self):
+ """Get compression statistics."""
+ from headroom.integrations.langchain.memory import HeadroomChatMessageHistory
+
+ mock_history = MagicMock()
+ mock_history.messages = []
+
+ wrapper = HeadroomChatMessageHistory(mock_history)
+ stats = wrapper.get_compression_stats()
+
+ assert "compression_count" in stats
+ assert "total_tokens_saved" in stats
+ assert stats["compression_count"] == 0
+
+
+class TestHeadroomDocumentCompressor:
+ """Tests for HeadroomDocumentCompressor retriever integration."""
+
+ def test_init(self):
+ """Initialize with defaults."""
+ from headroom.integrations.langchain.retriever import HeadroomDocumentCompressor
+
+ compressor = HeadroomDocumentCompressor()
+
+ assert compressor.max_documents == 10
+ assert compressor.min_relevance == 0.0
+ assert compressor.prefer_diverse is False
+
+ def test_init_custom(self):
+ """Initialize with custom settings."""
+ from headroom.integrations.langchain.retriever import HeadroomDocumentCompressor
+
+ compressor = HeadroomDocumentCompressor(
+ max_documents=5,
+ min_relevance=0.5,
+ prefer_diverse=True,
+ )
+
+ assert compressor.max_documents == 5
+ assert compressor.min_relevance == 0.5
+ assert compressor.prefer_diverse is True
+
+ def test_compress_passthrough_under_limit(self):
+ """Pass through when under max_documents."""
+ from headroom.integrations.langchain.retriever import HeadroomDocumentCompressor
+
+ compressor = HeadroomDocumentCompressor(max_documents=10)
+
+ docs = [
+ Document(page_content="Python is a programming language."),
+ Document(page_content="JavaScript runs in browsers."),
+ ]
+
+ result = compressor.compress_documents(docs, "What is Python?")
+
+ assert len(result) == 2
+
+ def test_compress_reduces_to_max(self):
+ """Compress when over max_documents."""
+ from headroom.integrations.langchain.retriever import HeadroomDocumentCompressor
+
+ compressor = HeadroomDocumentCompressor(max_documents=2)
+
+ docs = [
+ Document(page_content="Python is a programming language."),
+ Document(page_content="Java is also a language."),
+ Document(page_content="Weather today is sunny."),
+ Document(page_content="Cats are cute animals."),
+ ]
+
+ result = compressor.compress_documents(docs, "programming language")
+
+ assert len(result) == 2
+
+ def test_compress_prefers_relevant(self):
+ """Keep most relevant documents."""
+ from headroom.integrations.langchain.retriever import HeadroomDocumentCompressor
+
+ compressor = HeadroomDocumentCompressor(max_documents=1)
+
+ docs = [
+ Document(page_content="Weather today is sunny."),
+ Document(page_content="Python programming tutorial basics."),
+ Document(page_content="Cats are cute animals."),
+ ]
+
+ result = compressor.compress_documents(docs, "Python tutorial")
+
+ assert len(result) == 1
+ assert "Python" in result[0].page_content
+
+ def test_metrics_tracked(self):
+ """Compression metrics are tracked."""
+ from headroom.integrations.langchain.retriever import HeadroomDocumentCompressor
+
+ compressor = HeadroomDocumentCompressor(max_documents=2)
+
+ docs = [
+ Document(page_content="Doc 1"),
+ Document(page_content="Doc 2"),
+ Document(page_content="Doc 3"),
+ ]
+
+ compressor.compress_documents(docs, "query")
+
+ metrics = compressor.last_metrics
+ assert metrics is not None
+ assert metrics.documents_before == 3
+ assert metrics.documents_after == 2
+ assert metrics.documents_removed == 1
+
+ def test_get_compression_stats(self):
+ """Get compression statistics."""
+ from headroom.integrations.langchain.retriever import HeadroomDocumentCompressor
+
+ compressor = HeadroomDocumentCompressor(max_documents=1)
+ docs = [Document(page_content="A"), Document(page_content="B")]
+
+ compressor.compress_documents(docs, "A")
+ stats = compressor.get_compression_stats()
+
+ assert "documents_before" in stats
+ assert "documents_after" in stats
+ assert "average_relevance" in stats
+
+
+class TestHeadroomToolWrapper:
+ """Tests for HeadroomToolWrapper agent integration."""
+
+ def test_init(self):
+ """Initialize wrapper."""
+ from headroom.integrations.langchain.agents import HeadroomToolWrapper
+
+ mock_tool = MagicMock()
+ mock_tool.name = "test_tool"
+ mock_tool.description = "A test tool"
+
+ wrapper = HeadroomToolWrapper(mock_tool)
+
+ assert wrapper.name == "test_tool"
+ assert wrapper.description == "A test tool"
+
+ def test_call_passthrough_small_output(self):
+ """Small outputs pass through without compression."""
+ from headroom.integrations.langchain.agents import HeadroomToolWrapper
+
+ mock_tool = MagicMock()
+ mock_tool.name = "test"
+ mock_tool.description = "test"
+ mock_tool.invoke.return_value = "small result"
+
+ wrapper = HeadroomToolWrapper(mock_tool, min_chars_to_compress=1000)
+ result = wrapper("query")
+
+ assert result == "small result"
+
+ def test_call_compresses_large_json(self):
+ """Large JSON outputs get compressed."""
+ from headroom.integrations.langchain.agents import HeadroomToolWrapper
+
+ mock_tool = MagicMock()
+ mock_tool.name = "search"
+ mock_tool.description = "search"
+
+ # Large JSON output
+ large_output = json.dumps([{"id": i, "data": "x" * 100} for i in range(50)])
+ mock_tool.invoke.return_value = large_output
+
+ wrapper = HeadroomToolWrapper(mock_tool, min_chars_to_compress=100)
+ result = wrapper("query")
+
+ # Should be smaller after compression
+ assert len(result) <= len(large_output)
+
+ def test_as_langchain_tool(self):
+ """Convert to LangChain tool."""
+ from headroom.integrations.langchain.agents import HeadroomToolWrapper
+
+ mock_tool = MagicMock()
+ mock_tool.name = "test"
+ mock_tool.description = "test tool"
+ mock_tool.invoke.return_value = "result"
+
+ wrapper = HeadroomToolWrapper(mock_tool)
+ lc_tool = wrapper.as_langchain_tool()
+
+ assert isinstance(lc_tool, StructuredTool)
+ assert lc_tool.name == "test"
+
+ def test_wrap_tools_with_headroom(self):
+ """Wrap multiple tools at once."""
+ from headroom.integrations.langchain.agents import wrap_tools_with_headroom
+
+ tools = []
+ for i in range(3):
+ mock = MagicMock()
+ mock.name = f"tool_{i}"
+ mock.description = f"Tool {i}"
+ mock.invoke.return_value = "result"
+ tools.append(mock)
+
+ wrapped = wrap_tools_with_headroom(tools)
+
+ assert len(wrapped) == 3
+ assert all(isinstance(t, StructuredTool) for t in wrapped)
+
+ def test_metrics_collector(self):
+ """Tool metrics are collected."""
+ from headroom.integrations.langchain.agents import (
+ HeadroomToolWrapper,
+ ToolMetricsCollector,
+ )
+
+ collector = ToolMetricsCollector()
+
+ mock_tool = MagicMock()
+ mock_tool.name = "test"
+ mock_tool.description = "test"
+ mock_tool.invoke.return_value = "result"
+
+ wrapper = HeadroomToolWrapper(mock_tool, metrics_collector=collector)
+ wrapper("query")
+
+ assert len(collector.metrics) == 1
+ assert collector.metrics[0].tool_name == "test"
+
+
+class TestHeadroomLangSmithCallbackHandler:
+ """Tests for LangSmith integration."""
+
+ def test_init(self):
+ """Initialize handler."""
+ from headroom.integrations.langchain.langsmith import (
+ HeadroomLangSmithCallbackHandler,
+ )
+
+ handler = HeadroomLangSmithCallbackHandler(auto_update_runs=False)
+
+ assert handler._auto_update is False
+ assert handler._pending_metrics == {}
+
+ def test_set_headroom_metrics(self):
+ """Set metrics for a run."""
+ from headroom.integrations.langchain.langsmith import (
+ HeadroomLangSmithCallbackHandler,
+ )
+
+ handler = HeadroomLangSmithCallbackHandler(auto_update_runs=False)
+
+ handler.set_headroom_metrics(
+ run_id="test-run-123",
+ tokens_before=1000,
+ tokens_after=800,
+ transforms_applied=["smart_crusher"],
+ )
+
+ assert "test-run-123" in handler._pending_metrics
+ metrics = handler._pending_metrics["test-run-123"]
+ assert metrics.tokens_before == 1000
+ assert metrics.tokens_after == 800
+ assert metrics.tokens_saved == 200
+ assert metrics.savings_percent == 20.0
+
+ def test_get_run_metrics(self):
+ """Get metrics for a specific run."""
+ from headroom.integrations.langchain.langsmith import (
+ HeadroomLangSmithCallbackHandler,
+ )
+
+ handler = HeadroomLangSmithCallbackHandler(auto_update_runs=False)
+ handler._run_metrics["run-1"] = {"headroom.tokens_saved": 100}
+
+ metrics = handler.get_run_metrics("run-1")
+ assert metrics["headroom.tokens_saved"] == 100
+
+ def test_get_summary(self):
+ """Get summary statistics."""
+ from headroom.integrations.langchain.langsmith import (
+ HeadroomLangSmithCallbackHandler,
+ )
+
+ handler = HeadroomLangSmithCallbackHandler(auto_update_runs=False)
+ handler._run_metrics = {
+ "run-1": {"headroom.tokens_saved": 100, "headroom.savings_percent": 20},
+ "run-2": {"headroom.tokens_saved": 200, "headroom.savings_percent": 30},
+ }
+
+ summary = handler.get_summary()
+ assert summary["total_runs"] == 2
+ assert summary["total_tokens_saved"] == 300
+ assert summary["average_savings_percent"] == 25.0
+
+ def test_reset(self):
+ """Reset clears all metrics."""
+ from headroom.integrations.langchain.langsmith import (
+ HeadroomLangSmithCallbackHandler,
+ )
+
+ handler = HeadroomLangSmithCallbackHandler(auto_update_runs=False)
+ handler._run_metrics = {"run-1": {}}
+ handler._pending_metrics = {"run-2": MagicMock()}
+
+ handler.reset()
+
+ assert handler._run_metrics == {}
+ assert handler._pending_metrics == {}
+
+
+class TestStreamingMetricsTracker:
+ """Tests for streaming metrics tracking."""
+
+ def test_init(self):
+ """Initialize tracker."""
+ from headroom.integrations.langchain.streaming import StreamingMetricsTracker
+
+ tracker = StreamingMetricsTracker(model="gpt-4o")
+
+ assert tracker._model == "gpt-4o"
+ assert tracker._content == ""
+ assert tracker._chunk_count == 0
+
+ def test_add_chunk_string(self):
+ """Add string chunks."""
+ from headroom.integrations.langchain.streaming import StreamingMetricsTracker
+
+ tracker = StreamingMetricsTracker()
+ tracker.add_chunk("Hello ")
+ tracker.add_chunk("world!")
+
+ assert tracker.content == "Hello world!"
+ assert tracker.chunk_count == 2
+
+ def test_add_chunk_with_content_attr(self):
+ """Add chunks with content attribute."""
+ from headroom.integrations.langchain.streaming import StreamingMetricsTracker
+
+ tracker = StreamingMetricsTracker()
+
+ chunk1 = MagicMock()
+ chunk1.content = "Hello "
+ chunk2 = MagicMock()
+ chunk2.content = "world!"
+
+ tracker.add_chunk(chunk1)
+ tracker.add_chunk(chunk2)
+
+ assert tracker.content == "Hello world!"
+
+ def test_output_tokens(self):
+ """Count output tokens."""
+ from headroom.integrations.langchain.streaming import StreamingMetricsTracker
+
+ tracker = StreamingMetricsTracker(model="gpt-4o")
+ tracker.add_chunk("Hello world, this is a test message.")
+
+ tokens = tracker.output_tokens
+ assert tokens > 0
+
+ def test_finish(self):
+ """Finish tracking and get metrics."""
+ from headroom.integrations.langchain.streaming import StreamingMetricsTracker
+
+ tracker = StreamingMetricsTracker()
+ tracker.add_chunk("Test content")
+ metrics = tracker.finish()
+
+ assert metrics.chunk_count == 1
+ assert metrics.content_length == len("Test content")
+ assert metrics.duration_ms is not None
+ assert metrics.end_time is not None
+
+ def test_reset(self):
+ """Reset tracker for reuse."""
+ from headroom.integrations.langchain.streaming import StreamingMetricsTracker
+
+ tracker = StreamingMetricsTracker()
+ tracker.add_chunk("Content")
+ tracker.finish()
+
+ tracker.reset()
+
+ assert tracker.content == ""
+ assert tracker.chunk_count == 0
+
+ def test_streaming_metrics_callback(self):
+ """Test context manager interface."""
+ from headroom.integrations.langchain.streaming import StreamingMetricsCallback
+
+ with StreamingMetricsCallback(model="gpt-4o") as tracker:
+ tracker.add_chunk("Hello")
+ tracker.add_chunk(" world")
+
+ # After context exit, metrics should be available
+ # (accessed via the callback object, not the tracker)
+
+ def test_track_streaming_response(self):
+ """Track a complete streaming response."""
+ from headroom.integrations.langchain.streaming import track_streaming_response
+
+ chunks = ["Hello ", "world", "!"]
+ content, metrics = track_streaming_response(iter(chunks), model="gpt-4o")
+
+ assert content == "Hello world!"
+ assert metrics.chunk_count == 3
+
+
+class TestAutoDetectProviderInChatModel:
+ """Tests for auto_detect_provider in HeadroomChatModel."""
+
+ def test_auto_detect_enabled_by_default(self):
+ """auto_detect_provider is True by default."""
+ from headroom.integrations import HeadroomChatModel
+
+ mock_model = MagicMock()
+ mock_model._llm_type = "test"
+ mock_model._identifying_params = {}
+ mock_model.__class__.__name__ = "ChatOpenAI"
+ mock_model.__class__.__module__ = "langchain_openai"
+
+ model = HeadroomChatModel(mock_model)
+ assert model.auto_detect_provider is True
+
+ def test_auto_detect_can_be_disabled(self):
+ """auto_detect_provider can be set to False."""
+ from headroom.integrations import HeadroomChatModel
+
+ mock_model = MagicMock()
+ mock_model._llm_type = "test"
+ mock_model._identifying_params = {}
+
+ model = HeadroomChatModel(mock_model, auto_detect_provider=False)
+ assert model.auto_detect_provider is False
+
+ def test_pipeline_uses_detected_provider(self):
+ """Pipeline uses auto-detected provider."""
+ from headroom.integrations import HeadroomChatModel
+ from headroom.providers import AnthropicProvider
+
+ mock_model = MagicMock()
+ mock_model._llm_type = "test"
+ mock_model._identifying_params = {}
+ mock_model.__class__.__name__ = "ChatAnthropic"
+ mock_model.__class__.__module__ = "langchain_anthropic"
+
+ model = HeadroomChatModel(mock_model)
+ _ = model.pipeline # Force lazy init
+
+ assert isinstance(model._provider, AnthropicProvider)
diff --git a/tests/test_integrations/mcp/__init__.py b/tests/test_integrations/mcp/__init__.py
new file mode 100644
index 000000000..e69de29bb
diff --git a/tests/test_integrations/test_mcp.py b/tests/test_integrations/mcp/test_server.py
similarity index 100%
rename from tests/test_integrations/test_mcp.py
rename to tests/test_integrations/mcp/test_server.py
diff --git a/uv.lock b/uv.lock
index bda7cd134..cf48453e8 100644
--- a/uv.lock
+++ b/uv.lock
@@ -6,6 +6,25 @@ resolution-markers = [
"python_full_version < '3.11'",
]
+[[package]]
+name = "accelerate"
+version = "1.12.0"
+source = { registry = "https://pypi.netflix.net/simple" }
+dependencies = [
+ { name = "huggingface-hub" },
+ { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version < '3.11'" },
+ { name = "numpy", version = "2.4.0", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version >= '3.11'" },
+ { name = "packaging" },
+ { name = "psutil" },
+ { name = "pyyaml" },
+ { name = "safetensors" },
+ { name = "torch" },
+]
+sdist = { url = "https://pypi.netflix.net/packages/19372078203/accelerate-1.12.0.tar.gz", hash = "sha256:70988c352feb481887077d2ab845125024b2a137a5090d6d7a32b57d03a45df6", size = 398399 }
+wheels = [
+ { url = "https://pypi.netflix.net/packages/19372078202/accelerate-1.12.0-py3-none-any.whl", hash = "sha256:3e2091cd341423207e2f084a6654b1efcd250dc326f2a37d6dde446e07cabb11", size = 380935 },
+]
+
[[package]]
name = "annotated-doc"
version = "0.0.4"
@@ -362,8 +381,8 @@ wheels = [
]
[[package]]
-name = "headroom"
-version = "0.2.0"
+name = "headroom-ai"
+version = "0.2.3"
source = { editable = "." }
dependencies = [
{ name = "pydantic" },
@@ -375,11 +394,18 @@ all = [
{ name = "fastapi" },
{ name = "httpx" },
{ name = "jinja2" },
+ { name = "llmlingua" },
{ name = "numpy", version = "2.2.6", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version < '3.11'" },
{ name = "numpy", version = "2.4.0", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version >= '3.11'" },
{ name = "sentence-transformers" },
+ { name = "torch" },
+ { name = "transformers" },
+ { name = "tree-sitter-language-pack" },
{ name = "uvicorn" },
]
+code = [
+ { name = "tree-sitter-language-pack" },
+]
dev = [
{ name = "anthropic" },
{ name = "mypy" },
@@ -389,6 +415,11 @@ dev = [
{ name = "pytest-cov" },
{ name = "ruff" },
]
+llmlingua = [
+ { name = "llmlingua" },
+ { name = "torch" },
+ { name = "transformers" },
+]
proxy = [
{ name = "fastapi" },
{ name = "httpx" },
@@ -407,9 +438,10 @@ reports = [
requires-dist = [
{ name = "anthropic", marker = "extra == 'dev'", specifier = ">=0.18.0" },
{ name = "fastapi", marker = "extra == 'proxy'", specifier = ">=0.100.0" },
- { name = "headroom", extras = ["relevance", "proxy", "reports"], marker = "extra == 'all'" },
+ { name = "headroom-ai", extras = ["relevance", "proxy", "reports", "llmlingua", "code"], marker = "extra == 'all'" },
{ name = "httpx", marker = "extra == 'proxy'", specifier = ">=0.24.0" },
{ name = "jinja2", marker = "extra == 'reports'", specifier = ">=3.0.0" },
+ { name = "llmlingua", marker = "extra == 'llmlingua'", specifier = ">=0.2.0" },
{ name = "mypy", marker = "extra == 'dev'", specifier = ">=1.0.0" },
{ name = "numpy", marker = "extra == 'relevance'", specifier = ">=1.24.0" },
{ name = "openai", marker = "extra == 'dev'", specifier = ">=1.0.0" },
@@ -420,6 +452,9 @@ requires-dist = [
{ name = "ruff", marker = "extra == 'dev'", specifier = ">=0.1.0" },
{ name = "sentence-transformers", marker = "extra == 'relevance'", specifier = ">=2.2.0" },
{ name = "tiktoken", specifier = ">=0.5.0" },
+ { name = "torch", marker = "extra == 'llmlingua'", specifier = ">=2.0.0" },
+ { name = "transformers", marker = "extra == 'llmlingua'", specifier = ">=4.30.0" },
+ { name = "tree-sitter-language-pack", marker = "extra == 'code'", specifier = ">=0.10.0" },
{ name = "uvicorn", marker = "extra == 'proxy'", specifier = ">=0.23.0" },
]
@@ -708,6 +743,24 @@ wheels = [
{ url = "https://pypi.netflix.net/packages/19544946795/librt-0.7.7-cp314-cp314t-win_arm64.whl", hash = "sha256:142c2cd91794b79fd0ce113bd658993b7ede0fe93057668c2f98a45ca00b7e91", size = 39724 },
]
+[[package]]
+name = "llmlingua"
+version = "0.2.2"
+source = { registry = "https://pypi.netflix.net/simple" }
+dependencies = [
+ { name = "accelerate" },
+ { name = "nltk" },
+ { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version < '3.11'" },
+ { name = "numpy", version = "2.4.0", source = { registry = "https://pypi.netflix.net/simple" }, marker = "python_full_version >= '3.11'" },
+ { name = "tiktoken" },
+ { name = "torch" },
+ { name = "transformers" },
+]
+sdist = { url = "https://pypi.netflix.net/packages/19606733170/llmlingua-0.2.2.tar.gz", hash = "sha256:1a0caedd8d5a65512a85dadb6bfda6f5b3c4b45e5cb9e7b1c6009573f9058572", size = 59753 }
+wheels = [
+ { url = "https://pypi.netflix.net/packages/19606733169/llmlingua-0.2.2-py3-none-any.whl", hash = "sha256:da55137efe0db78063b3395396efe8a0dcfe4ae5a09aea0d503c34b7bf1d800c", size = 30536 },
+]
+
[[package]]
name = "markupsafe"
version = "3.0.3"
@@ -882,6 +935,21 @@ wheels = [
{ url = "https://pypi.netflix.net/packages/19441125158/networkx-3.6.1-py3-none-any.whl", hash = "sha256:d47fbf302e7d9cbbb9e2555a0d267983d2aa476bac30e90dfbe5669bd57f3762", size = 2068504 },
]
+[[package]]
+name = "nltk"
+version = "3.9.2"
+source = { registry = "https://pypi.netflix.net/simple" }
+dependencies = [
+ { name = "click" },
+ { name = "joblib" },
+ { name = "regex" },
+ { name = "tqdm" },
+]
+sdist = { url = "https://pypi.netflix.net/packages/19152095449/nltk-3.9.2.tar.gz", hash = "sha256:0f409e9b069ca4177c1903c3e843eef90c7e92992fa4931ae607da6de49e1419", size = 2887629 }
+wheels = [
+ { url = "https://pypi.netflix.net/packages/19152095448/nltk-3.9.2-py3-none-any.whl", hash = "sha256:1e209d2b3009110635ed9709a67a1a3e33a10f799490fa71cf4bec218c11c88a", size = 1513404 },
+]
+
[[package]]
name = "numpy"
version = "2.2.6"
@@ -1225,6 +1293,34 @@ wheels = [
{ url = "https://pypi.netflix.net/packages/18687957486/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538 },
]
+[[package]]
+name = "psutil"
+version = "7.2.1"
+source = { registry = "https://pypi.netflix.net/simple" }
+sdist = { url = "https://pypi.netflix.net/packages/19533562506/psutil-7.2.1.tar.gz", hash = "sha256:f7583aec590485b43ca601dd9cea0dcd65bd7bb21d30ef4ddbf4ea6b5ed1bdd3", size = 490253 }
+wheels = [
+ { url = "https://pypi.netflix.net/packages/19533562496/psutil-7.2.1-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:ba9f33bb525b14c3ea563b2fd521a84d2fa214ec59e3e6a2858f78d0844dd60d", size = 129624 },
+ { url = "https://pypi.netflix.net/packages/19533562497/psutil-7.2.1-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:81442dac7abfc2f4f4385ea9e12ddf5a796721c0f6133260687fec5c3780fa49", size = 130132 },
+ { url = "https://pypi.netflix.net/packages/19533562498/psutil-7.2.1-cp313-cp313t-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ea46c0d060491051d39f0d2cff4f98d5c72b288289f57a21556cc7d504db37fc", size = 180612 },
+ { url = "https://pypi.netflix.net/packages/19533562499/psutil-7.2.1-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:35630d5af80d5d0d49cfc4d64c1c13838baf6717a13effb35869a5919b854cdf", size = 183201 },
+ { url = "https://pypi.netflix.net/packages/19533562500/psutil-7.2.1-cp313-cp313t-win_amd64.whl", hash = "sha256:923f8653416604e356073e6e0bccbe7c09990acef442def2f5640dd0faa9689f", size = 139081 },
+ { url = "https://pypi.netflix.net/packages/19533562501/psutil-7.2.1-cp313-cp313t-win_arm64.whl", hash = "sha256:cfbe6b40ca48019a51827f20d830887b3107a74a79b01ceb8cc8de4ccb17b672", size = 134767 },
+ { url = "https://pypi.netflix.net/packages/19533562502/psutil-7.2.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:494c513ccc53225ae23eec7fe6e1482f1b8a44674241b54561f755a898650679", size = 129716 },
+ { url = "https://pypi.netflix.net/packages/19533562503/psutil-7.2.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:3fce5f92c22b00cdefd1645aa58ab4877a01679e901555067b1bd77039aa589f", size = 130133 },
+ { url = "https://pypi.netflix.net/packages/19533562504/psutil-7.2.1-cp314-cp314t-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:93f3f7b0bb07711b49626e7940d6fe52aa9940ad86e8f7e74842e73189712129", size = 181518 },
+ { url = "https://pypi.netflix.net/packages/19533562505/psutil-7.2.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d34d2ca888208eea2b5c68186841336a7f5e0b990edec929be909353a202768a", size = 184348 },
+ { url = "https://pypi.netflix.net/packages/19533563921/psutil-7.2.1-cp314-cp314t-win_amd64.whl", hash = "sha256:2ceae842a78d1603753561132d5ad1b2f8a7979cb0c283f5b52fb4e6e14b1a79", size = 140400 },
+ { url = "https://pypi.netflix.net/packages/19533563922/psutil-7.2.1-cp314-cp314t-win_arm64.whl", hash = "sha256:08a2f175e48a898c8eb8eace45ce01777f4785bc744c90aa2cc7f2fa5462a266", size = 135430 },
+ { url = "https://pypi.netflix.net/packages/19533563923/psutil-7.2.1-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:b2e953fcfaedcfbc952b44744f22d16575d3aa78eb4f51ae74165b4e96e55f42", size = 128137 },
+ { url = "https://pypi.netflix.net/packages/19533563924/psutil-7.2.1-cp36-abi3-macosx_11_0_arm64.whl", hash = "sha256:05cc68dbb8c174828624062e73078e7e35406f4ca2d0866c272c2410d8ef06d1", size = 128947 },
+ { url = "https://pypi.netflix.net/packages/19533563925/psutil-7.2.1-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5e38404ca2bb30ed7267a46c02f06ff842e92da3bb8c5bfdadbd35a5722314d8", size = 154694 },
+ { url = "https://pypi.netflix.net/packages/19533563926/psutil-7.2.1-cp36-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ab2b98c9fc19f13f59628d94df5cc4cc4844bc572467d113a8b517d634e362c6", size = 156136 },
+ { url = "https://pypi.netflix.net/packages/19533563927/psutil-7.2.1-cp36-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:f78baafb38436d5a128f837fab2d92c276dfb48af01a240b861ae02b2413ada8", size = 148108 },
+ { url = "https://pypi.netflix.net/packages/19533565348/psutil-7.2.1-cp36-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:99a4cd17a5fdd1f3d014396502daa70b5ec21bf4ffe38393e152f8e449757d67", size = 147402 },
+ { url = "https://pypi.netflix.net/packages/19533565349/psutil-7.2.1-cp37-abi3-win_amd64.whl", hash = "sha256:b1b0671619343aa71c20ff9767eced0483e4fc9e1f489d50923738caf6a03c17", size = 136938 },
+ { url = "https://pypi.netflix.net/packages/19533565350/psutil-7.2.1-cp37-abi3-win_arm64.whl", hash = "sha256:0d67c1822c355aa6f7314d92018fb4268a76668a536f133599b91edd48759442", size = 133836 },
+]
+
[[package]]
name = "pydantic"
version = "2.12.5"
@@ -2193,6 +2289,115 @@ wheels = [
{ url = "https://pypi.netflix.net/packages/19387983499/transformers-4.57.3-py3-none-any.whl", hash = "sha256:c77d353a4851b1880191603d36acb313411d3577f6e2897814f333841f7003f4", size = 11993463 },
]
+[[package]]
+name = "tree-sitter"
+version = "0.25.2"
+source = { registry = "https://pypi.netflix.net/simple" }
+sdist = { url = "https://pypi.netflix.net/packages/19129803294/tree-sitter-0.25.2.tar.gz", hash = "sha256:fe43c158555da46723b28b52e058ad444195afd1db3ca7720c59a254544e9c20", size = 177961 }
+wheels = [
+ { url = "https://pypi.netflix.net/packages/19129800490/tree_sitter-0.25.2-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:72a510931c3c25f134aac2daf4eb4feca99ffe37a35896d7150e50ac3eee06c7", size = 146749 },
+ { url = "https://pypi.netflix.net/packages/19129800491/tree_sitter-0.25.2-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:44488e0e78146f87baaa009736886516779253d6d6bac3ef636ede72bc6a8234", size = 137766 },
+ { url = "https://pypi.netflix.net/packages/19129800492/tree_sitter-0.25.2-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c2f8e7d6b2f8489d4a9885e3adcaef4bc5ff0a275acd990f120e29c4ab3395c5", size = 599809 },
+ { url = "https://pypi.netflix.net/packages/19129800493/tree_sitter-0.25.2-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:20b570690f87f1da424cd690e51cc56728d21d63f4abd4b326d382a30353acc7", size = 627676 },
+ { url = "https://pypi.netflix.net/packages/19129800494/tree_sitter-0.25.2-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:a0ec41b895da717bc218a42a3a7a0bfcfe9a213d7afaa4255353901e0e21f696", size = 624281 },
+ { url = "https://pypi.netflix.net/packages/19129800495/tree_sitter-0.25.2-cp310-cp310-win_amd64.whl", hash = "sha256:7712335855b2307a21ae86efe949c76be36c6068d76df34faa27ce9ee40ff444", size = 127295 },
+ { url = "https://pypi.netflix.net/packages/19129800496/tree_sitter-0.25.2-cp310-cp310-win_arm64.whl", hash = "sha256:a925364eb7fbb9cdce55a9868f7525a1905af512a559303bd54ef468fd88cb37", size = 113991 },
+ { url = "https://pypi.netflix.net/packages/19129800497/tree_sitter-0.25.2-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:b8ca72d841215b6573ed0655b3a5cd1133f9b69a6fa561aecad40dca9029d75b", size = 146752 },
+ { url = "https://pypi.netflix.net/packages/19129800498/tree_sitter-0.25.2-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:cc0351cfe5022cec5a77645f647f92a936b38850346ed3f6d6babfbeeeca4d26", size = 137765 },
+ { url = "https://pypi.netflix.net/packages/19129800499/tree_sitter-0.25.2-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1799609636c0193e16c38f366bda5af15b1ce476df79ddaae7dd274df9e44266", size = 604643 },
+ { url = "https://pypi.netflix.net/packages/19129800500/tree_sitter-0.25.2-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3e65ae456ad0d210ee71a89ee112ac7e72e6c2e5aac1b95846ecc7afa68a194c", size = 632229 },
+ { url = "https://pypi.netflix.net/packages/19129800501/tree_sitter-0.25.2-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:49ee3c348caa459244ec437ccc7ff3831f35977d143f65311572b8ba0a5f265f", size = 629861 },
+ { url = "https://pypi.netflix.net/packages/19129800502/tree_sitter-0.25.2-cp311-cp311-win_amd64.whl", hash = "sha256:56ac6602c7d09c2c507c55e58dc7026b8988e0475bd0002f8a386cce5e8e8adc", size = 127304 },
+ { url = "https://pypi.netflix.net/packages/19129800503/tree_sitter-0.25.2-cp311-cp311-win_arm64.whl", hash = "sha256:b3d11a3a3ac89bb8a2543d75597f905a9926f9c806f40fcca8242922d1cc6ad5", size = 113990 },
+ { url = "https://pypi.netflix.net/packages/19129801135/tree_sitter-0.25.2-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:ddabfff809ffc983fc9963455ba1cecc90295803e06e140a4c83e94c1fa3d960", size = 146941 },
+ { url = "https://pypi.netflix.net/packages/19129801136/tree_sitter-0.25.2-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:c0c0ab5f94938a23fe81928a21cc0fac44143133ccc4eb7eeb1b92f84748331c", size = 137699 },
+ { url = "https://pypi.netflix.net/packages/19129801137/tree_sitter-0.25.2-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:dd12d80d91d4114ca097626eb82714618dcdfacd6a5e0955216c6485c350ef99", size = 607125 },
+ { url = "https://pypi.netflix.net/packages/19129801138/tree_sitter-0.25.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b43a9e4c89d4d0839de27cd4d6902d33396de700e9ff4c5ab7631f277a85ead9", size = 635418 },
+ { url = "https://pypi.netflix.net/packages/19129801139/tree_sitter-0.25.2-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:fbb1706407c0e451c4f8cc016fec27d72d4b211fdd3173320b1ada7a6c74c3ac", size = 631250 },
+ { url = "https://pypi.netflix.net/packages/19129801140/tree_sitter-0.25.2-cp312-cp312-win_amd64.whl", hash = "sha256:6d0302550bbe4620a5dc7649517c4409d74ef18558276ce758419cf09e578897", size = 127156 },
+ { url = "https://pypi.netflix.net/packages/19129801141/tree_sitter-0.25.2-cp312-cp312-win_arm64.whl", hash = "sha256:0c8b6682cac77e37cfe5cf7ec388844957f48b7bd8d6321d0ca2d852994e10d5", size = 113984 },
+ { url = "https://pypi.netflix.net/packages/19129801142/tree_sitter-0.25.2-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:0628671f0de69bb279558ef6b640bcfc97864fe0026d840f872728a86cd6b6cd", size = 146926 },
+ { url = "https://pypi.netflix.net/packages/19129801143/tree_sitter-0.25.2-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:f5ddcd3e291a749b62521f71fc953f66f5fd9743973fd6dd962b092773569601", size = 137712 },
+ { url = "https://pypi.netflix.net/packages/19129801144/tree_sitter-0.25.2-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bd88fbb0f6c3a0f28f0a68d72df88e9755cf5215bae146f5a1bdc8362b772053", size = 607873 },
+ { url = "https://pypi.netflix.net/packages/19129801145/tree_sitter-0.25.2-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b878e296e63661c8e124177cc3084b041ba3f5936b43076d57c487822426f614", size = 636313 },
+ { url = "https://pypi.netflix.net/packages/19129801146/tree_sitter-0.25.2-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:d77605e0d353ba3fe5627e5490f0fbfe44141bafa4478d88ef7954a61a848dae", size = 631370 },
+ { url = "https://pypi.netflix.net/packages/19129803321/tree_sitter-0.25.2-cp313-cp313-win_amd64.whl", hash = "sha256:463c032bd02052d934daa5f45d183e0521ceb783c2548501cf034b0beba92c9b", size = 127157 },
+ { url = "https://pypi.netflix.net/packages/19129803322/tree_sitter-0.25.2-cp313-cp313-win_arm64.whl", hash = "sha256:b3f63a1796886249bd22c559a5944d64d05d43f2be72961624278eff0dcc5cb8", size = 113975 },
+ { url = "https://pypi.netflix.net/packages/19129803323/tree_sitter-0.25.2-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:65d3c931013ea798b502782acab986bbf47ba2c452610ab0776cf4a8ef150fc0", size = 146776 },
+ { url = "https://pypi.netflix.net/packages/19129803324/tree_sitter-0.25.2-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:bda059af9d621918efb813b22fb06b3fe00c3e94079c6143fcb2c565eb44cb87", size = 137732 },
+ { url = "https://pypi.netflix.net/packages/19129803325/tree_sitter-0.25.2-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:eac4e8e4c7060c75f395feec46421eb61212cb73998dbe004b7384724f3682ab", size = 609456 },
+ { url = "https://pypi.netflix.net/packages/19129803326/tree_sitter-0.25.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:260586381b23be33b6191a07cea3d44ecbd6c01aa4c6b027a0439145fcbc3358", size = 636772 },
+ { url = "https://pypi.netflix.net/packages/19129803327/tree_sitter-0.25.2-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:7d2ee1acbacebe50ba0f85fff1bc05e65d877958f00880f49f9b2af38dce1af0", size = 631522 },
+ { url = "https://pypi.netflix.net/packages/19129803328/tree_sitter-0.25.2-cp314-cp314-win_amd64.whl", hash = "sha256:4973b718fcadfb04e59e746abfbb0288694159c6aeecd2add59320c03368c721", size = 130864 },
+ { url = "https://pypi.netflix.net/packages/19129803329/tree_sitter-0.25.2-cp314-cp314-win_arm64.whl", hash = "sha256:b8d4429954a3beb3e844e2872610d2a4800ba4eb42bb1990c6a4b1949b18459f", size = 117470 },
+]
+
+[[package]]
+name = "tree-sitter-c-sharp"
+version = "0.23.1"
+source = { registry = "https://pypi.netflix.net/simple" }
+sdist = { url = "https://pypi.netflix.net/packages/18519163555/tree_sitter_c_sharp-0.23.1.tar.gz", hash = "sha256:322e2cfd3a547a840375276b2aea3335fa6458aeac082f6c60fec3f745c967eb", size = 1317728 }
+wheels = [
+ { url = "https://pypi.netflix.net/packages/18519163548/tree_sitter_c_sharp-0.23.1-cp39-abi3-macosx_10_9_x86_64.whl", hash = "sha256:2b612a6e5bd17bb7fa2aab4bb6fc1fba45c94f09cb034ab332e45603b86e32fd", size = 372235 },
+ { url = "https://pypi.netflix.net/packages/18519163549/tree_sitter_c_sharp-0.23.1-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:1a8b98f62bc53efcd4d971151950c9b9cd5cbe3bacdb0cd69fdccac63350d83e", size = 419046 },
+ { url = "https://pypi.netflix.net/packages/18519163550/tree_sitter_c_sharp-0.23.1-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:986e93d845a438ec3c4416401aa98e6a6f6631d644bbbc2e43fcb915c51d255d", size = 415999 },
+ { url = "https://pypi.netflix.net/packages/18519163551/tree_sitter_c_sharp-0.23.1-cp39-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a8024e466b2f5611c6dc90321f232d8584893c7fb88b75e4a831992f877616d2", size = 402830 },
+ { url = "https://pypi.netflix.net/packages/18519163552/tree_sitter_c_sharp-0.23.1-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:7f9bf876866835492281d336b9e1f9626ab668737f74e914c31d285261507da7", size = 397880 },
+ { url = "https://pypi.netflix.net/packages/18519163553/tree_sitter_c_sharp-0.23.1-cp39-abi3-win_amd64.whl", hash = "sha256:ae9a9e859e8f44e2b07578d44f9a220d3fa25b688966708af6aa55d42abeebb3", size = 377562 },
+ { url = "https://pypi.netflix.net/packages/18519163554/tree_sitter_c_sharp-0.23.1-cp39-abi3-win_arm64.whl", hash = "sha256:c81548347a93347be4f48cb63ec7d60ef4b0efa91313330e69641e49aa5a08c5", size = 375157 },
+]
+
+[[package]]
+name = "tree-sitter-embedded-template"
+version = "0.25.0"
+source = { registry = "https://pypi.netflix.net/simple" }
+sdist = { url = "https://pypi.netflix.net/packages/19023467751/tree_sitter_embedded_template-0.25.0.tar.gz", hash = "sha256:7d72d5e8a1d1d501a7c90e841b51f1449a90cc240be050e4fb85c22dab991d50", size = 14114 }
+wheels = [
+ { url = "https://pypi.netflix.net/packages/19023467743/tree_sitter_embedded_template-0.25.0-cp310-abi3-macosx_10_9_x86_64.whl", hash = "sha256:fa0d06467199aeb33fb3d6fa0665bf9b7d5a32621ffdaf37fd8249f8a8050649", size = 10266 },
+ { url = "https://pypi.netflix.net/packages/19023467744/tree_sitter_embedded_template-0.25.0-cp310-abi3-macosx_11_0_arm64.whl", hash = "sha256:fc7aacbc2985a5d7e7fe7334f44dffe24c38fb0a8295c4188a04cf21a3d64a73", size = 10650 },
+ { url = "https://pypi.netflix.net/packages/19023467745/tree_sitter_embedded_template-0.25.0-cp310-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:a7c88c3dd8b94b3c9efe8ae071ff6b1b936a27ac5f6e651845c3b9631fa4c1c2", size = 18268 },
+ { url = "https://pypi.netflix.net/packages/19023467746/tree_sitter_embedded_template-0.25.0-cp310-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:025f7ca84218dcd8455efc901bdbcc2689fb694f3a636c0448e322a23d4bc96b", size = 19068 },
+ { url = "https://pypi.netflix.net/packages/19023467747/tree_sitter_embedded_template-0.25.0-cp310-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:b5dc1aef6ffa3fae621fe037d85dd98948b597afba20df29d779c426be813ee5", size = 18518 },
+ { url = "https://pypi.netflix.net/packages/19023467748/tree_sitter_embedded_template-0.25.0-cp310-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:d0a35cfe634c44981a516243bc039874580e02a2990669313730187ce83a5bc6", size = 18267 },
+ { url = "https://pypi.netflix.net/packages/19023467749/tree_sitter_embedded_template-0.25.0-cp310-abi3-win_amd64.whl", hash = "sha256:3e05a4ac013d54505e75ae48e1a0e9db9aab19949fe15d9f4c7345b11a84a069", size = 13049 },
+ { url = "https://pypi.netflix.net/packages/19023467750/tree_sitter_embedded_template-0.25.0-cp310-abi3-win_arm64.whl", hash = "sha256:2751d402179ac0e83f2065b249d8fe6df0718153f1636bcb6a02bde3e5730db9", size = 11978 },
+]
+
+[[package]]
+name = "tree-sitter-language-pack"
+version = "0.13.0"
+source = { registry = "https://pypi.netflix.net/simple" }
+dependencies = [
+ { name = "tree-sitter" },
+ { name = "tree-sitter-c-sharp" },
+ { name = "tree-sitter-embedded-template" },
+ { name = "tree-sitter-yaml" },
+]
+sdist = { url = "https://pypi.netflix.net/packages/19391792931/tree_sitter_language_pack-0.13.0.tar.gz", hash = "sha256:032034c5e27b1f6e00730b9e7c2dbc8203b4700d0c681fd019d6defcf61183ec", size = 51353370 }
+wheels = [
+ { url = "https://pypi.netflix.net/packages/19391792760/tree_sitter_language_pack-0.13.0-cp310-abi3-macosx_10_15_universal2.whl", hash = "sha256:0e7eae812b40a2dc8a12eb2f5c55e130eb892706a0bee06215dd76affeb00d07", size = 32991857 },
+ { url = "https://pypi.netflix.net/packages/19391792761/tree_sitter_language_pack-0.13.0-cp310-abi3-manylinux2014_aarch64.whl", hash = "sha256:7fdacf383418a845b20772118fcb53ad245f9c5d409bd07dae16acec65151756", size = 20092989 },
+ { url = "https://pypi.netflix.net/packages/19391792762/tree_sitter_language_pack-0.13.0-cp310-abi3-manylinux2014_x86_64.whl", hash = "sha256:0d4f261fce387ae040dae7e4d1c1aca63d84c88320afcc0961c123bec0be8377", size = 19952029 },
+ { url = "https://pypi.netflix.net/packages/19391792845/tree_sitter_language_pack-0.13.0-cp310-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:78f369dc4d456c5b08d659939e662c2f9b9fba8c0ec5538a1f973e01edfcf04d", size = 19944614 },
+ { url = "https://pypi.netflix.net/packages/19391792846/tree_sitter_language_pack-0.13.0-cp310-abi3-win_amd64.whl", hash = "sha256:1cdbc88a03dacd47bec69e56cc20c48eace1fbb6f01371e89c3ee6a2e8f34db1", size = 16896852 },
+]
+
+[[package]]
+name = "tree-sitter-yaml"
+version = "0.7.2"
+source = { registry = "https://pypi.netflix.net/simple" }
+sdist = { url = "https://pypi.netflix.net/packages/19176087043/tree_sitter_yaml-0.7.2.tar.gz", hash = "sha256:756db4c09c9d9e97c81699e8f941cb8ce4e51104927f6090eefe638ee567d32c", size = 84882 }
+wheels = [
+ { url = "https://pypi.netflix.net/packages/19176087035/tree_sitter_yaml-0.7.2-cp310-abi3-macosx_10_9_x86_64.whl", hash = "sha256:7e269ddcfcab8edb14fbb1f1d34eed1e1e26888f78f94eedfe7cc98c60f8bc9f", size = 43898 },
+ { url = "https://pypi.netflix.net/packages/19176087036/tree_sitter_yaml-0.7.2-cp310-abi3-macosx_11_0_arm64.whl", hash = "sha256:0807b7966e23ddf7dddc4545216e28b5a58cdadedcecca86b8d8c74271a07870", size = 44691 },
+ { url = "https://pypi.netflix.net/packages/19176087037/tree_sitter_yaml-0.7.2-cp310-abi3-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:f1a5c60c98b6c4c037aae023569f020d0c489fad8dc26fdfd5510363c9c29a41", size = 91430 },
+ { url = "https://pypi.netflix.net/packages/19176087038/tree_sitter_yaml-0.7.2-cp310-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:88636d19d0654fd24f4f242eaaafa90f6f5ebdba8a62e4b32d251ed156c51a2a", size = 92428 },
+ { url = "https://pypi.netflix.net/packages/19176087039/tree_sitter_yaml-0.7.2-cp310-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:1d2e8f0bb14aa4537320952d0f9607eef3021d5aada8383c34ebeece17db1e06", size = 90580 },
+ { url = "https://pypi.netflix.net/packages/19176087040/tree_sitter_yaml-0.7.2-cp310-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:74ca712c50fc9d7dbc68cb36b4a7811d6e67a5466b5a789f19bf8dd6084ef752", size = 90455 },
+ { url = "https://pypi.netflix.net/packages/19176087041/tree_sitter_yaml-0.7.2-cp310-abi3-win_amd64.whl", hash = "sha256:7587b5ca00fc4f9a548eff649697a3b395370b2304b399ceefa2087d8a6c9186", size = 45514 },
+ { url = "https://pypi.netflix.net/packages/19176087042/tree_sitter_yaml-0.7.2-cp310-abi3-win_arm64.whl", hash = "sha256:f63c227b18e7ce7587bce124578f0bbf1f890ac63d3e3cd027417574273642c4", size = 44065 },
+]
+
[[package]]
name = "triton"
version = "3.5.1"