Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

Add Agno integration documentation

chopratejasaaaca9fa6a89
diff --git a/README.md b/README.md
index 92d35e241..69f98d74a 100644
--- a/README.md
+++ b/README.md
@@ -34,7 +34,7 @@
- **Content-aware** - code, logs, JSON each handled optimally
- **Provider caching** - automatic prefix optimization for cache hits
- **Persistent memory** - remember across conversations with zero-latency extraction
-- **Framework native** - LangChain, MCP, agents supported
+- **Framework native** - LangChain, Agno, MCP, agents supported
---
@@ -89,6 +89,30 @@ response = llm.invoke("Hello!")
See the full [LangChain Integration Guide](docs/langchain.md) for memory, retrievers, agents, and more.
+### Option 3: Agno Integration
+
+```bash
+pip install "headroom-ai[agno]"
+```
+
+```python
+from agno.agent import Agent
+from agno.models.openai import OpenAIChat
+from headroom.integrations.agno import HeadroomAgnoModel
+
+# Wrap your model - that's it!
+model = HeadroomAgnoModel(OpenAIChat(id="gpt-4o"))
+agent = Agent(model=model)
+
+# Use exactly like before
+response = agent.run("Hello!")
+
+# Check savings
+print(f"Tokens saved: {model.total_tokens_saved}")
+```
+
+See the full [Agno Integration Guide](docs/agno.md) for hooks, multi-provider support, and more.
+
---
## Framework Integrations
@@ -96,6 +120,7 @@ See the full [LangChain Integration Guide](docs/langchain.md) for memory, retrie
| Framework | Integration | Docs |
|-----------|-------------|------|
| **LangChain** | `HeadroomChatModel`, memory, retrievers, agents | [Guide](docs/langchain.md) |
+| **Agno** | `HeadroomAgnoModel`, hooks, multi-provider | [Guide](docs/agno.md) |
| **MCP** | Tool output compression for Claude | [Guide](docs/ccr.md) |
| **Any OpenAI Client** | Proxy server | [Guide](docs/proxy.md) |
@@ -112,6 +137,7 @@ See the full [LangChain Integration Guide](docs/langchain.md) for memory, retrie
| **RollingWindow** | Manages context limits without breaking tools | [Transforms](docs/transforms.md) |
| **CCR** | Reversible compression with automatic retrieval | [CCR Guide](docs/ccr.md) |
| **LangChain** | Memory, retrievers, agents, streaming | [LangChain](docs/langchain.md) |
+| **Agno** | Agent framework integration with hooks | [Agno](docs/agno.md) |
| **Text Utilities** | Opt-in compression for search/logs | [Text Compression](docs/text-compression.md) |
| **LLMLingua-2** | ML-based 20x compression (opt-in) | [LLMLingua](docs/llmlingua.md) |
| **Code-Aware** | AST-based code compression (tree-sitter) | [Transforms](docs/transforms.md) |
@@ -160,6 +186,7 @@ New models auto-supported via naming pattern detection.
pip install headroom-ai # SDK only
pip install "headroom-ai[proxy]" # Proxy server
pip install "headroom-ai[langchain]" # LangChain integration
+pip install "headroom-ai[agno]" # Agno agent framework
pip install "headroom-ai[code]" # AST-based code compression
pip install "headroom-ai[llmlingua]" # ML-based compression
pip install "headroom-ai[all]" # Everything
@@ -176,6 +203,7 @@ pip install "headroom-ai[all]" # Everything
| [Memory Guide](docs/memory.md) | Persistent memory for LLMs |
| [Compression Guide](docs/compression.md) | Universal compression with ML detection |
| [LangChain Integration](docs/langchain.md) | Full LangChain support |
+| [Agno Integration](docs/agno.md) | Full Agno agent framework support |
| [SDK Guide](docs/sdk.md) | Fine-grained control |
| [Proxy Guide](docs/proxy.md) | Production deployment |
| [Configuration](docs/configuration.md) | All options |
diff --git a/docs/README.md b/docs/README.md
index 4fdba15cf..f96622e79 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -15,6 +15,7 @@ Welcome to the Headroom documentation.
| Framework | Description |
|-----------|-------------|
| [LangChain](langchain.md) | Chat models, memory, retrievers, agents, streaming |
+| [Agno](agno.md) | Model wrapper, hooks, multi-provider support |
| MCP | See [CCR Guide](ccr.md) for tool compression |
## Core Concepts
diff --git a/docs/agno.md b/docs/agno.md
new file mode 100644
index 000000000..6dec29975
--- /dev/null
+++ b/docs/agno.md
@@ -0,0 +1,445 @@
+# Agno Integration
+
+Headroom integrates with [Agno](https://github.com/agno-agi/agno) (formerly Phidata) to provide automatic context optimization for AI agents. This guide covers model wrapping, observability hooks, and multi-provider support.
+
+---
+
+## Installation
+
+```bash
+pip install "headroom-ai[agno]"
+```
+
+This installs Headroom with Agno support. You'll also need Agno itself:
+
+```bash
+pip install agno
+```
+
+---
+
+## Quick Start
+
+```python
+from agno.agent import Agent
+from agno.models.openai import OpenAIChat
+from headroom.integrations.agno import HeadroomAgnoModel
+
+# Wrap your model
+model = HeadroomAgnoModel(OpenAIChat(id="gpt-4o"))
+
+# Create agent as usual
+agent = Agent(model=model)
+
+# Use exactly like before
+response = agent.run("What's the capital of France?")
+
+# Check savings
+print(f"Tokens saved: {model.total_tokens_saved}")
+print(model.get_savings_summary())
+# {'total_requests': 1, 'total_tokens_saved': 245, 'average_savings_percent': 12.3}
+```
+
+---
+
+## Integration Patterns
+
+### 1. Basic Model Wrapping
+
+The simplest integration - wrap any Agno model with `HeadroomAgnoModel`:
+
+```python
+from agno.models.openai import OpenAIChat
+from agno.models.anthropic import Claude
+from agno.models.google import Gemini
+from headroom.integrations.agno import HeadroomAgnoModel
+
+# Works with any Agno model
+openai_model = HeadroomAgnoModel(OpenAIChat(id="gpt-4o"))
+claude_model = HeadroomAgnoModel(Claude(id="claude-3-5-sonnet-20241022"))
+gemini_model = HeadroomAgnoModel(Gemini(id="gemini-2.0-flash"))
+
+# Each automatically uses the correct provider for accurate token counting
+```
+
+**Why this matters**: Headroom automatically detects the underlying provider and applies the correct tokenizer for accurate optimization metrics.
+
+### 2. Agent with Observability Hooks
+
+Use hooks for detailed tracking without modifying your model:
+
+```python
+from agno.agent import Agent
+from agno.models.openai import OpenAIChat
+from headroom.integrations.agno import (
+ HeadroomAgnoModel,
+ HeadroomPreHook,
+ HeadroomPostHook,
+)
+
+# Model wrapper for optimization
+model = HeadroomAgnoModel(OpenAIChat(id="gpt-4o"))
+
+# Hooks for observability
+pre_hook = HeadroomPreHook()
+post_hook = HeadroomPostHook(token_alert_threshold=10000)
+
+agent = Agent(
+ model=model,
+ pre_hooks=[pre_hook],
+ post_hooks=[post_hook],
+)
+
+# Run agent
+response = agent.run("Analyze this large dataset...")
+
+# Check metrics from model
+print(f"Tokens saved: {model.total_tokens_saved}")
+
+# Check observability from hooks
+print(f"Post-hook summary: {post_hook.get_summary()}")
+print(f"Alerts triggered: {post_hook.alerts}")
+```
+
+**Why this matters**: Hooks provide observability into agent behavior and can alert when token usage exceeds thresholds.
+
+### 3. Convenience Hook Factory
+
+Use `create_headroom_hooks()` to create matched hook pairs:
+
+```python
+from headroom.integrations.agno import create_headroom_hooks
+
+pre_hook, post_hook = create_headroom_hooks(
+ token_alert_threshold=5000,
+ log_level="DEBUG",
+)
+
+agent = Agent(
+ model=model,
+ pre_hooks=[pre_hook],
+ post_hooks=[post_hook],
+)
+```
+
+### 4. Custom Configuration
+
+Pass a `HeadroomConfig` for fine-grained control:
+
+```python
+from headroom import HeadroomConfig, HeadroomMode
+from headroom.integrations.agno import HeadroomAgnoModel
+
+config = HeadroomConfig(
+ default_mode=HeadroomMode.OPTIMIZE,
+ # Add other configuration options as needed
+)
+
+model = HeadroomAgnoModel(
+ wrapped_model=OpenAIChat(id="gpt-4o"),
+ config=config,
+)
+```
+
+### 5. Standalone Message Optimization
+
+Optimize messages without wrapping a model:
+
+```python
+from headroom.integrations.agno import optimize_messages
+
+messages = [
+ {"role": "system", "content": "You are a helpful assistant."},
+ {"role": "user", "content": "Analyze this large JSON: " + large_json},
+]
+
+optimized_messages, metrics = optimize_messages(messages, model="gpt-4o")
+
+print(f"Tokens saved: {metrics['tokens_saved']}")
+print(f"Transforms applied: {metrics['transforms_applied']}")
+```
+
+### 6. Async Operations
+
+Full async support for high-throughput applications:
+
+```python
+import asyncio
+from headroom.integrations.agno import HeadroomAgnoModel
+
+async def process_async():
+ model = HeadroomAgnoModel(OpenAIChat(id="gpt-4o"))
+
+ # Async response
+ response = await model.aresponse(messages)
+
+ # Async streaming
+ async for chunk in model.aresponse_stream(messages):
+ print(chunk, end="", flush=True)
+
+ print(f"\nTokens saved: {model.total_tokens_saved}")
+
+asyncio.run(process_async())
+```
+
+---
+
+## Real-World Examples
+
+### Example 1: Tool-Heavy Agent
+
+```python
+from agno.agent import Agent
+from agno.models.openai import OpenAIChat
+from agno.tools.duckduckgo import DuckDuckGoTools
+from headroom.integrations.agno import HeadroomAgnoModel
+
+# Wrap model for optimization
+model = HeadroomAgnoModel(OpenAIChat(id="gpt-4o"))
+
+# Agent with search tools
+agent = Agent(
+ model=model,
+ tools=[DuckDuckGoTools()],
+ show_tool_calls=True,
+)
+
+# Tool outputs get compressed automatically
+response = agent.run("Research the latest AI developments and summarize")
+
+# Impact: Tool outputs (often 10K+ tokens) compressed by 70-90%
+print(f"Tokens saved: {model.total_tokens_saved}")
+print(model.get_savings_summary())
+```
+
+### Example 2: Multi-Model Routing
+
+```python
+from agno.models.openai import OpenAIChat
+from agno.models.anthropic import Claude
+from headroom.integrations.agno import HeadroomAgnoModel
+
+# Different models for different tasks
+fast_model = HeadroomAgnoModel(OpenAIChat(id="gpt-4o-mini"))
+powerful_model = HeadroomAgnoModel(Claude(id="claude-3-5-sonnet-20241022"))
+
+# Use fast model for simple tasks
+simple_agent = Agent(model=fast_model)
+
+# Use powerful model for complex reasoning
+complex_agent = Agent(model=powerful_model)
+
+# Each tracks its own metrics
+print(f"Fast model saved: {fast_model.total_tokens_saved}")
+print(f"Powerful model saved: {powerful_model.total_tokens_saved}")
+```
+
+### Example 3: Production Monitoring
+
+```python
+from agno.agent import Agent
+from headroom.integrations.agno import (
+ HeadroomAgnoModel,
+ create_headroom_hooks,
+)
+
+model = HeadroomAgnoModel(OpenAIChat(id="gpt-4o"))
+pre_hook, post_hook = create_headroom_hooks(
+ token_alert_threshold=50000, # Alert on large requests
+ log_level="WARNING",
+)
+
+agent = Agent(
+ model=model,
+ pre_hooks=[pre_hook],
+ post_hooks=[post_hook],
+)
+
+# Run multiple requests
+for query in user_queries:
+ response = agent.run(query)
+
+# Check for alerts
+if post_hook.alerts:
+ print(f"WARNING: {len(post_hook.alerts)} requests exceeded threshold")
+ for alert in post_hook.alerts:
+ print(f" - {alert}")
+
+# Summary stats
+summary = post_hook.get_summary()
+print(f"Total requests: {summary['total_requests']}")
+print(f"Average tokens: {summary['average_tokens']}")
+```
+
+### Example 4: Reset for New Sessions
+
+```python
+model = HeadroomAgnoModel(OpenAIChat(id="gpt-4o"))
+
+# Session 1
+agent.run("First conversation...")
+print(f"Session 1 savings: {model.get_savings_summary()}")
+
+# Reset for new session
+model.reset()
+
+# Session 2 - metrics start fresh
+agent.run("Second conversation...")
+print(f"Session 2 savings: {model.get_savings_summary()}")
+```
+
+---
+
+## Supported Providers
+
+HeadroomAgnoModel automatically detects the provider from the wrapped model:
+
+| Provider | Agno Models | Auto-Detected |
+|----------|-------------|---------------|
+| **OpenAI** | `OpenAIChat`, `OpenAILike` | Yes |
+| **Anthropic** | `Claude`, `AwsBedrock` | Yes |
+| **Google** | `Gemini`, `VertexAI` | Yes |
+| **Cohere** | `Cohere`, `CohereChat` | Yes |
+| **Groq** | `Groq` | Yes (OpenAI-compatible) |
+| **Mistral** | `Mistral` | Yes (OpenAI-compatible) |
+| **Together** | `Together` | Yes (OpenAI-compatible) |
+| **Ollama** | `Ollama` | Yes (OpenAI-compatible) |
+
+To disable auto-detection:
+
+```python
+model = HeadroomAgnoModel(
+ wrapped_model=some_model,
+ auto_detect_provider=False, # Falls back to OpenAI tokenizer
+)
+```
+
+---
+
+## Configuration Reference
+
+### HeadroomAgnoModel
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `wrapped_model` | Any | Required | The Agno model to wrap |
+| `config` | `HeadroomConfig` | `None` | Custom configuration |
+| `auto_detect_provider` | `bool` | `True` | Auto-detect provider for token counting |
+
+**Properties:**
+- `wrapped_model` - Access the underlying Agno model
+- `total_tokens_saved` - Running total of tokens saved
+- `metrics_history` - List of last 100 `OptimizationMetrics`
+
+**Methods:**
+- `response(messages, **kwargs)` - Sync response with optimization
+- `response_stream(messages, **kwargs)` - Sync streaming response
+- `aresponse(messages, **kwargs)` - Async response
+- `aresponse_stream(messages, **kwargs)` - Async streaming
+- `get_savings_summary()` - Returns dict with stats
+- `reset()` - Clear all metrics
+
+### HeadroomPreHook
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `config` | `HeadroomConfig` | `None` | Configuration (for future use) |
+| `model` | `str` | `"gpt-4o"` | Model name for estimation |
+
+### HeadroomPostHook
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `log_level` | `str` | `"INFO"` | Logging level |
+| `token_alert_threshold` | `int` | `None` | Alert if tokens exceed this |
+
+**Properties:**
+- `total_requests` - Number of requests tracked
+- `alerts` - List of alert messages
+
+**Methods:**
+- `get_summary()` - Returns dict with request stats
+- `reset()` - Clear history and alerts
+
+### create_headroom_hooks()
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `config` | `HeadroomConfig` | `None` | Config for pre-hook |
+| `model` | `str` | `"gpt-4o"` | Model for pre-hook |
+| `log_level` | `str` | `"INFO"` | Log level for post-hook |
+| `token_alert_threshold` | `int` | `None` | Alert threshold for post-hook |
+
+Returns: `tuple[HeadroomPreHook, HeadroomPostHook]`
+
+---
+
+## Import Reference
+
+```python
+# Main integration
+from headroom.integrations.agno import HeadroomAgnoModel
+
+# Hooks
+from headroom.integrations.agno import HeadroomPreHook
+from headroom.integrations.agno import HeadroomPostHook
+from headroom.integrations.agno import create_headroom_hooks
+
+# Utilities
+from headroom.integrations.agno import optimize_messages
+from headroom.integrations.agno import agno_available
+from headroom.integrations.agno import get_headroom_provider
+from headroom.integrations.agno import get_model_name_from_agno
+
+# Or import everything from parent
+from headroom.integrations import (
+ HeadroomAgnoModel,
+ HeadroomPreHook,
+ HeadroomPostHook,
+ create_headroom_hooks,
+)
+```
+
+---
+
+## Troubleshooting
+
+### Check if Agno is Available
+
+```python
+from headroom.integrations.agno import agno_available
+
+if agno_available():
+ from headroom.integrations.agno import HeadroomAgnoModel
+else:
+ print("Install agno: pip install agno")
+```
+
+### Provider Detection Issues
+
+If auto-detection fails, check the detected provider:
+
+```python
+from headroom.integrations.agno import get_headroom_provider, get_model_name_from_agno
+
+model = OpenAIChat(id="gpt-4o")
+provider = get_headroom_provider(model)
+model_name = get_model_name_from_agno(model)
+
+print(f"Detected provider: {type(provider).__name__}")
+print(f"Model name: {model_name}")
+```
+
+### Metrics Not Updating
+
+Ensure you're checking the correct object:
+
+```python
+# Model metrics (optimization)
+print(model.total_tokens_saved) # Actual savings
+
+# Hook metrics (observability)
+print(post_hook.get_summary()) # Request tracking
+```
+
+Note: Hooks track request counts, not token savings. Use the model wrapper for optimization metrics.