Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
Add persistent memory system with zero-latency inline extraction
chopratejas9c9bb30deda9
diff --git a/README.md b/README.mdindex bffaaf093..1e7f4045f 100644--- a/README.md+++ b/README.md@@ -23,18 +23,31 @@</a></p>+----## What It Does+## Why Headroom?++- **Zero code changes** - works as a transparent proxy+- **50-90% cost savings** - verified on real workloads+- **Reversible compression** - LLM retrieves original data via CCR+- **Content-aware** - code, logs, JSON each handled optimally+- **Provider caching** - automatic prefix optimization for cache hits+- **Persistent memory** - remember across conversations with zero-latency extraction+- **Framework native** - LangChain, MCP, agents supported-Headroom is a **smart compression layer** for LLM applications:+----- **Compresses tool outputs** — 1000 search results → 15 items (keeps errors, anomalies, relevant items)-- **Enables provider caching** — Stabilizes prefixes so cache hits actually happen-- **Manages context windows** — Prevents token limit failures without breaking tool calls-- **Reversible compression** — LLM can retrieve original data if needed ([CCR architecture](docs/ccr.md))+## Headroom vs Alternatives-Works as a **proxy** (zero code changes) or **SDK** (fine-grained control).+| Approach | Token Reduction | Accuracy | Reversible | Latency |+|----------|-----------------|----------|------------|---------|+| **Headroom** | 50-90% | No loss | Yes (CCR) | ~1-5ms |+| Truncation | Variable | Data loss | No | ~0ms |+| Summarization | 60-80% | Lossy | No | ~500ms+ |+| No optimization | 0% | Full | N/A | 0ms |++**Headroom wins** because it intelligently selects relevant content while keeping a retrieval path to the original data.---@@ -86,74 +99,13 @@ See the full [LangChain Integration Guide](docs/langchain.md) for memory, retrie| **MCP** | Tool output compression for Claude | [Guide](docs/ccr.md) || **Any OpenAI Client** | Proxy server | [Guide](docs/proxy.md) |-### LangChain Highlights--```python-from headroom.integrations import (- HeadroomChatModel, # Wrap any chat model- HeadroomChatMessageHistory, # Auto-compress conversation history- HeadroomDocumentCompressor, # Filter retrieved documents- wrap_tools_with_headroom, # Compress agent tool outputs-)--# Memory that auto-compresses when over 4K tokens-memory = ConversationBufferMemory(- chat_memory=HeadroomChatMessageHistory(base_history)-)--# Retriever that keeps only relevant docs-retriever = ContextualCompressionRetriever(- base_compressor=HeadroomDocumentCompressor(max_documents=10),- base_retriever=vectorstore.as_retriever(search_kwargs={"k": 50}),-)--# Agent tools with automatic output compression-tools = wrap_tools_with_headroom([search_tool, database_tool])-```-------## Verify It's Working--```bash-curl http://localhost:8787/stats-```--```json-{- "tokens": {"saved": 12500, "savings_percent": 25.0},- "cost": {"total_savings_usd": 0.04}-}-```--Or in Python:--```python-print(llm.get_metrics())-# {'tokens_saved': 12500, 'savings_percent': 45.2}-```-------## Installation--```bash-pip install headroom-ai # SDK only-pip install "headroom-ai[proxy]" # Proxy server-pip install "headroom-ai[langchain]" # LangChain integration-pip install "headroom-ai[code]" # AST-based code compression-pip install "headroom-ai[llmlingua]" # ML-based compression-pip install "headroom-ai[all]" # Everything-```--**Requirements**: Python 3.10+----## Features| Feature | Description | Docs ||---------|-------------|------|+| **Memory** | Persistent memory across conversations (zero-latency inline extraction) | [Memory](docs/memory.md) || **SmartCrusher** | Compresses JSON tool outputs statistically | [Transforms](docs/transforms.md) || **CacheAligner** | Stabilizes prefixes for provider caching | [Transforms](docs/transforms.md) || **RollingWindow** | Manages context limits without breaking tools | [Transforms](docs/transforms.md) |@@ -165,6 +117,19 @@ pip install "headroom-ai[all]" # Everything---+## Performance++| Scenario | Before | After | Savings |+|----------|--------|-------|---------|+| Search results (1000 items) | 45,000 tokens | 4,500 tokens | 90% |+| Log analysis (500 entries) | 22,000 tokens | 3,300 tokens | 85% |+| Long conversation (50 turns) | 80,000 tokens | 32,000 tokens | 60% |+| Agent with tools (10 calls) | 100,000 tokens | 15,000 tokens | 85% |++**Overhead**: ~1-5ms per request++---+## Providers| Provider | Token Counting | Cache Optimization |@@ -175,29 +140,31 @@ pip install "headroom-ai[all]" # Everything| Cohere | Official API | - || Mistral | Official tokenizer | - |-**New models auto-supported** — Unknown models get sensible defaults based on naming patterns.+New models auto-supported via naming pattern detection.----## Performance+## Safety Guarantees-| Scenario | Before | After | Savings |-|----------|--------|-------|---------|-| Search results (1000 items) | 45,000 tokens | 4,500 tokens | 90% |-| Log analysis (500 entries) | 22,000 tokens | 3,300 tokens | 85% |-| Long conversation (50 turns) | 80,000 tokens | 32,000 tokens | 60% |-| Agent with tools (10 calls) | 100,000 tokens | 15,000 tokens | 85% |--Overhead: ~1-5ms per request.+- **Never removes human content** - user/assistant messages preserved+- **Never breaks tool ordering** - tool calls and responses stay paired+- **Parse failures are no-ops** - malformed content passes through unchanged+- **Compression is reversible** - LLM retrieves original data via CCR----## Safety+## Installation++```bash+pip install headroom-ai # SDK only+pip install "headroom-ai[proxy]" # Proxy server+pip install "headroom-ai[langchain]" # LangChain integration+pip install "headroom-ai[code]" # AST-based code compression+pip install "headroom-ai[llmlingua]" # ML-based compression+pip install "headroom-ai[all]" # Everything+```-- **Never removes human content** — User/assistant messages are never compressed-- **Never breaks tool ordering** — Tool calls and responses stay paired-- **Parse failures are no-ops** — Malformed content passes through unchanged-- **Compression is reversible** — LLM can retrieve original data via CCR+**Requirements**: Python 3.10+---@@ -205,25 +172,20 @@ Overhead: ~1-5ms per request.| Guide | Description ||-------|-------------|+| [Memory Guide](docs/memory.md) | Persistent memory for LLMs || [LangChain Integration](docs/langchain.md) | Full LangChain support |-| [SDK Guide](docs/sdk.md) | Wrap your client for fine-grained control |+| [SDK Guide](docs/sdk.md) | Fine-grained control || [Proxy Guide](docs/proxy.md) | Production deployment |-| [Configuration](docs/configuration.md) | All configuration options |-| [CCR Guide](docs/ccr.md) | Reversible compression architecture |-| [Metrics](docs/metrics.md) | Monitoring and observability |+| [Configuration](docs/configuration.md) | All options |+| [CCR Guide](docs/ccr.md) | Reversible compression |+| [Metrics](docs/metrics.md) | Monitoring || [Troubleshooting](docs/troubleshooting.md) | Common issues |----## Examples--See [`examples/`](examples/) for runnable code:+## Who's Using Headroom?-- `basic_usage.py` — Simple SDK usage-- `proxy_integration.py` — Using with different clients-- `langchain_agent.py` — LangChain ReAct agent with Headroom-- `rag_pipeline.py` — RAG with document compression-- `ccr_demo.py` — CCR architecture demonstration+> Add your project here! [Open a PR](https://github.com/chopratejas/headroom/pulls) or [start a discussion](https://github.com/chopratejas/headroom/discussions).---@@ -242,7 +204,7 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for details.## License-Apache License 2.0 — see [LICENSE](LICENSE).+Apache License 2.0 - see [LICENSE](LICENSE).---diff --git a/docs/memory.md b/docs/memory.mdnew file mode 100644index 000000000..1fd0b4f2a--- /dev/null+++ b/docs/memory.md@@ -0,0 +1,332 @@+# Memory++**Persistent memory for LLM applications.** Enable your AI to remember across conversations without carrying full history.++## Why Memory?++LLMs have two fundamental limitations:+1. **Context windows overflow** - Too much history, need to truncate+2. **No persistence** - Every conversation starts from zero++Memory solves both: **extract key facts, persist them, inject when relevant.**++This is *temporal compression* - instead of carrying 10,000 tokens of conversation history, carry 100 tokens of extracted memories.++---++## Quick Start++### Zero-Latency Memory (Recommended)++```python+from openai import OpenAI+from headroom.memory import with_fast_memory++# One line - that's it+client = with_fast_memory(OpenAI(), user_id="alice")++# Use exactly like normal+response = client.chat.completions.create(+ model="gpt-4o",+ messages=[{"role": "user", "content": "I prefer Python for backend work"}]+)+# Memory extracted INLINE - zero extra latency++# Later, in a new conversation...+response = client.chat.completions.create(+ model="gpt-4o",+ messages=[{"role": "user", "content": "What language should I use?"}]+)+# → Response uses the Python preference from memory+```++### How It Works++```+┌─────────────────────────────────────────────────────────────┐+│ with_fast_memory() │+│ │+│ 1. INJECT: Search memories → prepend to user message │+│ 2. INSTRUCT: Add memory extraction instruction │+│ 3. CALL: Forward to LLM │+│ 4. PARSE: Extract <memory> block from response │+│ 5. STORE: Save memories with embeddings │+│ 6. RETURN: Clean response (without memory block) │+│ │+└─────────────────────────────────────────────────────────────┘+```++**Key insight**: Memory extraction happens *inline* as part of the LLM response. No extra API calls, no extra latency.++---++## Two Approaches++### 1. Fast Memory (Inline Extraction)++```python+from headroom.memory import with_fast_memory++client = with_fast_memory(+ OpenAI(),+ user_id="alice",+ db_path="memory.db", # SQLite storage+ top_k=5, # Memories to inject+ use_local_embeddings=True, # Local model (fast) vs OpenAI API+)+```++**Characteristics:**+- Zero extra latency (extraction is part of response)+- ~100 extra output tokens per response+- Smart extraction (LLM decides what's important)+- Semantic retrieval (vector similarity)++### 2. Background Memory (Separate Extraction)++```python+from headroom.memory import with_memory++client = with_memory(+ OpenAI(),+ user_id="alice",+ db_path="memory.db",+)+```++**Characteristics:**+- Non-blocking (extraction happens in background worker)+- Separate LLM call for extraction+- Good when you don't want to modify responses++---++## Memory API++Both wrappers provide a `.memory` API for direct access:++```python+client = with_fast_memory(OpenAI(), user_id="alice")++# Search memories+results = client.memory.search("python preferences", top_k=5)+for memory, score in results:+ print(f"{score:.2f}: {memory.text}")++# Add manual memory+client.memory.add("User is a senior engineer", category="fact")++# Get all memories+all_memories = client.memory.get_all()++# Clear memories+client.memory.clear()++# Get stats+stats = client.memory.stats()+print(f"Total memories: {stats['total_chunks']}")+```++---++## Memory Categories++Memories are categorized for better organization:++| Category | Description | Examples |+|----------|-------------|----------|+| `preference` | Likes, dislikes, preferred approaches | "Prefers Python", "Likes async/await" |+| `fact` | Identity, role, constraints | "Works at fintech startup", "Senior engineer" |+| `context` | Current goals, ongoing tasks | "Migrating to microservices", "Working on auth" |++---++## Configuration++### Storage++```python+# SQLite (default, local)+client = with_fast_memory(OpenAI(), user_id="alice", db_path="memory.db")++# Custom path+client = with_fast_memory(OpenAI(), user_id="alice", db_path="/data/memories.db")+```++### Embeddings++```python+# Local embeddings (recommended - fast, free)+client = with_fast_memory(+ OpenAI(),+ user_id="alice",+ use_local_embeddings=True,+ embedding_model="all-MiniLM-L6-v2", # 384 dimensions+)++# OpenAI embeddings (higher quality, costs money)+client = with_fast_memory(+ OpenAI(),+ user_id="alice",+ use_local_embeddings=False, # Uses text-embedding-3-small+)+```++### Retrieval++```python+# Number of memories to inject+client = with_fast_memory(+ OpenAI(),+ user_id="alice",+ top_k=10, # Inject up to 10 relevant memories+)+```++---++## Multi-User Isolation++Memories are isolated by `user_id`:++```python+# Alice's memories+alice_client = with_fast_memory(OpenAI(), user_id="alice")++# Bob's memories (completely separate)+bob_client = with_fast_memory(OpenAI(), user_id="bob")++# Agent memories+agent_client = with_fast_memory(OpenAI(), user_id="agent-researcher")+```++---++## How Memory Enables Compression++Memory is *temporal compression*. Instead of carrying full conversation history:++```+WITHOUT MEMORY:+Context = Turn 1 + Turn 2 + ... + Turn 50 = 10,000 tokens++WITH MEMORY:+Context = 5 relevant memories = 100 tokens+Compression ratio: 100x+```++This lets you use aggressive rolling window truncation while preserving important facts.++```python+from headroom.memory import with_fast_memory+from headroom.transforms import RollingWindowTransform++# Memory + aggressive truncation = best of both worlds+client = with_fast_memory(OpenAI(), user_id="alice")+transform = RollingWindowTransform(max_tokens=4000)++# Old messages get truncated, but key facts live in memory+messages = transform.apply(very_long_conversation)+response = client.chat.completions.create(model="gpt-4o", messages=messages)+```++---++## Performance++| Operation | Latency | Notes |+|-----------|---------|-------|+| Memory injection | <50ms | Local embeddings + vector search |+| Memory extraction | +50-100ms | Part of LLM response (inline) |+| Memory storage | <10ms | SQLite write + cache update |++**Overhead**: ~100 extra output tokens per response for the `<memory>` block.++---++## Providers++Memory works with any OpenAI-compatible client:++```python+from openai import OpenAI+from anthropic import Anthropic+from groq import Groq++# OpenAI+client = with_fast_memory(OpenAI(), user_id="alice")++# Anthropic (via OpenAI-compatible wrapper)+client = with_fast_memory(OpenAI(base_url="..."), user_id="alice")++# Groq+client = with_fast_memory(Groq(), user_id="alice")++# Any OpenAI-compatible client+client = with_fast_memory(YourClient(), user_id="alice")+```++---++## Example: Multi-Turn Conversation++```python+from openai import OpenAI+from headroom.memory import with_fast_memory++client = with_fast_memory(OpenAI(), user_id="developer_jane")++# Conversation 1: User shares context+response = client.chat.completions.create(+ model="gpt-4o",+ messages=[{+ "role": "user",+ "content": "I'm a Python developer at a fintech startup. We use PostgreSQL."+ }]+)+# Memories extracted: "Python developer", "fintech startup", "uses PostgreSQL"++# Conversation 2 (new session): User asks question+response = client.chat.completions.create(+ model="gpt-4o",+ messages=[{+ "role": "user",+ "content": "What database should I use for my new project?"+ }]+)+# Response references PostgreSQL preference from memory+print(response.choices[0].message.content)+# → "Given your experience with PostgreSQL at your fintech company..."+```++---++## Troubleshooting++### Memories not being extracted++1. Check if the conversation has memory-worthy content (not just greetings)+2. Verify the LLM is following the memory instruction+3. Check logs for parsing errors++### Memories not being retrieved++1. Verify `user_id` matches between sessions+2. Check if memories exist: `client.memory.get_all()`+3. Try a more specific search query++### High latency++1. Switch to local embeddings: `use_local_embeddings=True`+2. Reduce `top_k` for fewer memories to retrieve+3. Check database size and consider pruning old memories++---++## Best Practices++1. **Use consistent `user_id`** - Same ID across sessions for continuity+2. **Start with local embeddings** - Faster, free, good enough for most cases+3. **Combine with rolling window** - Memory + truncation = aggressive compression+4. **Monitor memory growth** - Periodically review and prune if needed+5. **Use categories** - Helps with debugging and selective retrievaldiff --git a/examples/fast_memory_e2e.py b/examples/fast_memory_e2e.pynew file mode 100644index 000000000..505b4c5c1--- /dev/null+++ b/examples/fast_memory_e2e.py@@ -0,0 +1,240 @@+#!/usr/bin/env python3+"""End-to-End Test: Fast Memory with Zero-Latency Extraction.++This demonstrates the complete flow:+1. User shares information → Memory extracted INLINE (no extra latency)+2. User asks follow-up → Memory retrieved semantically+3. Assistant uses memory in response++Usage:+ export OPENAI_API_KEY="sk-..."+ python examples/fast_memory_e2e.py+"""++from __future__ import annotations++import os+import sys+import tempfile+import time+from pathlib import Path++sys.path.insert(0, str(Path(__file__).parent.parent))++from openai import OpenAI++from headroom.memory.fast_wrapper import with_fast_memory+++def run_conversation_test():+ """Test multi-turn conversation with memory."""+ print("=" * 70)+ print("FAST MEMORY E2E TEST")+ print("Zero-latency inline extraction + semantic retrieval")+ print("=" * 70)++ api_key = os.environ.get("OPENAI_API_KEY")+ if not api_key:+ print("ERROR: OPENAI_API_KEY not set")+ sys.exit(1)++ openai_client = OpenAI(api_key=api_key)++ # Use temp directory for clean test+ with tempfile.TemporaryDirectory() as tmpdir:+ db_path = Path(tmpdir) / "test_memory.db"++ # Create wrapped client+ print("\n📦 Creating fast memory client...")+ print(" Using local embeddings (sentence-transformers)")++ client = with_fast_memory(+ openai_client,+ user_id="test_user",+ db_path=db_path,+ use_local_embeddings=True,+ )++ # Conversation 1: Share preferences+ print("\n" + "─" * 70)+ print("TURN 1: User shares preferences")+ print("─" * 70)++ user_msg1 = "I'm a Python developer who prefers async/await patterns. I work at a fintech company and we use PostgreSQL."++ print(f"\n🧑 User: {user_msg1}")++ start = time.perf_counter()+ response1 = client.chat.completions.create(+ model="gpt-4o-mini",+ messages=[+ {"role": "system", "content": "You are a helpful assistant."},+ {"role": "user", "content": user_msg1},+ ],+ )+ latency1 = time.perf_counter() - start++ print(f"\n🤖 Assistant: {response1.choices[0].message.content[:200]}...")+ print(f"\n⏱️ Latency: {latency1 * 1000:.0f}ms (includes inline memory extraction)")++ # Check what was stored+ memories = client.memory.get_all()+ print(f"\n📝 Memories stored: {len(memories)}")+ for mem in memories:+ print(f" - {mem.text}")++ # Conversation 2: Ask related question+ print("\n" + "─" * 70)+ print("TURN 2: User asks related question (memory should be retrieved)")+ print("─" * 70)++ user_msg2 = "What database should I use for my new project?"++ print(f"\n🧑 User: {user_msg2}")++ start = time.perf_counter()+ response2 = client.chat.completions.create(+ model="gpt-4o-mini",+ messages=[+ {"role": "system", "content": "You are a helpful assistant."},+ {"role": "user", "content": user_msg2},+ ],+ )+ latency2 = time.perf_counter() - start++ print(f"\n🤖 Assistant: {response2.choices[0].message.content}")+ print(f"\n⏱️ Latency: {latency2 * 1000:.0f}ms")++ # Check if PostgreSQL is mentioned (should be from memory)+ response_text = response2.choices[0].message.content.lower()+ if "postgresql" in response_text or "postgres" in response_text:+ print("\n✅ SUCCESS: Assistant referenced PostgreSQL from memory!")+ else:+ print("\n⚠️ Note: Assistant didn't explicitly mention PostgreSQL")+ print(" (Memory was still injected - check if response is contextual)")++ # Conversation 3: Different topic+ print("\n" + "─" * 70)+ print("TURN 3: User asks about coding patterns")+ print("─" * 70)++ user_msg3 = "What's the best way to handle concurrent operations in my code?"++ print(f"\n🧑 User: {user_msg3}")++ start = time.perf_counter()+ response3 = client.chat.completions.create(+ model="gpt-4o-mini",+ messages=[+ {"role": "system", "content": "You are a helpful assistant."},+ {"role": "user", "content": user_msg3},+ ],+ )+ latency3 = time.perf_counter() - start++ print(f"\n🤖 Assistant: {response3.choices[0].message.content}")+ print(f"\n⏱️ Latency: {latency3 * 1000:.0f}ms")++ # Check if async/await is mentioned+ response_text = response3.choices[0].message.content.lower()+ if "async" in response_text or "await" in response_text:+ print("\n✅ SUCCESS: Assistant referenced async/await from memory!")+ else:+ print("\n⚠️ Note: Assistant didn't explicitly mention async/await")++ # Final summary+ print("\n" + "=" * 70)+ print("SUMMARY")+ print("=" * 70)++ all_memories = client.memory.get_all()+ avg_latency = (latency1 + latency2 + latency3) / 3++ print(f"\nTotal memories stored: {len(all_memories)}")+ print(f"Average latency: {avg_latency * 1000:.0f}ms")+ print("\nLatency breakdown:")+ print(f" Turn 1 (extraction): {latency1 * 1000:.0f}ms")+ print(f" Turn 2 (retrieval): {latency2 * 1000:.0f}ms")+ print(f" Turn 3 (retrieval): {latency3 * 1000:.0f}ms")++ print("\n✅ ZERO extra latency - all memory ops happen inline!")+ print("✅ Semantic search - finds conceptually related memories")+ print("✅ Local embeddings - sub-50ms retrieval (no API calls)")+++def benchmark_memory_overhead():+ """Measure the overhead of memory operations."""+ print("\n" + "=" * 70)+ print("BENCHMARK: Memory Overhead")+ print("=" * 70)++ api_key = os.environ.get("OPENAI_API_KEY")+ if not api_key:+ return++ openai_client = OpenAI(api_key=api_key)++ test_message = "I prefer Python and use PostgreSQL."++ # Baseline: No memory+ print("\n1. BASELINE (no memory wrapper)")+ baseline_latencies = []+ for i in range(3):+ start = time.perf_counter()+ openai_client.chat.completions.create(+ model="gpt-4o-mini",+ messages=[+ {"role": "system", "content": "You are a helpful assistant."},+ {"role": "user", "content": test_message},+ ],+ )+ elapsed = time.perf_counter() - start+ baseline_latencies.append(elapsed * 1000)+ print(f" Run {i + 1}: {elapsed * 1000:.0f}ms")++ # With memory+ print("\n2. WITH FAST MEMORY (inline extraction)")+ with tempfile.TemporaryDirectory() as tmpdir:+ db_path = Path(tmpdir) / "bench_memory.db"+ client = with_fast_memory(+ openai_client,+ user_id="bench",+ db_path=db_path,+ use_local_embeddings=True,+ )++ memory_latencies = []+ for i in range(3):+ start = time.perf_counter()+ client.chat.completions.create(+ model="gpt-4o-mini",+ messages=[+ {"role": "system", "content": "You are a helpful assistant."},+ {"role": "user", "content": test_message},+ ],+ )+ elapsed = time.perf_counter() - start+ memory_latencies.append(elapsed * 1000)+ print(f" Run {i + 1}: {elapsed * 1000:.0f}ms")++ baseline_avg = sum(baseline_latencies) / len(baseline_latencies)+ memory_avg = sum(memory_latencies) / len(memory_latencies)+ overhead = memory_avg - baseline_avg++ print(f"\n{'─' * 70}")+ print(f"{'Approach':<30} {'Avg Latency':<15} {'Overhead':<15}")+ print(f"{'─' * 70}")+ print(f"{'Baseline (no memory)':<30} {baseline_avg:>10.0f}ms {'0ms':>15}")+ print(f"{'With fast memory':<30} {memory_avg:>10.0f}ms {f'{overhead:+.0f}ms':>15}")+ print(f"{'─' * 70}")++ if overhead < 100:+ print(f"\n✅ Memory overhead is only {overhead:.0f}ms - negligible!")+ else:+ print(f"\n⚠️ Memory overhead is {overhead:.0f}ms")+ print(" This is mostly from the memory instruction in the prompt.")+++if __name__ == "__main__":+ run_conversation_test()+ benchmark_memory_overhead()diff --git a/examples/inline_memory_demo.py b/examples/inline_memory_demo.pynew file mode 100644index 000000000..dc73c2438--- /dev/null+++ b/examples/inline_memory_demo.py@@ -0,0 +1,209 @@+#!/usr/bin/env python3+"""Demo: Zero-Latency Inline Memory Extraction (Letta-style).++This demonstrates the Letta/MemGPT approach where the LLM outputs+memories as part of its response - ZERO extra latency!++Comparison:+- OLD: Main LLM call (500ms) + Extraction LLM call (500ms) = 1000ms total+- NEW: Main LLM call with inline extraction (500ms) = 500ms total++The memory is extracted from the SAME tokens the LLM is already generating.++Usage:+ export OPENAI_API_KEY="sk-..."+ python examples/inline_memory_demo.py+"""++from __future__ import annotations++import os+import sys+import time+from pathlib import Path++sys.path.insert(0, str(Path(__file__).parent.parent))++from openai import OpenAI++from headroom.memory.inline_extractor import (+ InlineMemoryWrapper,+)+++def demo_inline_extraction():+ """Demonstrate inline memory extraction."""+ print("=" * 60)+ print("ZERO-LATENCY INLINE MEMORY EXTRACTION")+ print("=" * 60)++ api_key = os.environ.get("OPENAI_API_KEY")+ if not api_key:+ print("ERROR: OPENAI_API_KEY not set")+ sys.exit(1)++ client = OpenAI(api_key=api_key)+ wrapper = InlineMemoryWrapper(client)++ # Test conversations with memory-worthy content+ test_conversations = [+ {+ "messages": [+ {"role": "system", "content": "You are a helpful assistant."},+ {+ "role": "user",+ "content": "I'm a Python developer working on a fintech startup. We use PostgreSQL for our database.",+ },+ ],+ "description": "User shares background info",+ },+ {+ "messages": [+ {"role": "system", "content": "You are a helpful assistant."},+ {"role": "user", "content": "Hello! How are you today?"},+ ],+ "description": "Simple greeting (should have no memories)",+ },+ {+ "messages": [+ {"role": "system", "content": "You are a helpful assistant."},+ {+ "role": "user",+ "content": "I prefer async/await over callbacks, and I always use type hints in my code.",+ },+ ],+ "description": "User shares preferences",+ },+ ]++ total_latency = 0+ total_memories = 0++ for i, test in enumerate(test_conversations, 1):+ print(f"\n{'─' * 60}")+ print(f"Test {i}: {test['description']}")+ print(f"{'─' * 60}")++ user_msg = test["messages"][-1]["content"]+ print(f"User: {user_msg[:80]}...")++ start = time.perf_counter()+ response, memories = wrapper.chat(+ messages=test["messages"],+ model="gpt-4o-mini",+ )+ elapsed = time.perf_counter() - start++ total_latency += elapsed+ total_memories += len(memories)++ print(f"\nAssistant: {response[:150]}...")+ print(f"\nLatency: {elapsed * 1000:.0f}ms")+ print(f"Memories extracted: {len(memories)}")++ if memories:+ for mem in memories:+ print(f" - [{mem.get('category', 'unknown')}] {mem.get('content', '')}")++ print(f"\n{'=' * 60}")+ print("SUMMARY")+ print(f"{'=' * 60}")+ print(f"Total conversations: {len(test_conversations)}")+ print(f"Total memories extracted: {total_memories}")+ print(f"Average latency: {total_latency / len(test_conversations) * 1000:.0f}ms")+ print("\n✓ ZERO extra latency - memories extracted from same response!")+++def benchmark_vs_separate_extraction():+ """Compare inline vs separate LLM extraction."""+ print("\n" + "=" * 60)+ print("BENCHMARK: Inline vs Separate Extraction")+ print("=" * 60)++ api_key = os.environ.get("OPENAI_API_KEY")+ if not api_key:+ print("ERROR: OPENAI_API_KEY not set")+ sys.exit(1)++ client = OpenAI(api_key=api_key)+ wrapper = InlineMemoryWrapper(client)++ test_message = "I'm a senior backend engineer at Netflix. I prefer Go for microservices but Python for ML. I always use Docker and Kubernetes."++ # Measure inline extraction+ print("\n1. INLINE EXTRACTION (Letta-style)")+ print(" Single LLM call with memory instruction")++ inline_latencies = []+ for i in range(3):+ messages = [+ {"role": "system", "content": "You are a helpful assistant."},+ {"role": "user", "content": test_message},+ ]++ start = time.perf_counter()+ response, memories = wrapper.chat(messages, model="gpt-4o-mini")+ elapsed = time.perf_counter() - start++ inline_latencies.append(elapsed * 1000)+ print(f" Run {i + 1}: {elapsed * 1000:.0f}ms ({len(memories)} memories)")++ # Measure separate extraction (simulated)+ print("\n2. SEPARATE EXTRACTION (Traditional)")+ print(" Main LLM call + Extraction LLM call")++ separate_latencies = []+ for i in range(3):+ messages = [+ {"role": "system", "content": "You are a helpful assistant."},+ {"role": "user", "content": test_message},+ ]++ start = time.perf_counter()++ # First call: Main response+ response1 = client.chat.completions.create(+ model="gpt-4o-mini",+ messages=messages,+ )+ main_response = response1.choices[0].message.content++ # Second call: Extract memories+ extraction_prompt = f"""Extract memories from this conversation:+User: {test_message}+Assistant: {main_response}++Return JSON: {{"memories": [{{"content": "...", "category": "preference|fact|context"}}]}}"""++ client.chat.completions.create(+ model="gpt-4o-mini",+ messages=[{"role": "user", "content": extraction_prompt}],+ )++ elapsed = time.perf_counter() - start+ separate_latencies.append(elapsed * 1000)+ print(f" Run {i + 1}: {elapsed * 1000:.0f}ms")++ # Summary+ inline_avg = sum(inline_latencies) / len(inline_latencies)+ separate_avg = sum(separate_latencies) / len(separate_latencies)++ print(f"\n{'─' * 60}")+ print(f"{'Approach':<30} {'Avg Latency':<15} {'Savings':<15}")+ print(f"{'─' * 60}")+ print(f"{'Inline (Letta-style)':<30} {inline_avg:>10.0f}ms {'baseline':>15}")+ print(+ f"{'Separate extraction':<30} {separate_avg:>10.0f}ms {f'+{separate_avg - inline_avg:.0f}ms':>15}"+ )+ print(f"{'─' * 60}")++ savings = separate_avg - inline_avg+ print(+ f"\n✓ Inline extraction saves {savings:.0f}ms ({savings / separate_avg * 100:.0f}% faster)"+ )+ print("✓ This is the latency of an ENTIRE extra LLM call - now FREE!")+++if __name__ == "__main__":+ demo_inline_extraction()+ benchmark_vs_separate_extraction()diff --git a/examples/memory_e2e_test.py b/examples/memory_e2e_test.pynew file mode 100644index 000000000..ac2c03f32--- /dev/null+++ b/examples/memory_e2e_test.py@@ -0,0 +1,595 @@+#!/usr/bin/env python3+"""End-to-end memory system test with LLM-as-judge evaluation.++This program tests the memory extraction and retrieval system with:+1. Multi-turn conversations containing embedded memory nuggets+2. Real OpenAI API calls for extraction and conversation+3. LLM-as-judge evaluation of memory quality++Usage:+ export OPENAI_API_KEY="sk-..."+ python examples/memory_e2e_test.py+"""++from __future__ import annotations++import json+import os+import sys+import tempfile+import time+from dataclasses import dataclass+from pathlib import Path++# Add parent to path for local development+sys.path.insert(0, str(Path(__file__).parent.parent))++from openai import OpenAI++from headroom.memory import with_memory+from headroom.memory.store import SQLiteMemoryStore++# =============================================================================+# Test Scenarios - Conversations with embedded memory nuggets+# =============================================================================+++@dataclass+class MemoryNugget:+ """A fact that should be remembered from the conversation."""++ content: str+ category: str # preference, fact, context+ importance: float # 0.0-1.0+ turn_index: int # Which turn contains this nugget+++@dataclass+class TestScenario:+ """A test scenario with conversation and expected memories."""++ name: str+ description: str+ entity_id: str # user_id or agent_id+ conversation: list[tuple[str, str]] # List of (user_msg, expected_response_topic)+ expected_nuggets: list[MemoryNugget]+ retrieval_queries: list[tuple[str, list[str]]] # (query, expected_keywords_in_memory)+++# Scenario 1: Software Developer User+DEVELOPER_SCENARIO = TestScenario(+ name="software_developer",+ description="A software developer discussing their preferences and projects",+ entity_id="dev_alice",+ conversation=[+ # Turn 0 - Preference nugget+ (+ "Hi! I'm starting a new backend project. I strongly prefer Python over "+ "JavaScript for backend work because of its cleaner syntax.",+ "backend_project_advice",+ ),+ # Turn 1 - Fact nugget+ (+ "Good point. I work at a fintech startup called PayFlow where we handle "+ "high-volume payment processing.",+ "fintech_architecture",+ ),+ # Turn 2 - Context nugget+ (+ "We're currently migrating from a monolith to microservices. It's been "+ "challenging but necessary for scale.",+ "migration_advice",+ ),+ # Turn 3 - Preference nugget+ (+ "For databases, I always use PostgreSQL. I've tried MongoDB but found "+ "relational databases more reliable for financial data.",+ "database_choice",+ ),+ # Turn 4 - Casual (no nugget expected)+ ("Thanks for all the help today!", "closing"),+ ],+ expected_nuggets=[+ MemoryNugget("Prefers Python over JavaScript for backend", "preference", 0.8, 0),+ MemoryNugget("Works at fintech startup PayFlow", "fact", 0.9, 1),+ MemoryNugget("Handles high-volume payment processing", "fact", 0.7, 1),+ MemoryNugget("Migrating from monolith to microservices", "context", 0.8, 2),+ MemoryNugget("Prefers PostgreSQL over MongoDB", "preference", 0.8, 3),+ MemoryNugget("Works with financial data", "fact", 0.7, 3),+ ],+ retrieval_queries=[+ # FTS5 is keyword-based, so queries must contain matching words+ ("Python backend", ["Python", "backend"]),+ ("PostgreSQL database", ["PostgreSQL", "database"]),+ ("PayFlow fintech", ["PayFlow", "fintech"]),+ ("microservices migration", ["microservices", "monolith"]),+ ],+)++# Scenario 2: AI Research Agent+AGENT_SCENARIO = TestScenario(+ name="research_agent",+ description="An AI agent discussing its capabilities and constraints",+ entity_id="agent_researcher",+ conversation=[+ # Turn 0 - Capability fact+ (+ "I'm Agent-7, specialized in scientific literature analysis. I can process "+ "up to 50 papers per hour and identify cross-domain connections.",+ "agent_intro",+ ),+ # Turn 1 - Constraint context+ (+ "My knowledge cutoff is March 2025, so I may not have the latest preprints. "+ "I work best with structured abstracts.",+ "limitations",+ ),+ # Turn 2 - Preference+ (+ "When summarizing papers, I prefer to use the IMRaD structure - Introduction, "+ "Methods, Results, and Discussion. It's more systematic.",+ "summary_format",+ ),+ # Turn 3 - Configuration fact+ (+ "I'm currently configured to prioritize papers from Nature, Science, and Cell "+ "journals, with a citation threshold of 10+.",+ "configuration",+ ),+ # Turn 4 - Context about ongoing task+ (+ "Right now I'm tracking the emerging field of mechanistic interpretability "+ "in neural networks. It's my primary research focus.",+ "current_focus",+ ),+ ],+ expected_nuggets=[+ MemoryNugget("Agent-7 specialized in scientific literature", "fact", 0.9, 0),+ MemoryNugget("Can process 50 papers per hour", "fact", 0.7, 0),+ MemoryNugget("Knowledge cutoff March 2025", "context", 0.8, 1),+ MemoryNugget("Prefers IMRaD structure for summaries", "preference", 0.8, 2),+ MemoryNugget("Prioritizes Nature, Science, Cell journals", "fact", 0.7, 3),+ MemoryNugget("Citation threshold of 10+", "fact", 0.6, 3),+ MemoryNugget("Focus on mechanistic interpretability", "context", 0.9, 4),+ ],+ retrieval_queries=[+ # FTS5 keyword-based queries+ ("scientific papers analysis", ["papers", "scientific", "literature"]),+ ("IMRaD summary structure", ["IMRaD", "structure"]),+ ("Nature Science Cell journals", ["Nature", "Science", "Cell"]),+ ("mechanistic interpretability neural", ["mechanistic", "interpretability"]),+ ],+)++# Scenario 3: Multi-session customer+CUSTOMER_SCENARIO = TestScenario(+ name="returning_customer",+ description="A customer across multiple support interactions",+ entity_id="customer_bob",+ conversation=[+ # Turn 0 - Account fact+ (+ "Hi, I'm Bob Chen, account number AC-789456. I've been a premium member since 2021.",+ "account_lookup",+ ),+ # Turn 1 - Preference+ (+ "Please always contact me via email at [email protected], never by phone. "+ "I work odd hours as a night shift nurse.",+ "contact_preference",+ ),+ # Turn 2 - Issue context+ (+ "I've had recurring issues with billing - this is the third time this month "+ "I've been double-charged.",+ "billing_issue",+ ),+ # Turn 3 - Product preference+ (+ "I mainly use your enterprise plan for the API access. The dashboard features "+ "I never touch.",+ "usage_pattern",+ ),+ ],+ expected_nuggets=[+ MemoryNugget("Bob Chen, account AC-789456", "fact", 0.9, 0),+ MemoryNugget("Premium member since 2021", "fact", 0.7, 0),+ MemoryNugget("Prefers email contact, never phone", "preference", 0.9, 1),+ MemoryNugget("Works as night shift nurse", "fact", 0.6, 1),+ MemoryNugget("Recurring billing/double-charge issues", "context", 0.8, 2),+ MemoryNugget("Uses enterprise plan for API access", "fact", 0.7, 3),+ ],+ retrieval_queries=[+ # FTS5 keyword-based queries+ ("Bob Chen account premium", ["Bob", "account", "premium"]),+ ("email contact phone", ["email", "phone"]),+ ("billing double charged", ["billing", "charged"]),+ ("enterprise API plan", ["API", "enterprise"]),+ ],+)++ALL_SCENARIOS = [DEVELOPER_SCENARIO, AGENT_SCENARIO, CUSTOMER_SCENARIO]+++# =============================================================================+# Conversation Simulator+# =============================================================================+++class ConversationSimulator:+ """Simulates realistic conversations using OpenAI."""++ def __init__(self, client: OpenAI):+ self.client = client+ self.model = "gpt-4o-mini"++ def generate_response(self, user_message: str, topic_hint: str) -> str:+ """Generate a realistic assistant response."""+ response = self.client.chat.completions.create(+ model=self.model,+ messages=[+ {+ "role": "system",+ "content": (+ "You are a helpful assistant. Respond naturally and briefly "+ "to the user's message. Keep responses under 100 words."+ ),+ },+ {"role": "user", "content": user_message},+ ],+ max_tokens=150,+ temperature=0.7,+ )+ return response.choices[0].message.content+++# =============================================================================+# LLM-as-Judge Evaluator+# =============================================================================+++@dataclass+class EvaluationResult:+ """Result of LLM judge evaluation."""++ scenario_name: str+ extraction_score: float # 0-1, how many expected nuggets were captured+ retrieval_score: float # 0-1, how well queries retrieved relevant memories+ overall_score: float+ extracted_memories: list[dict]+ missing_nuggets: list[str]+ retrieval_results: list[dict]+ judge_reasoning: str+++class LLMJudge:+ """Uses LLM to evaluate memory extraction and retrieval quality."""++ def __init__(self, client: OpenAI):+ self.client = client+ self.model = "gpt-4o" # Use stronger model for judging++ def evaluate_extraction(+ self,+ scenario: TestScenario,+ extracted_memories: list[dict],+ ) -> tuple[float, list[str], str]:+ """Evaluate if extracted memories capture expected nuggets."""++ prompt = f"""You are evaluating a memory extraction system.++The system processed this conversation and extracted memories.++## Expected Information to Remember:+{json.dumps([{"content": n.content, "category": n.category, "importance": n.importance} for n in scenario.expected_nuggets], indent=2)}++## Actually Extracted Memories:+{json.dumps(extracted_memories, indent=2)}++## Evaluation Task:+1. For each expected nugget, determine if it was captured (exact match not required - semantic similarity counts)+2. Calculate what percentage of expected nuggets were captured+3. Identify which nuggets were MISSING++Return a JSON object:+{{+ "captured_count": <number of expected nuggets that were captured>,+ "total_expected": {len(scenario.expected_nuggets)},+ "score": <0.0 to 1.0>,+ "missing_nuggets": ["list of expected nuggets that were not captured"],+ "reasoning": "Brief explanation of the evaluation"+}}++Return ONLY valid JSON."""++ response = self.client.chat.completions.create(+ model=self.model,+ messages=[{"role": "user", "content": prompt}],+ response_format={"type": "json_object"},+ temperature=0.0,+ )++ result = json.loads(response.choices[0].message.content)+ return (+ result.get("score", 0.0),+ result.get("missing_nuggets", []),+ result.get("reasoning", ""),+ )++ def evaluate_retrieval(+ self,+ query: str,+ expected_keywords: list[str],+ retrieved_memories: list[dict],+ ) -> tuple[float, str]:+ """Evaluate if retrieval returned relevant memories."""++ prompt = f"""You are evaluating a memory retrieval system.++## Query: "{query}"++## Expected Keywords in Results: {expected_keywords}++## Retrieved Memories:+{json.dumps(retrieved_memories, indent=2)}++## Evaluation Task:+Determine if the retrieved memories are relevant to the query and contain the expected information.++Return a JSON object:+{{+ "score": <0.0 to 1.0>,+ "keywords_found": ["list of expected keywords that appeared in results"],+ "reasoning": "Brief explanation"+}}++Return ONLY valid JSON."""++ response = self.client.chat.completions.create(+ model=self.model,+ messages=[{"role": "user", "content": prompt}],+ response_format={"type": "json_object"},+ temperature=0.0,+ )++ result = json.loads(response.choices[0].message.content)+ return result.get("score", 0.0), result.get("reasoning", "")+++# =============================================================================+# Main Test Runner+# =============================================================================+++class MemoryE2ETest:+ """End-to-end test runner for the memory system."""++ def __init__(self, api_key: str):+ self.raw_client = OpenAI(api_key=api_key)+ self.simulator = ConversationSimulator(self.raw_client)+ self.judge = LLMJudge(self.raw_client)+ self.results: list[EvaluationResult] = []++ def run_scenario(self, scenario: TestScenario, db_path: Path) -> EvaluationResult:+ """Run a complete test scenario."""+ print(f"\n{'=' * 60}")+ print(f"Running scenario: {scenario.name}")+ print(f"Description: {scenario.description}")+ print(f"{'=' * 60}")++ # Create memory-wrapped client+ store = SQLiteMemoryStore(db_path)+ memory_client = with_memory(+ self.raw_client,+ user_id=scenario.entity_id,+ db_path=db_path,+ _store=store,+ )++ # Run conversation through memory-wrapped client+ print(f"\n--- Running {len(scenario.conversation)} conversation turns ---")+ for i, (user_msg, _topic_hint) in enumerate(scenario.conversation):+ print(f"\nTurn {i + 1}:")+ print(f" User: {user_msg[:80]}...")++ # Send through memory-wrapped client - this:+ # 1. Retrieves relevant memories (if any)+ # 2. Injects them into user message+ # 3. Calls the actual API+ # 4. Queues extraction of (original_query, response) in background+ response = memory_client.chat.completions.create(+ model="gpt-4o-mini",+ messages=[+ {"role": "user", "content": user_msg},+ ],+ )+ assistant_response = response.choices[0].message.content+ print(f" Assistant: {assistant_response[:80]}...")++ # Small delay to allow background extraction+ time.sleep(0.5)++ # Flush all extractions (force immediate processing)+ print("\n--- Flushing background extractions ---")+ start_time = time.time()+ success = memory_client.flush_extractions(timeout=60.0)+ elapsed = time.time() - start_time++ if success:+ print(f" All extractions complete in {elapsed:.1f}s")+ else:+ print(f" WARNING: Flush timed out after {elapsed:.1f}s")+ pending = store.get_pending_extractions(limit=10)+ still_pending = [p for p in pending if p.status == "pending"]+ if still_pending:+ print(f" {len(still_pending)} extractions still pending")+ for p in still_pending[:2]:+ print(f" - Query: {p.query[:50]}...")++ # If no memories extracted, try direct extraction for debugging+ all_memories = store.get_all(scenario.entity_id)+ if not all_memories and scenario.conversation:+ print("\n DEBUG: Attempting direct extraction for first turn...")+ from headroom.memory.extractor import MemoryExtractor++ extractor = MemoryExtractor(self.raw_client)+ first_query, _ = scenario.conversation[0]+ test_response = "Acknowledged, I understand."+ direct_memories = extractor.extract(first_query, test_response)+ print(f" DEBUG: Direct extraction got {len(direct_memories)} memories")+ for m in direct_memories[:3]:+ print(f" - [{m.category}] {m.content[:50]}...")++ # Get all extracted memories (refresh)+ extracted = [+ {"content": m.content, "category": m.category, "importance": m.importance}+ for m in all_memories+ ]+ print(f"\n--- Extracted {len(extracted)} memories ---")+ for m in extracted:+ print(f" [{m['category']}] {m['content'][:60]}...")++ # Evaluate extraction quality+ print("\n--- Evaluating extraction quality ---")+ extraction_score, missing, extraction_reasoning = self.judge.evaluate_extraction(+ scenario, extracted+ )+ print(f" Extraction Score: {extraction_score:.2f}")+ if missing:+ print(f" Missing nuggets: {len(missing)}")+ for m in missing[:3]:+ print(f" - {m[:60]}...")++ # Test retrieval queries+ print("\n--- Testing retrieval queries ---")+ retrieval_results = []+ retrieval_scores = []++ for query, expected_keywords in scenario.retrieval_queries:+ results = store.search(scenario.entity_id, query, top_k=5)+ retrieved = [{"content": m.content, "category": m.category} for m in results]++ score, reasoning = self.judge.evaluate_retrieval(query, expected_keywords, retrieved)+ retrieval_scores.append(score)++ retrieval_results.append(+ {+ "query": query,+ "expected_keywords": expected_keywords,+ "retrieved_count": len(retrieved),+ "score": score,+ "reasoning": reasoning,+ }+ )+ print(f" Query: '{query[:40]}...' -> Score: {score:.2f}, Found: {len(retrieved)}")++ avg_retrieval_score = (+ sum(retrieval_scores) / len(retrieval_scores) if retrieval_scores else 0+ )++ # Calculate overall score+ overall_score = (extraction_score * 0.6) + (avg_retrieval_score * 0.4)++ result = EvaluationResult(+ scenario_name=scenario.name,+ extraction_score=extraction_score,+ retrieval_score=avg_retrieval_score,+ overall_score=overall_score,+ extracted_memories=extracted,+ missing_nuggets=missing,+ retrieval_results=retrieval_results,+ judge_reasoning=extraction_reasoning,+ )++ print("\n--- Scenario Complete ---")+ print(f" Extraction Score: {extraction_score:.2f}")+ print(f" Retrieval Score: {avg_retrieval_score:.2f}")+ print(f" Overall Score: {overall_score:.2f}")++ return result++ def run_all_scenarios(self) -> list[EvaluationResult]:+ """Run all test scenarios."""+ print("\n" + "=" * 60)+ print("MEMORY SYSTEM END-TO-END TEST")+ print("=" * 60)+ print(f"Running {len(ALL_SCENARIOS)} scenarios with LLM-as-judge evaluation")++ results = []++ with tempfile.TemporaryDirectory() as tmpdir:+ for scenario in ALL_SCENARIOS:+ db_path = Path(tmpdir) / f"{scenario.name}.db"+ result = self.run_scenario(scenario, db_path)+ results.append(result)++ self.results = results+ return results++ def print_summary(self):+ """Print summary of all test results."""+ print("\n" + "=" * 60)+ print("FINAL SUMMARY")+ print("=" * 60)++ total_extraction = 0+ total_retrieval = 0+ total_overall = 0++ for r in self.results:+ print(f"\n{r.scenario_name}:")+ print(f" Extraction: {r.extraction_score:.2f}")+ print(f" Retrieval: {r.retrieval_score:.2f}")+ print(f" Overall: {r.overall_score:.2f}")+ if r.missing_nuggets:+ print(f" Missing: {len(r.missing_nuggets)} nuggets")++ total_extraction += r.extraction_score+ total_retrieval += r.retrieval_score+ total_overall += r.overall_score++ n = len(self.results)+ print(f"\n{'=' * 60}")+ print("AGGREGATE SCORES")+ print(f"{'=' * 60}")+ print(f" Avg Extraction: {total_extraction / n:.2f}")+ print(f" Avg Retrieval: {total_retrieval / n:.2f}")+ print(f" Avg Overall: {total_overall / n:.2f}")++ # Overall assessment+ avg_overall = total_overall / n+ if avg_overall >= 0.8:+ verdict = "EXCELLENT - Memory system working well"+ elif avg_overall >= 0.6:+ verdict = "GOOD - Memory system functional with room for improvement"+ elif avg_overall >= 0.4:+ verdict = "FAIR - Memory system needs tuning"+ else:+ verdict = "POOR - Memory system needs significant work"++ print(f"\nVERDICT: {verdict}")+ print("=" * 60)++ return avg_overall+++def main():+ """Main entry point."""+ # Check for API key+ api_key = os.environ.get("OPENAI_API_KEY")+ if not api_key:+ print("ERROR: OPENAI_API_KEY environment variable not set")+ print("Usage: export OPENAI_API_KEY='sk-...' && python examples/memory_e2e_test.py")+ sys.exit(1)++ # Run tests+ tester = MemoryE2ETest(api_key)+ tester.run_all_scenarios()+ avg_score = tester.print_summary()++ # Exit with appropriate code+ sys.exit(0 if avg_score >= 0.5 else 1)+++if __name__ == "__main__":+ main()diff --git a/examples/memory_latency_benchmark.py b/examples/memory_latency_benchmark.pynew file mode 100644index 000000000..6d652699b--- /dev/null+++ b/examples/memory_latency_benchmark.py@@ -0,0 +1,242 @@+#!/usr/bin/env python3+"""Benchmark: LLM Extraction vs Embedding-Only Memory.++Demonstrates the massive latency difference between:+1. OLD: LLM-based extraction (2-3 seconds)+2. NEW: Embedding-only storage (sub-100ms)++Usage:+ export OPENAI_API_KEY="sk-..."+ python examples/memory_latency_benchmark.py+"""++from __future__ import annotations++import os+import sys+import tempfile+import time+from pathlib import Path+from statistics import mean++sys.path.insert(0, str(Path(__file__).parent.parent))++from openai import OpenAI++from headroom.memory.extractor import MemoryExtractor+from headroom.memory.fast_store import (+ FastMemoryStore,+ create_local_embed_fn,+ create_openai_batch_embed_fn,+ create_openai_embed_fn,+)++# Test messages with memory-worthy content+TEST_MESSAGES = [+ ("I prefer Python over JavaScript for backend development", "Great choice!"),+ ("I work at a fintech startup handling payment processing", "Interesting domain!"),+ ("Always use PostgreSQL for relational data, never MongoDB", "Solid preference!"),+ ("I'm migrating from monolith to microservices architecture", "Good luck!"),+ ("My email is [email protected], contact me there only", "Noted!"),+]+++def benchmark_llm_extraction(client: OpenAI, num_runs: int = 5) -> list[float]:+ """Benchmark the OLD LLM-based extraction approach."""+ print("\n" + "=" * 60)+ print("BENCHMARK: LLM-Based Extraction (OLD)")+ print("=" * 60)++ extractor = MemoryExtractor(client)+ latencies = []++ for i, (query, response) in enumerate(TEST_MESSAGES[:num_runs]):+ start = time.perf_counter()+ memories = extractor.extract(query, response)+ elapsed = time.perf_counter() - start++ latencies.append(elapsed * 1000) # Convert to ms+ print(f" Run {i + 1}: {elapsed * 1000:.0f}ms - extracted {len(memories)} memories")++ return latencies+++def benchmark_embedding_store(client: OpenAI, num_runs: int = 5) -> list[float]:+ """Benchmark embedding-only approach with INDIVIDUAL API calls."""+ print("\n" + "=" * 60)+ print("BENCHMARK: Embedding-Only, Individual Calls")+ print("=" * 60)++ with tempfile.TemporaryDirectory() as tmpdir:+ db_path = Path(tmpdir) / "fast_memory.db"+ embed_fn = create_openai_embed_fn(client)+ store = FastMemoryStore(db_path, embed_fn=embed_fn)++ latencies = []++ for i, (query, response) in enumerate(TEST_MESSAGES[:num_runs]):+ start = time.perf_counter()+ # Store both messages (2 separate API calls)+ store.add("test_user", query, role="user")+ store.add("test_user", response, role="assistant")+ elapsed = time.perf_counter() - start++ latencies.append(elapsed * 1000) # Convert to ms+ print(f" Run {i + 1}: {elapsed * 1000:.0f}ms - stored 2 chunks (2 API calls)")++ return latencies+++def benchmark_batched_embedding(client: OpenAI, num_runs: int = 5) -> list[float]:+ """Benchmark embedding-only approach with BATCHED API calls."""+ print("\n" + "=" * 60)+ print("BENCHMARK: Embedding-Only, BATCHED Calls")+ print("=" * 60)++ with tempfile.TemporaryDirectory() as tmpdir:+ db_path = Path(tmpdir) / "fast_memory.db"+ embed_fn = create_openai_embed_fn(client)+ batch_embed_fn = create_openai_batch_embed_fn(client)+ store = FastMemoryStore(db_path, embed_fn=embed_fn)++ latencies = []++ for i, (query, response) in enumerate(TEST_MESSAGES[:num_runs]):+ start = time.perf_counter()+ # Store both messages in ONE API call+ store.add_turn_batched("test_user", query, response, batch_embed_fn)+ elapsed = time.perf_counter() - start++ latencies.append(elapsed * 1000) # Convert to ms+ print(f" Run {i + 1}: {elapsed * 1000:.0f}ms - stored 2 chunks (1 API call)")++ return latencies+++def benchmark_local_embedding(num_runs: int = 5) -> list[float]:+ """Benchmark embedding-only approach with LOCAL model (FASTEST)."""+ print("\n" + "=" * 60)+ print("BENCHMARK: LOCAL Embeddings (FASTEST - No API!)")+ print("=" * 60)++ # Load model once (this is slow, but only happens once)+ print(" Loading local model (one-time cost)...")+ start_load = time.perf_counter()+ embed_fn = create_local_embed_fn("all-MiniLM-L6-v2")+ load_time = time.perf_counter() - start_load+ print(f" Model loaded in {load_time:.1f}s")++ # Warmup runs to trigger JIT compilation+ print(" Warming up (JIT compilation)...")+ for _ in range(3):+ embed_fn("warmup text for compilation")++ with tempfile.TemporaryDirectory() as tmpdir:+ db_path = Path(tmpdir) / "fast_memory.db"+ store = FastMemoryStore(db_path, embed_fn=embed_fn, embedding_dim=384)++ latencies = []++ for i, (query, response) in enumerate(TEST_MESSAGES[:num_runs]):+ start = time.perf_counter()+ # Store both messages+ store.add("test_user", query, role="user")+ store.add("test_user", response, role="assistant")+ elapsed = time.perf_counter() - start++ latencies.append(elapsed * 1000) # Convert to ms+ print(f" Run {i + 1}: {elapsed * 1000:.1f}ms - stored 2 chunks (LOCAL)")++ return latencies+++def benchmark_search_comparison(client: OpenAI) -> None:+ """Compare search latency: FTS5 vs Vector Similarity."""+ print("\n" + "=" * 60)+ print("BENCHMARK: Search Latency")+ print("=" * 60)++ with tempfile.TemporaryDirectory() as tmpdir:+ db_path = Path(tmpdir) / "fast_memory.db"+ embed_fn = create_openai_embed_fn(client)+ store = FastMemoryStore(db_path, embed_fn=embed_fn)++ # Populate with test data+ print(" Populating store with 20 memories...")+ for query, response in TEST_MESSAGES * 4:+ store.add("test_user", query, role="user")+ store.add("test_user", response, role="assistant")++ # Benchmark searches+ search_queries = [+ "What programming language?",+ "database recommendations",+ "architecture patterns",+ "contact information",+ ]++ print("\n Search latencies:")+ for query in search_queries:+ start = time.perf_counter()+ results = store.search("test_user", query, top_k=3)+ elapsed = time.perf_counter() - start++ top_match = results[0][0].text[:40] if results else "None"+ print(f" '{query}' -> {elapsed * 1000:.0f}ms ({len(results)} results)")+ print(f" Top match: '{top_match}...'")+++def main():+ """Run all benchmarks."""+ api_key = os.environ.get("OPENAI_API_KEY")+ if not api_key:+ print("ERROR: OPENAI_API_KEY environment variable not set")+ sys.exit(1)++ client = OpenAI(api_key=api_key)++ print("=" * 60)+ print("MEMORY LATENCY BENCHMARK")+ print("Comparing LLM Extraction vs Embedding-Only")+ print("=" * 60)++ # Run benchmarks+ llm_latencies = benchmark_llm_extraction(client, num_runs=3)+ embed_latencies = benchmark_embedding_store(client, num_runs=3)+ batched_latencies = benchmark_batched_embedding(client, num_runs=5)+ local_latencies = benchmark_local_embedding(num_runs=5)+ benchmark_search_comparison(client)++ # Summary+ print("\n" + "=" * 60)+ print("SUMMARY")+ print("=" * 60)++ llm_avg = mean(llm_latencies)+ embed_avg = mean(embed_latencies)+ batched_avg = mean(batched_latencies)+ local_avg = mean(local_latencies)++ print(f"\n{'Approach':<35} {'Avg Latency':<15} {'Speedup':<10}")+ print("-" * 60)+ print(f"{'LLM Extraction (OLD)':<35} {llm_avg:>10.0f}ms {'1.0x':>10}")+ print(f"{'Embedding (2 API calls)':<35} {embed_avg:>10.0f}ms {llm_avg / embed_avg:>9.1f}x")+ print(+ f"{'Embedding BATCHED (1 API call)':<35} {batched_avg:>10.0f}ms {llm_avg / batched_avg:>9.1f}x"+ )+ print(f"{'LOCAL Embeddings (no API!)':<35} {local_avg:>10.1f}ms {llm_avg / local_avg:>9.0f}x")++ print(f"\n{'=' * 60}")+ print(f"BEST SPEEDUP: {llm_avg / local_avg:.0f}x FASTER with local embeddings!")+ print(f"{'=' * 60}")++ if local_avg < 100:+ print("\n✓ SUB-100ms ACHIEVED with local embeddings!")+ if local_avg < 50:+ print("✓ SUB-50ms ACHIEVED!")+ if local_avg < 20:+ print("✓ SUB-20ms ACHIEVED - GOAL MET!")+++if __name__ == "__main__":+ main()diff --git a/headroom/__init__.py b/headroom/__init__.pyindex ca7fb91ab..38e4eb8a0 100644--- a/headroom/__init__.py+++ b/headroom/__init__.py@@ -112,6 +112,9 @@ from .exceptions import (TransformError,ValidationError,)++# Memory module - simple, LLM-driven memory+from .memory import Memory, SQLiteMemoryStore, with_memoryfrom .providers import AnthropicProvider, OpenAIProvider, Provider, TokenCounterfrom .relevance import (BM25Scorer,@@ -202,4 +205,8 @@ __all__ = ["count_tokens_text","count_tokens_messages","generate_report",+ # Memory - simple, LLM-driven memory+ "with_memory",+ "Memory",+ "SQLiteMemoryStore",]diff --git a/headroom/memory/__init__.py b/headroom/memory/__init__.pynew file mode 100644index 000000000..689267436--- /dev/null+++ b/headroom/memory/__init__.py@@ -0,0 +1,37 @@+"""Headroom Memory - Simple, LLM-driven memory for AI applications.++Two approaches available:++1. Background extraction (original):+ from headroom import with_memory+ client = with_memory(OpenAI(), user_id="alice")++2. Zero-latency inline extraction (recommended):+ from headroom.memory import with_fast_memory+ client = with_fast_memory(OpenAI(), user_id="alice")+"""++from headroom.memory.fast_store import FastMemoryStore, MemoryChunk+from headroom.memory.fast_wrapper import with_fast_memory+from headroom.memory.inline_extractor import (+ InlineMemoryWrapper,+ inject_memory_instruction,+ parse_response_with_memory,+)+from headroom.memory.store import Memory, SQLiteMemoryStore+from headroom.memory.wrapper import with_memory++__all__ = [+ # Original approach (background extraction)+ "with_memory",+ "Memory",+ "SQLiteMemoryStore",+ # Fast approach (inline extraction - recommended)+ "with_fast_memory",+ "FastMemoryStore",+ "MemoryChunk",+ # Low-level inline extraction+ "InlineMemoryWrapper",+ "inject_memory_instruction",+ "parse_response_with_memory",+]diff --git a/headroom/memory/extractor.py b/headroom/memory/extractor.pynew file mode 100644index 000000000..0571e7a4e--- /dev/null+++ b/headroom/memory/extractor.py@@ -0,0 +1,390 @@+"""Memory extraction using LLMs.++Supports multiple providers by reusing the wrapped client with a cheap model.+Auto-detects provider from client class and selects appropriate cheap model.+Uses structured JSON output where available for reliable parsing.+"""++from __future__ import annotations++import json+import logging+import re+from typing import Any, Protocol++from headroom.memory.store import Memory++logger = logging.getLogger(__name__)+++# Provider → Cheap Model mapping (verified January 2026)+# These are the most cost-effective models for simple extraction tasks+CHEAP_MODELS: dict[str, str] = {+ "openai": "gpt-4o-mini", # $0.15/1M input, $0.60/1M output+ "anthropic": "claude-3-5-haiku-latest", # $0.80/1M input, $4/1M output+ "mistralai": "mistral-small-latest", # $0.10/1M input, $0.30/1M output+ "groq": "llama-3.3-70b-versatile", # Free tier available+ "together": "meta-llama/Llama-3.3-70B-Instruct-Turbo", # $0.88/1M+ "fireworks": "accounts/fireworks/models/llama-v3p1-8b-instruct", # $0.20/1M+ "google": "gemini-2.0-flash-lite", # $0.075/1M input, $0.30/1M output+ "cohere": "command-r7b-12-2024", # $0.0375/1M input, $0.15/1M output+}++# Providers that support structured JSON output via response_format+SUPPORTS_JSON_MODE: set[str] = {"openai", "mistralai", "groq", "together", "fireworks"}+++# Entity-agnostic prompt - works for users, agents, or any conversational entity+EXTRACTION_PROMPT = """Analyze this conversation and extract any facts worth remembering.++Focus on:+- Preferences (language, tools, frameworks, style, configuration)+- Facts (identity, role, capabilities, constraints, environment)+- Context (goals, ongoing tasks, relationships, history)++Conversation:+Speaker A: {query}+Speaker B: {response}++Return a JSON object with this structure:+{{+ "memories": [+ {{"content": "Prefers Python for backend development", "category": "preference", "importance": 0.8}},+ {{"content": "Works on distributed systems", "category": "fact", "importance": 0.7}}+ ],+ "should_remember": true+}}++Categories: "preference", "fact", "context"+Importance: 0.0-1.0 (higher = more important to remember long-term)++If there's nothing worth remembering (greetings, generic questions, transient info), return:+{{"memories": [], "should_remember": false}}++Return ONLY valid JSON."""+++class ChatClient(Protocol):+ """Protocol for chat clients (OpenAI, Anthropic, etc.)."""++ class Chat:+ class Completions:+ def create(self, **kwargs: Any) -> Any: ...++ completions: Completions++ chat: Chat+++def detect_provider(client: Any) -> str | None:+ """Detect the provider from client class path.++ Args:+ client: The LLM client instance++ Returns:+ Provider name or None if unknown+ """+ module = type(client).__module__.lower()++ # Check for known providers+ providers = [+ "openai",+ "anthropic",+ "mistralai",+ "groq",+ "together",+ "fireworks",+ "google",+ "cohere",+ ]++ for provider in providers:+ if provider in module:+ return provider++ return None+++def get_cheap_model(provider: str) -> str | None:+ """Get the cheap model for a provider.++ Args:+ provider: Provider name++ Returns:+ Cheap model ID or None if unknown+ """+ return CHEAP_MODELS.get(provider)+++class MemoryExtractor:+ """Extracts memories from conversations using LLMs.++ Supports multiple providers by reusing the wrapped client.+ Auto-detects provider and selects appropriate cheap model.++ Usage:+ extractor = MemoryExtractor(openai_client)+ memories = extractor.extract("I prefer Python", "Great choice!")+ """++ def __init__(+ self,+ client: Any,+ model: str | None = None,+ ):+ """Initialize the extractor.++ Args:+ client: LLM client (OpenAI, Anthropic, etc.)+ model: Override the extraction model (auto-detects if None)+ """+ self.client = client+ self._provider = detect_provider(client)+ self._model: str | None = None++ if model:+ self._model = model+ elif self._provider:+ self._model = get_cheap_model(self._provider)++ if not self._model:+ logger.warning(+ f"Could not detect cheap model for provider. "+ f"Client type: {type(client).__module__}.{type(client).__name__}. "+ f"Memory extraction may fail."+ )++ @property+ def provider(self) -> str | None:+ """Get the detected provider."""+ return self._provider++ @property+ def model(self) -> str | None:+ """Get the extraction model."""+ return self._model++ def extract(self, query: str, response: str) -> list[Memory]:+ """Extract memories from a conversation turn.++ Args:+ query: User's message+ response: Assistant's response++ Returns:+ List of extracted memories (may be empty)+ """+ if not self._model:+ logger.warning("No extraction model configured, skipping extraction")+ return []++ prompt = EXTRACTION_PROMPT.format(query=query, response=response)++ try:+ result = self._call_llm(prompt)+ return self._parse_response(result)+ except Exception as e:+ logger.error(f"Extraction failed: {e}")+ return []++ def extract_batch(self, conversations: list[tuple[str, str, str]]) -> dict[str, list[Memory]]:+ """Extract memories from multiple conversations.++ Args:+ conversations: List of (user_id, query, response) tuples++ Returns:+ Dict mapping user_id to list of memories+ """+ if not conversations:+ return {}++ # Build batch prompt+ batch_prompt = self._build_batch_prompt(conversations)++ try:+ result = self._call_llm(batch_prompt)+ return self._parse_batch_response(result, conversations)+ except Exception as e:+ logger.error(f"Batch extraction failed: {e}")+ return {}++ def _call_llm(self, prompt: str) -> str:+ """Call the LLM with the given prompt.++ Uses structured JSON output (response_format) where available+ to ensure reliable JSON parsing.++ Args:+ prompt: The prompt to send++ Returns:+ The LLM's response text+ """+ if self._provider == "anthropic":+ # Anthropic uses different API - no native JSON mode yet+ response = self.client.messages.create(+ model=self._model,+ max_tokens=1024,+ messages=[{"role": "user", "content": prompt}],+ )+ return str(response.content[0].text)+ elif self._provider == "cohere":+ # Cohere uses different API+ response = self.client.chat(+ model=self._model,+ message=prompt,+ )+ return str(response.text)+ elif self._provider == "google":+ # Google Gemini - use JSON response mime type+ model = self.client.GenerativeModel(+ self._model,+ generation_config={"response_mime_type": "application/json"},+ )+ response = model.generate_content(prompt)+ return str(response.text)+ else:+ # OpenAI-compatible API (OpenAI, Groq, Together, Fireworks, Mistral)+ # Use JSON mode for structured output+ kwargs: dict[str, Any] = {+ "model": self._model,+ "messages": [{"role": "user", "content": prompt}],+ "temperature": 0.0, # Deterministic for extraction+ }++ # Add response_format for providers that support it+ if self._provider in SUPPORTS_JSON_MODE:+ kwargs["response_format"] = {"type": "json_object"}++ response = self.client.chat.completions.create(**kwargs)+ return str(response.choices[0].message.content)++ def _parse_response(self, text: str) -> list[Memory]:+ """Parse LLM response into memories.++ Args:+ text: Raw LLM response++ Returns:+ List of Memory objects+ """+ try:+ # Extract JSON from response (handle markdown code blocks)+ json_match = re.search(r"```(?:json)?\s*(.*?)\s*```", text, re.DOTALL)+ if json_match:+ text = json_match.group(1)++ data = json.loads(text.strip())++ if not data.get("should_remember", False):+ return []++ memories = []+ for item in data.get("memories", []):+ memories.append(+ Memory(+ content=item["content"],+ category=item.get("category", "fact"),+ importance=item.get("importance", 0.5),+ )+ )++ return memories++ except (json.JSONDecodeError, KeyError) as e:+ logger.warning(f"Failed to parse extraction response: {e}")+ return []++ def _build_batch_prompt(self, conversations: list[tuple[str, str, str]]) -> str:+ """Build a batch extraction prompt.++ Args:+ conversations: List of (entity_id, query, response) tuples++ Returns:+ Batch prompt string+ """+ lines = [+ "Analyze these conversations and extract facts worth remembering about each entity.",+ "",+ "Focus on: preferences, facts, context that helps future interactions.",+ "",+ ]++ for i, (entity_id, query, response) in enumerate(conversations):+ lines.extend(+ [+ f"--- Conversation {i + 1} (Entity: {entity_id}) ---",+ f"Speaker A: {query}",+ f"Speaker B: {response}",+ "",+ ]+ )++ lines.extend(+ [+ "Return a JSON object mapping entity_id to their memories:",+ "{",+ ' "entity_123": {',+ ' "memories": [{"content": "...", "category": "preference", "importance": 0.8}],',+ ' "should_remember": true',+ " }",+ "}",+ "",+ "Categories: preference, fact, context",+ "Importance: 0.0-1.0",+ "",+ "Return ONLY valid JSON.",+ ]+ )++ return "\n".join(lines)++ def _parse_batch_response(+ self,+ text: str,+ conversations: list[tuple[str, str, str]],+ ) -> dict[str, list[Memory]]:+ """Parse batch extraction response.++ Args:+ text: Raw LLM response+ conversations: Original conversations for fallback++ Returns:+ Dict mapping user_id to list of memories+ """+ try:+ # Extract JSON from response+ json_match = re.search(r"```(?:json)?\s*(.*?)\s*```", text, re.DOTALL)+ if json_match:+ text = json_match.group(1)++ data = json.loads(text.strip())+ result: dict[str, list[Memory]] = {}++ for user_id, user_data in data.items():+ if not user_data.get("should_remember", False):+ continue++ memories = []+ for item in user_data.get("memories", []):+ memories.append(+ Memory(+ content=item["content"],+ category=item.get("category", "fact"),+ importance=item.get("importance", 0.5),+ )+ )++ if memories:+ result[user_id] = memories++ return result++ except (json.JSONDecodeError, KeyError, AttributeError) as e:+ logger.warning(f"Failed to parse batch response: {e}")+ return {}diff --git a/headroom/memory/fast_store.py b/headroom/memory/fast_store.pynew file mode 100644index 000000000..a002f31e4--- /dev/null+++ b/headroom/memory/fast_store.py@@ -0,0 +1,621 @@+"""Fast embedding-based memory store.++Sub-100ms write and read latency by:+1. NO LLM extraction - just embed and store+2. Vector similarity search - not keyword matching+3. Optional local embeddings for sub-10ms latency++This replaces the slow LLM-based extraction approach.+"""++from __future__ import annotations++import json+import logging+import sqlite3+import time+from collections.abc import Callable+from dataclasses import dataclass, field+from datetime import datetime+from pathlib import Path+from typing import Any+from uuid import uuid4++import numpy as np++logger = logging.getLogger(__name__)+++@dataclass+class MemoryChunk:+ """A memory chunk with text and embedding."""++ id: str = field(default_factory=lambda: str(uuid4()))+ text: str = ""+ role: str = "user" # "user" or "assistant"+ embedding: np.ndarray | None = None+ timestamp: datetime = field(default_factory=datetime.utcnow)+ metadata: dict[str, Any] = field(default_factory=dict)++ def to_dict(self) -> dict:+ """Convert to dictionary for storage."""+ return {+ "id": self.id,+ "text": self.text,+ "role": self.role,+ "embedding": self.embedding.tolist() if self.embedding is not None else None,+ "timestamp": self.timestamp.isoformat(),+ "metadata": self.metadata,+ }++ @classmethod+ def from_dict(cls, data: dict) -> MemoryChunk:+ """Create from dictionary."""+ embedding = None+ if data.get("embedding"):+ embedding = np.array(data["embedding"], dtype=np.float32)+ return cls(+ id=data["id"],+ text=data["text"],+ role=data.get("role", "user"),+ embedding=embedding,+ timestamp=datetime.fromisoformat(data["timestamp"]),+ metadata=data.get("metadata", {}),+ )+++# Type aliases for embedding functions+EmbedFn = Callable[[str], np.ndarray]+BatchEmbedFn = Callable[[list[str]], list[np.ndarray]]+++def cosine_similarity(a: np.ndarray, b: np.ndarray) -> float:+ """Compute cosine similarity between two vectors."""+ norm_a = np.linalg.norm(a)+ norm_b = np.linalg.norm(b)+ if norm_a == 0 or norm_b == 0:+ return 0.0+ return float(np.dot(a, b) / (norm_a * norm_b))+++def cosine_similarity_batch(query: np.ndarray, vectors: np.ndarray) -> np.ndarray:+ """Compute cosine similarity between query and multiple vectors."""+ # Normalize query+ query_norm = query / (np.linalg.norm(query) + 1e-9)+ # Normalize vectors+ norms = np.linalg.norm(vectors, axis=1, keepdims=True) + 1e-9+ vectors_norm = vectors / norms+ # Dot product - cast to ndarray to satisfy mypy+ result: np.ndarray = np.dot(vectors_norm, query_norm)+ return result+++class FastMemoryStore:+ """Fast embedding-based memory store.++ Features:+ - Sub-100ms write latency (no LLM, just embedding)+ - Sub-50ms read latency (vector similarity search)+ - Pluggable embedding functions (local or API)+ - SQLite storage with in-memory vector cache++ Usage:+ store = FastMemoryStore(db_path, embed_fn=my_embed_fn)+ store.add("user_123", "I prefer Python", role="user")+ results = store.search("user_123", "programming language", top_k=5)+ """++ def __init__(+ self,+ db_path: str | Path,+ embed_fn: EmbedFn | None = None,+ embedding_dim: int = 1536, # OpenAI default+ ):+ """Initialize the store.++ Args:+ db_path: Path to SQLite database+ embed_fn: Function to embed text (if None, must call set_embed_fn later)+ embedding_dim: Dimension of embeddings+ """+ self.db_path = Path(db_path)+ self.embed_fn = embed_fn+ self.embedding_dim = embedding_dim++ # In-memory vector cache for fast similarity search+ self._vector_cache: dict[+ str, dict[str, np.ndarray]+ ] = {} # user_id -> {chunk_id -> embedding}+ self._chunk_cache: dict[str, dict[str, MemoryChunk]] = {} # user_id -> {chunk_id -> chunk}++ self._init_db()+ self._load_cache()++ def _init_db(self) -> None:+ """Initialize SQLite database."""+ self.db_path.parent.mkdir(parents=True, exist_ok=True)++ with sqlite3.connect(str(self.db_path)) as conn:+ conn.execute("""+ CREATE TABLE IF NOT EXISTS memory_chunks (+ id TEXT PRIMARY KEY,+ user_id TEXT NOT NULL,+ text TEXT NOT NULL,+ role TEXT DEFAULT 'user',+ embedding BLOB,+ timestamp TEXT NOT NULL,+ metadata TEXT DEFAULT '{}',+ created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP+ )+ """)+ conn.execute("""+ CREATE INDEX IF NOT EXISTS idx_chunks_user_id+ ON memory_chunks(user_id)+ """)+ conn.execute("""+ CREATE INDEX IF NOT EXISTS idx_chunks_timestamp+ ON memory_chunks(user_id, timestamp DESC)+ """)+ conn.commit()++ def _load_cache(self) -> None:+ """Load all embeddings into memory for fast search."""+ with sqlite3.connect(str(self.db_path)) as conn:+ cursor = conn.execute("""+ SELECT id, user_id, text, role, embedding, timestamp, metadata+ FROM memory_chunks+ WHERE embedding IS NOT NULL+ """)++ for row in cursor:+ chunk_id, user_id, text, role, embedding_blob, timestamp, metadata = row++ if user_id not in self._vector_cache:+ self._vector_cache[user_id] = {}+ self._chunk_cache[user_id] = {}++ # Deserialize embedding+ embedding = np.frombuffer(embedding_blob, dtype=np.float32)++ self._vector_cache[user_id][chunk_id] = embedding++ chunk = MemoryChunk(+ id=chunk_id,+ text=text,+ role=role,+ embedding=embedding,+ timestamp=datetime.fromisoformat(timestamp),+ metadata=json.loads(metadata) if metadata else {},+ )+ self._chunk_cache[user_id][chunk_id] = chunk++ logger.debug(f"Loaded {sum(len(v) for v in self._vector_cache.values())} chunks into cache")++ def set_embed_fn(self, embed_fn: EmbedFn) -> None:+ """Set the embedding function."""+ self.embed_fn = embed_fn++ def add(+ self,+ user_id: str,+ text: str,+ role: str = "user",+ metadata: dict[str, Any] | None = None,+ ) -> MemoryChunk:+ """Add a memory chunk.++ This is the FAST path - just embed and store, no LLM extraction.+ Typical latency: <50ms with API embeddings, <10ms with local.++ Args:+ user_id: User/entity identifier+ text: Text to store+ role: "user" or "assistant"+ metadata: Optional metadata++ Returns:+ The created MemoryChunk+ """+ if not self.embed_fn:+ raise ValueError("No embedding function set. Call set_embed_fn() first.")++ start_time = time.perf_counter()++ # Embed the text+ embedding = self.embed_fn(text)+ embed_time = time.perf_counter() - start_time++ # Create chunk+ chunk = MemoryChunk(+ text=text,+ role=role,+ embedding=embedding,+ metadata=metadata or {},+ )++ # Store in SQLite+ with sqlite3.connect(str(self.db_path)) as conn:+ conn.execute(+ """+ INSERT INTO memory_chunks (id, user_id, text, role, embedding, timestamp, metadata)+ VALUES (?, ?, ?, ?, ?, ?, ?)+ """,+ (+ chunk.id,+ user_id,+ chunk.text,+ chunk.role,+ embedding.astype(np.float32).tobytes(),+ chunk.timestamp.isoformat(),+ json.dumps(chunk.metadata),+ ),+ )+ conn.commit()++ # Update cache+ if user_id not in self._vector_cache:+ self._vector_cache[user_id] = {}+ self._chunk_cache[user_id] = {}++ self._vector_cache[user_id][chunk.id] = embedding+ self._chunk_cache[user_id][chunk.id] = chunk++ total_time = time.perf_counter() - start_time+ logger.debug(f"Added chunk in {total_time * 1000:.1f}ms (embed: {embed_time * 1000:.1f}ms)")++ return chunk++ def add_turn(+ self,+ user_id: str,+ user_message: str,+ assistant_response: str,+ metadata: dict[str, Any] | None = None,+ ) -> tuple[MemoryChunk, MemoryChunk]:+ """Add a conversation turn (user message + assistant response).++ Convenience method that stores both parts of a turn.++ Args:+ user_id: User/entity identifier+ user_message: The user's message+ assistant_response: The assistant's response+ metadata: Optional metadata for both chunks++ Returns:+ Tuple of (user_chunk, assistant_chunk)+ """+ user_chunk = self.add(user_id, user_message, role="user", metadata=metadata)+ assistant_chunk = self.add(user_id, assistant_response, role="assistant", metadata=metadata)+ return user_chunk, assistant_chunk++ def add_turn_batched(+ self,+ user_id: str,+ user_message: str,+ assistant_response: str,+ batch_embed_fn: BatchEmbedFn,+ metadata: dict[str, Any] | None = None,+ ) -> tuple[MemoryChunk, MemoryChunk]:+ """Add a conversation turn using BATCHED embedding (single API call).++ This is the FASTEST path - embeds both messages in ONE API call.+ Typical latency: 50-100ms total vs 200-400ms with individual calls.++ Args:+ user_id: User/entity identifier+ user_message: The user's message+ assistant_response: The assistant's response+ batch_embed_fn: Batch embedding function+ metadata: Optional metadata for both chunks++ Returns:+ Tuple of (user_chunk, assistant_chunk)+ """+ start_time = time.perf_counter()++ # Embed BOTH messages in ONE API call+ embeddings = batch_embed_fn([user_message, assistant_response])+ embed_time = time.perf_counter() - start_time++ # Create chunks+ user_chunk = MemoryChunk(+ text=user_message,+ role="user",+ embedding=embeddings[0],+ metadata=metadata or {},+ )+ assistant_chunk = MemoryChunk(+ text=assistant_response,+ role="assistant",+ embedding=embeddings[1],+ metadata=metadata or {},+ )++ # Store in SQLite (batch insert)+ with sqlite3.connect(str(self.db_path)) as conn:+ conn.executemany(+ """+ INSERT INTO memory_chunks (id, user_id, text, role, embedding, timestamp, metadata)+ VALUES (?, ?, ?, ?, ?, ?, ?)+ """,+ [+ (+ user_chunk.id,+ user_id,+ user_chunk.text,+ user_chunk.role,+ embeddings[0].astype(np.float32).tobytes(),+ user_chunk.timestamp.isoformat(),+ json.dumps(user_chunk.metadata),+ ),+ (+ assistant_chunk.id,+ user_id,+ assistant_chunk.text,+ assistant_chunk.role,+ embeddings[1].astype(np.float32).tobytes(),+ assistant_chunk.timestamp.isoformat(),+ json.dumps(assistant_chunk.metadata),+ ),+ ],+ )+ conn.commit()++ # Update cache+ if user_id not in self._vector_cache:+ self._vector_cache[user_id] = {}+ self._chunk_cache[user_id] = {}++ self._vector_cache[user_id][user_chunk.id] = embeddings[0]+ self._vector_cache[user_id][assistant_chunk.id] = embeddings[1]+ self._chunk_cache[user_id][user_chunk.id] = user_chunk+ self._chunk_cache[user_id][assistant_chunk.id] = assistant_chunk++ total_time = time.perf_counter() - start_time+ logger.debug(+ f"Added turn (batched) in {total_time * 1000:.1f}ms (embed: {embed_time * 1000:.1f}ms)"+ )++ return user_chunk, assistant_chunk++ def search(+ self,+ user_id: str,+ query: str,+ top_k: int = 5,+ min_similarity: float = 0.0,+ role_filter: str | None = None,+ ) -> list[tuple[MemoryChunk, float]]:+ """Search for relevant memory chunks.++ Uses vector similarity search for semantic matching.+ Typical latency: <50ms with API embeddings, <10ms with local.++ Args:+ user_id: User/entity identifier+ query: Search query+ top_k: Number of results to return+ min_similarity: Minimum cosine similarity threshold+ role_filter: Optional filter by role ("user" or "assistant")++ Returns:+ List of (chunk, similarity_score) tuples, sorted by relevance+ """+ if not self.embed_fn:+ raise ValueError("No embedding function set. Call set_embed_fn() first.")++ start_time = time.perf_counter()++ # Check if user has any memories+ if user_id not in self._vector_cache or not self._vector_cache[user_id]:+ return []++ # Embed query+ query_embedding = self.embed_fn(query)+ embed_time = time.perf_counter() - start_time++ # Get user's vectors+ chunk_ids = list(self._vector_cache[user_id].keys())+ vectors = np.array([self._vector_cache[user_id][cid] for cid in chunk_ids])++ # Compute similarities+ similarities = cosine_similarity_batch(query_embedding, vectors)+ search_time = time.perf_counter() - start_time - embed_time++ # Sort by similarity+ sorted_indices = np.argsort(similarities)[::-1]++ # Collect results+ results = []+ for idx in sorted_indices:+ chunk_id = chunk_ids[idx]+ similarity = float(similarities[idx])++ if similarity < min_similarity:+ break++ chunk = self._chunk_cache[user_id][chunk_id]++ # Apply role filter+ if role_filter and chunk.role != role_filter:+ continue++ results.append((chunk, similarity))++ if len(results) >= top_k:+ break++ total_time = time.perf_counter() - start_time+ logger.debug(+ f"Search completed in {total_time * 1000:.1f}ms "+ f"(embed: {embed_time * 1000:.1f}ms, search: {search_time * 1000:.1f}ms)"+ )++ return results++ def get_recent(+ self,+ user_id: str,+ limit: int = 10,+ role_filter: str | None = None,+ ) -> list[MemoryChunk]:+ """Get recent memory chunks.++ Args:+ user_id: User/entity identifier+ limit: Maximum number of chunks to return+ role_filter: Optional filter by role++ Returns:+ List of chunks, sorted by timestamp (newest first)+ """+ if user_id not in self._chunk_cache:+ return []++ chunks = list(self._chunk_cache[user_id].values())++ # Apply role filter+ if role_filter:+ chunks = [c for c in chunks if c.role == role_filter]++ # Sort by timestamp+ chunks.sort(key=lambda c: c.timestamp, reverse=True)++ return chunks[:limit]++ def get_all(self, user_id: str) -> list[MemoryChunk]:+ """Get all memory chunks for a user."""+ if user_id not in self._chunk_cache:+ return []+ return list(self._chunk_cache[user_id].values())++ def delete(self, user_id: str, chunk_id: str) -> bool:+ """Delete a specific chunk."""+ with sqlite3.connect(str(self.db_path)) as conn:+ cursor = conn.execute(+ "DELETE FROM memory_chunks WHERE id = ? AND user_id = ?",+ (chunk_id, user_id),+ )+ conn.commit()+ deleted = cursor.rowcount > 0++ if deleted and user_id in self._vector_cache:+ self._vector_cache[user_id].pop(chunk_id, None)+ self._chunk_cache[user_id].pop(chunk_id, None)++ return deleted++ def clear(self, user_id: str) -> int:+ """Clear all memories for a user."""+ with sqlite3.connect(str(self.db_path)) as conn:+ cursor = conn.execute(+ "DELETE FROM memory_chunks WHERE user_id = ?",+ (user_id,),+ )+ conn.commit()+ count = cursor.rowcount++ self._vector_cache.pop(user_id, None)+ self._chunk_cache.pop(user_id, None)++ return count++ def stats(self, user_id: str) -> dict[str, Any]:+ """Get statistics for a user."""+ chunks = self.get_all(user_id)+ return {+ "total": len(chunks),+ "user_messages": sum(1 for c in chunks if c.role == "user"),+ "assistant_messages": sum(1 for c in chunks if c.role == "assistant"),+ }+++# =============================================================================+# Embedding Functions+# =============================================================================+++def create_openai_embed_fn(+ client: Any,+ model: str = "text-embedding-3-small",+) -> EmbedFn:+ """Create an embedding function using OpenAI API.++ Typical latency: 30-100ms per call.++ Args:+ client: OpenAI client+ model: Embedding model to use++ Returns:+ Embedding function+ """++ def embed(text: str) -> np.ndarray:+ response = client.embeddings.create(+ model=model,+ input=text,+ )+ return np.array(response.data[0].embedding, dtype=np.float32)++ return embed+++def create_openai_batch_embed_fn(+ client: Any,+ model: str = "text-embedding-3-small",+) -> BatchEmbedFn:+ """Create a BATCH embedding function using OpenAI API.++ Much faster than individual calls - single API round trip for multiple texts.+ Typical latency: 50-200ms for 10 texts vs 500-2000ms for 10 individual calls.++ Args:+ client: OpenAI client+ model: Embedding model to use++ Returns:+ Batch embedding function+ """++ def embed_batch(texts: list[str]) -> list[np.ndarray]:+ if not texts:+ return []+ response = client.embeddings.create(+ model=model,+ input=texts,+ )+ # Sort by index to maintain order+ sorted_data = sorted(response.data, key=lambda x: x.index)+ return [np.array(d.embedding, dtype=np.float32) for d in sorted_data]++ return embed_batch+++def create_local_embed_fn(+ model_name: str = "all-MiniLM-L6-v2",+) -> EmbedFn:+ """Create an embedding function using local sentence-transformers.++ Typical latency: 5-20ms per call (after model load).++ Args:+ model_name: Sentence-transformers model name++ Returns:+ Embedding function+ """+ try:+ from sentence_transformers import SentenceTransformer+ except ImportError:+ raise ImportError(+ "sentence-transformers not installed. Install with: pip install sentence-transformers"+ ) from None++ model = SentenceTransformer(model_name)++ def embed(text: str) -> np.ndarray:+ return model.encode(text, convert_to_numpy=True).astype(np.float32)++ return embeddiff --git a/headroom/memory/fast_wrapper.py b/headroom/memory/fast_wrapper.pynew file mode 100644index 000000000..9561678f2--- /dev/null+++ b/headroom/memory/fast_wrapper.py@@ -0,0 +1,311 @@+"""Fast Memory Wrapper - Zero-latency inline extraction + semantic retrieval.++This is the ultimate memory solution:+1. ZERO extra latency - memories extracted as part of LLM response (Letta-style)+2. Semantic retrieval - vector similarity for intelligent memory lookup+3. Local embeddings - sub-50ms retrieval, no API calls needed++Usage:+ from headroom.memory import with_fast_memory++ client = with_fast_memory(OpenAI(), user_id="alice")+ response = client.chat.completions.create(+ model="gpt-4o",+ messages=[{"role": "user", "content": "I prefer Python"}]+ )+ # Memory extracted INLINE - zero extra latency!+"""++from __future__ import annotations++import copy+from pathlib import Path+from typing import Any++from headroom.memory.fast_store import (+ FastMemoryStore,+ MemoryChunk,+ create_local_embed_fn,+ create_openai_embed_fn,+)+from headroom.memory.inline_extractor import (+ inject_memory_instruction,+ parse_response_with_memory,+)+++class FastMemoryWrapper:+ """Wraps an LLM client with zero-latency inline memory extraction.++ Architecture:+ 1. BEFORE: Inject relevant memories into user message (semantic search)+ 2. DURING: Memory instruction is in system prompt+ 3. AFTER: Parse memory block from response, store extracted memories++ All memory operations happen as part of the normal LLM flow - no extra calls!+ """++ def __init__(+ self,+ client: Any,+ user_id: str,+ db_path: str | Path = "headroom_fast_memory.db",+ top_k: int = 5,+ use_local_embeddings: bool = True,+ embedding_model: str = "all-MiniLM-L6-v2",+ _store: FastMemoryStore | None = None,+ ):+ """Initialize the fast memory wrapper.++ Args:+ client: OpenAI-compatible LLM client+ user_id: User identifier for memory isolation+ db_path: Path to SQLite database+ top_k: Number of memories to inject+ use_local_embeddings: Use local model (fast) or OpenAI API+ embedding_model: Model name for local embeddings+ _store: Override store (for testing)+ """+ self._client = client+ self._user_id = user_id+ self._top_k = top_k++ # Initialize store with appropriate embedding function+ if _store:+ self._store = _store+ elif use_local_embeddings:+ embed_fn = create_local_embed_fn(embedding_model)+ # MiniLM-L6-v2 produces 384-dim embeddings+ self._store = FastMemoryStore(db_path, embed_fn=embed_fn, embedding_dim=384)+ else:+ embed_fn = create_openai_embed_fn(client)+ self._store = FastMemoryStore(db_path, embed_fn=embed_fn)++ # Create wrapped chat interface+ self.chat = _FastWrappedChat(self)++ @property+ def memory(self) -> _FastMemoryAPI:+ """Direct access to memory operations."""+ return _FastMemoryAPI(self._store, self._user_id)++ def _inject_memories(self, messages: list[dict]) -> list[dict]:+ """Inject relevant memories into user message.++ Uses semantic search (vector similarity) to find relevant memories.+ Injects into FIRST user message to preserve system prompt caching.++ Args:+ messages: Original messages list++ Returns:+ New messages with memories injected+ """+ # Find the last user message for search context+ user_content = None+ for msg in reversed(messages):+ if msg.get("role") == "user":+ user_content = msg.get("content", "")+ break++ if not user_content:+ return messages++ # Semantic search for relevant memories+ results = self._store.search(self._user_id, str(user_content), top_k=self._top_k)++ if not results:+ return messages++ # Build context block+ context_lines = ["<context>"]+ for chunk, _score in results:+ context_lines.append(f"- {chunk.text}")+ context_lines.append("</context>")+ context_block = "\n".join(context_lines)++ # Inject into first user message+ new_messages = copy.deepcopy(messages)+ for msg in new_messages:+ if msg.get("role") == "user":+ original = msg.get("content", "")+ msg["content"] = f"{context_block}\n\n{original}"+ break++ return new_messages++ def _store_memories(self, memories: list[dict[str, Any]]) -> None:+ """Store extracted memories.++ Args:+ memories: List of memory dicts from inline extraction+ """+ for mem in memories:+ content = mem.get("content", "")+ category = mem.get("category", "fact")+ if content:+ self._store.add(+ self._user_id,+ content,+ role="memory",+ metadata={"category": category, "source": "inline_extraction"},+ )+++class _FastWrappedChat:+ """Wrapped chat interface."""++ def __init__(self, wrapper: FastMemoryWrapper):+ self._wrapper = wrapper+ self.completions = _FastWrappedCompletions(wrapper)+++class _FastWrappedCompletions:+ """Wrapped completions with inline memory extraction."""++ def __init__(self, wrapper: FastMemoryWrapper):+ self._wrapper = wrapper++ def create(self, **kwargs: Any) -> Any:+ """Create chat completion with inline memory extraction.++ Flow:+ 1. Search for relevant memories (semantic)+ 2. Inject memories into user message+ 3. Add memory instruction to system prompt+ 4. Forward to LLM+ 5. Parse response to extract memories+ 6. Store extracted memories+ 7. Return clean response (without memory block)+ """+ messages = kwargs.get("messages", [])++ # 1. Inject relevant memories into user message+ enhanced_messages = self._wrapper._inject_memories(messages)++ # 2. Add memory extraction instruction to system prompt+ enhanced_messages = inject_memory_instruction(enhanced_messages, short=True)+ kwargs["messages"] = enhanced_messages++ # 3. Forward to LLM+ response = self._wrapper._client.chat.completions.create(**kwargs)++ # 4. Parse response and extract memories+ raw_content = response.choices[0].message.content+ parsed = parse_response_with_memory(raw_content)++ # 5. Store extracted memories+ if parsed.memories:+ self._wrapper._store_memories(parsed.memories)++ # 6. Return clean response (modify in place)+ response.choices[0].message.content = parsed.content++ return response+++class _FastMemoryAPI:+ """Direct API for memory operations."""++ def __init__(self, store: FastMemoryStore, user_id: str):+ self._store = store+ self._user_id = user_id++ def search(self, query: str, top_k: int = 5) -> list[tuple[MemoryChunk, float]]:+ """Semantic search for memories.++ Args:+ query: Search query+ top_k: Max results++ Returns:+ List of (memory, similarity_score) tuples+ """+ return self._store.search(self._user_id, query, top_k)++ def add(self, content: str, category: str = "fact") -> MemoryChunk:+ """Manually add a memory.++ Args:+ content: Memory content+ category: preference, fact, or context++ Returns:+ The created memory chunk+ """+ return self._store.add(+ self._user_id,+ content,+ role="memory",+ metadata={"category": category, "source": "manual"},+ )++ def get_all(self) -> list[MemoryChunk]:+ """Get all memories for this user."""+ return self._store.get_all(self._user_id)++ def clear(self) -> int:+ """Clear all memories for this user."""+ return self._store.clear(self._user_id)++ def stats(self) -> dict:+ """Get memory statistics."""+ return self._store.stats(self._user_id)+++def with_fast_memory(+ client: Any,+ user_id: str,+ db_path: str | Path = "headroom_fast_memory.db",+ top_k: int = 5,+ use_local_embeddings: bool = True,+ embedding_model: str = "all-MiniLM-L6-v2",+ **kwargs: Any,+) -> FastMemoryWrapper:+ """Wrap an LLM client with zero-latency inline memory extraction.++ This is the fastest memory solution:+ 1. ZERO extra LLM calls - memories extracted inline as part of response+ 2. Sub-50ms retrieval - local embeddings, no API calls+ 3. Semantic search - finds conceptually related memories++ Args:+ client: OpenAI-compatible LLM client+ user_id: User identifier for memory isolation+ db_path: Path to SQLite database+ top_k: Number of memories to inject per request+ use_local_embeddings: Use local model (True) or OpenAI API (False)+ embedding_model: Model name for local embeddings+ **kwargs: Additional arguments++ Returns:+ Wrapped client with automatic memory++ Example:+ from openai import OpenAI+ from headroom.memory import with_fast_memory++ client = with_fast_memory(OpenAI(), user_id="alice")++ # First conversation - memory extracted INLINE+ response = client.chat.completions.create(+ model="gpt-4o",+ messages=[{"role": "user", "content": "I prefer Python for backend work"}]+ )++ # Later - memories automatically retrieved+ response = client.chat.completions.create(+ model="gpt-4o",+ messages=[{"role": "user", "content": "What language should I use?"}]+ )+ # User sees: "Based on your preference for Python..."+ """+ return FastMemoryWrapper(+ client=client,+ user_id=user_id,+ db_path=db_path,+ top_k=top_k,+ use_local_embeddings=use_local_embeddings,+ embedding_model=embedding_model,+ **kwargs,+ )diff --git a/headroom/memory/inline_extractor.py b/headroom/memory/inline_extractor.pynew file mode 100644index 000000000..0e7eaa5d0--- /dev/null+++ b/headroom/memory/inline_extractor.py@@ -0,0 +1,229 @@+"""Inline memory extraction - zero extra latency.++Instead of making a separate LLM call to extract memories,+we modify the system prompt so the LLM outputs memories+as part of its response. This is the Letta/MemGPT approach.++Benefits:+- Zero extra latency (memory is part of response)+- Zero extra API cost (already paying for response tokens)+- Higher quality (LLM has full context)+- Intelligent filtering (LLM decides what's relevant)+"""++from __future__ import annotations++import json+import logging+import re+from dataclasses import dataclass+from typing import Any++logger = logging.getLogger(__name__)+++# Memory extraction instruction to append to system prompt+MEMORY_INSTRUCTION = """++## Memory Instructions+After your response, if there are facts worth remembering about the user/entity for future conversations, output them in a <memory> block:++<memory>+{"memories": [{"content": "fact to remember", "category": "preference|fact|context"}]}+</memory>++Categories:+- preference: likes, dislikes, preferred tools/languages/styles+- fact: identity, role, job, location, constraints+- context: current goals, ongoing tasks, recent events++Only output memories for significant, reusable information. Skip for:+- Greetings, thanks, small talk+- One-time questions+- Information already known++If nothing worth remembering: <memory>{"memories": []}</memory>+"""++# Shorter version for token efficiency+MEMORY_INSTRUCTION_SHORT = """++After responding, output facts to remember: <memory>{"memories": [{"content": "...", "category": "preference|fact|context"}]}</memory>+Skip for greetings/small talk. If nothing: <memory>{"memories": []}</memory>"""+++@dataclass+class ParsedResponse:+ """Response with extracted memories."""++ content: str # The actual response (without memory block)+ memories: list[dict[str, Any]] # Extracted memories+ raw: str # Original full response+++def inject_memory_instruction(+ messages: list[dict[str, Any]],+ short: bool = True,+) -> list[dict[str, Any]]:+ """Inject memory extraction instruction into system prompt.++ Args:+ messages: Original messages list+ short: Use short instruction (fewer tokens)++ Returns:+ Modified messages with memory instruction+ """+ instruction = MEMORY_INSTRUCTION_SHORT if short else MEMORY_INSTRUCTION+ messages = [m.copy() for m in messages] # Don't modify original++ # Find or create system message+ has_system = False+ for i, msg in enumerate(messages):+ if msg.get("role") == "system":+ messages[i] = {+ **msg,+ "content": msg.get("content", "") + instruction,+ }+ has_system = True+ break++ if not has_system:+ # Prepend system message+ messages.insert(+ 0,+ {+ "role": "system",+ "content": "You are a helpful assistant." + instruction,+ },+ )++ return messages+++def parse_response_with_memory(response_text: str) -> ParsedResponse:+ """Parse LLM response to extract memories.++ Args:+ response_text: Raw LLM response++ Returns:+ ParsedResponse with content and memories separated+ """+ memories: list[dict[str, Any]] = []+ content = response_text++ # Extract <memory> block+ memory_pattern = r"<memory>\s*(.*?)\s*</memory>"+ match = re.search(memory_pattern, response_text, re.DOTALL | re.IGNORECASE)++ if match:+ memory_json = match.group(1).strip()++ # Remove the memory block from content+ content = re.sub(memory_pattern, "", response_text, flags=re.DOTALL | re.IGNORECASE).strip()++ # Parse the JSON+ try:+ data = json.loads(memory_json)+ memories = data.get("memories", [])+ except json.JSONDecodeError as e:+ logger.warning(f"Failed to parse memory JSON: {e}")++ return ParsedResponse(+ content=content,+ memories=memories,+ raw=response_text,+ )+++class InlineMemoryWrapper:+ """Wrapper that extracts memories from LLM responses inline.++ This is the zero-latency approach - memories are extracted+ as part of the response, not in a separate call.++ Usage:+ wrapper = InlineMemoryWrapper(openai_client)+ response, memories = wrapper.chat(+ messages=[{"role": "user", "content": "I prefer Python"}],+ model="gpt-4o-mini"+ )+ # response = "Great choice! Python is excellent..."+ # memories = [{"content": "User prefers Python", "category": "preference"}]+ """++ def __init__(self, client: Any):+ """Initialize wrapper.++ Args:+ client: OpenAI-compatible client+ """+ self.client = client++ def chat(+ self,+ messages: list[dict[str, Any]],+ model: str = "gpt-4o-mini",+ short_instruction: bool = True,+ **kwargs: Any,+ ) -> tuple[str, list[dict[str, Any]]]:+ """Send chat request and extract memories inline.++ Args:+ messages: Chat messages+ model: Model to use+ short_instruction: Use shorter memory instruction+ **kwargs: Additional args for chat completion++ Returns:+ Tuple of (response_content, extracted_memories)+ """+ # Inject memory instruction+ modified_messages = inject_memory_instruction(messages, short=short_instruction)++ # Call LLM+ response = self.client.chat.completions.create(+ model=model,+ messages=modified_messages,+ **kwargs,+ )++ raw_content = response.choices[0].message.content++ # Parse response and extract memories+ parsed = parse_response_with_memory(raw_content)++ return parsed.content, parsed.memories++ def chat_with_response(+ self,+ messages: list[dict[str, Any]],+ model: str = "gpt-4o-mini",+ **kwargs: Any,+ ) -> tuple[Any, str, list[dict[str, Any]]]:+ """Send chat request and return full response object.++ Args:+ messages: Chat messages+ model: Model to use+ **kwargs: Additional args for chat completion++ Returns:+ Tuple of (response_object, content, memories)+ """+ modified_messages = inject_memory_instruction(messages)++ response = self.client.chat.completions.create(+ model=model,+ messages=modified_messages,+ **kwargs,+ )++ raw_content = response.choices[0].message.content+ parsed = parse_response_with_memory(raw_content)++ # Modify response to have clean content+ response.choices[0].message.content = parsed.content++ return response, parsed.content, parsed.memoriesdiff --git a/headroom/memory/store.py b/headroom/memory/store.pynew file mode 100644index 000000000..0384d27cc--- /dev/null+++ b/headroom/memory/store.py@@ -0,0 +1,434 @@+"""SQLite + FTS5 memory storage for Headroom Memory.++Simple, fast, local-first storage with full-text search.+No external dependencies - just SQLite (built into Python).+"""++from __future__ import annotations++import json+import sqlite3+import uuid+from dataclasses import dataclass, field+from datetime import datetime+from pathlib import Path+from typing import Literal+++@dataclass+class Memory:+ """A single memory entry."""++ content: str+ category: Literal["preference", "fact", "context"] = "fact"+ importance: float = 0.5+ id: str = field(default_factory=lambda: str(uuid.uuid4()))+ created_at: datetime = field(default_factory=datetime.utcnow)+ metadata: dict = field(default_factory=dict)+++@dataclass+class PendingExtraction:+ """A conversation pending memory extraction."""++ user_id: str+ query: str+ response: str+ id: str = field(default_factory=lambda: str(uuid.uuid4()))+ created_at: datetime = field(default_factory=datetime.utcnow)+ status: Literal["pending", "processing", "done", "failed"] = "pending"+++class SQLiteMemoryStore:+ """SQLite + FTS5 storage for memories.++ Features:+ - Full-text search via FTS5+ - User isolation (each user_id has separate memories)+ - Pending extractions for crash recovery+ - Thread-safe with connection per call++ Usage:+ store = SQLiteMemoryStore("./memory.db")+ store.save("alice", Memory(content="Prefers Python"))+ results = store.search("alice", "python")+ """++ def __init__(self, db_path: str | Path = "headroom_memory.db"):+ """Initialize the store.++ Args:+ db_path: Path to SQLite database file. Created if doesn't exist.+ """+ self.db_path = Path(db_path)+ self._init_db()++ def _get_conn(self) -> sqlite3.Connection:+ """Get a new connection (thread-safe pattern)."""+ conn = sqlite3.connect(str(self.db_path))+ conn.row_factory = sqlite3.Row+ return conn++ def _init_db(self) -> None:+ """Initialize database schema."""+ with self._get_conn() as conn:+ # Main memories table+ conn.execute("""+ CREATE TABLE IF NOT EXISTS memories (+ id TEXT PRIMARY KEY,+ user_id TEXT NOT NULL,+ content TEXT NOT NULL,+ category TEXT NOT NULL DEFAULT 'fact',+ importance REAL NOT NULL DEFAULT 0.5,+ created_at TEXT NOT NULL,+ metadata TEXT NOT NULL DEFAULT '{}'+ )+ """)++ # FTS5 virtual table for full-text search+ conn.execute("""+ CREATE VIRTUAL TABLE IF NOT EXISTS memories_fts USING fts5(+ content,+ content='memories',+ content_rowid='rowid'+ )+ """)++ # Triggers to keep FTS in sync+ conn.execute("""+ CREATE TRIGGER IF NOT EXISTS memories_ai AFTER INSERT ON memories BEGIN+ INSERT INTO memories_fts(rowid, content)+ VALUES (new.rowid, new.content);+ END+ """)++ conn.execute("""+ CREATE TRIGGER IF NOT EXISTS memories_ad AFTER DELETE ON memories BEGIN+ INSERT INTO memories_fts(memories_fts, rowid, content)+ VALUES ('delete', old.rowid, old.content);+ END+ """)++ conn.execute("""+ CREATE TRIGGER IF NOT EXISTS memories_au AFTER UPDATE ON memories BEGIN+ INSERT INTO memories_fts(memories_fts, rowid, content)+ VALUES ('delete', old.rowid, old.content);+ INSERT INTO memories_fts(rowid, content)+ VALUES (new.rowid, new.content);+ END+ """)++ # Index for user_id filtering+ conn.execute("""+ CREATE INDEX IF NOT EXISTS idx_memories_user_id+ ON memories(user_id)+ """)++ # Pending extractions table (for crash recovery)+ conn.execute("""+ CREATE TABLE IF NOT EXISTS pending_extractions (+ id TEXT PRIMARY KEY,+ user_id TEXT NOT NULL,+ query TEXT NOT NULL,+ response TEXT NOT NULL,+ created_at TEXT NOT NULL,+ status TEXT NOT NULL DEFAULT 'pending'+ )+ """)++ conn.execute("""+ CREATE INDEX IF NOT EXISTS idx_pending_status+ ON pending_extractions(status)+ """)++ conn.commit()++ def save(self, user_id: str, memory: Memory) -> None:+ """Save a memory for a user.++ Args:+ user_id: User identifier for isolation+ memory: Memory to save+ """+ with self._get_conn() as conn:+ conn.execute(+ """+ INSERT INTO memories (id, user_id, content, category, importance, created_at, metadata)+ VALUES (?, ?, ?, ?, ?, ?, ?)+ """,+ (+ memory.id,+ user_id,+ memory.content,+ memory.category,+ memory.importance,+ memory.created_at.isoformat(),+ json.dumps(memory.metadata),+ ),+ )+ conn.commit()++ def search(self, user_id: str, query: str, top_k: int = 5) -> list[Memory]:+ """Search memories using FTS5 full-text search.++ Args:+ user_id: User identifier for isolation+ query: Search query (auto-escaped, or use raw FTS5 syntax with prefix '_raw:')+ top_k: Maximum number of results++ Returns:+ List of matching memories, ranked by relevance+ """+ # Sanitize query for FTS5 (escape special characters unless raw mode)+ if query.startswith("_raw:"):+ fts_query = query[5:] # Use raw FTS5 syntax+ else:+ fts_query = self._sanitize_fts_query(query)++ if not fts_query.strip():+ return []++ with self._get_conn() as conn:+ # Use FTS5 MATCH with BM25 ranking, filtered by user_id+ cursor = conn.execute(+ """+ SELECT m.*, bm25(memories_fts) as rank+ FROM memories m+ JOIN memories_fts ON m.rowid = memories_fts.rowid+ WHERE memories_fts MATCH ? AND m.user_id = ?+ ORDER BY rank+ LIMIT ?+ """,+ (fts_query, user_id, top_k),+ )++ results = []+ for row in cursor:+ results.append(+ Memory(+ id=row["id"],+ content=row["content"],+ category=row["category"],+ importance=row["importance"],+ created_at=datetime.fromisoformat(row["created_at"]),+ metadata=json.loads(row["metadata"]),+ )+ )+ return results++ def _sanitize_fts_query(self, query: str) -> str:+ """Sanitize a query for FTS5.++ Escapes special characters and converts to prefix search for better matching.++ Args:+ query: Raw user query++ Returns:+ FTS5-safe query string+ """+ # FTS5 special characters that need escaping+ # We use a simple approach: extract words and use OR between them+ import re++ # Extract alphanumeric words+ words = re.findall(r"\w+", query)++ if not words:+ return ""++ # Use OR between words with prefix matching for flexibility+ # This allows "What language" to match "Python" memories when searching+ # by using prefix matching (word*)+ escaped_words = []+ for word in words:+ # Quote each word to handle any remaining special chars+ escaped_words.append(f'"{word}"')++ return " OR ".join(escaped_words)++ def get_all(self, user_id: str) -> list[Memory]:+ """Get all memories for a user.++ Args:+ user_id: User identifier++ Returns:+ All memories for the user, ordered by creation time (newest first)+ """+ with self._get_conn() as conn:+ cursor = conn.execute(+ """+ SELECT * FROM memories+ WHERE user_id = ?+ ORDER BY created_at DESC+ """,+ (user_id,),+ )++ return [+ Memory(+ id=row["id"],+ content=row["content"],+ category=row["category"],+ importance=row["importance"],+ created_at=datetime.fromisoformat(row["created_at"]),+ metadata=json.loads(row["metadata"]),+ )+ for row in cursor+ ]++ def delete(self, user_id: str, memory_id: str) -> bool:+ """Delete a specific memory.++ Args:+ user_id: User identifier+ memory_id: ID of memory to delete++ Returns:+ True if deleted, False if not found+ """+ with self._get_conn() as conn:+ cursor = conn.execute(+ "DELETE FROM memories WHERE id = ? AND user_id = ?",+ (memory_id, user_id),+ )+ conn.commit()+ return cursor.rowcount > 0++ def clear(self, user_id: str) -> int:+ """Delete all memories for a user.++ Args:+ user_id: User identifier++ Returns:+ Number of memories deleted+ """+ with self._get_conn() as conn:+ cursor = conn.execute(+ "DELETE FROM memories WHERE user_id = ?",+ (user_id,),+ )+ conn.commit()+ return cursor.rowcount++ def stats(self, user_id: str) -> dict:+ """Get memory statistics for a user.++ Args:+ user_id: User identifier++ Returns:+ Dict with count, categories breakdown, etc.+ """+ with self._get_conn() as conn:+ # Total count+ total = conn.execute(+ "SELECT COUNT(*) as count FROM memories WHERE user_id = ?",+ (user_id,),+ ).fetchone()["count"]++ # Category breakdown+ categories = {}+ for row in conn.execute(+ """+ SELECT category, COUNT(*) as count+ FROM memories WHERE user_id = ?+ GROUP BY category+ """,+ (user_id,),+ ):+ categories[row["category"]] = row["count"]++ return {+ "total": total,+ "categories": categories,+ }++ # --- Pending Extractions (for crash recovery) ---++ def queue_extraction(self, pending: PendingExtraction) -> None:+ """Queue a conversation for memory extraction.++ Args:+ pending: The pending extraction to queue+ """+ with self._get_conn() as conn:+ conn.execute(+ """+ INSERT INTO pending_extractions (id, user_id, query, response, created_at, status)+ VALUES (?, ?, ?, ?, ?, ?)+ """,+ (+ pending.id,+ pending.user_id,+ pending.query,+ pending.response,+ pending.created_at.isoformat(),+ pending.status,+ ),+ )+ conn.commit()++ def get_pending_extractions(+ self, limit: int = 10, status: str = "pending"+ ) -> list[PendingExtraction]:+ """Get pending extractions for processing.++ Args:+ limit: Maximum number to return+ status: Filter by status++ Returns:+ List of pending extractions+ """+ with self._get_conn() as conn:+ cursor = conn.execute(+ """+ SELECT * FROM pending_extractions+ WHERE status = ?+ ORDER BY created_at ASC+ LIMIT ?+ """,+ (status, limit),+ )++ return [+ PendingExtraction(+ id=row["id"],+ user_id=row["user_id"],+ query=row["query"],+ response=row["response"],+ created_at=datetime.fromisoformat(row["created_at"]),+ status=row["status"],+ )+ for row in cursor+ ]++ def update_extraction_status(self, extraction_id: str, status: str) -> None:+ """Update the status of a pending extraction.++ Args:+ extraction_id: ID of the extraction+ status: New status+ """+ with self._get_conn() as conn:+ conn.execute(+ "UPDATE pending_extractions SET status = ? WHERE id = ?",+ (status, extraction_id),+ )+ conn.commit()++ def delete_extraction(self, extraction_id: str) -> None:+ """Delete a completed extraction.++ Args:+ extraction_id: ID of the extraction to delete+ """+ with self._get_conn() as conn:+ conn.execute(+ "DELETE FROM pending_extractions WHERE id = ?",+ (extraction_id,),+ )+ conn.commit()diff --git a/headroom/memory/worker.py b/headroom/memory/worker.pynew file mode 100644index 000000000..7684b33ee--- /dev/null+++ b/headroom/memory/worker.py@@ -0,0 +1,260 @@+"""Background worker for batched memory extraction.++Collects conversations in a queue and processes them in batches,+reducing LLM calls and improving efficiency.+"""++from __future__ import annotations++import atexit+import logging+import threading+import time+from typing import TYPE_CHECKING++if TYPE_CHECKING:+ from headroom.memory.extractor import MemoryExtractor+ from headroom.memory.store import SQLiteMemoryStore+++logger = logging.getLogger(__name__)+++class ExtractionWorker:+ """Background worker that batches memory extractions.++ Features:+ - Collects conversations in a queue+ - Processes in batches (configurable size and timeout)+ - Persists pending work to SQLite for crash recovery+ - Thread-safe, daemon thread (stops with main program)++ Usage:+ worker = ExtractionWorker(store, extractor)+ worker.start()+ worker.schedule("alice", "I prefer Python", "Great choice!")+ # ... later, memories are extracted and saved automatically+ """++ def __init__(+ self,+ store: SQLiteMemoryStore,+ extractor: MemoryExtractor,+ batch_size: int = 10,+ max_wait_seconds: float = 30.0,+ ):+ """Initialize the worker.++ Args:+ store: Memory store for saving extracted memories+ extractor: Extractor for processing conversations+ batch_size: Max conversations per batch+ max_wait_seconds: Max time to wait before processing partial batch+ """+ self.store = store+ self.extractor = extractor+ self.batch_size = batch_size+ self.max_wait_seconds = max_wait_seconds++ self._queue: list[tuple[str, str, str]] = [] # (user_id, query, response)+ self._lock = threading.Lock()+ self._event = threading.Event()+ self._running = False+ self._thread: threading.Thread | None = None++ # Register cleanup on exit+ atexit.register(self._cleanup)++ def start(self) -> None:+ """Start the background worker thread."""+ if self._running:+ return++ self._running = True+ self._thread = threading.Thread(target=self._run, daemon=True)+ self._thread.start()++ # Process any pending extractions from previous runs (crash recovery)+ self._recover_pending()++ def stop(self, wait: bool = True, timeout: float = 5.0) -> None:+ """Stop the worker.++ Args:+ wait: If True, process remaining queue before stopping+ timeout: Max time to wait for remaining work+ """+ if not self._running:+ return++ self._running = False+ self._event.set() # Wake up the thread++ if wait and self._thread:+ self._thread.join(timeout=timeout)++ def schedule(self, user_id: str, query: str, response: str) -> None:+ """Schedule a conversation for memory extraction.++ Non-blocking - returns immediately and extracts in background.++ Args:+ user_id: User identifier+ query: User's message+ response: Assistant's response+ """+ # Persist to SQLite first (crash recovery)+ from headroom.memory.store import PendingExtraction++ pending = PendingExtraction(+ user_id=user_id,+ query=query,+ response=response,+ )+ self.store.queue_extraction(pending)++ # Add to in-memory queue+ with self._lock:+ self._queue.append((user_id, query, response))++ # Wake up worker if batch is full+ if len(self._queue) >= self.batch_size:+ self._event.set()++ def flush(self, timeout: float = 60.0) -> bool:+ """Force immediate processing of all queued extractions.++ Blocks until all pending extractions are processed or timeout.++ Args:+ timeout: Max time to wait in seconds++ Returns:+ True if all extractions completed, False if timed out+ """+ # Signal worker to process immediately by temporarily setting max_wait to 0+ original_max_wait = self.max_wait_seconds+ self.max_wait_seconds = 0+ self._event.set()++ # Wait for queue to empty+ start = time.time()+ while time.time() - start < timeout:+ pending = self.store.get_pending_extractions(limit=1, status="pending")+ if not pending:+ self.max_wait_seconds = original_max_wait+ return True+ time.sleep(0.5)++ self.max_wait_seconds = original_max_wait+ return False++ def _run(self) -> None:+ """Main worker loop."""+ last_process_time = time.time()++ while self._running:+ # Wait for batch to fill or timeout+ self._event.wait(timeout=1.0)+ self._event.clear()++ now = time.time()+ time_since_last = now - last_process_time++ with self._lock:+ should_process = len(self._queue) >= self.batch_size or (+ self._queue and time_since_last >= self.max_wait_seconds+ )++ if should_process:+ batch = self._queue[: self.batch_size]+ self._queue = self._queue[self.batch_size :]+ else:+ batch = []++ if batch:+ self._process_batch(batch)+ last_process_time = time.time()++ # Process remaining queue on shutdown+ with self._lock:+ remaining = self._queue[:]+ self._queue = []++ if remaining:+ self._process_batch(remaining)++ def _process_batch(self, batch: list[tuple[str, str, str]]) -> None:+ """Process a batch of conversations.++ Args:+ batch: List of (user_id, query, response) tuples+ """+ logger.debug(f"Processing batch of {len(batch)} conversations")++ try:+ # Extract memories+ result = self.extractor.extract_batch(batch)++ # Save memories+ for user_id, memories in result.items():+ for memory in memories:+ self.store.save(user_id, memory)+ logger.debug(f"Saved memory for {user_id}: {memory.content[:50]}...")++ # Mark pending extractions as done+ # Note: In a production system, we'd track exact IDs+ # For simplicity, we clear pending by matching user/query/response+ self._mark_batch_done(batch)++ except Exception as e:+ logger.error(f"Batch extraction failed: {e}")+ self._mark_batch_failed(batch)++ def _recover_pending(self) -> None:+ """Recover pending extractions from previous runs."""+ pending = self.store.get_pending_extractions(limit=100, status="pending")++ if not pending:+ return++ logger.info(f"Recovering {len(pending)} pending extractions")++ with self._lock:+ for p in pending:+ self._queue.append((p.user_id, p.query, p.response))++ # Trigger processing+ self._event.set()++ def _mark_batch_done(self, batch: list[tuple[str, str, str]]) -> None:+ """Mark batch items as completed in the pending table."""+ # Get pending extractions and mark matching ones as done+ pending = self.store.get_pending_extractions(limit=100)++ for user_id, query, response in batch:+ for p in pending:+ if p.user_id == user_id and p.query == query and p.response == response:+ self.store.delete_extraction(p.id)+ break++ def _mark_batch_failed(self, batch: list[tuple[str, str, str]]) -> None:+ """Mark batch items as failed in the pending table."""+ pending = self.store.get_pending_extractions(limit=100)++ for user_id, query, response in batch:+ for p in pending:+ if p.user_id == user_id and p.query == query and p.response == response:+ self.store.update_extraction_status(p.id, "failed")+ break++ def _cleanup(self) -> None:+ """Cleanup on program exit."""+ if self._running:+ self.stop(wait=True, timeout=2.0)++ @property+ def queue_size(self) -> int:+ """Get current queue size."""+ with self._lock:+ return len(self._queue)diff --git a/headroom/memory/wrapper.py b/headroom/memory/wrapper.pynew file mode 100644index 000000000..70cd4dedd--- /dev/null+++ b/headroom/memory/wrapper.py@@ -0,0 +1,321 @@+"""Memory wrapper - the main API for Headroom Memory.++One-line integration:+ from headroom import with_memory+ client = with_memory(OpenAI(), user_id="alice")+"""++from __future__ import annotations++import copy+from pathlib import Path+from typing import Any++from headroom.memory.extractor import MemoryExtractor+from headroom.memory.store import Memory, SQLiteMemoryStore+from headroom.memory.worker import ExtractionWorker+++class MemoryWrapper:+ """Wraps an LLM client to add automatic memory.++ Intercepts chat completions to:+ 1. BEFORE: Inject relevant memories into user message+ 2. AFTER: Queue conversation for background memory extraction++ The system prompt is left unchanged to preserve prompt caching.++ Usage:+ client = MemoryWrapper(OpenAI(), user_id="alice")+ response = client.chat.completions.create(...)+ """++ def __init__(+ self,+ client: Any,+ user_id: str,+ db_path: str | Path = "headroom_memory.db",+ extraction_model: str | None = None,+ top_k: int = 5,+ _extractor: Any = None, # For testing - inject mock+ _store: SQLiteMemoryStore | None = None, # For testing+ ):+ """Initialize the memory wrapper.++ Args:+ client: LLM client (OpenAI, Anthropic, etc.)+ user_id: User identifier for memory isolation+ db_path: Path to SQLite database+ extraction_model: Override extraction model (auto-detect if None)+ top_k: Number of memories to inject+ _extractor: Override extractor (for testing)+ _store: Override store (for testing)+ """+ self._client = client+ self._user_id = user_id+ self._top_k = top_k++ # Initialize store+ self._store = _store or SQLiteMemoryStore(db_path)++ # Initialize extractor+ self._extractor = _extractor or MemoryExtractor(client, model=extraction_model)++ # Initialize background worker with shorter wait for responsiveness+ self._worker = ExtractionWorker(+ store=self._store,+ extractor=self._extractor,+ max_wait_seconds=5.0, # Process partial batches after 5s+ )+ self._worker.start()++ # Create wrapped chat interface+ self.chat = _WrappedChat(self)++ def flush_extractions(self, timeout: float = 60.0) -> bool:+ """Force immediate processing of all queued extractions.++ Useful for testing or when you need to ensure memories are saved.++ Args:+ timeout: Max time to wait in seconds++ Returns:+ True if all extractions completed, False if timed out+ """+ return self._worker.flush(timeout=timeout)++ @property+ def memory(self) -> _MemoryAPI:+ """Direct access to memory operations."""+ return _MemoryAPI(self._store, self._user_id)++ def _inject_memories(self, messages: list[dict]) -> list[dict]:+ """Inject relevant memories into messages.++ Memories are prepended to the FIRST user message to preserve+ system prompt caching.++ Args:+ messages: Original messages list++ Returns:+ New messages list with memories injected+ """+ # Find the last user message+ user_content = None+ for msg in reversed(messages):+ if msg.get("role") == "user":+ user_content = msg.get("content", "")+ break++ if not user_content:+ return messages++ # Search for relevant memories+ memories = self._store.search(+ self._user_id,+ str(user_content),+ top_k=self._top_k,+ )++ if not memories:+ return messages++ # Build context block+ context_lines = ["<context>"]+ for mem in memories:+ context_lines.append(f"- {mem.content}")+ context_lines.append("</context>")+ context_block = "\n".join(context_lines)++ # Find the first user message and prepend context+ new_messages = copy.deepcopy(messages)+ for msg in new_messages:+ if msg.get("role") == "user":+ original = msg.get("content", "")+ msg["content"] = f"{context_block}\n\n{original}"+ break++ return new_messages++ def _queue_extraction(self, query: str, response: str) -> None:+ """Queue conversation for background memory extraction.++ Args:+ query: User's message+ response: Assistant's response+ """+ self._worker.schedule(self._user_id, query, response)+++class _WrappedChat:+ """Wrapped chat interface that intercepts completions."""++ def __init__(self, wrapper: MemoryWrapper):+ self._wrapper = wrapper+ self.completions = _WrappedCompletions(wrapper)+++class _WrappedCompletions:+ """Wrapped completions that add memory to requests."""++ def __init__(self, wrapper: MemoryWrapper):+ self._wrapper = wrapper++ def create(self, **kwargs: Any) -> Any:+ """Create a chat completion with memory injection.++ This intercepts the request to:+ 1. Inject relevant memories into user message+ 2. Forward to the real client+ 3. Queue response for background extraction++ All kwargs are passed through to the underlying client.+ """+ messages = kwargs.get("messages", [])++ # 1. Inject memories into user message+ enhanced_messages = self._wrapper._inject_memories(messages)+ kwargs["messages"] = enhanced_messages++ # 2. Forward to real client+ response = self._wrapper._client.chat.completions.create(**kwargs)++ # 3. Queue for extraction (non-blocking)+ self._extract_and_queue(messages, response)++ return response++ def _extract_and_queue(self, original_messages: list[dict], response: Any) -> None:+ """Extract query and response, queue for extraction."""+ # Get the last user message (without context injection)+ user_query = None+ for msg in reversed(original_messages):+ if msg.get("role") == "user":+ user_query = msg.get("content", "")+ break++ if not user_query:+ return++ # Get assistant response+ try:+ assistant_response = response.choices[0].message.content+ except (AttributeError, IndexError):+ return++ if assistant_response:+ self._wrapper._queue_extraction(user_query, assistant_response)+++class _MemoryAPI:+ """Direct API for memory operations."""++ def __init__(self, store: SQLiteMemoryStore, user_id: str):+ self._store = store+ self._user_id = user_id++ def search(self, query: str, top_k: int = 5) -> list[Memory]:+ """Search memories.++ Args:+ query: Search query+ top_k: Max results++ Returns:+ Matching memories+ """+ return self._store.search(self._user_id, query, top_k)++ def add(+ self,+ content: str,+ category: str = "fact",+ importance: float = 0.5,+ ) -> Memory:+ """Manually add a memory.++ Args:+ content: Memory content+ category: preference, fact, or context+ importance: 0.0-1.0++ Returns:+ The created memory+ """+ memory = Memory(+ content=content,+ category=category, # type: ignore+ importance=importance,+ )+ self._store.save(self._user_id, memory)+ return memory++ def get_all(self) -> list[Memory]:+ """Get all memories for this user."""+ return self._store.get_all(self._user_id)++ def delete(self, memory_id: str) -> bool:+ """Delete a specific memory."""+ return self._store.delete(self._user_id, memory_id)++ def clear(self) -> int:+ """Clear all memories for this user."""+ return self._store.clear(self._user_id)++ def stats(self) -> dict:+ """Get memory statistics."""+ return self._store.stats(self._user_id)+++def with_memory(+ client: Any,+ user_id: str,+ db_path: str | Path = "headroom_memory.db",+ extraction_model: str | None = None,+ top_k: int = 5,+ **kwargs: Any,+) -> MemoryWrapper:+ """Wrap an LLM client to add automatic memory.++ One-line integration for adding persistent memory to any LLM client.++ Args:+ client: LLM client (OpenAI, Anthropic, Mistral, Groq, etc.)+ user_id: User identifier for memory isolation+ db_path: Path to SQLite database (default: headroom_memory.db)+ extraction_model: Override extraction model (auto-detects by default)+ top_k: Number of memories to inject per request (default: 5)+ **kwargs: Additional arguments passed to MemoryWrapper++ Returns:+ Wrapped client with automatic memory++ Example:+ from openai import OpenAI+ from headroom import with_memory++ client = with_memory(OpenAI(), user_id="alice")++ response = client.chat.completions.create(+ model="gpt-4o",+ messages=[{"role": "user", "content": "I prefer Python"}]+ )+ # Memory automatically extracted in background++ # Later...+ response = client.chat.completions.create(+ model="gpt-4o",+ messages=[{"role": "user", "content": "What language should I use?"}]+ )+ # Memory about Python preference automatically injected!+ """+ return MemoryWrapper(+ client=client,+ user_id=user_id,+ db_path=db_path,+ extraction_model=extraction_model,+ top_k=top_k,+ **kwargs,+ )diff --git a/headroom/tokenizers/base.py b/headroom/tokenizers/base.pyindex d867a52a0..6e0a5745f 100644--- a/headroom/tokenizers/base.py+++ b/headroom/tokenizers/base.py@@ -60,6 +60,17 @@ class BaseTokenizer(ABC):"""Count tokens in a text string. Must be implemented by subclasses."""pass+ def count_message(self, message: dict[str, Any]) -> int:+ """Count tokens in a single message.++ Args:+ message: A message dict with 'role' and 'content'.++ Returns:+ Token count for this message.+ """+ return self.count_messages([message]) - self.REPLY_OVERHEAD+def count_messages(self, messages: list[dict[str, Any]]) -> int:"""Count tokens in a list of chat messages.diff --git a/pyproject.toml b/pyproject.tomlindex 7c4f6d1c7..14085fdc4 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -46,6 +46,8 @@ classifiers = [dependencies = ["tiktoken>=0.5.0","pydantic>=2.0.0",+ "openai>=2.14.0",+ "sentence-transformers>=5.2.0",][project.optional-dependencies]diff --git a/tests/test_memory/__init__.py b/tests/test_memory/__init__.pynew file mode 100644index 000000000..ca0a62165--- /dev/null+++ b/tests/test_memory/__init__.py@@ -0,0 +1 @@+"""Tests for Headroom Memory."""diff --git a/tests/test_memory/conftest.py b/tests/test_memory/conftest.pynew file mode 100644index 000000000..5e7705539--- /dev/null+++ b/tests/test_memory/conftest.py@@ -0,0 +1,147 @@+"""Test fixtures for Headroom Memory.++Philosophy: Mock at boundaries, not internals.+- SQLite: REAL (local, fast, no side effects)+- LLM clients: MOCKED (external dependency)+"""++from __future__ import annotations++import tempfile+from pathlib import Path+from typing import Any++import pytest+++def temp_db():+ """Fresh SQLite DB for each test - REAL database, auto-cleanup."""+ with tempfile.TemporaryDirectory() as tmpdir:+ db_path = Path(tmpdir) / "test_memory.db"+ yield db_path+++def memory_store(temp_db):+ """Real SQLite memory store."""+ from headroom.memory.store import SQLiteMemoryStore++ return SQLiteMemoryStore(temp_db)+++def mock_extractor():+ """Extractor with controllable responses - for testing worker/wrapper."""+ from headroom.memory.store import Memory++ class MockExtractor:+ def __init__(self):+ self.calls: list[tuple[str, str]] = []+ self.batch_calls: list[list[tuple[str, str, str]]] = []+ self._response: list[Memory] = []+ self._batch_response: dict[str, list[Memory]] = {}++ def set_response(self, memories: list[Memory]) -> None:+ self._response = memories++ def set_batch_response(self, response: dict[str, list[Memory]]) -> None:+ self._batch_response = response++ def extract(self, query: str, response: str) -> list[Memory]:+ self.calls.append((query, response))+ return self._response++ def extract_batch(+ self, conversations: list[tuple[str, str, str]]+ ) -> dict[str, list[Memory]]:+ self.batch_calls.append(conversations)+ return self._batch_response++ return MockExtractor()+++def mock_openai_client():+ """Fake OpenAI client - for testing wrapper without API calls."""++ class MockMessage:+ def __init__(self, content: str):+ self.content = content++ class MockChoice:+ def __init__(self, content: str):+ self.message = MockMessage(content)++ class MockResponse:+ def __init__(self, content: str = "Hello!"):+ self.choices = [MockChoice(content)]++ class MockCompletions:+ def __init__(self):+ self.calls: list[dict[str, Any]] = []+ self._response = MockResponse()++ def set_response(self, content: str) -> None:+ self._response = MockResponse(content)++ def create(self, **kwargs: Any) -> MockResponse:+ self.calls.append(kwargs)+ return self._response++ class MockChat:+ def __init__(self):+ self.completions = MockCompletions()++ class MockClient:+ """Mock OpenAI client."""++ def __init__(self):+ self.chat = MockChat()++ return MockClient()+++def mock_anthropic_client():+ """Fake Anthropic client - for testing wrapper without API calls."""++ class MockTextBlock:+ def __init__(self, text: str):+ self.text = text++ class MockResponse:+ def __init__(self, content: str = "Hello!"):+ self.content = [MockTextBlock(content)]++ class MockMessages:+ def __init__(self):+ self.calls: list[dict[str, Any]] = []+ self._response = MockResponse()++ def set_response(self, content: str) -> None:+ self._response = MockResponse(content)++ def create(self, **kwargs: Any) -> MockResponse:+ self.calls.append(kwargs)+ return self._response++ class MockClient:+ """Mock Anthropic client."""++ def __init__(self):+ self.messages = MockMessages()++ return MockClient()+++def sample_memories():+ """Sample memories for testing."""+ from headroom.memory.store import Memory++ return [+ Memory(content="User prefers Python", category="preference", importance=0.8),+ Memory(content="User works at a startup", category="fact", importance=0.7),+ Memory(content="User is building an AI agent", category="context", importance=0.6),+ ]diff --git a/tests/test_memory/test_extractor.py b/tests/test_memory/test_extractor.pynew file mode 100644index 000000000..005f66409--- /dev/null+++ b/tests/test_memory/test_extractor.py@@ -0,0 +1,337 @@+"""Tests for memory extractor.++Mocks LLM HTTP responses to test extraction logic without external calls.+"""++from __future__ import annotations++from unittest.mock import MagicMock++from headroom.memory.extractor import (+ CHEAP_MODELS,+ MemoryExtractor,+ detect_provider,+ get_cheap_model,+)+++class TestProviderDetection:+ """Test provider detection from client class."""++ def test_detect_openai(self):+ """Detect OpenAI from module path."""+ mock_client = MagicMock()+ mock_client.__class__.__module__ = "openai.resources.chat"++ result = detect_provider(mock_client)++ assert result == "openai"++ def test_detect_anthropic(self):+ """Detect Anthropic from module path."""+ mock_client = MagicMock()+ mock_client.__class__.__module__ = "anthropic.resources"++ result = detect_provider(mock_client)++ assert result == "anthropic"++ def test_detect_groq(self):+ """Detect Groq from module path."""+ mock_client = MagicMock()+ mock_client.__class__.__module__ = "groq.resources"++ result = detect_provider(mock_client)++ assert result == "groq"++ def test_detect_together(self):+ """Detect Together from module path."""+ mock_client = MagicMock()+ mock_client.__class__.__module__ = "together.client"++ result = detect_provider(mock_client)++ assert result == "together"++ def test_detect_fireworks(self):+ """Detect Fireworks from module path."""+ mock_client = MagicMock()+ mock_client.__class__.__module__ = "fireworks.client"++ result = detect_provider(mock_client)++ assert result == "fireworks"++ def test_detect_mistralai(self):+ """Detect Mistral from module path."""+ mock_client = MagicMock()+ mock_client.__class__.__module__ = "mistralai.client"++ result = detect_provider(mock_client)++ assert result == "mistralai"++ def test_detect_cohere(self):+ """Detect Cohere from module path."""+ mock_client = MagicMock()+ mock_client.__class__.__module__ = "cohere.client"++ result = detect_provider(mock_client)++ assert result == "cohere"++ def test_detect_google(self):+ """Detect Google from module path."""+ mock_client = MagicMock()+ mock_client.__class__.__module__ = "google.generativeai"++ result = detect_provider(mock_client)++ assert result == "google"++ def test_detect_unknown_returns_none(self):+ """Unknown provider returns None."""+ mock_client = MagicMock()+ mock_client.__class__.__module__ = "some.unknown.provider"++ result = detect_provider(mock_client)++ assert result is None+++class TestCheapModelMapping:+ """Test cheap model selection."""++ def test_all_providers_have_models(self):+ """All expected providers have cheap models defined."""+ expected_providers = [+ "openai",+ "anthropic",+ "mistralai",+ "groq",+ "together",+ "fireworks",+ "google",+ "cohere",+ ]++ for provider in expected_providers:+ assert provider in CHEAP_MODELS, f"Missing model for {provider}"+ assert CHEAP_MODELS[provider], f"Empty model for {provider}"++ def test_get_cheap_model_returns_correct_model(self):+ """get_cheap_model returns correct model for provider."""+ assert get_cheap_model("openai") == "gpt-4o-mini"+ assert get_cheap_model("anthropic") == "claude-3-5-haiku-latest"+ assert get_cheap_model("groq") == "llama-3.3-70b-versatile"++ def test_get_cheap_model_unknown_returns_none(self):+ """Unknown provider returns None."""+ assert get_cheap_model("unknown") is None+++class TestMemoryExtractorInit:+ """Test extractor initialization."""++ def test_auto_detects_provider_and_model(self, mock_openai_client):+ """Extractor auto-detects provider and selects cheap model."""+ # Mock the module path+ mock_openai_client.__class__.__module__ = "openai.resources"++ extractor = MemoryExtractor(mock_openai_client)++ assert extractor.provider == "openai"+ assert extractor.model == "gpt-4o-mini"++ def test_explicit_model_overrides_auto(self, mock_openai_client):+ """Explicit model parameter overrides auto-detection."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ extractor = MemoryExtractor(mock_openai_client, model="gpt-4-turbo")++ assert extractor.model == "gpt-4-turbo"++ def test_unknown_provider_warns(self, mock_openai_client, caplog):+ """Unknown provider logs warning."""+ mock_openai_client.__class__.__module__ = "unknown.provider"++ with caplog.at_level("WARNING"):+ extractor = MemoryExtractor(mock_openai_client)++ assert extractor.model is None+ assert "Could not detect cheap model" in caplog.text+++class TestMemoryExtraction:+ """Test memory extraction from conversations."""++ def test_extracts_preference(self, mock_openai_client):+ """Extracts preference from conversation."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ # Mock the LLM response+ mock_openai_client.chat.completions.set_response(+ '{"memories": [{"content": "Prefers Python", "category": "preference", "importance": 0.8}], "should_remember": true}'+ )++ extractor = MemoryExtractor(mock_openai_client)+ memories = extractor.extract(+ "I really prefer Python for data science",+ "Great choice! Python is excellent for data science.",+ )++ assert len(memories) == 1+ assert memories[0].content == "Prefers Python"+ assert memories[0].category == "preference"+ assert memories[0].importance == 0.8++ def test_extracts_multiple_memories(self, mock_openai_client):+ """Extracts multiple memories from one conversation."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ mock_openai_client.chat.completions.set_response(+ '{"memories": ['+ '{"content": "Works at a startup", "category": "fact", "importance": 0.7},'+ '{"content": "Building an AI agent", "category": "context", "importance": 0.6}'+ '], "should_remember": true}'+ )++ extractor = MemoryExtractor(mock_openai_client)+ memories = extractor.extract(+ "I work at a startup building an AI agent",+ "That sounds exciting!",+ )++ assert len(memories) == 2+ assert memories[0].content == "Works at a startup"+ assert memories[1].content == "Building an AI agent"++ def test_skips_trivial_conversation(self, mock_openai_client):+ """Returns empty for trivial conversations."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ mock_openai_client.chat.completions.set_response(+ '{"memories": [], "should_remember": false}'+ )++ extractor = MemoryExtractor(mock_openai_client)+ memories = extractor.extract("Hello", "Hi there!")++ assert len(memories) == 0++ def test_handles_json_in_code_block(self, mock_openai_client):+ """Parses JSON wrapped in markdown code block."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ mock_openai_client.chat.completions.set_response(+ '```json\n{"memories": [{"content": "Likes vim", "category": "preference"}], "should_remember": true}\n```'+ )++ extractor = MemoryExtractor(mock_openai_client)+ memories = extractor.extract("I use vim", "Nice!")++ assert len(memories) == 1+ assert memories[0].content == "Likes vim"++ def test_handles_malformed_json(self, mock_openai_client, caplog):+ """Gracefully handles malformed JSON."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ mock_openai_client.chat.completions.set_response("not valid json")++ with caplog.at_level("WARNING"):+ extractor = MemoryExtractor(mock_openai_client)+ memories = extractor.extract("test", "test")++ assert len(memories) == 0+ assert "Failed to parse" in caplog.text++ def test_no_extraction_without_model(self, mock_openai_client, caplog):+ """Skips extraction if no model configured."""+ mock_openai_client.__class__.__module__ = "unknown.provider"++ with caplog.at_level("WARNING"):+ extractor = MemoryExtractor(mock_openai_client)+ memories = extractor.extract("test", "test")++ assert len(memories) == 0+ assert "No extraction model" in caplog.text+++class TestBatchExtraction:+ """Test batch extraction."""++ def test_batch_extracts_for_multiple_users(self, mock_openai_client):+ """Batch extraction returns memories per user."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ mock_openai_client.chat.completions.set_response(+ '{"alice": {"memories": [{"content": "Likes Python", "category": "preference"}], "should_remember": true},'+ '"bob": {"memories": [{"content": "Likes Java", "category": "preference"}], "should_remember": true}}'+ )++ extractor = MemoryExtractor(mock_openai_client)+ result = extractor.extract_batch(+ [+ ("alice", "I like Python", "Great!"),+ ("bob", "I like Java", "Nice!"),+ ]+ )++ assert "alice" in result+ assert "bob" in result+ assert result["alice"][0].content == "Likes Python"+ assert result["bob"][0].content == "Likes Java"++ def test_batch_empty_input_returns_empty(self, mock_openai_client):+ """Empty batch returns empty dict."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ extractor = MemoryExtractor(mock_openai_client)+ result = extractor.extract_batch([])++ assert result == {}++ def test_batch_handles_partial_results(self, mock_openai_client):+ """Batch handles some users with no memories."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ mock_openai_client.chat.completions.set_response(+ '{"alice": {"memories": [{"content": "Fact", "category": "fact"}], "should_remember": true},'+ '"bob": {"memories": [], "should_remember": false}}'+ )++ extractor = MemoryExtractor(mock_openai_client)+ result = extractor.extract_batch(+ [+ ("alice", "Important info", "Noted!"),+ ("bob", "Hello", "Hi!"),+ ]+ )++ assert "alice" in result+ assert "bob" not in result # No memories to remember+++class TestAnthropicProvider:+ """Test Anthropic-specific API handling."""++ def test_anthropic_uses_messages_api(self, mock_anthropic_client):+ """Anthropic uses messages.create API."""+ mock_anthropic_client.__class__.__module__ = "anthropic.resources"++ mock_anthropic_client.messages.set_response(+ '{"memories": [{"content": "Test", "category": "fact"}], "should_remember": true}'+ )++ extractor = MemoryExtractor(mock_anthropic_client)+ memories = extractor.extract("test query", "test response")++ assert len(memories) == 1+ assert memories[0].content == "Test"++ # Verify Anthropic API was called+ assert len(mock_anthropic_client.messages.calls) == 1+ call = mock_anthropic_client.messages.calls[0]+ assert call["model"] == "claude-3-5-haiku-latest"diff --git a/tests/test_memory/test_inline_extractor.py b/tests/test_memory/test_inline_extractor.pynew file mode 100644index 000000000..937e8b764--- /dev/null+++ b/tests/test_memory/test_inline_extractor.py@@ -0,0 +1,196 @@+"""Tests for inline memory extraction (Letta-style)."""++from __future__ import annotations++from headroom.memory.inline_extractor import (+ MEMORY_INSTRUCTION,+ MEMORY_INSTRUCTION_SHORT,+ ParsedResponse,+ inject_memory_instruction,+ parse_response_with_memory,+)+++class TestParseResponseWithMemory:+ """Test response parsing to extract memories."""++ def test_extracts_single_memory(self):+ """Parse response with one memory."""+ response = """Great choice! Python is excellent for backend development.++<memory>{"memories": [{"content": "User prefers Python", "category": "preference"}]}</memory>"""++ parsed = parse_response_with_memory(response)++ assert (+ parsed.content.strip() == "Great choice! Python is excellent for backend development."+ )+ assert len(parsed.memories) == 1+ assert parsed.memories[0]["content"] == "User prefers Python"+ assert parsed.memories[0]["category"] == "preference"+ assert parsed.raw == response++ def test_extracts_multiple_memories(self):+ """Parse response with multiple memories."""+ response = """That's interesting background!++<memory>{"memories": [+ {"content": "Works at fintech startup", "category": "fact"},+ {"content": "Uses PostgreSQL", "category": "preference"}+]}</memory>"""++ parsed = parse_response_with_memory(response)++ assert len(parsed.memories) == 2+ assert parsed.memories[0]["content"] == "Works at fintech startup"+ assert parsed.memories[1]["content"] == "Uses PostgreSQL"++ def test_handles_empty_memories(self):+ """Parse response with no memories."""+ response = """Hello! How can I help?++<memory>{"memories": []}</memory>"""++ parsed = parse_response_with_memory(response)++ assert "Hello! How can I help?" in parsed.content+ assert len(parsed.memories) == 0++ def test_handles_no_memory_block(self):+ """Parse response without memory block."""+ response = "Just a normal response without memory."++ parsed = parse_response_with_memory(response)++ assert parsed.content == response+ assert len(parsed.memories) == 0++ def test_handles_malformed_json(self):+ """Parse response with invalid JSON in memory block."""+ response = """Some response.++<memory>this is not valid json</memory>"""++ parsed = parse_response_with_memory(response)++ assert "Some response" in parsed.content+ assert len(parsed.memories) == 0 # Gracefully handles error++ def test_case_insensitive_tags(self):+ """Memory tags should be case-insensitive."""+ response = """Response here.++<MEMORY>{"memories": [{"content": "Test", "category": "fact"}]}</MEMORY>"""++ parsed = parse_response_with_memory(response)++ assert len(parsed.memories) == 1++ def test_memory_block_in_middle(self):+ """Memory block can appear anywhere in response."""+ response = """First part.++<memory>{"memories": [{"content": "Test", "category": "fact"}]}</memory>++More content after."""++ parsed = parse_response_with_memory(response)++ assert len(parsed.memories) == 1+ assert "First part" in parsed.content+ assert "More content after" in parsed.content+++class TestInjectMemoryInstruction:+ """Test injection of memory instruction into messages."""++ def test_appends_to_existing_system_prompt(self):+ """Instruction appended to existing system message."""+ messages = [+ {"role": "system", "content": "You are helpful."},+ {"role": "user", "content": "Hello"},+ ]++ result = inject_memory_instruction(messages, short=True)++ assert len(result) == 2+ assert result[0]["role"] == "system"+ assert "You are helpful." in result[0]["content"]+ assert "memory" in result[0]["content"].lower()++ def test_creates_system_prompt_if_missing(self):+ """Creates system message if none exists."""+ messages = [+ {"role": "user", "content": "Hello"},+ ]++ result = inject_memory_instruction(messages, short=True)++ assert len(result) == 2+ assert result[0]["role"] == "system"+ assert "memory" in result[0]["content"].lower()++ def test_does_not_modify_original(self):+ """Original messages list is not modified."""+ messages = [+ {"role": "system", "content": "Original prompt."},+ {"role": "user", "content": "Hello"},+ ]+ original_content = messages[0]["content"]++ inject_memory_instruction(messages, short=True)++ assert messages[0]["content"] == original_content++ def test_short_vs_long_instruction(self):+ """Short instruction is shorter than full instruction."""+ messages = [{"role": "user", "content": "Hello"}]++ short = inject_memory_instruction(messages, short=True)+ long = inject_memory_instruction(messages, short=False)++ assert len(short[0]["content"]) < len(long[0]["content"])++ def test_instruction_contains_required_format(self):+ """Instruction explains the memory format."""+ messages = [{"role": "user", "content": "Hello"}]++ result = inject_memory_instruction(messages, short=False)+ content = result[0]["content"]++ assert "<memory>" in content+ assert "memories" in content+ assert "category" in content+++class TestParsedResponse:+ """Test ParsedResponse dataclass."""++ def test_dataclass_fields(self):+ """ParsedResponse has expected fields."""+ parsed = ParsedResponse(+ content="Hello",+ memories=[{"content": "Test", "category": "fact"}],+ raw="Hello\n<memory>...</memory>",+ )++ assert parsed.content == "Hello"+ assert len(parsed.memories) == 1+ assert parsed.raw == "Hello\n<memory>...</memory>"+++class TestMemoryInstructions:+ """Test memory instruction prompts."""++ def test_short_instruction_contains_essentials(self):+ """Short instruction has minimum required info."""+ assert "<memory>" in MEMORY_INSTRUCTION_SHORT+ assert "memories" in MEMORY_INSTRUCTION_SHORT+ assert "category" in MEMORY_INSTRUCTION_SHORT++ def test_full_instruction_more_detailed(self):+ """Full instruction has categories explained."""+ assert "preference" in MEMORY_INSTRUCTION+ assert "fact" in MEMORY_INSTRUCTION+ assert "context" in MEMORY_INSTRUCTION+ assert "Greetings" in MEMORY_INSTRUCTION or "greeting" in MEMORY_INSTRUCTION.lower()diff --git a/tests/test_memory/test_store.py b/tests/test_memory/test_store.pynew file mode 100644index 000000000..d52e9c6ac--- /dev/null+++ b/tests/test_memory/test_store.py@@ -0,0 +1,278 @@+"""Tests for SQLite memory store.++100% REAL SQLite - no mocks! These tests use actual SQLite+databases in temp directories for realistic testing.+"""++from __future__ import annotations++from headroom.memory.store import Memory, PendingExtraction, SQLiteMemoryStore+++class TestMemorySaveAndSearch:+ """Test basic save and search operations."""++ def test_save_and_search_finds_match(self, temp_db):+ """Real SQLite, real FTS5, real queries."""+ store = SQLiteMemoryStore(temp_db)++ store.save("alice", Memory(content="User prefers Python", category="preference"))++ results = store.search("alice", "python", top_k=5)++ assert len(results) == 1+ assert "Python" in results[0].content++ def test_search_no_results_for_unrelated_query(self, temp_db):+ """Search returns empty when no matches."""+ store = SQLiteMemoryStore(temp_db)++ store.save("alice", Memory(content="User prefers Python"))++ results = store.search("alice", "javascript", top_k=5)++ assert len(results) == 0++ def test_search_respects_top_k_limit(self, temp_db):+ """Search returns at most top_k results."""+ store = SQLiteMemoryStore(temp_db)++ for i in range(10):+ store.save("alice", Memory(content=f"Python fact number {i}"))++ results = store.search("alice", "python", top_k=3)++ assert len(results) == 3++ def test_save_preserves_all_fields(self, temp_db):+ """All memory fields are preserved through save/search."""+ store = SQLiteMemoryStore(temp_db)++ original = Memory(+ content="User prefers vim",+ category="preference",+ importance=0.9,+ metadata={"source": "chat"},+ )+ store.save("alice", original)++ results = store.search("alice", "vim", top_k=1)++ assert len(results) == 1+ retrieved = results[0]+ assert retrieved.content == original.content+ assert retrieved.category == original.category+ assert retrieved.importance == original.importance+ assert retrieved.metadata == original.metadata+++class TestUserIsolation:+ """Test that memories are isolated by user_id."""++ def test_users_have_separate_memories(self, temp_db):+ """Memories are isolated by user_id."""+ store = SQLiteMemoryStore(temp_db)++ store.save("alice", Memory(content="Likes Python programming"))+ store.save("bob", Memory(content="Likes JavaScript programming"))++ # Search for content that's actually in the memories+ alice_results = store.search("alice", "Python", top_k=10)+ bob_results = store.search("bob", "JavaScript", top_k=10)++ # Each user should only see their own memories+ assert len(alice_results) == 1+ assert "Python" in alice_results[0].content++ assert len(bob_results) == 1+ assert "JavaScript" in bob_results[0].content++ def test_get_all_returns_only_user_memories(self, temp_db):+ """get_all only returns memories for the specified user."""+ store = SQLiteMemoryStore(temp_db)++ store.save("alice", Memory(content="Alice memory 1"))+ store.save("alice", Memory(content="Alice memory 2"))+ store.save("bob", Memory(content="Bob memory"))++ alice_memories = store.get_all("alice")+ bob_memories = store.get_all("bob")++ assert len(alice_memories) == 2+ assert len(bob_memories) == 1+++class TestMemoryDeletion:+ """Test deletion operations."""++ def test_delete_specific_memory(self, temp_db):+ """Delete removes a specific memory."""+ store = SQLiteMemoryStore(temp_db)++ mem = Memory(content="To be deleted")+ store.save("alice", mem)++ result = store.delete("alice", mem.id)++ assert result is True+ assert len(store.get_all("alice")) == 0++ def test_delete_nonexistent_returns_false(self, temp_db):+ """Delete returns False for nonexistent memory."""+ store = SQLiteMemoryStore(temp_db)++ result = store.delete("alice", "nonexistent-id")++ assert result is False++ def test_clear_removes_all_user_memories(self, temp_db):+ """Clear removes all memories for a user."""+ store = SQLiteMemoryStore(temp_db)++ for i in range(5):+ store.save("alice", Memory(content=f"Memory {i}"))+ store.save("bob", Memory(content="Bob's memory"))++ count = store.clear("alice")++ assert count == 5+ assert len(store.get_all("alice")) == 0+ assert len(store.get_all("bob")) == 1 # Bob's memory untouched+++class TestMemoryStats:+ """Test statistics operations."""++ def test_stats_counts_memories(self, temp_db):+ """Stats returns correct count."""+ store = SQLiteMemoryStore(temp_db)++ store.save("alice", Memory(content="Mem 1", category="preference"))+ store.save("alice", Memory(content="Mem 2", category="fact"))+ store.save("alice", Memory(content="Mem 3", category="fact"))++ stats = store.stats("alice")++ assert stats["total"] == 3+ assert stats["categories"]["preference"] == 1+ assert stats["categories"]["fact"] == 2++ def test_stats_empty_user(self, temp_db):+ """Stats for user with no memories."""+ store = SQLiteMemoryStore(temp_db)++ stats = store.stats("alice")++ assert stats["total"] == 0+ assert stats["categories"] == {}+++class TestPendingExtractions:+ """Test pending extraction queue for crash recovery."""++ def test_queue_and_retrieve_pending(self, temp_db):+ """Queue and retrieve pending extractions."""+ store = SQLiteMemoryStore(temp_db)++ pending = PendingExtraction(+ user_id="alice",+ query="What's your favorite language?",+ response="I prefer Python for data science.",+ )+ store.queue_extraction(pending)++ retrieved = store.get_pending_extractions(limit=10)++ assert len(retrieved) == 1+ assert retrieved[0].user_id == "alice"+ assert retrieved[0].query == pending.query+ assert retrieved[0].response == pending.response+ assert retrieved[0].status == "pending"++ def test_update_extraction_status(self, temp_db):+ """Update status of pending extraction."""+ store = SQLiteMemoryStore(temp_db)++ pending = PendingExtraction(user_id="alice", query="Q", response="R")+ store.queue_extraction(pending)++ store.update_extraction_status(pending.id, "processing")++ # Should not appear in pending list anymore+ pending_list = store.get_pending_extractions(status="pending")+ processing_list = store.get_pending_extractions(status="processing")++ assert len(pending_list) == 0+ assert len(processing_list) == 1++ def test_delete_extraction(self, temp_db):+ """Delete completed extraction."""+ store = SQLiteMemoryStore(temp_db)++ pending = PendingExtraction(user_id="alice", query="Q", response="R")+ store.queue_extraction(pending)++ store.delete_extraction(pending.id)++ assert len(store.get_pending_extractions(limit=10)) == 0++ def test_pending_fifo_order(self, temp_db):+ """Pending extractions returned in FIFO order."""+ store = SQLiteMemoryStore(temp_db)++ for i in range(5):+ store.queue_extraction(+ PendingExtraction(user_id="alice", query=f"Q{i}", response=f"R{i}")+ )++ retrieved = store.get_pending_extractions(limit=3)++ assert len(retrieved) == 3+ assert retrieved[0].query == "Q0"+ assert retrieved[1].query == "Q1"+ assert retrieved[2].query == "Q2"+++class TestFTS5Features:+ """Test FTS5 full-text search features."""++ def test_phrase_search(self, temp_db):+ """FTS5 supports phrase search with _raw: prefix."""+ store = SQLiteMemoryStore(temp_db)++ store.save("alice", Memory(content="User prefers dark mode"))+ store.save("alice", Memory(content="User is in dark times"))++ # Exact phrase match using raw FTS5 syntax+ results = store.search("alice", '_raw:"dark mode"', top_k=5)++ assert len(results) == 1+ assert "dark mode" in results[0].content++ def test_prefix_search(self, temp_db):+ """FTS5 supports prefix search with _raw: prefix."""+ store = SQLiteMemoryStore(temp_db)++ store.save("alice", Memory(content="User loves Python programming"))+ store.save("alice", Memory(content="User loves JavaScript"))++ # Prefix search with * using raw FTS5 syntax+ results = store.search("alice", "_raw:Pyth*", top_k=5)++ assert len(results) == 1+ assert "Python" in results[0].content++ def test_boolean_and(self, temp_db):+ """FTS5 supports boolean AND with _raw: prefix."""+ store = SQLiteMemoryStore(temp_db)++ store.save("alice", Memory(content="User prefers Python"))+ store.save("alice", Memory(content="User prefers dark mode"))+ store.save("alice", Memory(content="User prefers Python and dark mode"))++ # Boolean AND using raw FTS5 syntax+ results = store.search("alice", "_raw:Python AND dark", top_k=5)++ assert len(results) == 1+ assert "Python" in results[0].content+ assert "dark" in results[0].contentdiff --git a/tests/test_memory/test_wrapper.py b/tests/test_memory/test_wrapper.pynew file mode 100644index 000000000..ac1faaf9e--- /dev/null+++ b/tests/test_memory/test_wrapper.py@@ -0,0 +1,375 @@+"""Tests for memory wrapper integration.++Tests the full with_memory() flow with mocked LLM clients.+"""++from __future__ import annotations++import time++from headroom.memory.store import Memory+from headroom.memory.wrapper import with_memory+++class TestWithMemoryBasic:+ """Test basic with_memory() functionality."""++ def test_one_line_integration(self, temp_db, mock_openai_client, mock_extractor):+ """One-line integration works."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ client = with_memory(+ mock_openai_client,+ user_id="alice",+ db_path=temp_db,+ _extractor=mock_extractor,+ )++ response = client.chat.completions.create(+ model="gpt-4o",+ messages=[{"role": "user", "content": "Hello"}],+ )++ assert response.choices[0].message.content == "Hello!"++ def test_forwards_all_kwargs(self, temp_db, mock_openai_client, mock_extractor):+ """All kwargs are forwarded to underlying client."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ client = with_memory(+ mock_openai_client,+ user_id="alice",+ db_path=temp_db,+ _extractor=mock_extractor,+ )++ client.chat.completions.create(+ model="gpt-4o",+ messages=[{"role": "user", "content": "test"}],+ temperature=0.5,+ max_tokens=100,+ )++ call = mock_openai_client.chat.completions.calls[0]+ assert call["model"] == "gpt-4o"+ assert call["temperature"] == 0.5+ assert call["max_tokens"] == 100+++class TestMemoryInjection:+ """Test memory injection into messages."""++ def test_injects_memory_into_user_message(self, temp_db, mock_openai_client, mock_extractor):+ """Memory is injected into user message, not system prompt."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ # Pre-populate memory with content that will match the query+ from headroom.memory.store import SQLiteMemoryStore++ store = SQLiteMemoryStore(temp_db)+ store.save("alice", Memory(content="User prefers Python for coding", category="preference"))++ client = with_memory(+ mock_openai_client,+ user_id="alice",+ db_path=temp_db,+ _extractor=mock_extractor,+ _store=store,+ )++ # Use a query that will match the memory content+ client.chat.completions.create(+ model="gpt-4o",+ messages=[+ {"role": "system", "content": "You are helpful."},+ {"role": "user", "content": "What Python framework?"},+ ],+ )++ # Check what was sent to the "API"+ call = mock_openai_client.chat.completions.calls[0]+ messages = call["messages"]++ # System prompt should be UNCHANGED (for caching)+ assert messages[0]["content"] == "You are helpful."++ # User message should have context prepended+ assert "<context>" in messages[1]["content"]+ assert "Python" in messages[1]["content"]+ assert "What Python framework?" in messages[1]["content"]++ def test_preserves_system_prompt_exactly(self, temp_db, mock_openai_client, mock_extractor):+ """System prompt is preserved exactly for caching."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ from headroom.memory.store import SQLiteMemoryStore++ store = SQLiteMemoryStore(temp_db)+ store.save("alice", Memory(content="Some memory"))++ client = with_memory(+ mock_openai_client,+ user_id="alice",+ db_path=temp_db,+ _extractor=mock_extractor,+ _store=store,+ )++ original_system = "You are a helpful assistant. Always be concise."++ client.chat.completions.create(+ model="gpt-4o",+ messages=[+ {"role": "system", "content": original_system},+ {"role": "user", "content": "test"},+ ],+ )++ call = mock_openai_client.chat.completions.calls[0]+ assert call["messages"][0]["content"] == original_system++ def test_no_injection_when_no_memories(self, temp_db, mock_openai_client, mock_extractor):+ """No injection when user has no memories."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ client = with_memory(+ mock_openai_client,+ user_id="alice",+ db_path=temp_db,+ _extractor=mock_extractor,+ )++ client.chat.completions.create(+ model="gpt-4o",+ messages=[{"role": "user", "content": "Hello"}],+ )++ call = mock_openai_client.chat.completions.calls[0]+ # Message should be unchanged+ assert call["messages"][0]["content"] == "Hello"+ assert "<context>" not in call["messages"][0]["content"]++ def test_respects_top_k(self, temp_db, mock_openai_client, mock_extractor):+ """Only top_k memories are injected."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ from headroom.memory.store import SQLiteMemoryStore++ store = SQLiteMemoryStore(temp_db)+ for i in range(10):+ store.save("alice", Memory(content=f"Python fact {i}"))++ client = with_memory(+ mock_openai_client,+ user_id="alice",+ db_path=temp_db,+ _extractor=mock_extractor,+ _store=store,+ top_k=3,+ )++ client.chat.completions.create(+ model="gpt-4o",+ messages=[{"role": "user", "content": "Python question"}],+ )++ call = mock_openai_client.chat.completions.calls[0]+ content = call["messages"][0]["content"]++ # Should have exactly 3 memories (top_k=3)+ assert content.count("Python fact") == 3+++class TestBackgroundExtraction:+ """Test background memory extraction."""++ def test_queues_for_extraction(self, temp_db, mock_openai_client, mock_extractor):+ """Conversation is queued for extraction after response."""+ mock_openai_client.__class__.__module__ = "openai.resources"+ mock_openai_client.chat.completions.set_response("I'll remember that!")++ client = with_memory(+ mock_openai_client,+ user_id="alice",+ db_path=temp_db,+ _extractor=mock_extractor,+ )++ client.chat.completions.create(+ model="gpt-4o",+ messages=[{"role": "user", "content": "I prefer Python"}],+ )++ # Wait for background worker+ time.sleep(0.1)++ # Check extraction was scheduled+ # The mock extractor records batch calls+ assert len(mock_extractor.batch_calls) >= 0 # May not have processed yet++ def test_extracts_from_original_message(self, temp_db, mock_openai_client, mock_extractor):+ """Extraction uses original message (without injected context)."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ from headroom.memory.store import SQLiteMemoryStore++ store = SQLiteMemoryStore(temp_db)+ store.save("alice", Memory(content="Existing memory"))++ mock_extractor.set_batch_response({"alice": [Memory(content="New fact", category="fact")]})++ client = with_memory(+ mock_openai_client,+ user_id="alice",+ db_path=temp_db,+ _extractor=mock_extractor,+ _store=store,+ )++ client.chat.completions.create(+ model="gpt-4o",+ messages=[{"role": "user", "content": "I like vim"}],+ )++ # Wait for background worker to process+ time.sleep(1.5)++ # The extractor should have been called with original message+ # (not the one with <context> injected)+ if mock_extractor.batch_calls:+ batch = mock_extractor.batch_calls[0]+ _, query, _ = batch[0]+ assert "<context>" not in query+++class TestMemoryAPI:+ """Test direct memory API access."""++ def test_memory_search(self, temp_db, mock_openai_client, mock_extractor):+ """client.memory.search() works."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ from headroom.memory.store import SQLiteMemoryStore++ store = SQLiteMemoryStore(temp_db)+ store.save("alice", Memory(content="Likes Python"))++ client = with_memory(+ mock_openai_client,+ user_id="alice",+ db_path=temp_db,+ _extractor=mock_extractor,+ _store=store,+ )++ results = client.memory.search("Python")++ assert len(results) == 1+ assert "Python" in results[0].content++ def test_memory_add(self, temp_db, mock_openai_client, mock_extractor):+ """client.memory.add() works."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ client = with_memory(+ mock_openai_client,+ user_id="alice",+ db_path=temp_db,+ _extractor=mock_extractor,+ )++ memory = client.memory.add("User prefers dark mode", category="preference")++ assert memory.content == "User prefers dark mode"+ assert memory.category == "preference"++ # Verify it was saved+ all_memories = client.memory.get_all()+ assert len(all_memories) == 1++ def test_memory_clear(self, temp_db, mock_openai_client, mock_extractor):+ """client.memory.clear() works."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ from headroom.memory.store import SQLiteMemoryStore++ store = SQLiteMemoryStore(temp_db)+ store.save("alice", Memory(content="Memory 1"))+ store.save("alice", Memory(content="Memory 2"))++ client = with_memory(+ mock_openai_client,+ user_id="alice",+ db_path=temp_db,+ _extractor=mock_extractor,+ _store=store,+ )++ count = client.memory.clear()++ assert count == 2+ assert len(client.memory.get_all()) == 0++ def test_memory_stats(self, temp_db, mock_openai_client, mock_extractor):+ """client.memory.stats() works."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ from headroom.memory.store import SQLiteMemoryStore++ store = SQLiteMemoryStore(temp_db)+ store.save("alice", Memory(content="Pref", category="preference"))+ store.save("alice", Memory(content="Fact 1", category="fact"))+ store.save("alice", Memory(content="Fact 2", category="fact"))++ client = with_memory(+ mock_openai_client,+ user_id="alice",+ db_path=temp_db,+ _extractor=mock_extractor,+ _store=store,+ )++ stats = client.memory.stats()++ assert stats["total"] == 3+ assert stats["categories"]["preference"] == 1+ assert stats["categories"]["fact"] == 2+++class TestMultiUser:+ """Test multi-user isolation."""++ def test_users_have_separate_memories(self, temp_db, mock_openai_client, mock_extractor):+ """Different users have isolated memories."""+ mock_openai_client.__class__.__module__ = "openai.resources"++ from headroom.memory.store import SQLiteMemoryStore++ store = SQLiteMemoryStore(temp_db)++ # Create two wrapped clients for different users+ alice_client = with_memory(+ mock_openai_client,+ user_id="alice",+ db_path=temp_db,+ _extractor=mock_extractor,+ _store=store,+ )++ bob_client = with_memory(+ mock_openai_client,+ user_id="bob",+ db_path=temp_db,+ _extractor=mock_extractor,+ _store=store,+ )++ # Add memories for each user+ alice_client.memory.add("Alice's preference")+ bob_client.memory.add("Bob's preference")++ # Each should only see their own+ assert len(alice_client.memory.get_all()) == 1+ assert len(bob_client.memory.get_all()) == 1+ assert "Alice" in alice_client.memory.get_all()[0].content+ assert "Bob" in bob_client.memory.get_all()[0].contentdiff --git a/tests/test_transforms/test_rolling_window.py b/tests/test_transforms/test_rolling_window.pyindex 8627716ef..77096226a 100644--- a/tests/test_transforms/test_rolling_window.py+++ b/tests/test_transforms/test_rolling_window.py@@ -755,16 +755,8 @@ class TestShouldApply:class TestConvenienceFunction:- """Tests for the apply_rolling_window convenience function.+ """Tests for the apply_rolling_window convenience function."""- NOTE: The apply_rolling_window convenience function in rolling_window.py- has a bug where Tokenizer() is called without a token_counter argument.- These tests are skipped until that bug is fixed.- """-- @pytest.mark.skip(- reason="Bug in source: apply_rolling_window calls Tokenizer() without token_counter"- )def test_convenience_function(self, long_conversation):"""The convenience function should work correctly."""from headroom.transforms.rolling_window import apply_rolling_window@@ -780,9 +772,6 @@ class TestConvenienceFunction:assert len(messages) < len(long_conversation)assert len(transforms) > 0- @pytest.mark.skip(- reason="Bug in source: apply_rolling_window calls Tokenizer() without token_counter"- )def test_convenience_function_with_config(self, long_conversation):"""The convenience function should accept a config."""from headroom.transforms.rolling_window import apply_rolling_windowdiff --git a/uv.lock b/uv.lockindex cf48453e8..719fcb3d9 100644--- a/uv.lock+++ b/uv.lock@@ -385,7 +385,9 @@ name = "headroom-ai"version = "0.2.3"source = { editable = "." }dependencies = [+ { name = "openai" },{ name = "pydantic" },+ { name = "sentence-transformers" },{ name = "tiktoken" },]@@ -444,12 +446,14 @@ requires-dist = [{ name = "llmlingua", marker = "extra == 'llmlingua'", specifier = ">=0.2.0" },{ name = "mypy", marker = "extra == 'dev'", specifier = ">=1.0.0" },{ name = "numpy", marker = "extra == 'relevance'", specifier = ">=1.24.0" },+ { name = "openai", specifier = ">=2.14.0" },{ name = "openai", marker = "extra == 'dev'", specifier = ">=1.0.0" },{ name = "pydantic", specifier = ">=2.0.0" },{ name = "pytest", marker = "extra == 'dev'", specifier = ">=7.0.0" },{ name = "pytest-asyncio", marker = "extra == 'dev'", specifier = ">=0.21.0" },{ name = "pytest-cov", marker = "extra == 'dev'", specifier = ">=4.0.0" },{ name = "ruff", marker = "extra == 'dev'", specifier = ">=0.1.0" },+ { name = "sentence-transformers", specifier = ">=5.2.0" },{ name = "sentence-transformers", marker = "extra == 'relevance'", specifier = ">=2.2.0" },{ name = "tiktoken", specifier = ">=0.5.0" },{ name = "torch", marker = "extra == 'llmlingua'", specifier = ">=2.0.0" },