Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
Commits on main
README: add back proof points — needle-in-haystack demo, benchmarks, architecture detail
99caff5 chopratejas committed 9 months ago
README: proxy as the hero quickstart, compress() for Python, integration table for existing proxies
f699375 chopratejas committed 9 months ago
Add one-function compress() API, ASGI middleware, LiteLLM callback
dde2f9f chopratejas committed 9 months ago
Add Compression Hooks — extension points for SaaS and advanced customization
7c2c55a chopratejas committed 9 months ago
Add Query Echo: re-inject user question after compressed tool outputs
d4d8dd0 chopratejas committed 9 months ago
Add compression summaries, multi-provider headers, Dockerfile fix
729cc03 chopratejas committed 9 months ago
Add pluggable adapter hooks for CCR, Storage, and TOIN backends
94a6cd8 chopratejas committed 9 months ago
Add tests for streaming resilience and concurrent session safety
636abd2 chopratejas committed 9 months ago
Add adaptive compression sizing and fix cost tracking bugs
e8911ef chopratejas committed 9 months ago
Add LiteLLM backend routing for OpenAI endpoint and Magika content detection
c90cf39 chopratejas committed 9 months ago
Fix token savings metrics mismatch between pipeline and server
5e3aa38 chopratejas committed 9 months ago
Use AWS API to dynamically fetch Bedrock inference profiles
b579ea6 chopratejas committed 9 months ago
fix: Handle BaseModelOutputWithPooling from SigLIP get_*_features()
f768f44 Prakersh Maheshwari committed 9 months ago
Merge pull request #25 from prakersh/fix/increase-request-body-size-limit
beddc99 Tejas Chopra committed 9 months ago
test: Update LLMLingua default model test to match new default
658dc6c Prakersh Maheshwari committed 9 months ago
fix: Increase MAX_REQUEST_BODY_SIZE from 10MB to 100MB to support image-heavy requests
bbe113a Prakersh Maheshwari committed 9 months ago