Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
Commits on main
fix(tokenizers): price Claude against a real BPE (tiktoken o200k) not a char estimate (#2543)
285176b Tejas Chopra committed 3 months ago
fix(proxy/perf): tokenizer-consistent token accounting + surface tool-schema savings (#2542)
1cc53c9 Tejas Chopra committed 3 months ago
fix(proxy/cost): warn once per model when pricing lookup fails (#2504) (#2535)
fa47637 Abhay Singh committed 3 months ago
fix(proxy/perf): count turn-hook message folds in token accounting (#2520)
c371d5a Tejas Chopra committed 3 months ago
fix(copilot): derive GHE credential host from API URL (#800) (#2511)
4a8157f Rod Boev committed 4 months ago
fix(grok): preserve business-seat auth while routing only inference (#2514)
e4076bb Rod Boev committed 4 months ago
fix(proxy): offload OpenAI and Gemini tokenizer counting off the event loop (#2498)
806d2e4 inix committed 4 months ago
refactor(wrap): retire tokensave; Serena is the code-memory MCP (#2499)
5d23a0a Tejas Chopra committed 4 months ago
fix(kompress): raise the default execution-slot wait (#2456)
5bd2266 Rod Boev committed 4 months ago
fix(learn): treat unreadable candidate paths as absent in project decode (#2446)
a09ba6c Parideboy committed 4 months ago
fix(proxy/output-shaping): tolerate a non-string system block text in steering (#2435)
3e97671 Abhay Singh committed 4 months ago
fix(proxy/streaming): tolerate malformed content in _response_to_sse (#2481)
77b26c0 Abhay Singh committed 4 months ago
fix(doctor): don't crash on a valid-but-non-object settings.json (#2482)
7524854 Abhay Singh committed 4 months ago
fix(backends): don't crash the OpenAI->Anthropic converter on empty choices (#2484)
43a7b57 Abhay Singh committed 4 months ago
fix(proxy/gemini): tolerate malformed parts on the compression path (#2486)
07cf547 Abhay Singh committed 4 months ago
docs: add troubleshooting entry for uv build cache errors (#2490)
f0975b8 Parideboy committed 4 months ago
fix(proxy/openai): don't record Codex WS savings without input accounting (#2493)
2195ba7 gglucass committed 4 months ago
docs: sync Vercel docs with current code and add in-depth proxy config (#2475)
5a0a5a7 Tejas Chopra committed 4 months ago
deps: bump the npm-minor-patch group across 3 directories with 7 updates (#2276)
961866b dependabot[bot] committed 4 months ago
deps: bump the cargo-minor-patch group with 10 updates (#2284)
3266ed7 dependabot[bot] committed 4 months ago
feat(proxy): make /v1/compress usable as a gateway/Kong sidecar (#2458)
1329ed7 Tejas Chopra committed 4 months ago
fix(transforms/cross-turn-dedup): don't renumber-fold zero-padded line prefixes (#2369)
f4070c4 Ehsan committed 4 months ago
fix(kompress): match all ONNX backends with startswith, not exact "onnx" (#2448)
c811007 Parideboy committed 4 months ago
fix(copilot): normalize subscription API routing (#2441) (#2455)
2eca5ee Rod Boev committed 4 months ago
fix(proxy): reassemble server_tool_use.input from streamed partial_json (#2449)
8c8fae0 Parideboy committed 4 months ago
feat(telemetry): record provider cache read/write/uncached tokens per request (#2450)
bec4cce Parideboy committed 4 months ago
feat(opencode): support Copilot subscription backend for headroom models (#2441) (#2445)
9089e7f Rod Boev committed 4 months ago
fix(proxy/metrics): move the savings-ledger append off the event loop (#2439)
4aac068 inix committed 4 months ago