Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Python agentaianthropicclaude-codecompressioncontext-engineeringcontext-windowcursorfastapilangchainllmmcpopenaiprompt-engineeringproxypythonragtoken-optimizationtokenstypescript
Commits on main
feat(beacon): hourly R2 compaction, per-strategy savings, and a stack that reports (#2853)
e0870ef Tejas Chopra committed 3 months ago
fix(proxy/openai): run response hooks on Responses, and bill their re-drives (#2872)
675d13f Tejas Chopra committed 3 months ago
perf(subscription): skip transcripts older than the window in compute_window_tokens (#2861)
91d6bf3 Romulo Reis committed 3 months ago
fix(anthropic): strip first-party tool search on custom upstreams (#2539)
7f6950b Rod Boev committed 3 months ago
fix(onnx): stop ONNX thread pools from spinning idle cores (#2495) (#2540)
5c561bd Abhay Singh committed 3 months ago
fix(openai): skip Responses tool-search deferral for clients that cannot execute it (#2696)
54ea28d Rod Boev committed 3 months ago
fix(wrap): stop the launch cwd from shadowing the installed package in the proxy subprocess (#2843)
c49be26 Abhay Singh committed 3 months ago
fix(install): use --userns=keep-id under Podman so bind-mount writes don't fail (#2846)
3488f8d Abhay Singh committed 3 months ago
fix(install): stop baking the host memory DB path into a container deployment (#2845)
14c4c9d Abhay Singh committed 3 months ago
fix(proxy/anthropic): inject headroom_retrieve whenever a CCR marker is present, not only for new markers (#2848)
3808f60 Abhay Singh committed 3 months ago
fix(memory): bound the TrafficLearner pending-pattern accumulator (memory leak) (#2579)
1f5feff Abhay Singh committed 3 months ago
fix(proxy/gemini): keep streaming-parity baseline so eligible_pct can't exceed 100 (#2824)
b97c7c6 Abhay Singh committed 3 months ago
fix(proxy): stop discarding compressed Codex WS later-frame payloads (#2823)
4ec416d Ashish Patel committed 3 months ago
perf: cut hot-path latency 27% (token-count memo, startup preloads, JSON scan memo) (#2838)
53af90d Tejas Chopra committed 3 months ago
fix(proxy): graceful shutdown and reliable Ctrl+C exit (#621)
17cdb18 Patrick A committed 3 months ago
fix(beacon): split session failures by status code (#2815)
2954e37 Tejas Chopra committed 3 months ago
Per-request backend selection for routing extensions (#2809)
c07da99 Tejas Chopra committed 3 months ago
fix(proxy): enable tool search by default and repair poisoned transcripts (#2807)
0237cbf Tejas Chopra committed 3 months ago
fix(opencode): don't preload a missing transport shim into child processes (#2806)
303e052 Tejas Chopra committed 3 months ago
feat(cli,pricing): add CLI extension seam and prompt-cache TTL pricing (#2802)
6ec3e34 Tejas Chopra committed 3 months ago
fix(deps): enforce audited transitive dependency floors (#2791)
64e2039 JD Davis committed 3 months ago
fix(beacon): report all-layers savings, not context-compression only (#2796)
e9a24f3 Tejas Chopra committed 3 months ago
fix(tokenizer): coerce non-string tool_call fields before counting (#2801)
b6f9877 Parideboy committed 3 months ago
fix(proxy): unwrap Hermes tool_call bridge in tool name map (#2717)
a97b824 pgjh committed 3 months ago
deps: bump postcss from 8.5.19 to 8.5.25 in /sdk/typescript (#2747)
267c2bd dependabot[bot] committed 3 months ago