Never stop coding. Free MIT AI gateway: one endpoint, 375 providers (150+ free), 1,400+ models Kimi, Claude, GPT, Gemini, GLM, DeepSeek, MiniMax. Works with Claude Code, Codex, Cursor, OpenCode, Cline & Copilot. Quota-aware auto-fallback, RTK+Caveman compression saves 15-95% tokens, MCP/A2A, Desktop/PWA. Built by 900+ contributors
TypeScript a2aai-agentsai-gatewayanthropicclaudeclaude-codeclinecodexcopilotcursordeepseekfree-aigeminikimillm-gatewaymcpopenaiopenai-proxyqwentoken-saver
docs(compression): document RTK+Caveman stacked savings ranges
diegosouzapw6424c825701a
diff --git a/README.md b/README.mdindex 90af73193d..d794d02d81 100644--- a/README.md+++ b/README.md@@ -2,7 +2,7 @@# 🚀 OmniRoute — The Free AI Gateway-### Never stop coding. Save 15-75% tokens with prompt compression + auto-fallback to **FREE & low-cost AI models**.+### Never stop coding. Save 15-95% eligible tokens with RTK+Caveman compression + auto-fallback to **FREE & low-cost AI models**._The most complete open-source AI proxy — **one endpoint**, **160+ providers**, **13 routing strategies**, zero downtime. Multi-platform: **Web**, **Desktop (Electron)**, **Mobile (PWA + Termux)**. Fully extensible via **MCP Server (37 tools)**, **A2A Protocol**, and **Memory/Skills** systems. Available in **40+ languages**._@@ -20,7 +20,7 @@ _The most complete open-source AI proxy — **one endpoint**, **160+ providers**<a href="https://trendshift.io/repositories/23589" target="_blank"><img src="https://trendshift.io/api/badge/repositories/23589" alt="diegosouzapw%2FOmniRoute | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>-[🚀 Quick Start](#-quick-start) • [💡 Features](#-key-features) • [🗜️ Compression](#%EF%B8%8F-prompt-compression--save-15-75-tokens-automatically) • [💰 Pricing](#-pricing-at-a-glance) • [🎯 Use Cases](#-use-cases--ready-made-combo-playbooks) • [🌍 Proxy](#-bypass-geographic-blocks--use-ai-from-any-country) • [❓ FAQ](#-frequently-asked-questions) • [📖 Docs](#-documentation) • [💬 WhatsApp](https://chat.whatsapp.com/JI7cDQ1GyaiDHhVBpLxf8b?mode=gi_t)+[🚀 Quick Start](#-quick-start) • [💡 Features](#-key-features) • [🗜️ Compression](#%EF%B8%8F-prompt-compression--save-15-95-eligible-tokens-automatically) • [💰 Pricing](#-pricing-at-a-glance) • [🎯 Use Cases](#-use-cases--ready-made-combo-playbooks) • [🌍 Proxy](#-bypass-geographic-blocks--use-ai-from-any-country) • [❓ FAQ](#-frequently-asked-questions) • [📖 Docs](#-documentation) • [💬 WhatsApp](https://chat.whatsapp.com/JI7cDQ1GyaiDHhVBpLxf8b?mode=gi_t)</div>@@ -219,7 +219,7 @@ _Connect any AI-powered IDE or CLI tool through OmniRoute — free API gateway f**OmniRoute solves all of this:**-✅ **Prompt Compression** — auto-compress prompts & tool outputs, save 15-75% tokens per request+✅ **Prompt Compression** — auto-compress prompts & tool outputs, save 15-95% eligible tokens per request with RTK+Caveman stacked mode✅ **Maximize subscriptions** — track quota, use every bit before reset✅ **Auto fallback** — Subscription → API Key → Cheap → Free, zero downtime✅ **Multi-account** — round-robin between accounts per provider@@ -410,7 +410,7 @@ Alibaba · Amazon Q · AssemblyAI · Baidu Qianfan · Baseten · Black Forest La↓┌──────────────────────────────────────────────────┐│ OmniRoute (Smart Router) │-│ • 🗜️ Prompt Compression (save 15-75% tokens) │+│ • 🗜️ Prompt Compression (save 15-95% eligible) ││ • Format translation (OpenAI ↔ Claude ↔ Gemini) ││ • Quota tracking + Embeddings + Images ││ • Auto token refresh + Rate limit management │@@ -424,14 +424,14 @@ Alibaba · Amazon Q · AssemblyAI · Baidu Qianfan · Baseten · Black Forest La│ ↓ budget limit└─→ [Tier 4: FREE] Qoder, Qwen, Kiro (unlimited)-Result: Never stop coding, minimal cost + 15-75% token savings+Result: Never stop coding, minimal cost + 15-95% eligible token savings```----## 🗜️ Prompt Compression — Save 15-75% Tokens Automatically+## 🗜️ Prompt Compression — Save 15-95% Eligible Tokens Automatically-> **Why use many token when few token do trick?** OmniRoute's built-in compression pipeline reduces token usage on every request — before it even reaches the provider. Inspired by [Caveman](https://github.com/JuliusBrussee/caveman) (⭐ 51K+).+> **Why use many token when few token do trick?** OmniRoute's built-in compression pipeline reduces token usage before requests reach the provider. It combines ideas from [RTK - Rust Token Killer](https://github.com/rtk-ai/rtk) and [Caveman](https://github.com/JuliusBrussee/caveman) (⭐ 51K+).### How It Works@@ -442,12 +442,12 @@ Every request passes through the compression pipeline **transparently** — no c│ Client sends │────▶│ OmniRoute Compression │────▶│ Provider ││ full prompt │ │ Pipeline (7 options) │ │ receives ││ (10,000 tok) │ │ │ │ compressed │-│ │ │ 🪶 Lite ........... ~15% │ │ (2,500 tok) │+│ │ │ 🪶 Lite ........... ~15% │ │ (~1,080 tok)││ │ │ 🪨 Standard ....... ~30% │ │ │-│ │ │ ⚡ Aggressive ..... ~50% │ │ 💰 75% saved │+│ │ │ ⚡ Aggressive ..... ~50% │ │ 💰 up to 95%││ │ │ 🔥 Ultra .......... ~75% │ │ │-│ │ │ 🧰 RTK ............ 20-70% │ │ │-│ │ │ 🔗 Stacked ........ 30-80% │ │ │+│ │ │ 🧰 RTK ............ 60-90% │ │ │+│ │ │ 🔗 Stacked ........ 78-95% │ │ │└──────────────────┘ └─────────────────────────────┘ └──────────────┘```@@ -460,8 +460,36 @@ Every request passes through the compression pipeline **transparently** — no c| **🪨 Standard (Caveman)** | ~30% | 30+ regex rules: filler removal, context condensation, structural compression, multi-turn dedup | Daily coding with Claude/Codex || **⚡ Aggressive** | ~50% | All standard + progressive message aging + tool result summarization + LLM-based compression | Long sessions with many tool calls || **🔥 Ultra** | ~75% | All aggressive + heuristic token pruning + stopword removal + score-based filtering | Maximum savings when tokens are scarce |-| **🧰 RTK** | 20-70% | 49 command-aware filters, RTK-style JSON DSL, verify gate, trust-gated custom filters | Shell/test/build/git output in agents |-| **🔗 Stacked** | 30-80% | Ordered engine pipeline, usually RTK first then Caveman | Mixed prompts with tool logs + prose |+| **🧰 RTK** | 60-90% | 49 command-aware filters, RTK-style JSON DSL, verify gate, trust-gated custom filters | Shell/test/build/git output in agents |+| **🔗 Stacked** | 78-95% | RTK first, then Caveman input condensation; ~89% with upstream average math | Mixed prompts with tool logs + prose |++### RTK + Caveman Savings Math++These numbers are based on the upstream project READMEs under `_references/_outros`:++| Source | Upstream claim used by OmniRoute docs |+| ------- | ------------------------------------------------------------------------------------------------------------------- |+| Caveman | `~75%` fewer output tokens; benchmark average `65%` output savings, range `22-87%`; `~46%` input compression tool |+| RTK | `60-90%` command-output token savings; sample session `~118,000 -> ~23,900` tokens, which is `79.7%` saved (`~80%`) |++For the default stacked compression combo, OmniRoute runs:++```txt+RTK -> Caveman+```++When both engines can act on the same tool/context payload, the savings compound:++```txt+combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)+average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%+range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%+```++Caveman output mode is separate from prompt compression. When enabled for responses, use Caveman's+own upstream output numbers: `65%` average, `~75%` headline, `22-87%` observed range. Total bill+savings depend on the prompt/output mix, but coding-agent sessions are often tool-context heavy, so+the `RTK -> Caveman` combo is the best default for maximum context savings.### Before & After (Standard/Caveman Mode)@@ -516,7 +544,7 @@ Auto-trigger: set `autoTriggerTokens` to automatically enable compression when aCompression combos can also assign a named compression pipeline to routing combos, so a coding combo can use RTK + Caveman while a paid subscription combo stays on lite mode.-> 🪨 **Fun fact:** The standard/caveman mode is inspired by [Caveman](https://github.com/JuliusBrussee/caveman) — the viral project that proved "caveman speak" cuts 65% of tokens while keeping 100% technical accuracy. OmniRoute takes this further with a **7-option pipeline** that goes from gentle whitespace cleanup through RTK tool-output filters and stacked multi-engine compression.+> 🪨 **Fun fact:** The standard/caveman mode is inspired by [Caveman](https://github.com/JuliusBrussee/caveman) — the viral project that reports 65% average output-token savings while keeping technical accuracy. OmniRoute takes this further with a **7-option pipeline** and a default `RTK -> Caveman` combo that can reach ~89% average savings on eligible tool/context payloads.📖 **Full compression documentation:** [`docs/COMPRESSION_GUIDE.md`](docs/COMPRESSION_GUIDE.md) • [`docs/RTK_COMPRESSION.md`](docs/RTK_COMPRESSION.md) • [`docs/COMPRESSION_ENGINES.md`](docs/COMPRESSION_ENGINES.md) • [`docs/COMPRESSION_RULES_FORMAT.md`](docs/COMPRESSION_RULES_FORMAT.md) • [`docs/COMPRESSION_LANGUAGE_PACKS.md`](docs/COMPRESSION_LANGUAGE_PACKS.md)@@ -941,7 +969,7 @@ Then in `/dashboard/media` → **Transcription** tab: upload any audio or video| ---------------------------------------------------------------------------------------------------- | -------------------------------- || 🧠 **Smart 4-Tier Fallback** — Subscription → API → Cheap → Free | Never stop coding, zero downtime || 🔄 **Format Translation** — OpenAI ↔ Claude ↔ Gemini ↔ Responses API | Works with ANY CLI tool |-| 🗜️ **Prompt Compression** — 7 options including Caveman, RTK, and stacked pipelines | Save 15-75% tokens automatically |+| 🗜️ **Prompt Compression** — 7 options including Caveman, RTK, and stacked pipelines | Save 15-95% eligible tokens || 🤖 **MCP Server** — 37 tools, 3 transports (stdio/SSE/HTTP), 10 scopes | IDE/agent tool integration || 🛡️ **Resilience Engine** — circuit breakers, cooldowns, TLS spoofing, anti-thundering herd | Auto-recovery from any failure || 🎵 **10 Multi-Modal APIs** — chat, embed, images, video, music, TTS, STT, moderation, rerank, search | One endpoint for everything |diff --git a/docs/COMPRESSION_ENGINES.md b/docs/COMPRESSION_ENGINES.mdindex da5c934cfb..81dc01c1ba 100644--- a/docs/COMPRESSION_ENGINES.md+++ b/docs/COMPRESSION_ENGINES.md@@ -13,7 +13,7 @@ OmniRoute compression is built around engine contracts. A mode can run one engin| `aggressive` | Caveman + history/tool summarizers | Long chat sessions || `ultra` | Caveman + pruning helpers | Context-limit recovery || `rtk` | RTK | Terminal, shell, build, test, and git output |-| `stacked` | Pipeline, default `rtk -> caveman` | Mixed tool logs and prose |+| `stacked` | Pipeline, default `rtk -> caveman` | Mixed tool logs and prose, max savings |## Engine Registry@@ -39,6 +39,11 @@ Caveman mode focuses on semantic condensation of normal prose:The dashboard surface is `Dashboard -> Context & Cache -> Caveman`.+Caveman upstream reports `~75%` fewer output tokens, `65%` average output savings in benchmarks+with a `22-87%` range, and a `~46%` input-compression tool. OmniRoute uses the Caveman input-side+number when documenting stacked prompt/context savings; Caveman output mode remains a separate+response-behavior feature.+## RTKRTK mode focuses on command and tool output:@@ -61,6 +66,9 @@ The dashboard surface is `Dashboard -> Context & Cache -> RTK`.Operational details for custom filters, trust, verify, and raw-output recovery live in[`RTK_COMPRESSION.md`](RTK_COMPRESSION.md).+RTK upstream reports `60-90%` savings for command-output compression. Its README example shows a+30-minute Claude Code session going from `~118,000` tokens to `~23,900`, or `79.7%` saved.+## Stacked PipelinesStacked mode runs pipeline steps in order. The default is:@@ -75,6 +83,14 @@ prose. RTK reduces noisy tool logs first, then Caveman compresses remaining natuPipeline steps are configured with `stackedPipeline` in compression settings or through compressioncombos.+When both engines reduce the same eligible payload, savings compound:++```txt+combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)+average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%+range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%+```+## Compression CombosCompression combos are named compression profiles that can be assigned to routing combos:diff --git a/docs/COMPRESSION_GUIDE.md b/docs/COMPRESSION_GUIDE.mdindex 9cfe6c8a15..88213bfc63 100644--- a/docs/COMPRESSION_GUIDE.md+++ b/docs/COMPRESSION_GUIDE.md@@ -1,6 +1,6 @@# 🗜️ Prompt Compression Guide — OmniRoute-> Save 15-75% on token costs automatically. For a quick overview, see the [README Compression section](../README.md#%EF%B8%8F-prompt-compression--save-15-75-tokens-automatically).+> Save 15-95% on eligible context automatically. For a quick overview, see the [README Compression section](../README.md#%EF%B8%8F-prompt-compression--save-15-95-eligible-tokens-automatically).## Overview@@ -19,8 +19,8 @@ Client Request→ Standard: Caveman-speak filler removal (~30%)→ Aggressive: History aging + summarization (~50%)→ Ultra: Heuristic pruning + code-block thinning (~75%)- → RTK: Command-aware terminal/tool-output filtering (20-70%)- → Stacked: Ordered multi-engine pipeline, usually RTK then Caveman (30-80%)+ → RTK: Command-aware terminal/tool-output filtering (60-90% upstream range)+ → Stacked: Ordered multi-engine pipeline, usually RTK then Caveman (78-95% eligible range)→ Compressed Request → Provider```@@ -79,7 +79,7 @@ Maximum compression for token-critical scenarios:**Best for:** When you're hitting context limits repeatedly.-### RTK Mode (20-70% savings)+### RTK Mode (60-90% upstream range)RTK mode is optimized for verbose tool outputs that appear in coding-agent sessions:@@ -94,7 +94,7 @@ RTK mode is optimized for verbose tool outputs that appear in coding-agent sessi**Best for:** Agent sessions with shell, build, test, git, grep, and file-output transcripts.-### Stacked Mode (30-80% savings)+### Stacked Mode (78-95% eligible range)Stacked mode runs multiple compression engines in a deterministic order. The default pipeline is:@@ -110,6 +110,36 @@ compression combos assigned to routing combos.---+## Upstream Savings Math++OmniRoute documents compression savings from two sources: upstream project benchmarks and+OmniRoute's own engine composition.++| Source | Upstream README number used here |+| ------- | --------------------------------------------------------------------------------------------------------------------- |+| Caveman | `~75%` fewer output tokens, `65%` benchmark average output savings, `22-87%` range, and `~46%` input compression tool |+| RTK | `60-90%` command-output savings; sample session `~118,000 -> ~23,900` tokens, or `79.7%` saved (`~80%`) |++For overlapping tool/context payloads, the default OmniRoute combo stacks the engines:++```txt+RTK -> Caveman+```++The combined savings are multiplicative, not additive:++```txt+combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)+average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%+range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%+```++That `78-95%` number applies when both RTK and Caveman can reduce the same input/context payload.+Caveman response output mode is separate: when enabled, use Caveman's own output savings (`65%`+average, `~75%` headline, `22-87%` range). Total billing savings depend on your prompt/output mix.++---+## Token Savings Visualization```@@ -118,8 +148,8 @@ With Lite: 40K tokens sent (15% saved — safe, always-on)With Standard: 33K tokens sent (30% saved — caveman-speak rules)With Aggressive: 24K tokens sent (50% saved — aging + summarization)With Ultra: 12K tokens sent (75% saved — heuristic pruning)-With RTK: varies by output (20-70% saved — command-aware filters)-With Stacked: varies by pipeline (30-80% saved — multi-engine execution)+With RTK: 19K-5K tokens sent (60-90% saved on command/tool output)+With Stacked: 10K-2.5K tokens sent (78-95% eligible RTK+Caveman range)```---@@ -229,9 +259,9 @@ Every compressed request includes stats in the server logs:## Acknowledgments-Standard mode compression rules are inspired by **[Caveman](https://github.com/JuliusBrussee/caveman)** by **[JuliusBrussee](https://github.com/JuliusBrussee)** (⭐ 51K+) — the viral "why use many token when few token do trick" project.+Standard mode compression rules are inspired by **[Caveman](https://github.com/JuliusBrussee/caveman)** by **[JuliusBrussee](https://github.com/JuliusBrussee)** (⭐ 51K+) — the viral "why use many token when few token do trick" project. Caveman reports `~75%` fewer output tokens, `65%` benchmark average output savings, a `22-87%` output range, and a `~46%` input-compression tool.-RTK mode is inspired by **[RTK - Rust Token Killer](https://github.com/rtk-ai/rtk)** by **[RTK AI](https://github.com/rtk-ai)** — the high-performance command-output compression project for terminal, build, test, git, and tool-output filtering.+RTK mode is inspired by **[RTK - Rust Token Killer](https://github.com/rtk-ai/rtk)** by **[RTK AI](https://github.com/rtk-ai)** — the high-performance command-output compression project for terminal, build, test, git, and tool-output filtering. RTK reports `60-90%` savings, with its README sample session showing `~80%` saved.---diff --git a/docs/FEATURES.md b/docs/FEATURES.mdindex ff86c872f1..4c8e9e455f 100644--- a/docs/FEATURES.md+++ b/docs/FEATURES.md@@ -119,7 +119,7 @@ Context & Cache now exposes dedicated pages for Caveman, RTK, and Compression Co- **Caveman** — language-aware rule packs, preview, output-mode controls, and analytics- **RTK** — command-aware compression for shell, git, test, build, package, Docker, infra, JSON, and stack-trace output-- **Compression Combos** — named pipelines such as `rtk -> caveman` assigned to routing combos+- **Compression Combos** — named pipelines such as `rtk -> caveman` assigned to routing combos; the default stacked math reaches `~89%` average and `78-95%` eligible-context savings when both engines apply- **Raw-output recovery** — optional redacted RTK raw-output pointers for debugging compressed failuresSee [Compression Guide](COMPRESSION_GUIDE.md), [RTK Compression](RTK_COMPRESSION.md), anddiff --git a/docs/RTK_COMPRESSION.md b/docs/RTK_COMPRESSION.mdindex 9f24139df3..f675c93d25 100644--- a/docs/RTK_COMPRESSION.md+++ b/docs/RTK_COMPRESSION.md@@ -12,9 +12,20 @@ rtk -> cavemanThat order compresses noisy machine output first, then lets Caveman condense remaining prose.+Upstream RTK reports `60-90%` command-output savings. Its README sample session goes from+`~118,000` standard tokens to `~23,900` RTK tokens, which is `79.7%` saved (`~80%`). OmniRoute uses+that upstream average for the stacked savings calculation with Caveman input compression:++```txt+RTK average: 80% saved+Caveman input: 46% saved+Stacked: 1 - (1 - 0.80) * (1 - 0.46) = 89.2% saved+Range: 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%+```+## What It Compresses-The built-in catalog currently ships 39 filters across these categories:+The built-in catalog currently ships 49 filters across these categories:| Category | Examples || --------- | ------------------------------------------------------------- |diff --git a/package.json b/package.jsonindex 24568eb1fb..12d4fc61f0 100644--- a/package.json+++ b/package.json@@ -1,7 +1,7 @@{"name": "omniroute","version": "3.7.9",- "description": "Smart AI Router with auto fallback — route to FREE & cheap models, zero downtime. Works with Cursor, Cline, Claude Desktop, Codex, and any OpenAI-compatible tool.",+ "description": "Unified AI router with 160+ providers, RTK+Caveman compression, auto fallback, MCP/A2A, desktop, PWA, and OpenAI-compatible APIs.","type": "module","bin": {"omniroute": "bin/omniroute.mjs",