Skip to content

Never stop coding. Free MIT AI gateway: one endpoint, 375 providers (150+ free), 1,400+ models Kimi, Claude, GPT, Gemini, GLM, DeepSeek, MiniMax. Works with Claude Code, Codex, Cursor, OpenCode, Cline & Copilot. Quota-aware auto-fallback, RTK+Caveman compression saves 15-95% tokens, MCP/A2A, Desktop/PWA. Built by 900+ contributors

docs(compression): document RTK+Caveman stacked savings ranges

diegosouzapw6424c825701a
diff --git a/README.md b/README.md
index 90af73193d..d794d02d81 100644
--- a/README.md
+++ b/README.md
@@ -2,7 +2,7 @@
# 🚀 OmniRoute — The Free AI Gateway
-### Never stop coding. Save 15-75% tokens with prompt compression + auto-fallback to **FREE & low-cost AI models**.
+### Never stop coding. Save 15-95% eligible tokens with RTK+Caveman compression + auto-fallback to **FREE & low-cost AI models**.
_The most complete open-source AI proxy — **one endpoint**, **160+ providers**, **13 routing strategies**, zero downtime. Multi-platform: **Web**, **Desktop (Electron)**, **Mobile (PWA + Termux)**. Fully extensible via **MCP Server (37 tools)**, **A2A Protocol**, and **Memory/Skills** systems. Available in **40+ languages**._
@@ -20,7 +20,7 @@ _The most complete open-source AI proxy — **one endpoint**, **160+ providers**
<a href="https://trendshift.io/repositories/23589" target="_blank"><img src="https://trendshift.io/api/badge/repositories/23589" alt="diegosouzapw%2FOmniRoute | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
-[🚀 Quick Start](#-quick-start) • [💡 Features](#-key-features) • [🗜️ Compression](#%EF%B8%8F-prompt-compression--save-15-75-tokens-automatically) • [💰 Pricing](#-pricing-at-a-glance) • [🎯 Use Cases](#-use-cases--ready-made-combo-playbooks) • [🌍 Proxy](#-bypass-geographic-blocks--use-ai-from-any-country) • [❓ FAQ](#-frequently-asked-questions) • [📖 Docs](#-documentation) • [💬 WhatsApp](https://chat.whatsapp.com/JI7cDQ1GyaiDHhVBpLxf8b?mode=gi_t)
+[🚀 Quick Start](#-quick-start) • [💡 Features](#-key-features) • [🗜️ Compression](#%EF%B8%8F-prompt-compression--save-15-95-eligible-tokens-automatically) • [💰 Pricing](#-pricing-at-a-glance) • [🎯 Use Cases](#-use-cases--ready-made-combo-playbooks) • [🌍 Proxy](#-bypass-geographic-blocks--use-ai-from-any-country) • [❓ FAQ](#-frequently-asked-questions) • [📖 Docs](#-documentation) • [💬 WhatsApp](https://chat.whatsapp.com/JI7cDQ1GyaiDHhVBpLxf8b?mode=gi_t)
</div>
@@ -219,7 +219,7 @@ _Connect any AI-powered IDE or CLI tool through OmniRoute — free API gateway f
**OmniRoute solves all of this:**
-✅ **Prompt Compression** — auto-compress prompts & tool outputs, save 15-75% tokens per request
+✅ **Prompt Compression** — auto-compress prompts & tool outputs, save 15-95% eligible tokens per request with RTK+Caveman stacked mode
✅ **Maximize subscriptions** — track quota, use every bit before reset
✅ **Auto fallback** — Subscription → API Key → Cheap → Free, zero downtime
✅ **Multi-account** — round-robin between accounts per provider
@@ -410,7 +410,7 @@ Alibaba · Amazon Q · AssemblyAI · Baidu Qianfan · Baseten · Black Forest La
↓
┌──────────────────────────────────────────────────┐
│ OmniRoute (Smart Router) │
-│ • 🗜️ Prompt Compression (save 15-75% tokens) │
+│ • 🗜️ Prompt Compression (save 15-95% eligible) │
│ • Format translation (OpenAI ↔ Claude ↔ Gemini) │
│ • Quota tracking + Embeddings + Images │
│ • Auto token refresh + Rate limit management │
@@ -424,14 +424,14 @@ Alibaba · Amazon Q · AssemblyAI · Baidu Qianfan · Baseten · Black Forest La
│ ↓ budget limit
└─→ [Tier 4: FREE] Qoder, Qwen, Kiro (unlimited)
-Result: Never stop coding, minimal cost + 15-75% token savings
+Result: Never stop coding, minimal cost + 15-95% eligible token savings
```
---
-## 🗜️ Prompt Compression — Save 15-75% Tokens Automatically
+## 🗜️ Prompt Compression — Save 15-95% Eligible Tokens Automatically
-> **Why use many token when few token do trick?** OmniRoute's built-in compression pipeline reduces token usage on every request — before it even reaches the provider. Inspired by [Caveman](https://github.com/JuliusBrussee/caveman) (⭐ 51K+).
+> **Why use many token when few token do trick?** OmniRoute's built-in compression pipeline reduces token usage before requests reach the provider. It combines ideas from [RTK - Rust Token Killer](https://github.com/rtk-ai/rtk) and [Caveman](https://github.com/JuliusBrussee/caveman) (⭐ 51K+).
### How It Works
@@ -442,12 +442,12 @@ Every request passes through the compression pipeline **transparently** — no c
│ Client sends │────▶│ OmniRoute Compression │────▶│ Provider │
│ full prompt │ │ Pipeline (7 options) │ │ receives │
│ (10,000 tok) │ │ │ │ compressed │
-│ │ │ 🪶 Lite ........... ~15% │ │ (2,500 tok) │
+│ │ │ 🪶 Lite ........... ~15% │ │ (~1,080 tok)│
│ │ │ 🪨 Standard ....... ~30% │ │ │
-│ │ │ ⚡ Aggressive ..... ~50% │ │ 💰 75% saved │
+│ │ │ ⚡ Aggressive ..... ~50% │ │ 💰 up to 95%│
│ │ │ 🔥 Ultra .......... ~75% │ │ │
-│ │ │ 🧰 RTK ............ 20-70% │ │ │
-│ │ │ 🔗 Stacked ........ 30-80% │ │ │
+│ │ │ 🧰 RTK ............ 60-90% │ │ │
+│ │ │ 🔗 Stacked ........ 78-95% │ │ │
└──────────────────┘ └─────────────────────────────┘ └──────────────┘
```
@@ -460,8 +460,36 @@ Every request passes through the compression pipeline **transparently** — no c
| **🪨 Standard (Caveman)** | ~30% | 30+ regex rules: filler removal, context condensation, structural compression, multi-turn dedup | Daily coding with Claude/Codex |
| **⚡ Aggressive** | ~50% | All standard + progressive message aging + tool result summarization + LLM-based compression | Long sessions with many tool calls |
| **🔥 Ultra** | ~75% | All aggressive + heuristic token pruning + stopword removal + score-based filtering | Maximum savings when tokens are scarce |
-| **🧰 RTK** | 20-70% | 49 command-aware filters, RTK-style JSON DSL, verify gate, trust-gated custom filters | Shell/test/build/git output in agents |
-| **🔗 Stacked** | 30-80% | Ordered engine pipeline, usually RTK first then Caveman | Mixed prompts with tool logs + prose |
+| **🧰 RTK** | 60-90% | 49 command-aware filters, RTK-style JSON DSL, verify gate, trust-gated custom filters | Shell/test/build/git output in agents |
+| **🔗 Stacked** | 78-95% | RTK first, then Caveman input condensation; ~89% with upstream average math | Mixed prompts with tool logs + prose |
+
+### RTK + Caveman Savings Math
+
+These numbers are based on the upstream project READMEs under `_references/_outros`:
+
+| Source | Upstream claim used by OmniRoute docs |
+| ------- | ------------------------------------------------------------------------------------------------------------------- |
+| Caveman | `~75%` fewer output tokens; benchmark average `65%` output savings, range `22-87%`; `~46%` input compression tool |
+| RTK | `60-90%` command-output token savings; sample session `~118,000 -> ~23,900` tokens, which is `79.7%` saved (`~80%`) |
+
+For the default stacked compression combo, OmniRoute runs:
+
+```txt
+RTK -> Caveman
+```
+
+When both engines can act on the same tool/context payload, the savings compound:
+
+```txt
+combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)
+average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%
+range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
+```
+
+Caveman output mode is separate from prompt compression. When enabled for responses, use Caveman's
+own upstream output numbers: `65%` average, `~75%` headline, `22-87%` observed range. Total bill
+savings depend on the prompt/output mix, but coding-agent sessions are often tool-context heavy, so
+the `RTK -> Caveman` combo is the best default for maximum context savings.
### Before & After (Standard/Caveman Mode)
@@ -516,7 +544,7 @@ Auto-trigger: set `autoTriggerTokens` to automatically enable compression when a
Compression combos can also assign a named compression pipeline to routing combos, so a coding combo can use RTK + Caveman while a paid subscription combo stays on lite mode.
-> 🪨 **Fun fact:** The standard/caveman mode is inspired by [Caveman](https://github.com/JuliusBrussee/caveman) — the viral project that proved "caveman speak" cuts 65% of tokens while keeping 100% technical accuracy. OmniRoute takes this further with a **7-option pipeline** that goes from gentle whitespace cleanup through RTK tool-output filters and stacked multi-engine compression.
+> 🪨 **Fun fact:** The standard/caveman mode is inspired by [Caveman](https://github.com/JuliusBrussee/caveman) — the viral project that reports 65% average output-token savings while keeping technical accuracy. OmniRoute takes this further with a **7-option pipeline** and a default `RTK -> Caveman` combo that can reach ~89% average savings on eligible tool/context payloads.
📖 **Full compression documentation:** [`docs/COMPRESSION_GUIDE.md`](docs/COMPRESSION_GUIDE.md) • [`docs/RTK_COMPRESSION.md`](docs/RTK_COMPRESSION.md) • [`docs/COMPRESSION_ENGINES.md`](docs/COMPRESSION_ENGINES.md) • [`docs/COMPRESSION_RULES_FORMAT.md`](docs/COMPRESSION_RULES_FORMAT.md) • [`docs/COMPRESSION_LANGUAGE_PACKS.md`](docs/COMPRESSION_LANGUAGE_PACKS.md)
@@ -941,7 +969,7 @@ Then in `/dashboard/media` → **Transcription** tab: upload any audio or video
| ---------------------------------------------------------------------------------------------------- | -------------------------------- |
| 🧠 **Smart 4-Tier Fallback** — Subscription → API → Cheap → Free | Never stop coding, zero downtime |
| 🔄 **Format Translation** — OpenAI ↔ Claude ↔ Gemini ↔ Responses API | Works with ANY CLI tool |
-| 🗜️ **Prompt Compression** — 7 options including Caveman, RTK, and stacked pipelines | Save 15-75% tokens automatically |
+| 🗜️ **Prompt Compression** — 7 options including Caveman, RTK, and stacked pipelines | Save 15-95% eligible tokens |
| 🤖 **MCP Server** — 37 tools, 3 transports (stdio/SSE/HTTP), 10 scopes | IDE/agent tool integration |
| 🛡️ **Resilience Engine** — circuit breakers, cooldowns, TLS spoofing, anti-thundering herd | Auto-recovery from any failure |
| 🎵 **10 Multi-Modal APIs** — chat, embed, images, video, music, TTS, STT, moderation, rerank, search | One endpoint for everything |
diff --git a/docs/COMPRESSION_ENGINES.md b/docs/COMPRESSION_ENGINES.md
index da5c934cfb..81dc01c1ba 100644
--- a/docs/COMPRESSION_ENGINES.md
+++ b/docs/COMPRESSION_ENGINES.md
@@ -13,7 +13,7 @@ OmniRoute compression is built around engine contracts. A mode can run one engin
| `aggressive` | Caveman + history/tool summarizers | Long chat sessions |
| `ultra` | Caveman + pruning helpers | Context-limit recovery |
| `rtk` | RTK | Terminal, shell, build, test, and git output |
-| `stacked` | Pipeline, default `rtk -> caveman` | Mixed tool logs and prose |
+| `stacked` | Pipeline, default `rtk -> caveman` | Mixed tool logs and prose, max savings |
## Engine Registry
@@ -39,6 +39,11 @@ Caveman mode focuses on semantic condensation of normal prose:
The dashboard surface is `Dashboard -> Context & Cache -> Caveman`.
+Caveman upstream reports `~75%` fewer output tokens, `65%` average output savings in benchmarks
+with a `22-87%` range, and a `~46%` input-compression tool. OmniRoute uses the Caveman input-side
+number when documenting stacked prompt/context savings; Caveman output mode remains a separate
+response-behavior feature.
+
## RTK
RTK mode focuses on command and tool output:
@@ -61,6 +66,9 @@ The dashboard surface is `Dashboard -> Context & Cache -> RTK`.
Operational details for custom filters, trust, verify, and raw-output recovery live in
[`RTK_COMPRESSION.md`](RTK_COMPRESSION.md).
+RTK upstream reports `60-90%` savings for command-output compression. Its README example shows a
+30-minute Claude Code session going from `~118,000` tokens to `~23,900`, or `79.7%` saved.
+
## Stacked Pipelines
Stacked mode runs pipeline steps in order. The default is:
@@ -75,6 +83,14 @@ prose. RTK reduces noisy tool logs first, then Caveman compresses remaining natu
Pipeline steps are configured with `stackedPipeline` in compression settings or through compression
combos.
+When both engines reduce the same eligible payload, savings compound:
+
+```txt
+combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)
+average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%
+range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
+```
+
## Compression Combos
Compression combos are named compression profiles that can be assigned to routing combos:
diff --git a/docs/COMPRESSION_GUIDE.md b/docs/COMPRESSION_GUIDE.md
index 9cfe6c8a15..88213bfc63 100644
--- a/docs/COMPRESSION_GUIDE.md
+++ b/docs/COMPRESSION_GUIDE.md
@@ -1,6 +1,6 @@
# 🗜️ Prompt Compression Guide — OmniRoute
-> Save 15-75% on token costs automatically. For a quick overview, see the [README Compression section](../README.md#%EF%B8%8F-prompt-compression--save-15-75-tokens-automatically).
+> Save 15-95% on eligible context automatically. For a quick overview, see the [README Compression section](../README.md#%EF%B8%8F-prompt-compression--save-15-95-eligible-tokens-automatically).
## Overview
@@ -19,8 +19,8 @@ Client Request
→ Standard: Caveman-speak filler removal (~30%)
→ Aggressive: History aging + summarization (~50%)
→ Ultra: Heuristic pruning + code-block thinning (~75%)
- → RTK: Command-aware terminal/tool-output filtering (20-70%)
- → Stacked: Ordered multi-engine pipeline, usually RTK then Caveman (30-80%)
+ → RTK: Command-aware terminal/tool-output filtering (60-90% upstream range)
+ → Stacked: Ordered multi-engine pipeline, usually RTK then Caveman (78-95% eligible range)
→ Compressed Request → Provider
```
@@ -79,7 +79,7 @@ Maximum compression for token-critical scenarios:
**Best for:** When you're hitting context limits repeatedly.
-### RTK Mode (20-70% savings)
+### RTK Mode (60-90% upstream range)
RTK mode is optimized for verbose tool outputs that appear in coding-agent sessions:
@@ -94,7 +94,7 @@ RTK mode is optimized for verbose tool outputs that appear in coding-agent sessi
**Best for:** Agent sessions with shell, build, test, git, grep, and file-output transcripts.
-### Stacked Mode (30-80% savings)
+### Stacked Mode (78-95% eligible range)
Stacked mode runs multiple compression engines in a deterministic order. The default pipeline is:
@@ -110,6 +110,36 @@ compression combos assigned to routing combos.
---
+## Upstream Savings Math
+
+OmniRoute documents compression savings from two sources: upstream project benchmarks and
+OmniRoute's own engine composition.
+
+| Source | Upstream README number used here |
+| ------- | --------------------------------------------------------------------------------------------------------------------- |
+| Caveman | `~75%` fewer output tokens, `65%` benchmark average output savings, `22-87%` range, and `~46%` input compression tool |
+| RTK | `60-90%` command-output savings; sample session `~118,000 -> ~23,900` tokens, or `79.7%` saved (`~80%`) |
+
+For overlapping tool/context payloads, the default OmniRoute combo stacks the engines:
+
+```txt
+RTK -> Caveman
+```
+
+The combined savings are multiplicative, not additive:
+
+```txt
+combined = 1 - (1 - RTK savings) * (1 - Caveman input savings)
+average = 1 - (1 - 0.80) * (1 - 0.46) = 89.2%
+range = 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
+```
+
+That `78-95%` number applies when both RTK and Caveman can reduce the same input/context payload.
+Caveman response output mode is separate: when enabled, use Caveman's own output savings (`65%`
+average, `~75%` headline, `22-87%` range). Total billing savings depend on your prompt/output mix.
+
+---
+
## Token Savings Visualization
```
@@ -118,8 +148,8 @@ With Lite: 40K tokens sent (15% saved — safe, always-on)
With Standard: 33K tokens sent (30% saved — caveman-speak rules)
With Aggressive: 24K tokens sent (50% saved — aging + summarization)
With Ultra: 12K tokens sent (75% saved — heuristic pruning)
-With RTK: varies by output (20-70% saved — command-aware filters)
-With Stacked: varies by pipeline (30-80% saved — multi-engine execution)
+With RTK: 19K-5K tokens sent (60-90% saved on command/tool output)
+With Stacked: 10K-2.5K tokens sent (78-95% eligible RTK+Caveman range)
```
---
@@ -229,9 +259,9 @@ Every compressed request includes stats in the server logs:
## Acknowledgments
-Standard mode compression rules are inspired by **[Caveman](https://github.com/JuliusBrussee/caveman)** by **[JuliusBrussee](https://github.com/JuliusBrussee)** (⭐ 51K+) — the viral "why use many token when few token do trick" project.
+Standard mode compression rules are inspired by **[Caveman](https://github.com/JuliusBrussee/caveman)** by **[JuliusBrussee](https://github.com/JuliusBrussee)** (⭐ 51K+) — the viral "why use many token when few token do trick" project. Caveman reports `~75%` fewer output tokens, `65%` benchmark average output savings, a `22-87%` output range, and a `~46%` input-compression tool.
-RTK mode is inspired by **[RTK - Rust Token Killer](https://github.com/rtk-ai/rtk)** by **[RTK AI](https://github.com/rtk-ai)** — the high-performance command-output compression project for terminal, build, test, git, and tool-output filtering.
+RTK mode is inspired by **[RTK - Rust Token Killer](https://github.com/rtk-ai/rtk)** by **[RTK AI](https://github.com/rtk-ai)** — the high-performance command-output compression project for terminal, build, test, git, and tool-output filtering. RTK reports `60-90%` savings, with its README sample session showing `~80%` saved.
---
diff --git a/docs/FEATURES.md b/docs/FEATURES.md
index ff86c872f1..4c8e9e455f 100644
--- a/docs/FEATURES.md
+++ b/docs/FEATURES.md
@@ -119,7 +119,7 @@ Context & Cache now exposes dedicated pages for Caveman, RTK, and Compression Co
- **Caveman** — language-aware rule packs, preview, output-mode controls, and analytics
- **RTK** — command-aware compression for shell, git, test, build, package, Docker, infra, JSON, and stack-trace output
-- **Compression Combos** — named pipelines such as `rtk -> caveman` assigned to routing combos
+- **Compression Combos** — named pipelines such as `rtk -> caveman` assigned to routing combos; the default stacked math reaches `~89%` average and `78-95%` eligible-context savings when both engines apply
- **Raw-output recovery** — optional redacted RTK raw-output pointers for debugging compressed failures
See [Compression Guide](COMPRESSION_GUIDE.md), [RTK Compression](RTK_COMPRESSION.md), and
diff --git a/docs/RTK_COMPRESSION.md b/docs/RTK_COMPRESSION.md
index 9f24139df3..f675c93d25 100644
--- a/docs/RTK_COMPRESSION.md
+++ b/docs/RTK_COMPRESSION.md
@@ -12,9 +12,20 @@ rtk -> caveman
That order compresses noisy machine output first, then lets Caveman condense remaining prose.
+Upstream RTK reports `60-90%` command-output savings. Its README sample session goes from
+`~118,000` standard tokens to `~23,900` RTK tokens, which is `79.7%` saved (`~80%`). OmniRoute uses
+that upstream average for the stacked savings calculation with Caveman input compression:
+
+```txt
+RTK average: 80% saved
+Caveman input: 46% saved
+Stacked: 1 - (1 - 0.80) * (1 - 0.46) = 89.2% saved
+Range: 1 - (1 - 0.60..0.90) * (1 - 0.46) = 78.4-94.6%
+```
+
## What It Compresses
-The built-in catalog currently ships 39 filters across these categories:
+The built-in catalog currently ships 49 filters across these categories:
| Category | Examples |
| --------- | ------------------------------------------------------------- |
diff --git a/package.json b/package.json
index 24568eb1fb..12d4fc61f0 100644
--- a/package.json
+++ b/package.json
@@ -1,7 +1,7 @@
{
"name": "omniroute",
"version": "3.7.9",
- "description": "Smart AI Router with auto fallback — route to FREE & cheap models, zero downtime. Works with Cursor, Cline, Claude Desktop, Codex, and any OpenAI-compatible tool.",
+ "description": "Unified AI router with 160+ providers, RTK+Caveman compression, auto fallback, MCP/A2A, desktop, PWA, and OpenAI-compatible APIs.",
"type": "module",
"bin": {
"omniroute": "bin/omniroute.mjs",