Skip to content

Make Codex and Claude Code subscriptions go further: a local model router that gives routine agent work to economical models and keeps your strongest model for the hard parts.

docs: explain smart routing cache protection and cost tradeoffs (#122)

澄潭b4c1a92765bc
diff --git a/apps/website/src/components/HomePage.astro b/apps/website/src/components/HomePage.astro
index 9936e869..2f483c35 100644
--- a/apps/website/src/components/HomePage.astro
+++ b/apps/website/src/components/HomePage.astro
@@ -109,7 +109,7 @@ const releaseKinds = [preferredArtifact && t('macOS 桌面应用', 'macOS Deskto
</section>
<section class="section wrap faq-section" id="faq"><div class="eyebrow section-label">{t('常见问题', 'Questions')}</div><h2>{t('常见问题', 'Frequently asked questions')}</h2>
- <details><summary>{t('切换模型会影响 KV Cache 吗?', 'How does model switching affect KV Cache?')}</summary><p>{t('KV Cache 保存模型已计算的注意力键和值,减少处理重复上下文时的计算。不同模型之间无法共享 KV Cache。同轮工具续接优先使用所选组内合格的当前模型,以便复用 KV Cache;新用户请求重新判断,也可以继续原模型。KV Cache 命中率取决于模型服务、配置和有效期。', 'KV Cache stores attention keys and values already computed by a model, reducing computation for repeated context. Different models cannot share this cache. Tool continuations prefer the eligible current model within the selected group to support KV Cache reuse. New user turns decide again and can keep the same model. The KV Cache hit rate depends on the provider, configuration, and cache lifetime.')}</p></details>
+ <details><summary>{t('切换模型会影响 KV Cache 吗?', 'How does model switching affect KV Cache?')}</summary><p>{t('会。HiRoute 在同一轮正常的连续工具调用中沿用路由决策,并优先保持所选组内的当前模型,让增长中的前缀持续复用缓存。新用户消息会重新判断任务;若发生切换,首次请求可能损失旧缓存,后续连续调用则可以复用新缓存。', 'Yes. During normal consecutive tool calls in the same turn, HiRoute reuses the routing decision and prefers the current model within the selected group, preserving reuse of the growing prefix. A new user message triggers reassessment; a switch may lose the old cache on the first request, while later calls can reuse the new cache.')}</p><p>{t('上下文压缩或重建后,部分缓存本来就需要重建,这时重新选模的额外损失可能更小。HiRoute 通过保持连续执行减少不必要的缓存开销,最终比较包含缓存重建、路由判断和重试的整单成本。具体影响与 Astra × Qwen 实验测算见下文。', 'After compaction or a history rebuild, some cache entries already need rebuilding, so reassessment at that boundary may incur a smaller additional loss. HiRoute preserves continuous execution to reduce unnecessary cache overhead. Whole-task costs include rebuilding, routing decisions, and retries; the article below explains the effects and Astra × Qwen calculations.')}</p><p><a class="text-link" href={en ? '/en/news/routing-cache-cost/' : '/news/routing-cache-cost/'}>{t('阅读缓存影响与成本分析:Astra × Qwen 3.8 Flash', 'Read the cache and cost analysis: Astra × Qwen 3.8 Flash')} →</a></p></details>
<details><summary>{t('上下文压缩一定会重新选模吗?', 'Does context compaction always trigger a new model choice?')}</summary><p>{t('要看是否仍能识别为同轮续接、此前冻结决策是否可复用。能够继承时保持本轮决策;历史重建使续接关系或决策无法复用时才再次判断。重新判断也可以选择原模型,客户端无需单独上报压缩事件。', 'It depends on whether HiRoute can still recognize the same-turn continuation and reuse the frozen decision. A reusable continuation inherits that decision. If a history rebuild breaks continuity or the decision cannot be reused, HiRoute judges again. The new choice can keep the same model; clients need no separate compaction event.')}</p></details>
<details id="history-faq"><summary>{t('决策模型能看到哪些执行历史?', 'What execution history can a decision model see?')}</summary><p>{t('HiRoute 提供简化的 Agent 轮次历史:用户问题、接受的回答、工具名称与粗粒度状态。内部任务类别、模型组与执行身份由 HiRoute 绑定,不要求模型猜测。历史被裁剪、淘汰或重启后可能不完整;评分目标阶段的证据完整性单独判断,缺失或部分评分不会视为零分。', 'HiRoute supplies simplified Agent-turn history: user input, accepted output, tool names, and coarse outcomes. HiRoute binds internal task categories, groups, and execution identities itself. Trimming, eviction, or restart may leave history incomplete; completeness of the assessed stage is tracked separately. Missing or partial assessments are not zero scores.')}</p></details>
<details><summary>{t('一定要部署 Jev 吗?', 'Must I deploy Jev?')}</summary><p>{t('不需要。在模型 → 决策模型中连接百炼、OpenRouter Jev、TypeSafe 或兼容接入点,就能使用内置判断。智能省钱也支持启发式规则;需要自有处理逻辑时才选择自定义扩展。', 'No. Connect Bailian, OpenRouter Jev, TypeSafe, or a compatible endpoint in Models → Decision models to use built-in judgment. Smart saving also supports heuristic rules. Choose a custom extension when you need your own processing logic.')}</p></details>
diff --git a/docs/code-map/decision-foundation.md b/docs/code-map/decision-foundation.md
index d91b6101..80982731 100644
--- a/docs/code-map/decision-foundation.md
+++ b/docs/code-map/decision-foundation.md
@@ -89,6 +89,7 @@ and [CLI entry tests](../../crates/cli/src/lib.rs) must remain aligned.
| Official optional Jev service | [Extension](../../decision-extensions/extensions/jev-decider), with the same v1 contract |
| Website decision pages and downloads | [Renderer](../../apps/website/src/lib/decision-docs.mjs), [content preparation](../../apps/website/scripts/prepare-content.mjs), [website checks](../../apps/website/tests/decision-docs.test.mjs) |
| UI/CLI instructions | [Website guides](../../apps/website/content/guides), [standalone CLI](../standalone-cli.md) |
+| Routing cost explanations and FAQ | [Bilingual articles](../../news/README.md), [homepage](../../apps/website/src/components/HomePage.astro), [frozen-evidence cache calculation](../../experiments/cache_cost_analysis.py) |
| Illustrations and native screenshots | [Component capture](../../apps/desktop/decision-docs.tsx), [fixture checks](../../apps/desktop/tests/decision-docs-data.test.mjs), [asset provenance](../../decision-extensions/assets/README.md) |
Documentation fixtures and native windows with constructed observation records
diff --git a/experiments/README.md b/experiments/README.md
index 26dcb651..f3154f97 100644
--- a/experiments/README.md
+++ b/experiments/README.md
@@ -64,6 +64,31 @@ python3 experiments/render_figures.py
It writes the canonical SVGs in `news/assets/`. Website preparation copies these
assets; the Markdown and news index remain the source of truth.
+## Recalculate cache-cost sensitivity
+
+For the cache-cost article's separate post-hoc sensitivity analysis, run:
+
+```sh
+python3 experiments/reproduce.py fetch
+python3 experiments/cache_cost_analysis.py
+```
+
+The calculator verifies the frozen evidence before using its prices, fixed FX,
+all-attempt usage and recorded delivery bounds. It preserves all three paired
+acceptance outcomes and computes an idealized all-input-cached Astra reference
+with each run's observed outputs held fixed. Prefix-loss examples are conditional
+calculations, not measured cache-hit rates. This analysis adds no model calls and
+changes neither the historical evidence nor its acceptance rules. Read the article
+in [English](../news/2026-10-14-routing-cache-cost.en.md) or
+[简体中文](../news/2026-10-14-routing-cache-cost.zh-CN.md).
+
+Observed per-turn request counts are reported separately from the conditional
+cache calculations. Writing counts use the tracked October 6 structured results
+(including interrupted turns separately); HTTPX counts use the frozen October 4
+evidence, distinguishing the whole native turn from its final stable Astra segment.
+The calculator includes the writing source hash and keeps decision reuse distinct
+from cache hits.
+
## Add an experiment
Create `cases/<name>/` with a README, task inputs, parameter/provenance record,
diff --git a/experiments/cache_cost_analysis.py b/experiments/cache_cost_analysis.py
new file mode 100644
index 00000000..12deb5d1
--- /dev/null
+++ b/experiments/cache_cost_analysis.py
@@ -0,0 +1,143 @@
+#!/usr/bin/env python3
+"""Recalculate the cache-cost article from frozen evidence; no network/model calls."""
+from decimal import Decimal, ROUND_FLOOR
+import json
+from statistics import median
+
+from reproduce import ROOT, RESULTS, assess, digest, read, require, verify_manifest
+
+
+def analyze():
+ verified = verify_manifest()
+ evidence = read(RESULTS / 'evidence.json')
+ deliveries = read(RESULTS / 'deliveries.json')
+ assessment = assess(deliveries)
+ basis = evidence['frozen_cost_basis']
+ fx = Decimal(basis['fx']['usd_per_eur']) / Decimal(basis['fx']['cny_per_eur'])
+ million = Decimal(1_000_000)
+ rates = {}
+ for model, record in basis['rates_per_million_tokens'].items():
+ require(record['currency'] in ('USD', 'CNY'), 'unsupported currency')
+ multiplier = fx if record['currency'] == 'CNY' else Decimal(1)
+ rates[model] = {}
+ for kind in ('input', 'cache_read', 'cache_write', 'output'):
+ low, high = map(Decimal, record[kind])
+ require(low == high, 'article needs a fixed tariff, not a price interval')
+ rates[model][kind] = low * multiplier
+
+ examples = []
+ for model, rate in rates.items():
+ for prefix in (100_000, 10_000):
+ examples.append({
+ 'model': model, 'lost_reusable_prefix_tokens': prefix,
+ 'ordinary_processing_premium_usd': Decimal(prefix) * (rate['input'] - rate['cache_read']) / million,
+ 'cache_write_premium_usd': Decimal(prefix) * (rate['cache_write'] - rate['cache_read']) / million,
+ 'hit_rate_loss_percentage_points_at_1m_input': Decimal(prefix) / million * 100,
+ })
+
+ primary = rates['gpt-6-astra']
+ writing_path = ROOT / 'results/decision-routing-20261006/results.json'
+ writing = read(writing_path)['writing']
+ writing_turns = [s for r in writing['replicates'] for s in r['stages']]
+ complete_counts = [s['request_count'] for s in writing_turns if s['native_completed']]
+ partial_counts = [s['request_count'] for s in writing_turns if not s['native_completed']]
+ require(len(writing_turns) == writing['registered_turns'], 'writing turn coverage changed')
+ require(len(complete_counts) == writing['normal_native_turns'], 'writing completion counts disagree')
+ httpx = read(ROOT / 'cases/unattended-engineering/results/2026-10-04/evidence.json')
+ require(httpx['single_native_turn'] and httpx['native_event_counts']['turn.started'] == 1,
+ 'HTTPX must remain a single native user turn')
+ segments = []
+ for request in sorted(httpx['requests'], key=lambda r: r['start_ms']):
+ require(request['decision_source'] in ('external_classifier', 'inherited'), 'unknown decision source')
+ if request['decision_source'] == 'external_classifier':
+ segments.append([])
+ require(bool(segments), 'continuation without opening decision')
+ segments[-1].append(request)
+ last_segment = segments[-1]
+ require(all(r['outcome'] == 'accepted' and r['models'] == ['gpt-6-astra'] for r in last_segment),
+ 'final HTTPX segment must contain only accepted Astra requests')
+ # Illustrative append-only user turn, not a measured experiment trace.
+ old_prefix, new_input, continuation_growth = 100_000, 1_000, 2_000
+ new_turn_examples = []
+ for calls in sorted({1, 5, 20, len(last_segment)}):
+ total = calls * (old_prefix + new_input) + continuation_growth * calls * (calls - 1) // 2
+ uncached = new_input + (calls - 1) * continuation_growth
+ new_turn_examples.append({
+ 'calls': calls, 'total_input_tokens': total,
+ 'no_switch_hit_percent': Decimal(total - uncached) / total * 100,
+ 'first_call_cold_switch_hit_percent': Decimal(total - uncached - old_prefix) / total * 100,
+ 'loss_percentage_points': Decimal(old_prefix) / total * 100,
+ 'append_only_single_switch_bound_percentage_points': Decimal(100) / calls,
+ })
+ directions = []
+ for source, destination in (('gpt-6-astra', 'qwen3.8-flash'), ('qwen3.8-flash', 'gpt-6-astra')):
+ warm = old_prefix * rates[source]['cache_read'] / million
+ cold = old_prefix * rates[destination]['cache_write'] / million
+ directions.append({
+ 'source': source, 'destination': destination,
+ 'source_warm_prefix_usd': warm, 'destination_cold_prefix_usd': cold,
+ 'old_prefix_cost_change_usd': cold - warm,
+ })
+ pairs = []
+ for pair in assessment['pairs']:
+ prefix = f"formal-draft98-v1-p{pair['pair']:02d}"
+ arms = {row['group']: row for row in deliveries['rows'] if row['id'].startswith(prefix + '-')}
+ strong, mixed = arms['strong'], arms['mixed']
+ attempts = [a for c in strong['components'] for a in c['accounting']['attempts']]
+ threshold = basis['rates_per_million_tokens']['gpt-6-astra']['long_input_threshold']
+ require(all(a['model'] == 'gpt-6-astra' and a['usage']['input_tokens'] <= threshold for a in attempts),
+ 'all-Astra standard-context assumptions changed')
+ inputs = sum(a['usage']['input_tokens'] for a in attempts)
+ outputs = sum(a['usage']['output_tokens'] for a in attempts)
+ decisions = sum((Decimal(c['accounting']['jev_reported_usd']) for c in strong['components']), Decimal(0))
+ ideal = (inputs * primary['cache_read'] + outputs * primary['output']) / million + decisions
+ mixed_upper = Decimal(mixed['usd']['upper'])
+ headroom = ideal - mixed_upper
+ pairs.append({
+ 'pair': pair['pair'], 'both_deliveries_accepted': pair['accepted'],
+ 'observed_savings_percent_floor': pair['conservative_savings_percent'],
+ 'mixed_usd_upper': mixed_upper, 'astra_input_tokens': inputs, 'astra_output_tokens': outputs,
+ 'astra_all_input_cached_hypothetical_usd': ideal,
+ 'hypothetical_savings_percent_floor': ((1 - mixed_upper / ideal) * 100).quantize(Decimal('.01'), rounding=ROUND_FLOOR),
+ 'additional_cost_break_even_usd': headroom,
+ 'extra_astra_cache_write_tokens_at_break_even': headroom * million / (primary['cache_write'] - primary['cache_read']),
+ })
+ return {
+ 'basis': 'Historical API-equivalent prices checked 2026-10-03; fixed 2026-09-30 FX, not current billing.',
+ 'scope': 'Post-hoc sensitivity calculation with observed outputs fixed; not measured cache-hit rates or a new run.',
+ 'verified_frozen_files': verified,
+ 'usd_per_cny': fx, 'usd_per_million_tokens': rates,
+ 'prefix_examples': examples, 'pairs': pairs,
+ 'observed_call_counts': {
+ 'unit': 'business-model requests per native user turn; not individual tool invocations',
+ 'writing_source': str(writing_path.relative_to(ROOT)), 'writing_source_sha256': digest(writing_path),
+ 'writing_completed': {
+ 'turns': len(complete_counts), 'requests': sum(complete_counts),
+ 'min': min(complete_counts), 'median': median(complete_counts), 'max': max(complete_counts),
+ 'mean': Decimal(sum(complete_counts)) / len(complete_counts),
+ },
+ 'writing_interrupted': {'turns': len(partial_counts), 'requests': sum(partial_counts)},
+ 'httpx': {
+ 'native_user_turns': httpx['native_event_counts']['turn.started'],
+ 'requests': len(httpx['requests']),
+ 'accepted': sum(r['outcome'] == 'accepted' for r in httpx['requests']),
+ 'cancelled': sum(r['outcome'] == 'cancelled' for r in httpx['requests']),
+ 'compactions': len(httpx['compactions']),
+ 'inherited_decisions': sum(r['decision_source'] == 'inherited' for r in httpx['requests']),
+ 'inherited_decisions_percent': Decimal(sum(r['decision_source'] == 'inherited' for r in httpx['requests'])) / len(httpx['requests']) * 100,
+ 'final_accepted_astra_segment_requests': len(last_segment),
+ },
+ },
+ 'new_user_turn_scenario': {
+ 'kind': 'conditional illustration; fixed input sequence, one cold switch, no expiration or compaction',
+ 'old_reusable_prefix_tokens': old_prefix, 'new_uncached_input_tokens': new_input,
+ 'uncached_tokens_added_per_continuation': continuation_growth,
+ 'hit_rate_examples': new_turn_examples, 'prefix_cost_by_direction': directions,
+ },
+ 'original_three_pair_gate_passed': assessment['original_three_pair_gate_passed'],
+ 'paid_model_calls': 0,
+ }
+
+
+if __name__ == '__main__':
+ print(json.dumps(analyze(), ensure_ascii=False, indent=2, default=str))
diff --git a/news/2026-10-14-routing-cache-cost.en.md b/news/2026-10-14-routing-cache-cost.en.md
new file mode 100644
index 00000000..1fed2892
--- /dev/null
+++ b/news/2026-10-14-routing-cache-cost.en.md
@@ -0,0 +1,168 @@
+# Model Switching Cache Effects and Smart Routing Costs
+
+Smart routing assigns work to different models. A natural concern follows: could the cache lost in a model switch consume the money saved?
+
+HiRoute reduces that overhead in two ways: **keep the model stable through consecutive tool calls within a turn, and reassess at context-rebuild boundaries where less reusable history may be lost.** For switches that do occur, cache rebuilding belongs in the whole-task bill.
+
+We explain the protections first, then use observed call counts and the Astra × Qwen experiment to calculate cache effects and total costs separately.
+
+## How HiRoute protects cache reuse
+
+KV Cache stores context states already computed by a model. Keeping the model and context prefix stable lets later requests read cached history and process new content. A destination without that cache must process the old prefix again, but subsequent calls can reuse the cache it builds.
+
+HiRoute's context-hold mechanism, ContextHold, recognizes session and history continuity and retains valid execution preferences:
+
+| Execution boundary | HiRoute behavior | Effect on caching |
+| --- | --- | --- |
+| Consecutive tool calls within a turn | When the turn is recognizable, history is continuous, and the decision is valid, reuse it and prefer the eligible current model in the selected group | Avoid selecting a model afresh on every tool round-trip; reuse the growing prefix |
+| New user message | Reassess the task, keeping or changing the model | Keeping the model can retain the old cache; switching requires accounting for the full reusable prefix lost |
+| Compaction or history rebuild | Keep a reusable decision; otherwise reassess | Some history already needs rebuilding, so additional losses may shrink to the remaining reusable prefix |
+
+Actual hits depend on upstream cache rules and lifetime; an unavailable candidate can still trigger failover. HiRoute uses continuity to reduce unnecessary switches. See the [model-routing guide](https://hiroute.ai/en/docs/model-routing/) for selection rules.
+
+## Spreading the loss from a new-user-message switch
+
+A new user message usually appends to history, leaving the old prefix reusable. Switching to a completely cold model can lose that cache on the first request. Keeping subsequent tool calls stable spreads the loss across the turn.
+
+The calculations use a token-weighted input hit rate. To isolate one switch, assume identical input sequences on both paths, a valid original cache, a fully cold destination on the first call, and normal prefix reuse afterward with no further switches, compaction, or expiration. Define:
+
+- `P`: old prefix tokens that would have hit.
+- `U`: the new user message and other uncached input tokens.
+- `n`: model calls in the turn, including the first call.
+- `g`: uncached tokens added by each tool continuation, held constant in this example.
+
+On the first request, staying with the model gives a hit rate of `P / (P + U)`; a cold switch gives zero. With a 100,000-token prefix and 1,000 new tokens, the request drops from **99.01% to 0%**.
+
+Input on call `i` is `P + U + (i − 1)g`, so:
+
+**Total turn input T = n(P + U) + g × n(n − 1) / 2.**
+
+Without switching, uncached input totals `U + (n − 1)g`. A first-call cold switch adds one loss of `P` tokens. Therefore:
+
+**Turn hit-rate reduction = P / T ≤ 1 / n.**
+
+The bound follows from every input containing `P`, so `T ≥ nP`. Under these conditions, fourteen consecutive calls bound the reduction at approximately **7.14 percentage points**, and twenty at **5 percentage points**.
+
+With `P = 100,000`, `U = 1,000`, and `g = 2,000`, the example becomes:
+
+| Calls in the turn | Total input T | No-switch hit rate | First-call cold-switch hit rate | Reduction |
+| --- | ---: | ---: | ---: | ---: |
+| 1 | 101,000 | 99.01% | 0.00% | 99.01 percentage points |
+| 5 | 525,000 | 98.29% | 79.24% | 19.05 percentage points |
+| 14 | 1,596,000 | 98.31% | 92.04% | 6.27 percentage points |
+| 20 | 2,400,000 | 98.38% | 94.21% | 4.17 percentage points |
+
+**Continuous execution spreads a large first-request loss across the execution stage.** Frequent switches between short turns with long histories remain costly. If the destination has a partially valid prefix, count only the difference; keeping the original model causes no switching loss. Multiple switches require summing their losses rather than applying the single-switch bound.
+
+### How many calls occurred in real experiments
+
+A turn runs from one user instruction to the end of that native Agent turn. A call is a business-model request: one response can contain several tool invocations, and routing-decision requests are counted separately.
+
+| Observed workload | Model calls |
+| --- | --- |
+| Drafting, reviewing, revising, and final review | 310 requests across 42 normally completed turns; 3–19 per turn, median 5.5, mean 7.38 |
+| Unattended Astra/Qwen HTTPX coding | One initial instruction generated 40 requests: 25 Qwen and 15 Astra |
+| The final stable Astra segment of that HTTPX task | 14 consecutive requests, all accepted; the first reassessed and 13 inherited the decision |
+
+Writing counts come from the [per-turn structured records](../experiments/results/decision-routing-20261006/results.json) for Qwen 3.8 Max and GLM 5.3. All 48 registered turns are retained, with 51 requests in the other six interrupted turns reported separately. The [HTTPX experiment](../experiments/cases/unattended-engineering/README.md) included two compactions, 30 accepted requests, and 10 cancellations; its final stable stretch of 14 calls supplies the continuous-execution example.
+
+Across HTTPX, **35/40 requests inherited the decision, or 87.5%**. This shows the protection in operation: most tool round-trips reused an existing choice. The 87.5% measures decision reuse; the earlier 6.27-point gap is a calculation using the observed count of 14 with assumed input sizes. Neither is a measured cache-hit rate. That stable segment began after compaction and illustrates execution length, rather than providing a controlled new-user-message switch.
+
+These samples give a practical sense of scale: writing turns often contain several requests, while long coding work can contain over a dozen consecutive requests. Different workloads offer different room to spread a switching loss.
+
+## Why switching after compaction can lose less
+
+Compaction replaces some history with a summary. Even the original model must process that new content, while unchanged system instructions or tool definitions may still hit. For example, Claude's [compaction and caching documentation](https://platform.claude.com/docs/en/build-with-claude/compaction-threshold#maximizing-cache-hits-with-system-prompts) describes retaining a system-prompt cache after compaction.
+
+When comparing staying versus switching after compaction, count only **the prefix that would still hit but is unavailable on the destination** as the additional loss `R`. Both paths must process the summary and new messages.
+
+Under the same single-switch model, **stage hit-rate reduction = R / T**. At one million total input tokens, losing a 100,000-token prefix reduces the rate by 10 percentage points; losing 10,000 reduces it by one point. This comparison fixes the denominator; real compaction also changes `T`.
+
+The protections address different terms: **continuous execution reduces the number of losses, while a rebuild boundary can reduce the prefix lost each time.** Long retained system instructions and tools still contribute to `R`. The additional impact becomes negligible when `R / T` is sufficiently small.
+
+## What the cache loss costs
+
+The calculations use the experiment's **API-equivalent prices frozen on October 3, 2026**, its **fixed September 30 exchange rate**, and the standard context tier. These are historical analytical references, not current quotes or subscription bills.
+
+| Per million tokens | Astra USD | Qwen 3.8 Flash CNY | Qwen equivalent USD |
+| --- | ---: | ---: | ---: |
+| Ordinary input | 10 | 0.8 | 0.119322 |
+| Cache read | 1 | 0.1 | 0.014915 |
+| Cache write | 12.5 | 1.25 | 0.186441 |
+| Output | 50 | 2.7 | 0.402712 |
+
+The conversion is `1.1355 / 7.6130` USD per CNY, calculated at full precision. Original records are under `evidence.json → frozen_cost_basis` in the [frozen evidence archive](https://github.com/higress-group/HiRoute/releases/tag/experiment-evidence-2026-10-04).
+
+First isolate caching by comparing a warm and cold execution on the same destination model. Let `p` be the first-processing price and `r` the cache-read price, both in USD per million tokens:
+
+**Additional cache cost K = R × (p − r) / 1,000,000.**
+
+Whether first processing uses the ordinary input or cache-write price depends on the provider. These are alternative billing cases, not two full prices to add together.
+
+| Destination and lost prefix | Premium at ordinary input price | Premium at cache-write price |
+| --- | ---: | ---: |
+| Astra, 100,000 tokens | $0.900 | $1.150 |
+| Astra, 10,000 tokens | $0.090 | $0.115 |
+| Qwen, 100,000 tokens | About $0.010441 | About $0.017153 |
+| Qwen, 10,000 tokens | About $0.001044 | About $0.001715 |
+
+Astra losing a reusable 10,000-token prefix adds $0.115 under the write-price case. Spread across twenty consecutive calls, that is $0.00575 per call. The total remains $0.115, while losing a full 100,000-token prefix can add $1.15. Preserving long-prefix reuse has real value.
+
+### Switching direction also changes the bill
+
+A cross-model comparison must include the price change. When a new user message arrives, the old-prefix portion changes by:
+
+**Old-prefix cost change = P × (destination first-processing price − original cache-read price) / 1,000,000.**
+
+For `P = 100,000`, with destination processing charged at the cache-write price:
+
+| Direction | Read old prefix on original model | Rebuild on destination | Cost change |
+| --- | ---: | ---: | ---: |
+| Astra → Qwen | $0.100000 | About $0.018644 | About $0.081356 less |
+| Qwen → Astra | About $0.001492 | $1.250000 | About $1.248508 more |
+
+**Even with a cache loss, processing the old prefix on Qwen can cost less; upgrading to Astra pays for stronger capability.** This table includes the cross-model price difference, so the preceding cache premium must not be added again. New input, outputs, routing decisions, and rework still belong in the whole-task bill.
+
+## Put cache overhead back into the whole-task cost
+
+The [Astra × Qwen experiment](https://hiroute.ai/en/news/astra-qwen-smart-routing/) assigned six source-verification components to Qwen and a critical decision to Astra. Two of three paired runs met the original whole-delivery standard in both arms. The mixed delivery that missed the first pair's gate and all results remain in the [experiment record](../experiments/cases/research-cost-quality/README.md). This component-routing comparison establishes a cost margin for the workload.
+
+Whole-task costs include all subject attempts, structural retries, and reported Jev decisions. Unreported cache-write quantities produce cost intervals, so the comparison uses the mixed upper bound against the all-Astra lower bound. Ordinary input, cache reads, and cache writes do not overlap; billed reasoning tokens are already included in output.
+
+| Pair accepted in both arms | Mixed upper bound | All-Astra lower bound | Conservative saving |
+| --- | ---: | ---: | ---: |
+| Pair 2 | $0.146806 | $1.838882 | 92.01% |
+| Pair 3 | $0.177957 | $2.067170 | 91.39% |
+
+### Even if every all-Astra input token hits
+
+Give all-Astra an ideal cache: charge every input token at the cache-read price, keeping observed output usage, while mixed routing stays at its recorded upper bound. This fixed-usage sensitivity calculation checks whether the cache discount changes the conclusion.
+
+Each all-Astra run used 116,383 input tokens, with 34,214 and 34,310 output tokens respectively:
+
+`Pair 2: (116,383 × 1 + 34,214 × 50) / 1,000,000 = $1.827083`
+
+`Pair 3: (116,383 × 1 + 34,310 × 50) / 1,000,000 = $1.831883`
+
+| Pair | Hypothetical all-input-cached Astra | Mixed upper bound | Remaining saving |
+| --- | ---: | ---: | ---: |
+| Pair 2 | $1.827083 | $0.146806 | 91.96% |
+| Pair 3 | $1.831883 | $0.177957 | 90.28% |
+
+The advantage survives for both tasks: input caching does not eliminate output charges, and Qwen's ordinary input price is below Astra's cache-read price at these rates. Calculations retain each model's different observed output volume, with savings calculated at full precision and rounded down to two percentage decimals.
+
+### Room for additional overhead
+
+Against the idealized all-Astra cost `C_A*` and mixed upper bound `C_M`, the break-even condition is:
+
+**New overhead X < C_A* − C_M.**
+
+The margins are approximately **$1.680277** and **$1.653926**. One extra $0.115 Astra prefix-write loss would not offset the savings in this example. `X` includes only cache rebuilding, decisions, or rework added beyond the recorded bill, avoiding double counting.
+
+Reproduce the call counts, costs, and hypothetical comparisons with the [offline calculator](../experiments/cache_cost_analysis.py), following the [reproduction instructions](../experiments/README.md). The records support these specific workloads; a real task should be assessed using its own context, call volume, and delivery quality.
+
+## Keep suitable work on economy and upgrade when needed
+
+HiRoute aims to let an economy model continue handling work suited to it, bringing in a primary model when stronger capability is needed. Complex work can start on primary, and insufficient economy-model performance can trigger an upgrade, avoiding repeated rework just to save on individual calls.
+
+Keep the model stable within a stage to accumulate cache benefits. Reassess at rebuild boundaries to reduce avoidable cache losses. Upgrade when needed to protect the delivery standard. **Smart routing has a total-cost advantage when savings from suitable economy-model work exceed additional cache rebuilding, routing decisions, and rework.**
diff --git a/news/2026-10-14-routing-cache-cost.zh-CN.md b/news/2026-10-14-routing-cache-cost.zh-CN.md
new file mode 100644
index 00000000..dd11b170
--- /dev/null
+++ b/news/2026-10-14-routing-cache-cost.zh-CN.md
@@ -0,0 +1,168 @@
+# 模型切换的缓存影响与智能路由成本分析
+
+智能路由能把工作分给不同模型,但用户也会关心:切换模型损失的缓存,会不会把省下的钱又花回去?
+
+HiRoute 从两个环节减少这笔开销:**同一轮连续工具调用保持模型稳定,让上下文持续复用缓存;上下文压缩或重建后重新选模,有机会减少仍可复用的旧前缀损失。** 对于确实发生的切换,还要把缓存重建费用计入整单,判断最终是否划算。
+
+下面先解释保护机制,再用真实调用次数和 Astra × Qwen 实验的费用数据,把缓存影响与总成本分别算清楚。
+
+## HiRoute 如何保护缓存
+
+KV Cache 保存模型已经计算过的上下文状态。保持模型和上下文前缀稳定,后续请求就有机会读取旧缓存,只处理新增内容。换到没有对应缓存的模型时,需要重新处理旧前缀;新缓存建立后,又可以被接下来的调用复用。
+
+HiRoute 的上下文保持机制(ContextHold)识别会话与历史连续性,并保留有效的执行偏好:
+
+| 执行节点 | HiRoute 的处理 | 对缓存的作用 |
+| --- | --- | --- |
+| 同一轮连续工具调用 | 同轮可识别、历史连续且决策有效时,沿用决策,优先保持所选组内合格的当前模型 | 避免每次工具往返都重新选模,持续复用增长中的前缀 |
+| 新用户消息 | 重新判断任务,可以继续原模型,也可以切换 | 继续原模型时可保留旧缓存;发生切换时,要计算完整可复用前缀的损失 |
+| 上下文压缩或重建 | 原决策仍可续用则保持,否则重新判断 | 部分历史本来就需要重建,额外损失可能缩小到剩余可复用前缀 |
+
+实际命中由上游的缓存规则和有效期决定,当前模型不可用时也可能故障接力。HiRoute 通过连续性减少不必要的切换;具体路由规则见[模型路由指南](https://hiroute.ai/docs/model-routing/)。
+
+## 新用户消息触发切换时 如何摊薄损失
+
+新用户消息通常只是追加历史,旧前缀仍可命中。若这时切到完全冷的模型,首次请求可能损失整段缓存;后续工具调用保持稳定,才能逐步摊薄这次损失。
+
+以下计算用 token 加权的输入命中率。为单独观察一次切换,假设两条路径有相同的输入序列,原缓存有效,目标模型首次完全冷启动,后续前缀正常复用且不再切换、压缩或过期。设:
+
+- `P`:原本可命中的旧前缀 token 数。
+- `U`:新用户消息及其他尚未缓存的输入 token 数。
+- `n`:本轮模型调用次数,包含首次调用。
+- `g`:每次工具续接新增的未缓存 token 数,本例按固定值计算。
+
+首次请求不切换时的命中率为 `P / (P + U)`,冷切换后为零。例如,旧前缀 100,000 token、新输入 1,000 token,当次命中率会从 **99.01% 降至 0%**。
+
+把后续调用一起计入,第 `i` 次输入是 `P + U + (i − 1)g`,因此:
+
+**本轮累计输入 T = n(P + U) + g × n(n − 1) / 2。**
+
+不切换时,累计未命中量为 `U + (n − 1)g`;仅首次冷切换时,多出一次 `P` 的未命中量。所以:
+
+**本轮命中率下降 = P / T ≤ 1 / n。**
+
+最后的不等式来自每次输入都包含前缀 `P`,即 `T ≥ nP`。在这些条件下,14 次连续调用的下降上界约为 **7.14 个百分点**,20 次为 **5 个百分点**。
+
+取 `P = 100,000`、`U = 1,000`、`g = 2,000`,得到更具体的例子:
+
+| 本轮调用次数 | 累计输入 T | 不切换命中率 | 仅首次冷切换命中率 | 下降幅度 |
+| --- | ---: | ---: | ---: | ---: |
+| 1 | 101,000 | 99.01% | 0.00% | 99.01 个百分点 |
+| 5 | 525,000 | 98.29% | 79.24% | 19.05 个百分点 |
+| 14 | 1,596,000 | 98.31% | 92.04% | 6.27 个百分点 |
+| 20 | 2,400,000 | 98.38% | 94.21% | 4.17 个百分点 |
+
+**保持连续调用,能把首次请求的明显损失摊薄到整个执行阶段。** 短轮次、长历史的频繁切换仍需谨慎。若目标模型已有部分有效前缀缓存,只计算两者之差;若继续原模型,切换损失为零。多次切换则累计各次损失,不能继续沿用“一次冷切换”的上界。
+
+### 真实实验里 一轮有多少次调用
+
+这里“一轮”指一条用户指令到原生 Agent 本轮结束,“一次调用”指一次业务模型请求。一次模型响应可以包含多个工具调用,路由判断请求另行计数。
+
+| 真实实验样本 | 观察到的调用次数 |
+| --- | --- |
+| 写稿、审稿、修订与终审 | 正常完成的 42 轮共 310 次;每轮 3–19 次,中位数 5.5 次,平均 7.38 次 |
+| Astra/Qwen 无人值守 HTTPX 编码 | 一条初始任务指令共 40 次请求:Qwen 25 次、Astra 15 次 |
+| 同一 HTTPX 任务最后的稳定 Astra 执行段 | 连续 14 次请求全部接受,首次重新判断后,13 次继承已有决策 |
+
+写作数据来自[逐轮结构化记录](../experiments/results/decision-routing-20261006/results.json),涉及 Qwen 3.8 Max 与 GLM 5.3;全部登记 48 轮,另 6 个中断轮次的 51 次请求单独保留。HTTPX 来自[已有编码实验](../experiments/cases/unattended-engineering/README.md),整轮含两次压缩、30 次接受和 10 次取消,因此采用最后稳定的 14 次调用作连续执行示例。
+
+HTTPX 全部请求中,**35/40 继承决策,占 87.5%**。这展示了保护机制的实际作用:多数工具往返复用已有选择。87.5% 是决策复用率;前表的 6.27 个百分点则是将真实观察到的 14 次调用代入假设输入规模后的推算,两者都不是实测缓存命中率。该稳定片段始于压缩后的重选,用来说明连续调用的长度,不是新用户消息切换的直接对照。
+
+这些样本给出的体感是:写作一轮往往有数次请求,长编码任务可以有十几次连续请求。任务类型不同,摊薄空间也不同。
+
+## 压缩后切换 为什么额外损失可能更小
+
+压缩会把部分历史替换成摘要。继续原模型也需要处理这份新内容,但系统提示词、工具定义等未变前缀可能仍可命中。例如,Claude 的[压缩与缓存说明](https://platform.claude.com/docs/en/build-with-claude/compaction-threshold#maximizing-cache-hits-with-system-prompts)介绍了压缩后继续复用系统提示词缓存的方式。
+
+因此,比较压缩后“保持原模型”和“切换模型”,只应把**原本还能命中、目标模型却没有的前缀**记作额外损失 `R`。摘要和新增消息是两条路径都需要处理的内容。
+
+沿用前面的单次切换模型,**阶段命中率下降 = R / T**。若阶段累计输入为 100 万 token,损失 10 万 token 前缀对应下降 10 个百分点,损失 1 万则对应 1 个百分点。这里固定总输入作对照;实际压缩也会改变 `T`。
+
+这就是两个保护环节的分工:**连续执行减少损失发生的次数,压缩边界有机会减小每次损失的前缀。** 保留的系统提示词和工具很长时,仍应按实际 `R` 计算;当 `R / T` 足够小,额外影响才可以近乎忽略。
+
+## 缓存损失具体值多少钱
+
+下面使用实验在 **2026 年 10 月 3 日冻结的 API 等价价格**和 **9 月 30 日固定汇率**,按标准上下文档位计算。这是历史分析口径,不是当前报价或订阅账单。
+
+| 每百万 token | Astra(美元) | Qwen 3.8 Flash(人民币) | Qwen 折合美元 |
+| --- | ---: | ---: | ---: |
+| 普通输入 | 10 | 0.8 | 0.119322 |
+| 缓存读取 | 1 | 0.1 | 0.014915 |
+| 缓存写入 | 12.5 | 1.25 | 0.186441 |
+| 输出 | 50 | 2.7 | 0.402712 |
+
+换算系数为 `1.1355 / 7.6130` 美元/人民币,计算使用完整精度;原始记录见[冻结证据包](https://github.com/higress-group/HiRoute/releases/tag/experiment-evidence-2026-10-04)的 `evidence.json → frozen_cost_basis`。
+
+先隔离缓存因素,比较同一目标模型有缓存和冷启动的差异。首次处理单价记为 `p`,缓存读取单价为 `r`,均按美元/百万 token:
+
+**额外缓存成本 K = R × (p − r) / 1,000,000。**
+
+首次处理按普通输入还是缓存写入收费,取决于服务方;两种价格是替代计费情形,不重复相加。
+
+| 目标模型与损失前缀 | 按普通输入处理的额外费用 | 按缓存写入处理的额外费用 |
+| --- | ---: | ---: |
+| Astra,100,000 token | $0.900 | $1.150 |
+| Astra,10,000 token | $0.090 | $0.115 |
+| Qwen,100,000 token | 约 $0.010441 | 约 $0.017153 |
+| Qwen,10,000 token | 约 $0.001044 | 约 $0.001715 |
+
+损失 1 万 token 前缀的 Astra 切换,按写入价多花 $0.115;分摊到 20 次连续调用,平均每次 $0.00575。总开销仍是 $0.115,而完整 10 万 token 前缀的损失可达 $1.15。这说明保持长前缀的复用有实际价值。
+
+### 切换方向也会改变费用
+
+跨模型比较还要计入模型单价变化。新用户消息到来时,旧前缀这部分的费用变化为:
+
+**旧前缀费用变化 = P ×(目标模型首次处理价 − 原模型缓存读取价)/ 1,000,000。**
+
+取 `P = 100,000`,目标模型首次处理按缓存写入价计费:
+
+| 切换方向 | 继续原模型读取旧前缀 | 切换后重建旧前缀 | 费用变化 |
+| --- | ---: | ---: | ---: |
+| Astra → Qwen | $0.100000 | 约 $0.018644 | 减少约 $0.081356 |
+| Qwen → Astra | 约 $0.001492 | $1.250000 | 增加约 $1.248508 |
+
+**同样损失缓存,转到 Qwen 的旧前缀处理费仍可能更低;升级 Astra 则为更强能力付费。** 这张表包含跨模型价格差,不能再叠加前表的缓存损失。新输入、输出、路由判断和返工还需进入整单计算。
+
+## 把缓存开销放回整单成本
+
+[Astra × Qwen 实验](https://hiroute.ai/news/astra-qwen-smart-routing/)让 Qwen 完成六份资料核验,Astra 处理关键决策。三个配对运行中,两对双方均达到原定整单验收标准;第一对未达标的混合结果和全部记录保留在[实验说明](../experiments/cases/research-cost-quality/README.md)。这是分任务成本对照,用于检验费用余量。
+
+整单金额包含全部受试尝试、结构性重试与已报告的 Jev 判断费用。未报告的缓存写入量按区间处理,所以采用混合成本上界对比全 Astra 下界;普通输入、缓存读取、缓存写入互不重复,输出已包含计费的推理 token。
+
+| 双方均达标的运行 | 混合成本上界 | 全 Astra 成本下界 | 保守降幅 |
+| --- | ---: | ---: | ---: |
+| 第 2 对 | $0.146806 | $1.838882 | 92.01% |
+| 第 3 对 | $0.177957 | $2.067170 | 91.39% |
+
+### 即使全 Astra 的输入全部命中缓存
+
+进一步给全 Astra 一个理想条件:所有输入按缓存读取价计费,输出保留实际用量;混合模式仍取已记录的成本上界。这是固定用量后的敏感性分析,检验缓存折扣能否改变结论。
+
+全 Astra 两次均有 116,383 个输入 token,输出分别为 34,214 和 34,310 个。因此:
+
+`第 2 对:(116,383 × 1 + 34,214 × 50) / 1,000,000 = $1.827083`
+
+`第 3 对:(116,383 × 1 + 34,310 × 50) / 1,000,000 = $1.831883`
+
+| 运行 | 全 Astra 全输入命中的假设成本 | 混合成本上界 | 仍节省 |
+| --- | ---: | ---: | ---: |
+| 第 2 对 | $1.827083 | $0.146806 | 91.96% |
+| 第 3 对 | $1.831883 | $0.177957 | 90.28% |
+
+两次任务的成本优势仍然存在:输入缓存不会免除输出费用,而冻结价格下 Qwen 的普通输入价也低于 Astra 的缓存读取价。计算保留各模型实际的不同输出用量,降幅使用完整精度并向下截取至两位百分数。
+
+### 还能容纳多少新增开销
+
+以上述理想化的全 Astra 成本 `C_A*` 对比混合上界 `C_M`,额外开销的盈亏平衡条件是:
+
+**新增开销 X < C_A* − C_M。**
+
+两次运行分别还有约 **$1.680277** 和 **$1.653926** 的余量。额外增加一次 $0.115 的 Astra 前缀写入损失,仍不足以抵消本例的收益。`X` 只计原账目之外新增的缓存重建、判断或返工费用,避免重复扣除。
+
+调用次数、费用和假设比较均可通过[离线计算入口](../experiments/cache_cost_analysis.py)复算,步骤见[实验复算说明](../experiments/README.md)。记录支持这些具体任务的结论;实际任务仍需用自己的上下文、调用量与交付质量核算。
+
+## 经济模型持续执行 在需要时升级主力
+
+HiRoute 的目标,是让适合经济模型的工作尽可能由经济模型持续完成,在任务需要更强能力时交给主力模型。任务复杂时可以从一开始就使用主力,经济模型表现不足时也可以升级,避免为了省单次调用费用而反复返工。
+
+阶段内保持模型稳定,持续积累缓存收益;重建边界重新判断,减少可避免的缓存损失;必要时升级能力,守住交付标准。**只要经济模型承担工作所节省的费用,超过额外缓存重建、路由判断与返工开销,智能路由就有总成本优势。**
diff --git a/news/README.md b/news/README.md
index 8e4c7d11..1311daa4 100644
--- a/news/README.md
+++ b/news/README.md
@@ -2,9 +2,12 @@
| Date | English | 简体中文 |
| --- | --- | --- |
+| 2026-10-14 | [Model switching, caching, and costs](2026-10-14-routing-cache-cost.en.md) | [模型切换的缓存影响与成本分析](2026-10-14-routing-cache-cost.zh-CN.md) |
| 2026-10-04 | [Smart routing: Astra × Qwen](2026-10-04-astra-qwen.en.md) | [智能路由省钱实战](2026-10-04-astra-qwen.zh-CN.md) |
| — | [HiRoute is open source](hiroute-open-source.en.md) | [HiRoute 正式开源](hiroute-open-source.zh-CN.md) |
Articles here are the canonical source for the bilingual website Articles pages. Add a dated Markdown pair and an entry in `index.json` for future posts. A localized `shortTitle` is optional for the compact homepage link.
-Omit `date` for a backfilled introduction whose original publication date is not recorded. It follows dated news in the article list and does not replace the latest dated article on the homepage. The open-source introduction is such a backfill; the existing Astra × Qwen case study remains the homepage entry.
+Omit `date` for a backfilled introduction whose original publication date is not recorded. It follows dated news in the article list and does not replace the latest dated article on the homepage. The open-source introduction is such a backfill.
+
+The cache-cost article uses frozen experiment prices and usage. Its conditional cache model and post-hoc cost sensitivity analysis are distinct from measured cache-hit rates. Recalculate its figures with `python3 experiments/cache_cost_analysis.py` from the repository root after fetching the public evidence archive; keep historical evidence unchanged.
diff --git a/news/index.json b/news/index.json
index 8b90f284..492fe1c0 100644
--- a/news/index.json
+++ b/news/index.json
@@ -1,4 +1,24 @@
[
+ {
+ "slug": "routing-cache-cost",
+ "date": "2026-10-14",
+ "title": {
+ "zh": "模型切换的缓存影响与智能路由成本分析",
+ "en": "Model Switching Cache Effects and Smart Routing Costs"
+ },
+ "shortTitle": {
+ "zh": "模型切换、缓存与整单成本",
+ "en": "Model switching, caching, and task cost"
+ },
+ "description": {
+ "zh": "解释 HiRoute 如何保护连续调用的缓存复用,用 Astra 与 Qwen 3.8 Flash 的实验价格和用量计算缓存损失、摊薄条件与整单成本余量。",
+ "en": "How HiRoute protects cache reuse, with calculations of cache losses, amortization, and whole-task cost margins using the Astra and Qwen 3.8 Flash experiment."
+ },
+ "files": {
+ "zh": "2026-10-14-routing-cache-cost.zh-CN.md",
+ "en": "2026-10-14-routing-cache-cost.en.md"
+ }
+ },
{
"slug": "astra-qwen-smart-routing",
"date": "2026-10-04",