The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.
fix(lib): correct stale model rates in the shared cost estimator and skill
Andres Tuul39416e3cdde4
diff --git a/docs/ja-JP/skills/cost-aware-llm-pipeline/SKILL.md b/docs/ja-JP/skills/cost-aware-llm-pipeline/SKILL.mdindex 97e95f06d..adb4b532c 100644--- a/docs/ja-JP/skills/cost-aware-llm-pipeline/SKILL.md+++ b/docs/ja-JP/skills/cost-aware-llm-pipeline/SKILL.md@@ -155,9 +155,9 @@ def process(text: str, config: Config, tracker: CostTracker) -> tuple[Result, Co| モデル | 入力($/1Mトークン) | 出力($/1Mトークン) | 相対コスト ||-------|---------------------|----------------------|---------------|-| Haiku 4.5 | $0.80 | $4.00 | 1x |-| Sonnet 4.6 | $3.00 | $15.00 | 約4x |-| Opus 4.5 | $15.00 | $75.00 | 約19x |+| Haiku 4.5 | $1.00 | $5.00 | 1x |+| Sonnet 4.6 | $3.00 | $15.00 | 3x |+| Opus 4.5 | $5.00 | $25.00 | 5x |## ベストプラクティスdiff --git a/docs/zh-CN/skills/cost-aware-llm-pipeline/SKILL.md b/docs/zh-CN/skills/cost-aware-llm-pipeline/SKILL.mdindex 9af5a8466..396171570 100644--- a/docs/zh-CN/skills/cost-aware-llm-pipeline/SKILL.md+++ b/docs/zh-CN/skills/cost-aware-llm-pipeline/SKILL.md@@ -155,9 +155,9 @@ def process(text: str, config: Config, tracker: CostTracker) -> tuple[Result, Co| 模型 | 输入(美元/百万令牌) | 输出(美元/百万令牌) | 相对成本 ||-------|---------------------|----------------------|---------------|-| Haiku 4.5 | $0.80 | $4.00 | 1x |-| Sonnet 4.6 | $3.00 | $15.00 | ~4x |-| Opus 4.5 | $15.00 | $75.00 | ~19x |+| Haiku 4.5 | $1.00 | $5.00 | 1x |+| Sonnet 4.6 | $3.00 | $15.00 | 3x |+| Opus 4.5 | $5.00 | $25.00 | 5x |## 最佳实践diff --git a/scripts/lib/cost-estimate.js b/scripts/lib/cost-estimate.jsindex a1651a8c9..0dfa9c834 100644--- a/scripts/lib/cost-estimate.js+++ b/scripts/lib/cost-estimate.js@@ -3,18 +3,51 @@/*** Shared cost estimation for ECC hooks.*- * Approximate per-1M-token blended rates (conservative defaults).+ * Published per-1M-token rates. Input and output only: callers pass two token+ * counts, so cache tiers are out of scope here.*/+// The previous table priced every `opus` model at $15/$75, which are Claude 3+// Opus era rates. Opus 4.5 and later bill at $5/$25, so every current-+// generation Opus estimate was exactly 3x real spend. `haiku` was $0.80/$4.00,+// which is Claude 3.5 Haiku: Haiku 4.5 bills at $1/$5, so Haiku was understated+// 1.25x. Fable and Mythos had no bucket at all and fell through to `sonnet`,+// understating them 3.3x.+//+// The legacy rows exist so that correcting the current generation does not+// reprice the old one. Claude 3 Haiku ($0.25/$1.25) is deliberately not+// modelled: Claude Code never ran it.const RATE_TABLE = {- haiku: { in: 0.8, out: 4.0 },+ haiku: { in: 1.0, out: 5.0 },+ haikuLegacy: { in: 0.8, out: 4.0 },sonnet: { in: 3.0, out: 15.0 },- opus: { in: 15.0, out: 75.0 }+ opus: { in: 5.0, out: 25.0 },+ opusLegacy: { in: 15.0, out: 75.0 },+ fable: { in: 10.0, out: 50.0 }};+// The only Opus models that really billed at $15/$75: Claude 3 Opus, Opus 4.0+// and Opus 4.1. Every spelling of each has to match, alias and dated snapshot+// alike, which is why the bare `opus-4-<date>` form is listed on its own:+// Opus 4.0's snapshot is `claude-opus-4-20250514`, with no minor segment, so+// an `opus-4-0` substring alone misses it and reprices a legacy estimate at a+// third of its real cost. The `[-@]` covers Vertex AI, which joins the date+// with `@` (`claude-opus-4@20250514`); Bedrock's+// `anthropic.claude-3-opus-20240229-v1:0` is caught by the first alternative.+//+// Opus 4.5 through Opus 5 are $5/$25 and take the default bucket, which also+// means a future Opus is assumed to be $5/$25. That assumption is the same+// shape of silent staleness this change fixes, so if Opus is ever repriced+// again, a new row belongs here rather than a rediscovery of this comment.+const LEGACY_OPUS_RE = /claude-3-opus|opus-4-0(?!\d)|opus-4-1(?!\d)|opus-4[-@]\d{8}/;++// Claude 3.5 Haiku, whose $0.80/$4.00 this table used to apply to all Haiku.+const LEGACY_HAIKU_RE = /3-5-haiku|haiku-3-5/;+/*** Estimate USD cost from token counts.- * @param {string} model - Model name (may contain "haiku", "sonnet", or "opus")+ * @param {string} model - Model name (may contain "haiku", "sonnet", "opus",+ * "fable" or "mythos"); anything else is priced at sonnet rates.* @param {number} inputTokens* @param {number} outputTokens* @returns {number} Estimated cost in USD (rounded to 6 decimal places)@@ -22,8 +55,13 @@ const RATE_TABLE = {function estimateCost(model, inputTokens, outputTokens) {const normalized = String(model || '').toLowerCase();let rates = RATE_TABLE.sonnet;- if (normalized.includes('haiku')) rates = RATE_TABLE.haiku;- if (normalized.includes('opus')) rates = RATE_TABLE.opus;+ if (normalized.includes('haiku')) {+ rates = LEGACY_HAIKU_RE.test(normalized) ? RATE_TABLE.haikuLegacy : RATE_TABLE.haiku;+ } else if (normalized.includes('fable') || normalized.includes('mythos')) {+ rates = RATE_TABLE.fable;+ } else if (normalized.includes('opus')) {+ rates = LEGACY_OPUS_RE.test(normalized) ? RATE_TABLE.opusLegacy : RATE_TABLE.opus;+ }const cost = (inputTokens / 1_000_000) * rates.in + (outputTokens / 1_000_000) * rates.out;return Math.round(cost * 1e6) / 1e6;diff --git a/skills/cost-aware-llm-pipeline/SKILL.md b/skills/cost-aware-llm-pipeline/SKILL.mdindex 139d10985..63b10f6b3 100644--- a/skills/cost-aware-llm-pipeline/SKILL.md+++ b/skills/cost-aware-llm-pipeline/SKILL.md@@ -156,9 +156,9 @@ def process(text: str, config: Config, tracker: CostTracker) -> tuple[Result, Co| Model | Input ($/1M tokens) | Output ($/1M tokens) | Relative Cost ||-------|---------------------|----------------------|---------------|-| Haiku 4.5 | $0.80 | $4.00 | 1x |-| Sonnet 4.6 | $3.00 | $15.00 | ~4x |-| Opus 4.5 | $15.00 | $75.00 | ~19x |+| Haiku 4.5 | $1.00 | $5.00 | 1x |+| Sonnet 4.6 | $3.00 | $15.00 | 3x |+| Opus 4.5 | $5.00 | $25.00 | 5x |## Best Practicesdiff --git a/tests/lib/cost-estimate.test.js b/tests/lib/cost-estimate.test.jsindex bcb5906bc..2aa60770f 100644--- a/tests/lib/cost-estimate.test.js+++ b/tests/lib/cost-estimate.test.js@@ -31,16 +31,25 @@ function runTests() {console.log('RATE_TABLE:');if (- test('RATE_TABLE has haiku, sonnet, opus keys', () => {- assert.ok(RATE_TABLE.haiku, 'Missing haiku');- assert.ok(RATE_TABLE.sonnet, 'Missing sonnet');- assert.ok(RATE_TABLE.opus, 'Missing opus');- assert.strictEqual(typeof RATE_TABLE.haiku.in, 'number');- assert.strictEqual(typeof RATE_TABLE.haiku.out, 'number');- assert.strictEqual(typeof RATE_TABLE.sonnet.in, 'number');- assert.strictEqual(typeof RATE_TABLE.sonnet.out, 'number');- assert.strictEqual(typeof RATE_TABLE.opus.in, 'number');- assert.strictEqual(typeof RATE_TABLE.opus.out, 'number');+ test('RATE_TABLE has a bucket per billing tier', () => {+ for (const key of ['haiku', 'haikuLegacy', 'sonnet', 'opus', 'opusLegacy', 'fable']) {+ assert.ok(RATE_TABLE[key], `Missing ${key}`);+ assert.strictEqual(typeof RATE_TABLE[key].in, 'number', `${key}.in not a number`);+ assert.strictEqual(typeof RATE_TABLE[key].out, 'number', `${key}.out not a number`);+ }+ })+ )+ passed++;+ else failed++;++ if (+ test('RATE_TABLE carries current published rates, not Claude 3 era rates', () => {+ assert.deepStrictEqual(RATE_TABLE.opus, { in: 5.0, out: 25.0 });+ assert.deepStrictEqual(RATE_TABLE.opusLegacy, { in: 15.0, out: 75.0 });+ assert.deepStrictEqual(RATE_TABLE.haiku, { in: 1.0, out: 5.0 });+ assert.deepStrictEqual(RATE_TABLE.haikuLegacy, { in: 0.8, out: 4.0 });+ assert.deepStrictEqual(RATE_TABLE.sonnet, { in: 3.0, out: 15.0 });+ assert.deepStrictEqual(RATE_TABLE.fable, { in: 10.0, out: 50.0 });}))passed++;@@ -50,9 +59,9 @@ function runTests() {console.log('\nestimateCost:');if (- test('opus 1M/1M tokens returns 90', () => {+ test('opus 1M/1M tokens returns 30', () => {const cost = estimateCost('opus', 1_000_000, 1_000_000);- assert.strictEqual(cost, 90);+ assert.strictEqual(cost, 30);}))passed++;@@ -68,9 +77,9 @@ function runTests() {else failed++;if (- test('haiku 1M/1M tokens returns 4.8', () => {+ test('haiku 1M/1M tokens returns 6', () => {const cost = estimateCost('haiku', 1_000_000, 1_000_000);- assert.strictEqual(cost, 4.8);+ assert.strictEqual(cost, 6);}))passed++;@@ -86,16 +95,74 @@ function runTests() {else failed++;if (- test('full model name claude-opus-4-6 uses opus rates', () => {+ test('full model name claude-opus-4-6 uses current opus rates', () => {const cost = estimateCost('claude-opus-4-6', 500, 200);- // (500 / 1_000_000) * 15 + (200 / 1_000_000) * 75 = 0.0075 + 0.015 = 0.0225- const expected = Math.round(0.0225 * 1e6) / 1e6;+ // (500 / 1_000_000) * 5 + (200 / 1_000_000) * 25 = 0.0025 + 0.005 = 0.0075+ const expected = Math.round(0.0075 * 1e6) / 1e6;assert.strictEqual(cost, expected);}))passed++;else failed++;+ // Every spelling of the three Opus models that really billed at $15/$75 has+ // to keep doing so. Opus 4.0's snapshot is `claude-opus-4-20250514`, with no+ // minor segment, so a bare `opus-4-0` substring misses it.+ for (const legacy of [+ 'claude-3-opus-20240229',+ 'anthropic.claude-3-opus-20240229-v1:0',+ 'claude-opus-4-20250514',+ 'claude-opus-4@20250514',+ 'claude-opus-4-0',+ 'claude-opus-4-1'+ ]) {+ if (+ test(`${legacy} keeps legacy opus rates`, () => {+ assert.strictEqual(estimateCost(legacy, 1_000_000, 1_000_000), 90);+ })+ )+ passed++;+ else failed++;+ }++ for (const current of ['claude-opus-4-5', 'claude-opus-4-7', 'claude-opus-4-8', 'claude-opus-5']) {+ if (+ test(`${current} uses current opus rates`, () => {+ assert.strictEqual(estimateCost(current, 1_000_000, 1_000_000), 30);+ })+ )+ passed++;+ else failed++;+ }++ if (+ test('claude-3-5-haiku keeps legacy haiku rates', () => {+ assert.strictEqual(estimateCost('claude-3-5-haiku-20241022', 1_000_000, 1_000_000), 4.8);+ })+ )+ passed++;+ else failed++;++ if (+ test('claude-haiku-4-5 uses current haiku rates', () => {+ assert.strictEqual(estimateCost('claude-haiku-4-5-20251001', 1_000_000, 1_000_000), 6);+ })+ )+ passed++;+ else failed++;++ // Fable and Mythos had no bucket at all and fell through to sonnet, which+ // understated them 3.3x.+ for (const model of ['claude-fable-5', 'claude-mythos-5']) {+ if (+ test(`${model} uses fable rates`, () => {+ assert.strictEqual(estimateCost(model, 1_000_000, 1_000_000), 60);+ })+ )+ passed++;+ else failed++;+ }+if (test('unknown model falls back to sonnet rates', () => {const cost = estimateCost('unknown-model', 1_000_000, 1_000_000);