The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.
merge: preserve cache guidance source 3173 with measured cost boundaries
haelyra0aebb36af29f
diff --cc manifests/install-modules.jsonindex 97761d2c6,884c7d39d..795458eda--- a/manifests/install-modules.json+++ b/manifests/install-modules.json@@@ -923,6 -843,6 +923,7 @@@"skills/dynamic-workflow-mode","skills/enterprise-agent-ops","skills/nanoclaw-repl",++ "skills/prompt-caching-strategy","skills/prompt-optimizer","skills/ralphinho-rfc-pipeline","skills/regex-vs-llm-structured-text",diff --cc package.jsonindex d11033a7d,70a31ff8d..933962b14--- a/package.json+++ b/package.json@@@ -316,6 -304,6 +316,7 @@@"skills/prediction-market-oracle-research/","skills/prediction-market-risk-review/","skills/project-flow-ops/",++ "skills/prompt-caching-strategy/","skills/prompt-optimizer/","skills/python-patterns/","skills/python-testing/",diff --cc skills/prompt-caching-strategy/SKILL.mdindex 000000000,fca3d8d3d..e075debd5mode 000000,100644..100644--- a/skills/prompt-caching-strategy/SKILL.md+++ b/skills/prompt-caching-strategy/SKILL.md@@@ -1,0 -1,147 +1,187 @@@+ ---+ name: prompt-caching-strategy+ description: >-+ Optimize LLM prompt caching hit rate to reduce API costs and improve latency.+ Analyzes prompt structure, identifies cacheable vs non-cacheable content,+ and recommends restructuring to maximize cache reuse.+ TRIGGER when: user says "prompt cache", "cache hit rate", "reduce LLM cost",+ "optimize token usage", "cheaper LLM calls", or asks about prompt caching+ strategies for OpenAI/Anthropic/other providers.+ DO NOT TRIGGER when: user just wants to optimize a single prompt's quality+ (use prompt-optimizer instead), or asks about general caching infrastructure.+ metadata:+ origin: community+ author: cyberspace-cs+ version: "1.0.0"+ ---++ # Prompt Caching Strategy+-Optimize your LLM application's prompt caching hit rate to cut costs by 50-90%-and reduce latency.++Arrange reusable prompt content to improve cache reuse, then measure input cost++and latency. Savings depend on model, platform, cache eligibility, writes,++retention, reuse and output volume. Runtime cache hits and savings for this skill++are **unmeasured**; the examples below validate arithmetic without provider calls.+-## Core Principles++## When to Use+-### What is Prompt Caching++Use for repeated API requests that share authorized instructions, tools or++reference material. First identify the exact provider endpoint, model, service++tier and caching mode. Consult its current documentation before selecting rates,++minimum prefix length or retention. Caching does not reduce context-window use.+-Providers (OpenAI, Anthropic, Google) cache prefixes of prompts. If the-beginning of your prompt matches a previous call, the provider reuses the-cached computation instead of re-processing it. This means:-- **Cheaper**: cached tokens cost 50-90% less-- **Faster**: cached prefixes skip processing time++## How It Works+-### The Golden Rule++### Preserve a stable prefix+-> **Cacheable content goes FIRST. Variable content goes LAST.**--The cache matches from the start of the prompt. Any change in the middle-invalidates the entire cache from that point onward.--## Prompt Structure for Maximum Cache Hit--### Optimal Ordering++Keep reusable instructions, tool schemas and examples stable; place changing++questions, timestamps and retrieved results after that content where the API++allows. Respect the provider's serialization order: Claude uses tools, system,++then messages. Matching prefixes need identical content, including tool schemas++and relevant request settings. A changed prefix can prevent reuse from that++point. A dynamic suffix may itself be cached later if reused unchanged.+-```-1. System prompt (static) ← cacheable-2. Tool definitions (static) ← cacheable-3. Few-shot examples (stable) ← cacheable-4. User context (slowly changing) ← partially cacheable-5. Current user input (dynamic) ← NOT cacheable++```text++Reusable prefix: approved tools, instructions, stable examples/reference++Changing suffix: current question, fresh retrieval results, request metadata+ ```+-### Anti-Pattern: Bad Ordering++Choose only relevant context. Padding prompts or broadening access just to meet++an eligibility threshold needs its own cost and quality justification.+-```-❌ System prompt at the end-❌ User input before tool definitions-❌ Timestamp or random ID early in the prompt-❌ Changing examples every call-```++### Preserve user and workspace boundaries+-## What to Cache vs What Not to Cache++Authorize content access before cache lookup or reuse. Scope application-managed++cache handles and accounting keys to the approved user, workspace, provider++account, model and content revision. Share only material explicitly approved for++that scope. A cache key is a routing/accounting hint, not an authorization gate++or a sandbox. Provider cache isolation does not replace application access checks.++Invalidate application references when permissions or content change; follow++provider retention/deletion controls. Avoid logging private prompt contents.++Context selection, capabilities, sandbox enforcement and evidence remain++independent of cache optimization.+-### High-Value Cache Content++## Provider Requirements and Usage Evidence+-| Content Type | Cache Worth | Notes |-| --- | --- | --- |-| System instructions | ⭐⭐⭐⭐⭐ | Always keep first |-| Tool/function definitions | ⭐⭐⭐⭐⭐ | Static, large, high reuse |-| Few-shot examples | ⭐⭐⭐⭐ | Keep stable, don't vary |-| Preamble / preamble text | ⭐⭐⭐⭐ | Brand voice, rules, constraints |-| Retrieval context (same session) | ⭐⭐⭐ | Cache per session, not global |--### Low-Value / Anti-Cache Content--| Content Type | Why Not Cache |-| --- | --- |-| Timestamps / dates | Changes every call, invalidates cache |-| Random IDs / nonces | Same issue |-| User-specific variable data | Changes per user |-| Real-time search results | Changes per query |-| Very short prompts | Overhead exceeds savings |--## Optimization Checklist--Before shipping your LLM application, run through this:--- [ ] **System prompt is at the very start** — nothing variable before it-- [ ] **Tool definitions are static** — not regenerated per call-- [ ] **Few-shot examples are stable** — not randomly selected-- [ ] **No timestamps/IDs early in the prompt** — move them to the end-- [ ] **Dynamic content is at the end** — user input, current context-- [ ] **Prompt length > 1024 tokens** — below this, caching overhead may not be worth it-- [ ] **High call frequency** — caching saves more when you call the same prompt many times-- [ ] **Provider supports caching** — OpenAI/Anthropic/Google all do, but check specifics--## Provider-Specific Notes--### OpenAI-- Caches the prefix automatically (no explicit API call needed)-- Minimum cacheable prefix: ~1024 tokens-- Cached tokens cost 50% less-- Cache TTL: ~5-10 minutes (auto-evicted)--### Anthropic-- Explicit caching with `cache_control` breakpoints-- You control exactly what gets cached-- Minimum cacheable prefix: ~1024 tokens-- Cached tokens cost 90% less (!!)-- Cache TTL: 5 minutes (default), extendable--### Google Gemini-- Automatic prefix caching-- Similar economics to OpenAI-- Check docs for current pricing details--## Cost Savings Calculator--Estimate your potential savings:++The linked documentation is authoritative; endpoint and SDK versions can expose++different usage field names. Record that version alongside counters.+++| Provider | Requirements to verify | Counters to record |++| --- | --- | --- |++| OpenAI | Model-specific automatic/explicit modes, minimum length, retention and write/read prices; earlier and newer models differ | Responses `usage.input_tokens_details.cached_tokens` and, when present, `cache_write_tokens`; input and output totals. Chat Completions uses `prompt_tokens_details.cached_tokens` |++| Claude | Supported platform, model minimum, `cache_control` support and TTL; cache writes may cost more than ordinary input | `input_tokens`, `cache_creation_input_tokens`, `cache_read_input_tokens`, `output_tokens`; split writes by TTL when rates differ |++| Gemini | Model minimum and API: Interactions supports implicit caching; explicit cache objects require GenerateContent | Interactions `usage.total_cached_tokens`; GenerateContent `usage_metadata` cache/input/output counters plus explicit-cache token count and retained duration |++++Use [OpenAI's prompt caching guide](https://developers.openai.com/api/docs/guides/prompt-caching)++and [pricing](https://developers.openai.com/api/docs/pricing/) for the selected++model. For Chat Completions, check the [usage schema](https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create).++Derive ordinary input from the total minus cached reads and any separately++reported writes; do not count the same tokens twice.++++Use [Claude's prompt caching guide](https://platform.claude.com/docs/en/build-with-claude/prompt-caching)++for current model/platform limits and TTL rates. Its `input_tokens` counts++ordinary input separately from cache reads and writes. A short prefix can produce++zero cache counters even when marked for caching. Do not assume every Claude++model or hosted platform uses the same minimum or read price.++++Use [Gemini's caching guide](https://ai.google.dev/gemini-api/docs/caching),++[GenerateContent caching guide](https://ai.google.dev/gemini-api/docs/generate-content/caching)++and [pricing](https://ai.google.dev/gemini-api/docs/pricing). Explicit-cache++storage is charged by token count and retained duration, separately from reads,++ordinary input and output. Implicit hits are not guaranteed. Verify billing++categories for the chosen endpoint rather than copying another provider's rates.++++## Cost Comparison Calculator++++Normalize actual usage into nonoverlapping categories and use prices in the same++currency per million tokens. Split write categories when TTL, modality, tier or++model prices differ. Storage uses million-token-hours; other applicable charges++such as tools and grounding need their own lines. Include all warmup, miss,++expiration, refresh and retry requests in the measured period.++++```text++baseline = (ordinary + written + read) * ordinary_input_rate / 1e6++ + output * output_rate / 1e6++actual = ordinary * ordinary_input_rate / 1e6++ + written * cache_write_rate / 1e6++ + read * cache_read_rate / 1e6++ + output * output_rate / 1e6++ + storage_token_hours * storage_rate / 1e6++savings = baseline - actual+ ```-Current monthly cost: $X-Average prompt length: Y tokens-Cache hit rate after optimization: Z%-Cached token discount: D% (e.g., 50% for OpenAI, 90% for Anthropic)+-Estimated monthly savings = $X × (Y / total_tokens) × Z% × D%++This counterfactual holds prompt and output volumes fixed. For explicit-cache++creation, include any separately billed creation input in the write category;++use the endpoint's documented rate. Set nonexistent categories to zero. Compare++actual before/after invoices separately if warmup requests or output differ.++Never apply a cache discount to the total bill or to output tokens.++++The executable calculator takes normalized counters, not raw provider responses:++++```javascript++function compareCacheCost(usage, rates) {++ const counts = ['uncached', 'written', 'read', 'output', 'storageTokenHours'];++ const prices = ['input', 'write', 'read', 'output', 'storage'];++ for (const [values, keys] of [[usage, counts], [rates, prices]]) {++ for (const key of keys) {++ if (!Number.isFinite(values[key]) || values[key] < 0) {++ throw new Error(`${key} must be a finite nonnegative number`);++ }++ }++ }++ const outputCost = usage.output * rates.output / 1e6;++ const storageCost = usage.storageTokenHours * rates.storage / 1e6;++ const baseline = (usage.uncached + usage.written + usage.read)++ * rates.input / 1e6 + outputCost;++ const actual = (usage.uncached * rates.input + usage.written * rates.write++ + usage.read * rates.read) / 1e6 + outputCost + storageCost;++ return { baseline, actual, savings: baseline - actual, outputCost, storageCost };++}+ ```+-Example:-- $1000/month bill, 60% of tokens are cacheable prefix-- After optimization: 80% cache hit rate, 90% discount (Anthropic)-- Savings = $1000 × 60% × 80% × 90% = **$432/month saved**--## When NOT to Use Caching--Caching is not always worth it:-- **Very short prompts** (< 1000 tokens): overhead > savings-- **One-off calls**: cache never gets reused-- **Rapidly changing content**: cache miss rate too high-- **Low traffic apps**: total dollar savings too small to matter++## Examples++++These are hypothetical prices and counts, not provider quotes or measured hits.++Ten calls each contain a 10,000-token stable prefix, 2,000-token changing suffix++and 2,000 output tokens. Assume one prefix write and nine full prefix reads.++++| Category | Tokens | Example rate per million | Cost |++| --- | ---: | ---: | ---: |++| Ordinary input | 20,000 | $3.00 | $0.0600 |++| Cache write | 10,000 | $3.75 | $0.0375 |++| Cache read | 90,000 | $0.30 | $0.0270 |++| Output | 20,000 | $15.00 | $0.3000 |++| Storage | 0 token-hours | $0.00 | $0.0000 |++++Baseline: 120,000 input tokens at $3 plus the same output = **$0.6600**.++With caching: **$0.4245**, savings **$0.2355**. The write premium is included;++the $0.3000 output bill remains unchanged. An output-heavy workload has the same++absolute input savings under these assumptions, a smaller fraction of total cost.++++If an explicit cache retains 10,000 tokens for two hours at a hypothetical++$1 per million-token-hour, add **$0.0200** storage. Actual cost becomes++**$0.4445**. Use the selected Gemini endpoint's prices for a real estimate.++++For a one-off call, that prefix write costs $0.0375 rather than $0.0300 ordinary++input: **$0.0075 more**, with no read savings. For a 500-token prefix below a++selected model's documented minimum, treat the request as uncached; claiming++future hits without counters would overstate savings. Repeated expiry or changing++prefixes can also turn expected savings into higher cost.++++## Verification Workflow++++1. Capture eligible prefix size and model/platform requirements from current docs.++2. Run the offline example and edge cases with++ `node tests/skills/prompt-caching-strategy.test.js` in the ECC repository.++3. If provider traffic is authorized, compare matched requests and aggregate++ actual cached-token counts, all input categories, output, storage, latency and++ realized cost by approved user/workspace. Record misses and rewrites too.++4. Report cached reads divided by total input tokens as a token hit rate; report++ request hit rate separately. Keep denominators and sampling period explicit.++5. If no traffic was measured, label hit rate, latency improvement and realized++ savings unmeasured. Offline arithmetic establishes no runtime cache behavior.++ ## Relationship to Other Skills+-- **prompt-optimizer**: Optimizes prompt quality/clarity. Use this first to get- a good prompt, then use this skill to optimize its cacheability.-- **context-budget**: Analyzes how much context your session uses. Use that to- find bloat; use this skill to make the remaining context cheaper.++- **prompt-optimizer** improves prompt quality before optimizing reuse.++- **context-budget** identifies unnecessary context; caching still consumes it.++- **cost-aware-llm-pipeline** compares caching with routing, retries and other costs.diff --cc tests/skills/prompt-caching-strategy.test.jsindex 000000000,000000000..ddc37627fnew file mode 100644--- /dev/null+++ b/tests/skills/prompt-caching-strategy.test.js@@@ -1,0 -1,0 +1,67 @@@++'use strict';++++const assert = require('assert');++const fs = require('fs');++const path = require('path');++const vm = require('vm');++++const root = path.resolve(__dirname, '../..');++const source = fs.readFileSync(path.join(root, 'skills/prompt-caching-strategy/SKILL.md'), 'utf8');++const snippet = source.match(/```javascript\n([\s\S]*?)\n```/);++assert.ok(snippet, 'The documented calculator must be executable offline');++const context = vm.createContext({});++vm.runInContext(`${snippet[1]}\nthis.calculate = compareCacheCost;`, context);++const calculate = context.calculate;++const rates = { input: 3, write: 3.75, read: 0.3, output: 15, storage: 1 };++const usage = { uncached: 20000, written: 10000, read: 90000, output: 20000, storageTokenHours: 0 };++const near = (actual, expected) => assert.ok(Math.abs(actual - expected) < 1e-10, `${actual} != ${expected}`);++let passed = 0;++function test(name, fn) {++ fn();++ passed += 1;++ console.log(`PASS ${name}`);++}++++test('ten-request example charges a write premium and unchanged output', () => {++ const costs = calculate(usage, rates);++ near(costs.baseline, 0.66);++ near(costs.actual, 0.4245);++ near(costs.savings, 0.2355);++ near(costs.outputCost, 0.3);++});++test('one-off cache write increases the bill', () => {++ const costs = calculate({ ...usage, uncached: 2000, written: 10000, read: 0, output: 2000 }, rates);++ near(costs.savings, -0.0075);++});++test('ineligible short prefix has no cache reads or writes', () => {++ const costs = calculate({ ...usage, uncached: 5000, written: 0, read: 0, output: 1000 }, rates);++ near(costs.savings, 0);++});++test('storage token-hours add cost separately', () => {++ const costs = calculate({ ...usage, storageTokenHours: 20000 }, rates);++ near(costs.storageCost, 0.02);++ near(costs.actual, 0.4445);++ near(costs.savings, 0.2155);++});++test('output-heavy workload does not multiply savings by the whole bill', () => {++ const costs = calculate({ ...usage, output: 1000000 }, rates);++ near(costs.outputCost, 15);++ near(costs.savings, 0.2355);++});++test('expiration/rewrite cost is charged on every write', () => {++ const costs = calculate({ ...usage, written: 100000, read: 0 }, rates);++ near(costs.savings, -0.075);++});++test('missing or invalid usage and rates fail clearly', () => {++ for (const invalid of [-1, NaN, Infinity, '3', undefined]) {++ assert.throws(() => calculate({ ...usage, read: invalid }, rates), /nonnegative/);++ assert.throws(() => calculate(usage, { ...rates, storage: invalid }), /nonnegative/);++ }++});++test('canonical module and package both distribute the skill', () => {++ const modules = JSON.parse(fs.readFileSync(path.join(root, 'manifests/install-modules.json'), 'utf8'));++ const pkg = JSON.parse(fs.readFileSync(path.join(root, 'package.json'), 'utf8'));++ assert.ok(modules.modules.find(module => module.id === 'agentic-patterns').paths.includes('skills/prompt-caching-strategy'));++ assert.ok(pkg.files.includes('skills/prompt-caching-strategy/'));++});++console.log(`${passed}/${passed} offline prompt caching checks passed; runtime cache hits unmeasured.`);