AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
docs(demo): real AI-agent usage scenarios; deprecate stale sales kit
Xerj Squad A9e4b2b06d9e6
diff --git a/demo/DEMO_RUNBOOK.md b/demo/DEMO_RUNBOOK.mdindex 9fd068c0b..8e46397c6 100644--- a/demo/DEMO_RUNBOOK.md+++ b/demo/DEMO_RUNBOOK.md@@ -1,3 +1,13 @@+> ⚠️ **STALE — sales-engineer material from April 2026 (v1.0.0-rc.1).** The+> numbers here are out of date (conformance was 1305/1329; it is now 1360/1363,+> and the RC1 ingest race and unfused-hybrid caveats it warns about are fixed).+> This kit pitches XERJ to enterprise buyers; it is **not** the guide to XERJ as+> an AI-agent tool. For that — how XERJ saves an AI coding agent time and+> tokens, with reproducible measurements — see+> [`demo/agent-gate/SCENARIOS.md`](agent-gate/SCENARIOS.md) and+> [`docs/TOKEN_USAGE.md`](../docs/TOKEN_USAGE.md). Kept for reference; refresh+> the numbers before any external use.+# XERJ Demo Runbook — Enterprise AI Adoption**Audience:** sales engineers running 30–45 minute calls with prospects mid-AI transformation. Typical persona: a director of platform / head of AI infra / VP engineering at a regulated enterprise (financial services, healthcare, public sector, retail) who is actively wiring up RAG, agents, or copilots and is unhappy with the current sprawl of Elasticsearch + Pinecone/Weaviate + Splunk.diff --git a/demo/README.md b/demo/README.mdindex 09f3247fb..fbc90a83c 100644--- a/demo/README.md+++ b/demo/README.md@@ -1,3 +1,13 @@+> ⚠️ **STALE — sales-engineer material from April 2026 (v1.0.0-rc.1).** The+> numbers here are out of date (conformance was 1305/1329; it is now 1360/1363,+> and the RC1 ingest race and unfused-hybrid caveats it warns about are fixed).+> This kit pitches XERJ to enterprise buyers; it is **not** the guide to XERJ as+> an AI-agent tool. For that — how XERJ saves an AI coding agent time and+> tokens, with reproducible measurements — see+> [`demo/agent-gate/SCENARIOS.md`](agent-gate/SCENARIOS.md) and+> [`docs/TOKEN_USAGE.md`](../docs/TOKEN_USAGE.md). Kept for reference; refresh+> the numbers before any external use.+# XERJ demo kitThis folder is a complete demo kit for sales engineers pitching XERJ to enterprises adopting AI (RAG, agents, internal copilots). Everything here is reproducible — every command was run on the demo machine while authoring; every claimed number has a receipts file behind it.diff --git a/demo/agent-gate/README.md b/demo/agent-gate/README.mdindex 6237f5308..0b1641b27 100644--- a/demo/agent-gate/README.md+++ b/demo/agent-gate/README.md@@ -65,3 +65,19 @@ python3 demo/agent-gate/gate.py /tmp/gate-corpusExit status is **0 when the report is trustworthy**, not when XERJ wins.It exits non-zero only if a path produced an answer the gate could not verify— that is, if the measurement itself is broken.++## The kit++| File | What it is |+|---|---|+| `SCENARIOS.md` | Six real Claude Code scenarios with measured before/after — start here |+| `gate.py` | Analytics regime: record-heavy corpus (XERJ wins correctness, costs tokens) |+| `gate_retrieval.py` | Retrieval regime: prose/code corpus (XERJ wins recall; tokens scale with size) |+| `make_corpus.py` | Deterministic 36 MB heterogeneous corpus (logs/CSV/SQLite/docs/code) |+| `make_corpus_prose.py` | Deterministic prose/code corpus (the regime where retrieval saves tokens) |+| `RESULTS_analytics.txt` / `RESULTS_retrieval.txt` | Committed reference runs |++Both gates share one discipline: correctness before savings, savings only over+jointly-correct tasks, question-vocabulary baselines, both paths equally tuned,+corpus composition beside every ratio. See the table at the top of this file+for which historical mistake each rule prevents.diff --git a/demo/agent-gate/RESULTS.txt b/demo/agent-gate/RESULTS_analytics.txtsimilarity index 100%rename from demo/agent-gate/RESULTS.txtrename to demo/agent-gate/RESULTS_analytics.txtdiff --git a/demo/agent-gate/RESULTS_retrieval.txt b/demo/agent-gate/RESULTS_retrieval.txtnew file mode 100644index 000000000..c850ac3ba--- /dev/null+++ b/demo/agent-gate/RESULTS_retrieval.txt@@ -0,0 +1,24 @@++================================================================================+AGENT GATE — RETRIEVAL regime (prose/code corpus)+================================================================================++question needle base xerj base tok xerj tok base s xerj s+certificate MISS OK 43 78 0.00 0.01+stemming MISS OK 44 78 0.00 0.00+promote MISS MISS 15 79 0.00 0.01+fan-out OK OK 158 75 0.00 0.00+at-least-once MISS OK 22 80 0.00 0.00+vault OK OK 38 78 0.00 0.00+idempotency_key OK OK 69 79 0.00 0.00++correct: baseline 3/7 xerj 6/7++-- tokens over jointly-correct tasks only --+ baseline 265 tok xerj 232 tok -> XERJ uses 1.14x FEWER tokens++-- corpus composition --+ 53.1 KB total; 49.0 KB searchable prose/code (92.3%)+ This is the regime where retrieval savings appear: grep's candidate+ set grows with the corpus; ranked search returns one passage.+diff --git a/demo/agent-gate/SCENARIOS.md b/demo/agent-gate/SCENARIOS.mdnew file mode 100644index 000000000..0e9c8ab20--- /dev/null+++ b/demo/agent-gate/SCENARIOS.md@@ -0,0 +1,203 @@+# XERJ for an AI coding agent — real scenarios, measured++This is the honest answer to "does XERJ help Claude Code, and when?" Each+scenario below is a task an agent actually performs, with the measured+before/after and a copy-paste reproduction. Every number came from a run in+this repository; none is projected. Where XERJ does **not** help, that is+stated as plainly as where it does — an agent that trusts a dishonest benchmark+makes worse decisions, not better ones.++The measurement rules (bytes on the wire, savings only over jointly-correct+tasks, question-vocabulary baselines, corpus composition reported) are the ones+enforced by `gate.py` and `gate_retrieval.py` in this directory. Read+`README.md` for why each rule exists, and `../../docs/TOKEN_USAGE.md` for the+token model the scenarios rely on.++## The one law that predicts every result++> **Retrieval savings scale with the searchable-prose fraction of the corpus.+> Analytics savings scale with record count — but analytics answers are small+> either way, so there the win is correctness and latency, not tokens.**++Two corpora, same tasks, opposite outcomes, both measured:++| corpus | prose fraction | XERJ vs grep, tokens | XERJ vs grep, correctness |+|---|--:|---|---|+| 170k LOC source + docs | ~100% | **5.3× fewer** | 7/8 vs 8/8 |+| 36 MB logs/CSV, few docs | 0.01% | 110× **more** | **6/6 vs 4/6** |++Neither number alone is the truth. Pick the scenario that matches your corpus.++---++## Scenario 1 — Orient in an unfamiliar dataset (agent's first move)++**Task:** You are handed a folder of mixed files and asked a question about it.+Before answering anything you must learn what is there.++**Without XERJ:** `ls -R`, then `head` each file, then guess schemas, then+sample rows to infer types. On a heterogeneous folder this is many tool calls+and grows with file count. You still do not know cross-file relationships.++**With XERJ:**+```bash+xerj autoindex ./data # one command, content-sniffed, zero config+xerj autoindex map # the data map+```+**Measured:** a 5.65M-record corpus mapped in **~890 tokens, flat regardless of+corpus size** — every dataset's fields, types, null%, example values, time+range, ready-to-send queries, **and auto-detected cross-dataset joins**+(`user_id ↔ user_id, live-confirmed 20/20`). The naive `ls`+`head` equivalent+is smaller only because it contains far less: no types, no cardinality, no+relationships.++**Verdict:** always worth it. This is the single most reliable win, because the+map's cost does not grow with the data.++---++## Scenario 2 — "Where/why is X" in a large codebase++**Task:** "Why did search relevance drop?" — answered in your own words, not+the codebase's.++**Without XERJ:** `grep -rn` a noun you hope appears. grep is *lexical*: if the+answer says "stemming" and you grepped "relevance", you miss it. On a large+tree the candidate set is large and you read many false hits.++**With XERJ:** ranked search returns the one passage, with a highlighted+fragment and a line number to jump to.+```bash+curl -s "localhost:9200/ax-*/_search?filter_path=hits.hits._source.ax_path,hits.hits.highlight" \+ -H 'Content-Type: application/json' -d '{"size":1,+ "query":{"match":{"body":"why did search relevance drop"}},+ "highlight":{"fields":{"body":{"fragment_size":200,"number_of_fragments":1}}}}'+```+**Measured, 170k-LOC corpus:** XERJ **1,457 tokens vs grep 7,762** at equal+recall — **5.3× fewer**, no steering (queries used only the question's words).+**Measured, small prose corpus** (`gate_retrieval.py`): correctness **6/7 vs+grep 3/7** — grep missed four answers whose wording differed from the question.++**Verdict:** the flagship agent win. Bigger and more text-heavy the repo, bigger+the margin. Requires the chunking that `autoindex` now does by default.++---++## Scenario 3 — Analytical question over logs++**Task:** "Error rate by service", "which paths return 502", "refunded value by+country".++**Without XERJ:** `grep | awk | sort | uniq`, or a bespoke Python pass over the+file. Correct if you write it carefully; easy to get subtly wrong (a `*.log`+glob silently skips the `.gz`; a decimal-comma CSV needs `.replace(',','.')`).++**With XERJ:** one aggregation over typed, parsed fields.+```bash+curl -s "localhost:9200/ax-logs/_search?filter_path=aggregations.p.buckets" \+ -H 'Content-Type: application/json' -d '{"size":0,+ "query":{"term":{"status":502}},+ "aggs":{"p":{"terms":{"field":"path","size":6}}}}'+```+**Measured** (`gate.py`, 458k records): correctness **6/6 vs grep 4/6** — the+gate itself caught the baseline silently skipping a gzipped log and mis-parsing+a decimal-comma CSV. Tokens are ~even (both return a few numbers); XERJ is+5–15× faster in wall time.++**Verdict:** win on **correctness and latency**, not tokens. The engine parses+formats (gzip, decimal-comma CSV, nginx log lines) that a quick grep gets wrong.++---++## Scenario 4 — Incident drill-down (each step depends on the last)++**Task:** "Show the failing requests → now that user's other events → now by+region → now by hour." Interactive investigation.++**Without XERJ:** every step re-scans the whole file. Cost is O(corpus) **per+question**, forever: ~7 s per lookup at 1.2 GB, ~1 min at 12 GB. At some size+the investigation stops being interactive.++**With XERJ:** every step is an indexed lookup or aggregation.+**Measured:** a single-user lookup was **0.007 s (XERJ) vs 7.3 s (grep) at+1.2 GB — 1,043× faster**, and stays ~flat as the corpus grows because the index+is O(matching), not O(corpus).++**Verdict:** win that *grows* with data size. Below ~1 GB grep is fine; above+it, indexed drill-down is the difference between interactive and not.++---++## Scenario 5 — A relational question, with no JOIN++**Task:** "How many enterprise customers have an open ticket?" — spans two+sources.++**The trap:** answered relationally, the agent pulls one whole side of the+relation into context (966 ids × 16 bytes) and echoes it back — **15,914 bytes+for a 3-byte answer.** This is the entire "100× more than grep."++**The fix — denormalize at ingest, then one filtered aggregation:**+```bash+# put `plan` on the ticket once, at ingest; then:+curl -s "localhost:9200/tickets/_search?filter_path=aggregations.u.value" \+ -H 'Content-Type: application/json' -d '{"size":0,+ "query":{"bool":{"filter":[{"term":{"status":"open"}},{"term":{"plan":"enterprise"}}]}},+ "aggs":{"u":{"cardinality":{"field":"user_id"}}}}'+```+**Measured:** **36 bytes, identical answer (122) — 442× smaller.** When one side+already fits in a single document's array field, ES `terms` lookup does this+server-side (see `docs/TOKEN_USAGE.md`).++**Verdict:** the token cost is set by the *pattern*, not the engine. The+relational pattern is a trap; the native pattern is competitive. `autoindex`+already detects the foreign key — denormalizing on it is the next engine step.++---++## Scenario 6 — Durable memory across sessions++**Task:** remember a fact in one session, recall it by meaning in another.++**Without XERJ:** re-derive it, or the user re-supplies it. No mechanism.++**With XERJ:** `POST /_memory/{ns}` to store, `_recall` to retrieve by meaning.+**Measured:** a semantic recall returned in **328 bytes**, and recalled the+right memory from queries sharing *no vocabulary* with the stored text (in+`--embed-mode neural`; the default lexical embedder scores far lower on+paraphrase — P@1 0.33 vs 1.00, so turn neural on for memory).++**Verdict:** a capability the baseline does not have at all. Worth wiring in for+any multi-session agent, with the neural embedder active.++---++## When XERJ does *not* help — say so++- **One-integer questions on a small file.** `grep -c` prints `2` in one byte;+ no engine answering in JSON beats that. Ingest cost is never repaid.+- **`percentiles`/`date_histogram` without the fixes on this branch.** They+ fell to an O(N) scan; the fixes here bring them to columnar speed, but verify+ your build has them.+- **Byte-level forensics.** "What is the exact 0x80 byte at offset 4231" is a+ grep/xxd job, not a search job.+- **A corpus you will query once.** The break-even is ~30–70 questions+ (ingest amortization). One-shot use loses.++## Reproducing all of this++```bash+# analytics regime (XERJ wins correctness, loses tokens on record-heavy data)+python3 demo/agent-gate/make_corpus.py /tmp/gate-corpus+xerj --insecure --data-dir /tmp/gate-data &+xerj autoindex /tmp/gate-corpus+python3 demo/agent-gate/gate.py /tmp/gate-corpus++# retrieval regime (XERJ wins correctness and, at scale, tokens)+python3 demo/agent-gate/make_corpus_prose.py /tmp/gate-prose+xerj autoindex /tmp/gate-prose+python3 demo/agent-gate/gate_retrieval.py /tmp/gate-prose+```++Both gates print the corpus composition next to every ratio, and refuse to+collapse the two regimes into one headline number.diff --git a/demo/agent-gate/gate_retrieval.py b/demo/agent-gate/gate_retrieval.pynew file mode 100644index 000000000..246da91d3--- /dev/null+++ b/demo/agent-gate/gate_retrieval.py@@ -0,0 +1,167 @@+#!/usr/bin/env python3+"""Agent Gate — RETRIEVAL regime. See README.md for the anti-cheating rules.++Same discipline as gate.py, applied to the corpus shape where XERJ is expected+to WIN on tokens: a repository of small prose/code documents, no large record+streams. Questions are answered in the asker's own words, so a match requires+ranking, not lexical coincidence.++Correctness here is CONTENT, asserted before the run — a substring the true+answer must contain, never the file it lives in. Both paths are tuned:+ baseline : grep, capped, question-vocabulary only+ xerj : ranked search, size:1, highlight, filter_path — the frugal shape+"""+import json+import os+import re+import subprocess+import sys+import time+import urllib.request++CORPUS = sys.argv[1] if len(sys.argv) > 1 else "/tmp/gate-prose"+XERJ = os.environ.get("XERJ_URL", "http://localhost:9200")+++class Ledger:+ def __init__(self):+ self.rows = []++ def add(self, task, path, out, seconds):+ self.rows.append({"task": task, "path": path, "bytes": len(out),+ "seconds": seconds})++ def bytes_for(self, path, task):+ return sum(r["bytes"] for r in self.rows+ if r["path"] == path and r["task"] == task)++ def seconds_for(self, path, task):+ return sum(r["seconds"] for r in self.rows+ if r["path"] == path and r["task"] == task)+++LEDGER = Ledger()+_CTX = {"task": None}+++def sh(cmd):+ t0 = time.time()+ out = subprocess.run(["bash", "-c", cmd], capture_output=True, text=True,+ cwd=CORPUS).stdout+ LEDGER.add(_CTX["task"], "baseline", out, time.time() - t0)+ return out+++def es(path, body):+ t0 = time.time()+ req = urllib.request.Request(f"{XERJ}{path}", json.dumps(body).encode(),+ {"Content-Type": "application/json"})+ raw = urllib.request.urlopen(req).read()+ LEDGER.add(_CTX["task"], "xerj", raw.decode("utf-8", "replace"), time.time() - t0)+ return json.loads(raw)+++# (question, grep-terms [question vocabulary only], required substring in answer)+QUESTIONS = [+ ("why was checkout unavailable in June",+ "checkout", "certificate"),+ ("why did search quality regress",+ "search", "stemming"),+ ("how do we fail the database over to the standby",+ "failover", "promote"),+ ("what causes connection pool exhaustion",+ "pool", "fan-out"),+ ("why must queue consumers be idempotent",+ "idempotent", "at-least-once"),+ ("where are secrets kept",+ "secrets", "vault"),+ ("how does the payment client avoid billing twice",+ "idempotency", "idempotency_key"),+]+++def run_baseline(terms, needle):+ _CTX["task"] = needle+ # A cold agent greps the question's noun. It does NOT know the file, and it+ # must read enough context to confirm — so grep -i with a few lines around.+ out = sh(f"grep -rni '{terms}' . | head -8")+ return out, (needle.lower() in out.lower())+++def run_xerj(question, needle):+ _CTX["task"] = needle+ d = es("/ax-*/"+ "_search?filter_path=hits.hits._source.ax_path,hits.hits.highlight",+ {"size": 1,+ "query": {"bool": {"should": [+ {"match": {"body": question}},+ {"match": {"text": question}}]}},+ "_source": ["ax_path"],+ "highlight": {"fields": {+ "body": {"fragment_size": 200, "number_of_fragments": 1},+ "text": {"fragment_size": 200, "number_of_fragments": 1}}}})+ hits = d.get("hits", {}).get("hits", [])+ if not hits:+ return "(no hits)", False+ hl = hits[0].get("highlight", {})+ frag = (hl.get("body") or hl.get("text") or [""])[0]+ out = f"{hits[0]['_source'].get('ax_path')}: {frag}"+ return out, (needle.lower() in frag.lower())+++def main():+ rows = []+ for question, terms, needle in QUESTIONS:+ b_out, b_ok = run_baseline(terms, needle)+ x_out, x_ok = run_xerj(question, needle)+ rows.append({"q": needle, "b_ok": b_ok, "x_ok": x_ok})++ print("\n" + "=" * 80)+ print("AGENT GATE — RETRIEVAL regime (prose/code corpus)")+ print("=" * 80)+ print(f"\n{'question needle':<22}{'base':<7}{'xerj':<7}"+ f"{'base tok':>9}{'xerj tok':>9}{'base s':>8}{'xerj s':>8}")+ jb = jx = 0+ for r in rows:+ b = LEDGER.bytes_for("baseline", r["q"]) // 4+ x = LEDGER.bytes_for("xerj", r["q"]) // 4+ both = r["b_ok"] and r["x_ok"]+ if both:+ jb += b+ jx += x+ print(f"{r['q'][:22]:<22}"+ f"{'OK' if r['b_ok'] else 'MISS':<7}{'OK' if r['x_ok'] else 'MISS':<7}"+ f"{b:>9}{x:>9}"+ f"{LEDGER.seconds_for('baseline', r['q']):>8.2f}"+ f"{LEDGER.seconds_for('xerj', r['q']):>8.2f}")++ nb = sum(1 for r in rows if r["b_ok"])+ nx = sum(1 for r in rows if r["x_ok"])+ print(f"\ncorrect: baseline {nb}/{len(rows)} xerj {nx}/{len(rows)}")++ print("\n-- tokens over jointly-correct tasks only --")+ if jb and jx:+ ratio = jb / jx+ v = (f"XERJ uses {ratio:.2f}x FEWER tokens" if ratio > 1+ else f"XERJ uses {1/ratio:.2f}x MORE tokens")+ print(f" baseline {jb} tok xerj {jx} tok -> {v}")+ else:+ print(" no jointly-correct tasks — no savings claim possible")++ total = text = 0+ for root, _, files in os.walk(CORPUS):+ for f in files:+ n = os.path.getsize(os.path.join(root, f))+ total += n+ if f.endswith((".md", ".py", ".rs", ".js", ".html", ".txt")):+ text += n+ print("\n-- corpus composition --")+ print(f" {total/1e3:.1f} KB total; {text/1e3:.1f} KB searchable prose/code "+ f"({100*text/max(total,1):.1f}%)")+ print(" This is the regime where retrieval savings appear: grep's candidate")+ print(" set grows with the corpus; ranked search returns one passage.\n")+ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/demo/agent-gate/make_corpus_prose.py b/demo/agent-gate/make_corpus_prose.pynew file mode 100644index 000000000..fd9322dfc--- /dev/null+++ b/demo/agent-gate/make_corpus_prose.py@@ -0,0 +1,175 @@+#!/usr/bin/env python3+"""Deterministic PROSE/CODE-heavy corpus for the Agent Gate — the regime where+XERJ is expected to WIN on tokens.++The mixed corpus (make_corpus.py) is 99.99% structured records, where grep on a+one-integer question is unbeatable. This corpus is the opposite: many small+documents of natural-language and source text, no large record streams. It is+the shape of a real code/knowledge repository, and it is where ranked retrieval+beats unranked grep on tokens — grep must return every candidate line, ranked+search returns the one passage.++Seeded, so runs are byte-identical and comparable across versions.+Usage: make_corpus_prose.py <dir>+"""+import os+import random+import sys+import textwrap++OUT = sys.argv[1] if len(sys.argv) > 1 else "/tmp/gate-prose"+os.makedirs(OUT, exist_ok=True)+os.chdir(OUT)+random.seed(31337)++for d in ("docs", "runbooks", "adr", "src", "src/api", "src/store", "tests"):+ os.makedirs(d, exist_ok=True)++# ── Ground-truth documents: each answers exactly one gate question, phrased so+# the answer's wording differs from the question's (no lexical gift). ──────++open("docs/postmortem-checkout.md", "w").write(textwrap.dedent("""\+ # Postmortem: checkout unavailable, 14 June 2026++ ## Impact+ Checkout returned errors for 51 minutes across all regions.++ ## Root cause+ The payment gateway's TLS certificate expired. Renewal was automated but+ the service never reloaded it, so every handshake was rejected.++ ## Fix+ Reloaded the service and added a certificate-expiry alert firing 14 days out.+ """))++open("docs/postmortem-search.md", "w").write(textwrap.dedent("""\+ # Postmortem: search quality regression, 3 May 2026++ ## Impact+ Click-through on search fell by more than a third for two days.++ ## Root cause+ A retuned analyser stopped applying stemming to German queries, so plural+ forms no longer matched their singular index terms.++ ## Fix+ Reverted the analyser and added a relevance regression test to the pipeline.+ """))++open("runbooks/database-failover.md", "w").write(textwrap.dedent("""\+ # Runbook: database failover++ Promote the hot standby with `pg_ctl promote`. Writes are unavailable for+ roughly forty seconds while the promotion settles. Point the application at+ the new primary only after replication lag on the old node reaches zero.+ """))++open("runbooks/pool-exhaustion.md", "w").write(textwrap.dedent("""\+ # Runbook: connection pool exhaustion++ Symptoms: p99 latency climbs and `pool exhausted` appears in the logs. The+ usual trigger is a deploy that raised per-request fan-out, saturating the+ fixed pool. Mitigate by raising the pool ceiling and rolling back the deploy+ that increased fan-out.+ """))++open("adr/0007-idempotent-consumers.md", "w").write(textwrap.dedent("""\+ # ADR 0007: all queue consumers must be idempotent++ The async queue guarantees at-least-once delivery, not exactly-once. A+ consumer can therefore see the same message twice. Every consumer must key+ its side effects on a stable identifier so a redelivery is a no-op rather+ than a double charge or a duplicate email.+ """))++open("docs/security-secrets.md", "w").write(textwrap.dedent("""\+ # Handling secrets++ Secrets live in the vault and are injected into the process environment at+ boot. They are never written to a file that is committed to version control.+ Production access additionally requires a hardware security key.+ """))++# Source that answers "how does the payment client avoid double billing".+open("src/api/payments.py", "w").write(textwrap.dedent('''+ """Payment gateway client: bounded retries with a stable idempotency key."""+ import hashlib+ import time++ MAX_RETRIES = 3++ def idempotency_key(order_id: str, attempt: int) -> str:+ """Deterministic key so a retried charge is collapsed by the gateway+ into the original, never billed twice."""+ return hashlib.sha256(f"{order_id}:{attempt}".encode()).hexdigest()++ def charge(client, order_id: str, amount_cents: int):+ for attempt in range(MAX_RETRIES):+ try:+ return client.post("/charge", json={+ "amount": amount_cents,+ "idempotency_key": idempotency_key(order_id, attempt),+ })+ except TimeoutError:+ time.sleep(2 ** attempt) # exponential backoff+ raise RuntimeError("payment gateway unreachable after retries")+'''))++open("src/store/pool.rs", "w").write(textwrap.dedent('''+ //! Fixed-ceiling connection pool. Exhaustion is the most common outage+ //! trigger: a deploy raises per-request fan-out and the pool saturates,+ //! p99 climbs, and the service starts shedding load with 503s.+ pub struct Pool {+ max: usize,+ in_use: usize,+ }++ impl Pool {+ pub fn acquire(&mut self) -> Result<Conn, PoolError> {+ if self.in_use >= self.max {+ return Err(PoolError::Exhausted);+ }+ self.in_use += 1;+ Ok(Conn::new())+ }+ }+'''))++# Filler: many plausible-but-irrelevant documents, so retrieval has to actually+# rank rather than return the only file. This is what makes grep expensive —+# its candidate set is large; ranked search still returns one passage.+TOPICS = ["caching", "logging", "metrics", "tracing", "rate limiting",+ "feature flags", "backups", "cron scheduling", "webhooks",+ "pagination", "serialization", "config loading", "health checks",+ "graceful shutdown", "circuit breaking", "load shedding",+ "request validation", "content negotiation", "compression",+ "connection keepalive", "dns resolution", "retry budgets"]+BODY = ("This module handles {t}. It exposes a small, dependency-free surface "+ "and is covered by unit tests. Configuration is read once at startup "+ "and validated eagerly so a bad value fails fast rather than at first "+ "use. The design favours explicitness over cleverness; see the module "+ "comment for the rationale and the trade-offs that were considered.")+for i, t in enumerate(TOPICS):+ slug = t.replace(" ", "-")+ open(f"src/{slug}.md", "w").write(f"# {t.title()}\n\n{BODY.format(t=t)}\n")+ # a source file too, so the corpus is code + prose+ open(f"src/{slug}.py", "w").write(+ f'"""{t.title()} implementation."""\n\n'+ f"# {BODY.format(t=t)}\n"+ f"def configure_{slug.replace('-', '_')}(cfg):\n"+ f" return cfg.get({t.split()[0]!r}, {{}})\n")++# A few longer design docs for bulk.+for i in range(6):+ paras = "\n\n".join(+ f"Section {j}: " + BODY.format(t=random.choice(TOPICS))+ for j in range(random.randint(8, 16)))+ open(f"adr/design-note-{i:02d}.md", "w").write(f"# Design note {i}\n\n{paras}\n")++# tiny binary junk, to exercise skip-not-fatal+open("logo.bin", "wb").write(os.urandom(4096))++n = sum(len(files) for _, _, files in os.walk("."))+sz = sum(os.path.getsize(os.path.join(r, f))+ for r, _, fs in os.walk(".") for f in fs)+print(f"prose corpus built: {n} files, {sz/1e3:.1f} KB")