Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

docs(readme,landing): AI-first front — agent prompt first, reference coding as the lead use case

xerj-orgb1b6a692837e
diff --git a/README.md b/README.md
index 4cfc36df5..b4ac8836b 100644
--- a/README.md
+++ b/README.md
@@ -4,29 +4,14 @@ XERJ is a search engine for AI agents. Point it at a folder and one command make
your code, docs, logs and PDFs queryable, so an agent asks questions instead of
reading files into its context window.
-## Install
-
-```sh
-curl -fsSL https://xerj.org/get | sh
-```
-
-Windows PowerShell:
-
-```powershell
-irm https://xerj.org/get.ps1 | iex
-```
-
-One static binary, no JVM, no dependencies. Prebuilt for Linux, macOS and Windows
-on x86-64 and arm64. You can also [build from source](#build-from-source). It
-speaks the Elasticsearch API, so existing clients, dashboards and tooling work
-against it unchanged.
+[![CI](https://github.com/xerj-org/xerj/actions/workflows/ci.yml/badge.svg)](https://github.com/xerj-org/xerj/actions/workflows/ci.yml)
+[![License: Apache-2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](./LICENSE)
+[![Release](https://img.shields.io/github/v/release/xerj-org/xerj?include_prereleases&sort=semver)](https://github.com/xerj-org/xerj/releases)
+[![ES conformance](https://img.shields.io/badge/ES%20conformance-1365%2F1368-brightgreen.svg)](https://xerj.org/benchmarks)
-First commands after install (the installer prints where `xerj` landed — add it to
-your PATH if needed): `xerj --insecure --data-dir ./data &`, wait until
-`http://localhost:9200` responds, then `xerj autoindex ~/my-project` — see
-[Index a folder](#index-a-folder).
+## Paste this to your AI agent
-### Or paste this to your AI agent
+The whole setup is one prompt:
```text
Install XERJ (docs: https://xerj.org/llms.txt), index this project's sources, and set up
@@ -34,10 +19,26 @@ reference coding: clone and index the open-source repos closest to what we're bu
and search how they solved a problem before writing code.
```
-One prompt is enough — [llms.txt](https://xerj.org/llms.txt) gives your agent the
-ordered steps: install, start the server, `xerj autoindex .`, query with any
-Elasticsearch client, and the reference-coding loop (clone similar OSS, index it,
-retrieve the mechanism before writing, cite what you use, respect licenses).
+[llms.txt](https://xerj.org/llms.txt) gives your agent the ordered steps: install,
+start the server, `xerj autoindex .`, query with any Elasticsearch client, and the
+reference-coding loop (clone similar OSS, index it, retrieve the mechanism before
+writing, cite what you use, respect licenses).
+
+Why you would: an agent working from memory retry-loops on any API it hasn't
+memorised, and grep only tells it where to look — the recovery is still reading
+source into context, up to 1.06M input tokens on one corpus in our measurements.
+An agent that queries XERJ reads the exact passage instead.
+
+The numbers, measured end-to-end on code the model had not memorised
+([the case study](https://xerj.org/case-studies/reference-coding.html), 8 tasks
+across 4 languages, 16 runs per arm, real `claude -p` token counts): **2.7× fewer
+output tokens** than grep-driven Claude Code (9,982 vs 26,477) at the same 16/16
+solve rate, **26× fewer** than working from memory alone (260,916 → 9,982), and
+**2.1× cheaper** ($1.58 vs $3.27). In a companion run on a Rust library the model
+had never seen: **9/9 correct with retrieval versus 0/9 from memory**. Beyond the
+lab, users report roughly **5× fewer tokens** across real product development —
+fewer retry loops, sharper requests
+([field reports](./user-feedback/11-reference-coding-field-reports/2026-08-11-token-savings-reports.md)).
More prompts that work on a fresh install:
@@ -48,12 +49,7 @@ More prompts that work on a fresh install:
- *"Use XERJ's `/_memory/notes` API as your long-term memory for this project: store what
you learn as you work, and recall it by meaning next session."*
-Worked, validated examples for each capability: [xerj.org/recipes](https://xerj.org/recipes).
-
-[![CI](https://github.com/xerj-org/xerj/actions/workflows/ci.yml/badge.svg)](https://github.com/xerj-org/xerj/actions/workflows/ci.yml)
-[![License: Apache-2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](./LICENSE)
-[![Release](https://img.shields.io/github/v/release/xerj-org/xerj?include_prereleases&sort=semver)](https://github.com/xerj-org/xerj/releases)
-[![ES conformance](https://img.shields.io/badge/ES%20conformance-1365%2F1368-brightgreen.svg)](https://xerj.org/benchmarks)
+Worked, validated examples for each capability: [xerj.org/docs/recipes](https://xerj.org/docs/recipes/).
<video src="https://xerj.org/xerj-biz-demo.mp4" poster="docs/media/demo-poster.png" controls muted playsinline width="840">
<a href="https://xerj.org/aise-demo.html"><img src="docs/media/demo-poster.png" width="840" alt="Watch the XERJ demo: boot, ingest, search, vectors, dashboards"></a>
@@ -63,6 +59,28 @@ Worked, validated examples for each capability: [xerj.org/recipes](https://xerj.
<a href="https://xerj.org/aise-demo.html">Watch it on xerj.org</a> or
<a href="https://xerj.org/playground/">try the live playground</a>.</sub>
+## Install by hand
+
+```sh
+curl -fsSL https://xerj.org/get | sh
+```
+
+Windows PowerShell:
+
+```powershell
+irm https://xerj.org/get.ps1 | iex
+```
+
+One static binary, no JVM, no dependencies. Prebuilt for Linux, macOS and Windows
+on x86-64 and arm64. You can also [build from source](#build-from-source). It
+speaks the Elasticsearch API, so existing clients, dashboards and tooling work
+against it unchanged.
+
+First commands after install (the installer prints where `xerj` landed — add it to
+your PATH if needed): `xerj --insecure --data-dir ./data &`, wait until
+`http://localhost:9200` responds, then `xerj autoindex ~/my-project` — see
+[Index a folder](#index-a-folder).
+
## Index a folder
Start the server, then point `autoindex` at anything:
@@ -122,6 +140,7 @@ load about half a percent of the tree.
## Use cases
+- **[Reference coding](https://xerj.org/case-studies/reference-coding.html)**: your coding agent retrieves how peer projects already solved it instead of re-deriving — measured 2.7× fewer output tokens than grep-driven coding (26× vs memory alone); users report ~5× in real development
- **[Code search and security audits](https://xerj.org/use-cases/code-security-audit.html)**: AST-aware indexing, so an agent finds a function instead of a line
- **[AI search and RAG](https://xerj.org/use-cases/ai-search-retrieval.html)**: full-text, vector and hybrid retrieval in one query, with no separate vector database
- **[Agent memory](https://xerj.org/use-cases/second-brain.html)**: durable recall with a knowledge graph over your own documents
diff --git a/landing/index.html b/landing/index.html
index 63a260db7..02c6f93e2 100644
--- a/landing/index.html
+++ b/landing/index.html
@@ -48,10 +48,25 @@
<span>2026</span>
</div>
- <h1 class="hero"><span class="accent">XERJ.</span><br>ONE ENGINE.<br>ZERO STACK.</h1>
+ <h1 class="hero"><span class="accent">XERJ.</span><br>STOP RE-DERIVING.<br>START LOOKING UP.</h1>
<div class="hero-install">
- <div class="hero-install-label">INSTALL · ONE COMMAND · SHA-256 CHECKSUMS</div>
+ <div class="hero-install-agent" style="margin-top:0;">
+ <span class="hero-install-label">PASTE THIS TO YOUR AI AGENT · THE WHOLE SETUP</span>
+ <div class="hero-install-cmd">
+ <code>Install XERJ (docs: https://xerj.org/llms.txt), index this project's sources, and set up reference coding: clone and index the open-source repos closest to what we're building, and search how they solved a problem before writing code.</code>
+ <button type="button" class="hero-install-copy" data-copy="Install XERJ (docs: https://xerj.org/llms.txt), index this project's sources, and set up reference coding: clone and index the open-source repos closest to what we're building, and search how they solved a problem before writing code.">COPY</button>
+ </div>
+ <p class="hero-install-agent-line">One prompt is enough — llms.txt gives your agent
+ the ordered steps: install, start the server, <code>xerj autoindex .</code>,
+ query with any Elasticsearch client, and the reference-coding loop.
+ Measured: <strong>2.7× fewer output tokens</strong> than grep-driven Claude Code,
+ <strong>26× fewer</strong> than memory alone —
+ <a href="/case-studies/reference-coding.html">the case study</a>. Users report
+ <strong>~5×</strong> across real product development —
+ <a href="https://github.com/xerj-org/xerj/blob/main/user-feedback/11-reference-coding-field-reports/2026-08-11-token-savings-reports.md">field reports</a>.</p>
+ </div>
+ <div class="hero-install-label" style="margin-top:var(--sp-3);">OR INSTALL BY HAND · ONE COMMAND · SHA-256 CHECKSUMS</div>
<div class="hero-install-cmd">
<code>curl -fsSL https://xerj.org/get | sh</code>
<button type="button" class="hero-install-copy" data-copy="curl -fsSL https://xerj.org/get | sh">COPY</button>
@@ -60,29 +75,20 @@
<span class="mono">WINDOWS · <code>irm https://xerj.org/get.ps1 | iex</code></span>
<button type="button" class="hero-install-copy" data-copy="irm https://xerj.org/get.ps1 | iex">COPY</button>
</div>
- <div class="hero-install-agent">
- <span class="hero-install-label">OR PASTE THIS TO YOUR AI AGENT</span>
- <div class="hero-install-cmd">
- <code>Install XERJ (docs: https://xerj.org/llms.txt), index this project's sources, and set up reference coding: clone and index the open-source repos closest to what we're building, and search how they solved a problem before writing code.</code>
- <button type="button" class="hero-install-copy" data-copy="Install XERJ (docs: https://xerj.org/llms.txt), index this project's sources, and set up reference coding: clone and index the open-source repos closest to what we're building, and search how they solved a problem before writing code.">COPY</button>
- </div>
- <p class="hero-install-agent-line">One prompt is enough — llms.txt gives your agent
- the ordered steps: install, start the server, <code>xerj autoindex .</code>,
- query with any Elasticsearch client, and the reference-coding loop.</p>
- </div>
</div>
<p class="lead">
- The unified search engine for AI — connect it, run
+ The AI-native search engine — connect it, run
<code class="inline">xerj&nbsp;autoindex</code>, and it works: any folder
becomes typed, queryable indices — <strong>zero configuration</strong>.
- Give agents <strong>long-term memory</strong>, <strong>semantic search
- &amp; RAG</strong>, <strong>hybrid</strong> BM25&nbsp;+&nbsp;vector
- retrieval, <strong>auto-embedding</strong> on ingest, and
- <strong>anomaly detection</strong> — no bolt-on vector database, no
- glue code. One binary that speaks the Elasticsearch wire protocol,
- released under <strong>Apache&#8209;2.0</strong>: no lock-in, no SSPL,
- no per-feature gates. Yours to run, fork, and build on.
+ Give agents <strong>reference coding</strong>, <strong>long-term
+ memory</strong>, <strong>semantic search &amp; RAG</strong>,
+ <strong>hybrid</strong> BM25&nbsp;+&nbsp;vector retrieval,
+ <strong>auto-embedding</strong> on ingest, and <strong>anomaly
+ detection</strong> — no bolt-on vector database, no glue code. One binary
+ that speaks the Elasticsearch wire protocol, released under
+ <strong>Apache&#8209;2.0</strong>: no lock-in, no SSPL, no per-feature
+ gates. Yours to run, fork, and build on.
</p>
</div>
@@ -112,9 +118,42 @@
<p class="viz-caption">Five LLM models tracked in real time. P95 latency, 60-second window. Ribbon depth encodes model generation. This is one XERJ query — not five dashboards stitched together.</p>
</section>
-<!-- 01 — POINT XERJ AT A FOLDER ------------------------------->
+<!-- 01 — REFERENCE CODING ------------------------------------->
+<section class="scene-section">
+ <div class="kicker"><span class="accent">01</span><span class="dash">·</span><span>REFERENCE CODING · THE CODING USE CASE</span></div>
+ <h2 class="scene">GREP RE-READS.<br><span class="accent">XERJ RETRIEVES.</span></h2>
+
+ <p class="wide">
+ A coding agent hitting an API it hasn't memorised invents method names,
+ fails to compile, and loops — spending <em>output</em> tokens, the
+ expensive kind, on every lap. Grep doesn't save it: grep tells the agent
+ where to look, and the recovery is still reading source into context —
+ up to <strong>1.06M input tokens</strong> on one corpus in our
+ measurements. Reference coding flips the loop: clone the open-source
+ repos closest to what you're building,
+ <code class="inline">xerj autoindex</code> them once, and the agent
+ retrieves the exact definition — with its contract — before writing.
+ Measured (8 tasks across 4 languages, 16 runs per arm, real
+ <code class="inline">claude -p</code> token counts): <strong>2.7× fewer
+ output tokens</strong> than grep-driven Claude Code at the same 16/16
+ solve rate, <strong>26× fewer</strong> than working from memory
+ (260,916 → 9,982), <strong>2.1× cheaper</strong> ($1.58 vs $3.27). In a
+ companion run on a Rust library the model had never seen:
+ <strong>9/9 with retrieval vs 0/9 from memory</strong>. The three helper
+ scripts below ship with the case study — the hero prompt sets them up.
+ </p>
+
+<pre class="code">$ xc-corpus.sh xerj-storage https://github.com/spacejam/sled … # clone the peers, once
+$ xc-index.sh xerj-storage # xerj autoindex, once per corpus
+$ xc.py xerj-storage "how does sled fsync its WAL segment on rotation?"
+→ the exact function, file:line, contract included — a passage to read, not a tree to grep</pre>
+
+ <p class="viz-caption"><a class="mono" href="/case-studies/reference-coding.html" style="color:var(--z-mute);">THE FULL CASE STUDY — EVERY NUMBER, EVERY TASK, THE EXACT PROMPT →</a></p>
+</section>
+
+<!-- 02 — POINT XERJ AT A FOLDER ------------------------------->
<section class="scene-section">
- <div class="kicker"><span class="accent">01</span><span class="dash">·</span><span>POINT XERJ AT A FOLDER · ZERO CONFIG</span></div>
+ <div class="kicker"><span class="accent">02</span><span class="dash">·</span><span>POINT XERJ AT A FOLDER · ZERO CONFIG</span></div>
<h2 class="scene">ONE COMMAND.<br><span class="accent">ANY FOLDER.</span></h2>
<p class="wide">
@@ -176,9 +215,9 @@ next: `xerj autoindex map --url http://localhost:9280` for the data map; search
<p class="viz-caption"><a class="mono" href="/docs/recipes/zero-config-autoindex.html" style="color:var(--z-mute);">THE FULL RECIPE, EVERY NUMBER TRACED TO A RUN →</a></p>
</section>
-<!-- 02 — BUILT FOR AGENTS ------------------------------------->
+<!-- 03 — BUILT FOR AGENTS ------------------------------------->
<section class="scene-section">
- <div class="kicker"><span class="accent">02</span><span class="dash">·</span><span>BUILT FOR AGENTS · MEMORY · RECIPES</span></div>
+ <div class="kicker"><span class="accent">03</span><span class="dash">·</span><span>BUILT FOR AGENTS · MEMORY · RECIPES</span></div>
<h2 class="scene">AGENTS ARE<br><span class="accent">THE CUSTOMER.</span></h2>
<p class="wide">
@@ -225,9 +264,9 @@ $ curl -sXPOST localhost:9200/_memory/agent-demo/_recall \
<p class="viz-caption"><a class="mono" href="/for-agents.html" style="color:var(--z-mute);">THE FULL AGENT SURFACE — SEVEN OPERATIONS + LLMS.TXT →</a></p>
</section>
-<!-- 03 — UNIFIED DATA LAYER ---------------------------------->
+<!-- 04 — UNIFIED DATA LAYER ---------------------------------->
<section class="scene-section">
- <div class="kicker"><span class="accent">03</span><span class="dash">·</span><span>XERJ REPLACES THREE SYSTEMS</span></div>
+ <div class="kicker"><span class="accent">04</span><span class="dash">·</span><span>XERJ REPLACES THREE SYSTEMS</span></div>
<h2 class="scene">LOGS.<br>VECTORS.<br><span class="accent">AGENT MEMORY.</span></h2>
<p class="wide">
@@ -256,9 +295,9 @@ $ curl -sXPOST localhost:9200/_memory/agent-demo/_recall \
</div>
</section>
-<!-- 04 — HYBRID SEARCH ---------------------------------------->
+<!-- 05 — HYBRID SEARCH ---------------------------------------->
<section class="scene-section">
- <div class="kicker"><span class="accent">04</span><span class="dash">·</span><span>XERJ NATIVE HYBRID SEARCH</span></div>
+ <div class="kicker"><span class="accent">05</span><span class="dash">·</span><span>XERJ NATIVE HYBRID SEARCH</span></div>
<h2 class="scene">ONE PASS.<br><span class="accent">ONE HOP.</span></h2>
<p class="wide">
@@ -305,10 +344,10 @@ $ curl -sXPOST localhost:9200/_memory/agent-demo/_recall \
</div>
</section>
-<!-- 05 — COST VISIBILITY -------------------------------------->
+<!-- 06 — COST VISIBILITY -------------------------------------->
<section class="scene-section section-split">
<div class="split-copy">
- <div class="kicker"><span class="accent">05</span><span class="dash">·</span><span>XERJ COST VISIBILITY</span></div>
+ <div class="kicker"><span class="accent">06</span><span class="dash">·</span><span>XERJ COST VISIBILITY</span></div>
<h2 class="scene">WATCH THE<br><span class="accent">MONEY MOVE.</span></h2>
<p class="wide">
Spend shouldn't be a monthly surprise. XERJ encodes cost directly
@@ -344,7 +383,7 @@ $ curl -sXPOST localhost:9200/_memory/agent-demo/_recall \
<!-- BENCHMARKS — SUPPORTING EVIDENCE -------------------------->
<section class="scene-section">
- <div class="kicker"><span class="accent">06</span><span class="dash">·</span><span>SUPPORTING EVIDENCE · XERJ vs ELASTICSEARCH 8.13.4</span></div>
+ <div class="kicker"><span class="accent">07</span><span class="dash">·</span><span>SUPPORTING EVIDENCE · XERJ vs ELASTICSEARCH 8.13.4</span></div>
<div class="ticker">
<div class="tk">
<div class="eyebrow">FULL QUERY MATRIX · CLOSED-LOOP · CACHE OFF</div>
@@ -367,7 +406,7 @@ $ curl -sXPOST localhost:9200/_memory/agent-demo/_recall \
<!-- ENGINE --------------------------------------------------->
<section class="scene-section" id="engine" style="border-top:0;">
- <div class="kicker"><span class="accent">07</span><span class="dash">·</span><span>INSIDE XERJ</span></div>
+ <div class="kicker"><span class="accent">08</span><span class="dash">·</span><span>INSIDE XERJ</span></div>
<h2 class="scene">ONE BINARY.<br>NO JVM.<br><span class="accent">NO COMPROMISES.</span></h2>
<p class="wide">
diff --git a/user-feedback/11-reference-coding-field-reports/2026-08-11-token-savings-reports.md b/user-feedback/11-reference-coding-field-reports/2026-08-11-token-savings-reports.md
new file mode 100644
index 000000000..43c7b97ba
--- /dev/null
+++ b/user-feedback/11-reference-coding-field-reports/2026-08-11-token-savings-reports.md
@@ -0,0 +1,28 @@
+# Field reports: token savings from reference coding (2026-08-11)
+
+**What is recorded here.** Users running the reference-coding loop (clone the
+peer repos, `xerj autoindex` them, retrieve before writing) in real product
+development report roughly **5× fewer tokens** end-to-end, attributing it to
+fewer retry loops on unfamiliar APIs and sharper, better-grounded requests.
+Relayed by the project owner on 2026-08-11 from direct user conversations.
+
+**What this is and is not.** This is field testimony, not a controlled run. It
+is recorded here so that every public occurrence of the number has a citable
+source, per the repository's honest-claims rules. Public copy must always
+attribute it as a user report ("users report"), never as a measurement.
+
+**How it relates to what we measured.** The controlled benchmark
+(`landing/case-studies/reference-coding.html`; raw data under
+`.claude/skills/xerj-code/measure/`) measured, on code the model had not
+memorised: 2.7× fewer output tokens task-for-task than grep-driven Claude Code
+(26,477 → 9,982), 26× fewer than working from memory alone (260,916 → 9,982),
+2.1× cheaper ($3.27 → $1.58), at the same 16/16 solve rate as native tooling.
+A real development mix sits between the 2.7× (tooling-assisted) and 26×
+(memory-loop) regimes, which is where the ~5× reports fall. The regimes have
+different baselines; do not present the bracket as a single like-for-like
+measurement.
+
+**Standing rule for reusing the number.** Quote it only as "users report ~5×",
+link or cite this file as provenance, and keep a measured number adjacent so
+the reader can see both. If future controlled end-to-end runs on real product
+work land, replace the testimony with the measurement.