AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Reproduce the second-brain use case, exactly
Every command below was run on branch feat/second-brain. The corpus is this
repository's own docs/ folder, so the whole thing reproduces from a clone.
Numbers (0.7 s, 364 links, …) are from one machine and one corpus; yours will
differ in the details, and the invariants (idempotence, evidence, replay)
should not.
Throughout, $REPO is the repository root and $DATA is a fresh, empty
directory for the brain's server data.
0. Build (scoped — never workspace-wide)
cd $REPO/engine
cargo build --release -j 32 -p xerj-engine -p xerj-api -p xerj-autoindex \
-p xerj-console-api -p xerj-server -p xerj-mcp
1. Run the brain
DATA=$(mktemp -d)
$REPO/engine/target/release/xerj brain $REPO/docs \
--url http://localhost:9340 --data-dir "$DATA" --no-open
Expected shape (values vary with the current state of docs/):
brain 'docs': 121 files (5 MB) under …/docs
booted xerj server (pid …) — data: …/data
phase A: 7 datasets inferred, 14 junk/skipped files
phase B: indexing 106 files with 8 workers → http://localhost:9340
✓ your second brain is ready — 121 files, 364 links, 0.7s
→ http://localhost:9340/_xerj-console/#/second-brain?brain=docs
Exit codes: 0 ready · 3 ready-with-junk (unreadable files recorded, never
fatal — expected on this corpus, which contains images and binaries) · 1
nothing indexable / server failure · 2 usage.
Idempotence check — run the identical command again. It must attach to the
already-running server, converge fast (0.14 s here), exit 0, and the link
count must not grow.
The booted server keeps its own admin key at $DATA/admin.key; every direct
HTTP call below needs it:
AUTH="Authorization: ApiKey $(cat "$DATA/admin.key")"
2. Claim 1 — evidence on every link
Pick a hub node from the overview, then ask for its neighborhood:
curl -s -H "$AUTH" 'http://localhost:9340/_graph/docs/overview' | python3 -m json.tool
# take an id from .hubs.in[0].id (call it $NODE)
curl -s -H "$AUTH" \
"http://localhost:9340/_graph/docs/ego?node=$NODE&include_nodes=true" \
| python3 -m json.tool
Verify one quote against the source file (this is the whole point):
# for an authored edge (detector mdlink@1 / wikilink@1 / href@1):
# evidence.quote must be verbatim text from evidence.source
grep -F "<paste evidence.quote here>" $REPO/docs/<evidence.source>
Notes on what you will see:
evidence.offsetis the byte offset within the section the link was found in, not within the whole file (contract §6.5).- Structural edges (
samedir@1,sequence@1) carry generated rationales ("a.md and b.md share directory …"), offset 0 — not extracted text.
3. Claim 2 — retire, then replay
# capture a moment before the retirement
T_BEFORE=$(date +%s%3N)
# assert a manual link (src/dst are node ids from step 2)
curl -s -H "$AUTH" -H 'content-type: application/json' \
-X POST http://localhost:9340/_graph/docs/link \
-d '{"src":"<id1>","dst":"<id2>","type":"manual"}'
# → 201 {"edge_id":"…", …}
# retire it
curl -s -H "$AUTH" -X DELETE http://localhost:9340/_graph/docs/link/<edge_id>
# → {"invalidated":true,"invalid_at":<ms>}
# now: the link is gone, and the response SAYS one link was excluded
curl -s -H "$AUTH" "http://localhost:9340/_graph/docs/ego?node=<id1>&types=manual"
# → 0 edges, "not_shown":{"expired_excluded":1,…}
# replay the moment before the retirement: the link comes back,
# with its later invalid_at visible
curl -s -H "$AUTH" \
"http://localhost:9340/_graph/docs/ego?node=<id1>&types=manual&as_of=$T_BEFORE"
# or ask now with tombstones included
curl -s -H "$AUTH" \
"http://localhost:9340/_graph/docs/ego?node=<id1>&types=manual&include_expired=true"
Also check that the 2-hop cap refuses honestly:
curl -s -H "$AUTH" "http://localhost:9340/_graph/docs/ego?node=<id1>&hops=3"
# → 400, "…not a graph database…"
4. The dashboard
Open http://localhost:9340/_xerj-console/#/second-brain?brain=docs in a
browser (the console handles its own login using the same bootstrap
credentials). What to look for: the ego ledger with links grouped by what
taught them, evidence on hover/click, the belief-time scrubber narrating
appeared/retired counts, and the AUTHORED vs STRUCTURAL split.
Automated equivalent (what our verification ran — the real data layer plus the real renderers for all nine panels, in Node, against the live server; 47/47 checks; needs Node ≥ 20):
XERJ_URL=http://localhost:9340 XERJ_BRAIN=docs \
XERJ_ADMIN_KEY_FILE="$DATA/admin.key" \
node $REPO/docs/usecases/second-brain/contract-check.mjs
Note: this proves the rendering logic and data contract, not real-browser pixels/interactions — those were not machine-verified.
5. The agent surface (MCP)
XERJ_URL=http://localhost:9340 \
XERJ_AUTH="ApiKey $(cat "$DATA/admin.key")" \
$REPO/engine/target/release/xerj-mcp
Then over stdio: tools/list must include xerj_brain_ego,
xerj_brain_link, xerj_brain_unlink, xerj_brain_overview; a
tools/call of xerj_brain_overview with {"brain":"docs"} must round-trip
with "contract": "xerj-second-brain/1" and edge counts that match step 2.
6. Test suites
cd $REPO/engine
cargo test --release -p xerj-engine --test graph_expand # 5
cargo test --release -p xerj-server --test brain_cli # 3
cargo test --release -p xerj-autoindex # 109
cargo test --release -p xerj-console-api # 76
cargo test --release -p xerj-mcp # 25
cargo test --release -p xerj-engine --lib # 241
cargo test --release -p xerj-api --lib # 84 pass, 1 fail
The one xerj-api failure (reindex_pages_past_10k_via_keyset) is
pre-existing on main (its commit is an ancestor of main) and unrelated to
the graph feature; it is listed so you are not surprised by it.
7. Cleanup
kill <server pid printed at boot> # or: pkill -f 'xerj.*9340'
rm -rf "$DATA"
Environment used for the published numbers
Linux x86-64, release build, 121 files / 5.7 MB corpus, fresh data dir, port 9340. Token economics and large-corpus scaling were not measured; treat any extrapolation as yours, not ours.