AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
docs(agents): teach 'xerj search' as the one-command query; drop the last xc.py-only pointer
xerj-orgae9584a424ed
diff --git a/AGENTS.md b/AGENTS.mdindex 077e25fe5..8abc88e1d 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -61,7 +61,7 @@ cannot push at all: [.github/AI_CONTRIBUTIONS.md](./.github/AI_CONTRIBUTIONS.md)## Ground rules for working in this repo- **Honest claims only.** Every public number traces to a verified run. The **default** embedding mode is *lexical* feature-hashing — never claim the default mode does neural/semantic understanding. A built-in **neural** BERT embedder ships in the binary but is **off unless activated** (`--embed-mode neural`, model auto-downloads on first use); only describe output as neural when that mode is actually running. kNN: unfiltered queries are HNSW-served with exact rescoring — recall is *measured* (recall@10 1.00 on the official bench query, 100-probe mean 0.976), never claim "recall 1.00 by construction" for the ANN path; filtered/nested/SQ8/small-index shapes run the exact brute-force scan; benchmark headline is the audited scorecard in `demo/playbooks/SCORECARD.md` (currently 55W / 26T / 4L, 3 N/A vs live ES 8.13.4 — the 4 losses are all read-under-write p99 cells); do not claim TB-scale end-to-end (server heap ticket: `demo/usecases/autoindex/scale/TICKET_server-unbounded-ingest-heap.md`).-- **Reference-code before you write.** For non-trivial engine work, retrieve how peer projects already solved the problem before writing your own version, and cite `file:line` for what you relied on. This repo dogfoods XERJ for it: reference projects (tantivy, meilisearch, quickwit, qdrant, usearch, hnswlib, sled, fjall, redb, ClickHouse) are indexed locally and queried with `xc.py <corpus> "<what you need>"`; the tooling and the pinned corpus definitions are in [`tools/xerj-code/`](tools/xerj-code/). Two rules bind regardless of tooling: **retrieved code is evidence, not authority** — adapt and attribute rather than paste; and **check the licence** — XERJ is Apache-2.0, so GPL-family sources (and Elasticsearch, which is AGPL/SSPL/Elastic-licensed) are approach-only, never copied. That second rule is also why XERJ can state it shares no code with Elasticsearch: ES is read for wire-protocol *semantics*, never for implementation. Honest scope: retrieval pays off on code a model has not memorised (internal, niche, post-cutoff) and is pure overhead on popular public libraries it already knows.+- **Reference-code before you write.** For non-trivial engine work, retrieve how peer projects already solved the problem before writing your own version, and cite `file:line` for what you relied on. This repo dogfoods XERJ for it: reference projects (tantivy, meilisearch, quickwit, qdrant, usearch, hnswlib, sled, fjall, redb, ClickHouse) are indexed locally and queried with one command — `xerj search "<what you need>"` against a running node (it detects the node and ranks symbol definitions first). For a shareable, pinned corpus definition across machines, the `xc-corpus.sh` / `xc.py` helpers in [`tools/xerj-code/`](tools/xerj-code/) wrap the same flow. Two rules bind regardless of tooling: **retrieved code is evidence, not authority** — adapt and attribute rather than paste; and **check the licence** — XERJ is Apache-2.0, so GPL-family sources (and Elasticsearch, which is AGPL/SSPL/Elastic-licensed) are approach-only, never copied. That second rule is also why XERJ can state it shares no code with Elasticsearch: ES is read for wire-protocol *semantics*, never for implementation. Honest scope: retrieval pays off on code a model has not memorised (internal, niche, post-cutoff) and is pure overhead on popular public libraries it already knows.- **Builds are scoped:** `cd engine && cargo build --release -j 32 -p <crate>` — never workspace-wide, never `cargo clean`.- **The hard gate:** the current CI ES-YAML conformance suite must stay at **0 failed**; pass and skip totals are reported by CI and grow as cases are added, so gate on failures, not on an exact total, before any engine change lands (see "Running the conformance tests" in the README).- **Git discipline:** non-trivial changes land with full commit bodies (motivation, before/after numbers, root cause, file pointers) — the git history is the project's engineering log; read it before re-deriving decisions.diff --git a/landing/llms.txt b/landing/llms.txtindex a97413f69..55bdb6d1d 100644--- a/landing/llms.txt+++ b/landing/llms.txt@@ -19,7 +19,7 @@ First run, in order (`autoindex` is a client of a running node at `http://localh1. `xerj --insecure --data-dir ~/xerj-data` — start a local dev node (no TLS, no auth). **Put the data dir OUTSIDE any folder you will autoindex**: `--data-dir ./data` inside a tree you then index makes the node index its own storage (`data/` is not in the built-in ignore list).2. `xerj autoindex <folder>` — make a folder searchable, zero config.-3. Query `http://localhost:9200` with any Elasticsearch client or plain HTTP.+3. Query it — `xerj search "<plain-English text>"` if you have a shell (one command: it finds the node, ranks definitions first, prints file:line + passage), or `POST http://localhost:9200/ax-*/_search` with any Elasticsearch client / plain HTTP.Then, from an unknown folder to an answered question: `xerj autoindex <folder>` (streaming extractors cover JSON/JSONL, CSV, structured logs, SQL dumps, SQLite, PDF, DOCX, HTML, XML, YAML, plain text and gzip variants) → `xerj autoindex map [--json]` for a briefing with per-field types, real example values and ready-to-send queries → `POST /ax-*/_search`. Index names are `<prefix>-<dataset-slug>` and `--prefix` defaults to `ax`, so give a corpus its own `--prefix` when you do not want it inside the shared `ax-*` wildcard. Re-run any time: the journal resumes and idempotent `_id`s never duplicate.