AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
release: cut 1.0.0-rc.14
xerj-org9b020903f704
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 73d7ca4b6..9160e83f7 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,8 +7,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+## [1.0.0-rc.14] - 2026-08-10+### Changed+- **One documented core and memory policy, and the worker knobs are live**+ ([#240](https://github.com/xerj-org/xerj/issues/240)). XERJ had no single+ answer to "how much of this machine may I take?", and the scattered answers it+ did have disagreed with each other — and, on Darwin, with the OS.+ `engine.flush_workers`, `engine.merge_workers` and `engine.search_workers`+ were documented but inert; they now drive real pool widths, and a value of `0`+ refuses startup instead of being silently reinterpreted. `--bulk-mb` above 24+ and `--workers 0` are likewise refused rather than clamped, so a request the+ engine will not honour fails loudly. Only hand-written configs are affected:+ the shipped `xerj.default.toml` has no `[engine]` section.++- **One 16 MiB file no longer costs 65 GB of RSS**+ ([#239](https://github.com/xerj-org/xerj/issues/239)). `split_sections` was+ quadratic in both time and memory: a 16 MiB text file with no blank line+ needed **65.6 GB of peak RSS and 16.3 s** to produce 16.8 MB of sections.+ Now linear — the same input takes **8.3 ms and 35 MB**.+- **BREAKING — `server.bind_address` now defaults to `127.0.0.1`, and acleartext node refuses to publish itself to the network**([#228](https://github.com/xerj-org/xerj/issues/228)). The old default was@@ -54,8 +73,78 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0instead, after the data directory, the `.xerj_*` system indices, the masterkey and a printed first-run `admin.key` already existed.+### Security++- **API key secrets are no longer stored in the clear**+ ([#201](https://github.com/xerj-org/xerj/issues/201)). `api_keys.json` held+ the secret verbatim. 0600 plus atomic rename is right for a secret a *process*+ owns, but a file has more readers than a process: a backup, a snapshot, a+ container layer, a support bundle, a decommissioned disk. Secrets are now+ stored as a salted SHA-256 (`$ssha256$<salt>$<digest>`) and compared in+ constant time, with existing plaintext records migrated on load. The fast hash+ is deliberate: the secret is two v4 UUIDs — 244 bits of CSPRNG output, never+ human-chosen — so offline guessing is out of reach whatever the hash costs,+ while this comparison runs on *every authenticated request*, where an Argon2id+ would add tens of milliseconds per request and hand out a free CPU-exhaustion+ DoS. The same release makes `_security/_authenticate` report the key's real+ identity instead of a hardcoded `superuser`, and gives the audit chain durable+ storage.++- **The node refuses to start when TLS is on but gRPC would answer in cleartext+ off-loopback** ([#229](https://github.com/xerj-org/xerj/issues/229)).+ `tls.enabled = true` encrypted two of the three data-plane listeners; the+ third never participated, because the gRPC server is built without tonic's+ `tls` feature. Nothing surfaced it — gRPC clients connected and worked+ identically either way — so an operator who enabled TLS and bound a network+ interface got no error, no failed handshake and no symptom, while API keys and+ document bodies crossed the network in the clear. Auth was never the gap;+ confidentiality was. Boot now fails closed unless+ `tls.allow_insecure_grpc_h2c` says otherwise.+### Fixed+- **`multi_match` returns the same hit set before and after `_flush`**+ ([#218](https://github.com/xerj-org/xerj/issues/218)). The memtable and+ segment paths disagreed on multi-token semantics, so flushing an index could+ change which documents a query matched — verified live on a one-document+ index where an OR query returned 0 hits before `_flush` and 1 after, with ES+ returning 1 throughout.++- **Dynamic string fields get Elasticsearch's default `.keyword` multi-field**+ ([#209](https://github.com/xerj-org/xerj/issues/209)). Dynamic mapping+ inferred every string as bare `text`, so `GET _mapping` never showed a+ `.keyword` sub-field and `_field_caps` never listed one. Anyone arriving from+ Elasticsearch — or any tool that reads `_field_caps` for field discovery,+ Kibana included — could not see that `category.keyword` existed.++- **`min_score` is honoured at `size:0`, and an external scalar N keeps its+ ghosts** ([#193](https://github.com/xerj-org/xerj/issues/193)). A+ `size:0 + min_score` body never materialised a hit, so the threshold had+ nothing to filter and `hits.total` reported the raw match count — measured at+ **65** where the same query at `size:100` counted **5**. The tie-break+ comparator item from that issue is deliberately not included here; it belongs+ with [#191](https://github.com/xerj-org/xerj/issues/191).++- **Index blocks can be removed, `read_only_allow_delete` means what its name+ says, and a blocked write answers 403 rather than 500.** Three compounding+ defects: `PUT /_settings` wrote only the display-side map while enforcement+ read the per-index `settings.json`, so a block was acknowledged, invisible to+ `GET /_settings`, and — with `_block` registered PUT-only — removable only by+ restarting with a hand-edited file. The one block that fires automatically did+ the inverse of its name, and clients hitting any of it were told the server+ had faulted.++- **`llms.txt` no longer tells agents six things the binary does not do.** The+ "Running an index for a human" section is executed, not just read: an agent+ follows it literally and then reports the result to its user. Corrected: the+ job-size line is not the first line printed; `--dry-run` does not write+ *nothing* (it creates and locks the state dir and appends a journal record);+ a dry run reuses a frozen plan instead of re-costing the job, so it is not a+ floor for the real run; exit `1` is the catch-all for every error, not+ "endpoint unreachable"; `status` needs the same explicit `--state-dir` the run+ used; and `--workers`/`--pdf-workers` bound the client-side extractor, not the+ server (with `--pdf-workers` capped to 1–4).+- **A failed index can now be inspected, deleted and retried without stoppingthe server** ([#206](https://github.com/xerj-org/xerj/issues/206)). An indexdirectory that refused to open at boot was recorded in a map that only@@ -143,6 +232,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Added+- **`autoindex` reports real progress, an honest percent and an ETA — no run is+ silent** ([#241](https://github.com/xerj-org/xerj/issues/241)). Between 47%+ and 64% of a typical run produced no output at all, which is the worst+ possible behaviour for the two audiences that matter: a person watching a+ laptop they cannot use, and an AI agent that has gone quiet on that person's+ behalf. Progress now covers phase, items done and total, rate and ETA, in both+ modes — a terminal gets a live redrawn line, a pipe gets periodic structured+ `xerj-progress` lines an agent can parse, and every run ends with a single+ `xerj-done` summary. The percent is bytes-based so a large file cannot make it+ stall at 99%, the first ETA is withheld for five seconds and labelled `rough`+ until there is enough evidence for it, and the line names the file currently+ being waited on. `--quiet` still produces exactly nothing on either stream,+ and `--progress none` is honoured everywhere — the resource policy's own+ startup notes were moved onto the same surface so they cannot leak into+ `--progress json`'s single parseable stream.+- **`/get` and `/get.ps1` are counted.** `functions/get.js` (a Cloudflare PagesFunction) serves the installer straight out of `landing/get`, unchanged, andrecords the request. The install path had never had a counter of any kind —diff --git a/engine/Cargo.lock b/engine/Cargo.lockindex 0c98c057d..3b18ec0b4 100644--- a/engine/Cargo.lock+++ b/engine/Cargo.lock@@ -6378,7 +6378,7 @@ dependencies = [[[package]]name = "xerj-ai"-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"dependencies = ["anyhow","candle-core",@@ -6399,7 +6399,7 @@ dependencies = [[[package]]name = "xerj-api"-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"dependencies = ["anyhow","async-trait",@@ -6429,7 +6429,7 @@ dependencies = [[[package]]name = "xerj-autoindex"-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"dependencies = ["anyhow","chrono",@@ -6471,7 +6471,7 @@ dependencies = [[[package]]name = "xerj-cluster"-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"dependencies = ["anyhow","async-trait",@@ -6496,7 +6496,7 @@ dependencies = [[[package]]name = "xerj-common"-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"dependencies = ["anyhow","byteorder",@@ -6515,7 +6515,7 @@ dependencies = [[[package]]name = "xerj-compress"-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"dependencies = ["byteorder","bytes",@@ -6528,7 +6528,7 @@ dependencies = [[[package]]name = "xerj-console-api"-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"dependencies = ["anyhow","arc-swap",@@ -6564,7 +6564,7 @@ dependencies = [[[package]]name = "xerj-engine"-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"dependencies = ["anyhow","arc-swap",@@ -6603,7 +6603,7 @@ dependencies = [[[package]]name = "xerj-fts"-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"dependencies = ["anyhow","bitpacking",@@ -6629,7 +6629,7 @@ dependencies = [[[package]]name = "xerj-logs"-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"dependencies = ["anyhow","byteorder",@@ -6647,7 +6647,7 @@ dependencies = [[[package]]name = "xerj-mcp"-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"dependencies = ["anyhow","reqwest",@@ -6657,7 +6657,7 @@ dependencies = [[[package]]name = "xerj-query"-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"dependencies = ["anyhow","base64 0.22.1",@@ -6672,7 +6672,7 @@ dependencies = [[[package]]name = "xerj-server"-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"dependencies = ["anyhow","axum",@@ -6718,7 +6718,7 @@ dependencies = [[[package]]name = "xerj-storage"-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"dependencies = ["anyhow","arc-swap",@@ -6748,7 +6748,7 @@ dependencies = [[[package]]name = "xerj-vector"-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"dependencies = ["anyhow","byteorder",@@ -6766,7 +6766,7 @@ dependencies = [[[package]]name = "xerj-wasm"-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"dependencies = ["anyhow","bytes",diff --git a/engine/Cargo.toml b/engine/Cargo.tomlindex ffe748be7..bbf394233 100644--- a/engine/Cargo.toml+++ b/engine/Cargo.toml@@ -21,7 +21,7 @@ members = [][workspace.package]-version = "1.0.0-rc.13"+version = "1.0.0-rc.14"edition = "2021"license = "Apache-2.0"authors = ["xerj Contributors"]