AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
release: cut v1.0.0-rc.5
xerj-org16d6df0e9da2
diff --git a/.github/workflows/release.yml b/.github/workflows/release.ymlindex c6fb1cd93..b716fe184 100644--- a/.github/workflows/release.yml+++ b/.github/workflows/release.yml@@ -11,11 +11,11 @@ on:# workflow's single linux-gnu build cannot, so a real tag never# discovers a broken target for the first time at release time.branches: [main]+ # Manual runs take no inputs: the version and the publish decision are both+ # derived from the ref you dispatch against (GITHUB_REF_NAME / `refs/tags/v*`).+ # Dispatch against the TAG ref to rebuild and republish a release; dispatch+ # against a branch for an artifacts-only cross-compile check.workflow_dispatch:- inputs:- tag:- description: 'Tag to build (e.g. v0.5.9)'- required: truepermissions:contents: writediff --git a/CHANGELOG.md b/CHANGELOG.mdindex 054dc4ca5..85a8cfeba 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,388 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+## [1.0.0-rc.5] - 2026-07-27++Fifth release candidate: the **real-client compatibility release**. Much of+it came from pointing real Elasticsearch and OpenSearch tooling at+XERJ — Kibana 8.13, Kibana OSS 7.10.2, OpenSearch Dashboards 2.11.1/3.6,+and their own shipped sample datasets (flights, eCommerce, logs) — and+fixing the wrong answers, 500s, and stalls those clients produced. The rest+is storage, ingest and `autoindex` work that did not come from client+testing. Headline for users: the query classes that silently matched+**zero** documents (booleans, `.keyword` multi-fields, `match_phrase` on+arrays — and keyword arrays before a flush, see the qualification below)+now return what ES returns; a real Kibana/OSD instance boots,+logs in, and saves objects against XERJ end to end; and a dashboard firing+several panel queries at once no longer stalls the node. ES-YAML+conformance holds at 1360 passed / 0 failed / 3 skipped. Zero-hit defects+found during this cycle that are **not** fixed here are listed under Known+limitations.++### Fixed — query semantics (classes that silently matched zero docs)++- **Keyword arrays (partial — memtable half only):** a `term`/`terms` query+ on a multi-valued keyword field only ever compared element `[0]` — the+ memtable stored just the first value and the segment keyword column is+ one ordinal per doc — so exact lookups returned silent false negatives+ while `terms` aggregations and `match` (which join all elements) looked+ correct. The memtable path is fixed: those doc-values lookups now bail+ for array fields and fall through to the array-aware source scan, so a+ `term` on a multi-valued keyword field is correct **before** a flush.+ Flushed segments are **not** fixed — the segment keyword column is still+ one ordinal per document, so after a flush a `term`/`terms` on a+ non-first array element can still silently miss. The complete fix needs+ multi-valued segment keyword columns (a storage-format change); the+ regression test+ (`xerj-engine` `test_term_matches_non_first_array_element`) is committed+ but `#[ignore]`d until that lands.+- **Booleans:** `term`/`match_phrase` on a boolean field undercounted to 0+ while a `terms` aggregation on the *same* field bucketed `true`/`false`+ correctly. The trigger was the count shortcut over memtable-resident+ data — i.e. any bulk import before an explicit `_flush`, which is what+ every real importer (OSD's own sample-data loader included) produces.+- **`.keyword` multi-fields:** the brute-force scan's field resolver had no+ multi-field fallback, so `term`/`match_phrase` on `category.keyword`,+ `manufacturer.keyword` and friends matched nothing — `_source` never+ contains a literal `"category.keyword"` key. It now strips the trailing+ segment and retries against the parent when the parent is a leaf value+ (guarded, so a genuinely absent nested/object field is still absent).+- **`match_phrase` / `match_phrase_prefix` on arrays:** both arms only+ handled scalar values, so an array-valued field never matched any+ phrase. ES semantics restored: the doc matches if any element does.+- **`match_phrase` with a non-string query value:** `{"query": true}` was a+ hard parse error surfaced as `search_phase_execution_exception` — the+ exact shape OSD's filter bar sends for a boolean filter pill, which+ broke every panel on the dashboard at once. Scalars are now coerced the+ way `match` already coerced them.+- **Empty query strings:** `match`/`match_phrase`/`match_phrase_prefix`+ with `""` returned 400; ES treats an empty analyzed query as zero terms.+ They now resolve to `match_none` (200, no hits) — this is what Kibana's+ saved-objects `_find?search=*` builds, so that endpoint works again.+- **`geo_point`:** aggregations and `geo_distance`/`geo_bounding_box`+ rejected string-encoded `{"lat": "50.03", "lon": "8.57"}` coordinates+ (the shape the flights sample data ships), and `geo_centroid`/+ `geo_bounds` used a flat lookup that never found a nested geo field.+ Both now behave like ES.+- **Aggregation scripts:** `terms` required a top-level `field` and+ returned `{"buckets": []}` whenever it was absent — any script-based+ terms aggregation silently produced no buckets. `script` is now a real+ key source alongside `field`.+- **Painless date accessors:** `doc['t'].value` returned a plain string, so+ `.getHour()`, `.getDayOfWeek()`, `.getYear()` and the rest of the common+ accessor set failed the whole script. They now parse and extract the+ requested UTC component.+- **Dynamic date mapping:** ISO-8601 strings are inferred as `date` instead+ of `text`; `date_detection: false` was accepted-but-ignored and is now+ honored; a non-date value written into an inferred date field returns+ `mapper_parsing_exception` instead of being silently stored and then+ being invisible to range queries, sorts, and time filters; and+ `date_detection` survives the `PUT /_mapping` merge path.+- **Segment-path parity:** projected FTS bool queries dropped+ `minimum_should_match` (so `match_bool_prefix` with `mm=3` matched every+ single-term hit after flush) and doc-values wildcards did not case-fold;+ nested kNN returned zero hits for any parent doc living in a segment+ because the reassembled `{_id, _seq_no, _source}` shape was not+ unwrapped. `_count` now resolves `terms` lookups.++### Fixed — API surface (what real clients call)++- **Aliases:** single-index endpoints (`GET /{alias}`, `/_mapping`,+ `/_settings`, …) never resolved an alias to its backing index, and+ aliases were never persisted at all — index data survived a restart but+ the alias pointing at it did not. Aliases now resolve everywhere ES+ accepts them and are written to `aliases.json` (atomic temp+rename).+ Together this unsticks a fresh OSD container looping on "Another+ OpenSearch Dashboards instance appears to be migrating the index".+- **`_field_caps`:** never listed declared multi-fields (so `fields=*`+ omitted every `.keyword` entry and index-pattern refresh reported+ "field not found"), and only special-cased the literal `*`/`_all`+ wildcards — any real glob such as `wiki-test*` returned empty caps.+ Both fixed; wildcard/comma index specs also now resolve on 9 further+ endpoints that previously 404'd, 400'd, or — for `_refresh` and+ `_cat/count` — silently reported success / zero documents while doing+ nothing (`_refresh` answered `{"_shards":{"successful":1,…}}` without+ refreshing; `_cat/count` returned 0, indistinguishable from a genuinely+ empty index).+- **`POST /{index}/_update`** never returned a `get` block, so every Kibana+ saved-object write crashed client-side reading `body.get._source`. It is+ now returned exactly when the caller passes `_source`/`_source_includes`/+ `_source_excludes`, matching ES (absent otherwise, so `_doc` responses+ are byte-identical to before).+- **`_source` vs `stored_fields`:** the implicit `_source` suppression that+ `stored_fields` triggers is now only a default — an explicit top-level+ `_source` in the same request wins, as in ES. This is what starved+ almost every column in OSD's Discover.+- **`_bulk`** reports each item's real `_seq_no` and `_version` instead of+ a wall-clock microsecond timestamp and a hardcoded `1`.+- **`_update_by_query` / `_delete_by_query`** honor+ `wait_for_completion=false` and return the async `{"task": …}` form;+ `GET /_tasks/{id}` reports completion with the final response.+- **`/_xpack`** respects `--compat-version` (it had its own hardcoded+ `8.13.0`, which Kibana OSS 7.10.2 surfaced as an opaque "license not+ available" refusal to start), and `/_xpack` + `/_xpack/usage` report the+ real auth state instead of `security.enabled: true` — so `--insecure`+ no longer makes Kibana render a login screen for auth that is off.+- **Missing HTTP verbs:** `_refresh`, `_analyze`, `_msearch` accept GET and+ `_clone`/`_shrink`/`_split` accept PUT, as ES does — the PUT `_clone`+ 405 was a fatal error during Kibana's saved-object migration.+- **Login-path endpoints:** HTTP Basic auth (Kibana's interactive realm),+ `POST /_security/profile/_activate`, `GET /_security/profile/{uid}`,+ `GET`/`POST /_security/user/_has_privileges`, and a real+ `GET`/`PUT`/`DELETE /_security/privilege` store with ES-shaped 404s —+ each of these was a 500 or a hang on the Kibana login and home pages.+- **`GET /_cat/templates/{pattern}`** (a 404 that crashed OSD) and+ **`POST /_index_template/_simulate`** (body-only template preview) added.++### Changed — index resolution (can break existing callers)++These three align index addressing with ES 8 and turn some+previously-succeeding requests into errors. Review any caller that relies+on the old behavior.++- **A comma-separated index spec now 404s when a concrete name is+ missing**, implementing ES's default `ignore_unavailable=false`. The+ missing name used to be dropped silently — `POST /real,typo/_refresh`+ answered `200` with `_shards.total: 1`. Wildcard/`_all` specs that match+ nothing are still a valid empty result (ES's `allow_no_indices` default).+- **Wildcard and `_all` expansion no longer sweeps in hidden+ (dot-prefixed) indices**, so an operation like `POST /*/_close` can no+ longer hit `.xerj_users`-class system indices. A dot-prefixed pattern+ opts back in, as ES's hidden addressing does.+- **`POST /_close` refuses a wildcard or `_all` target outright** with ES+ 8's `action.destructive_requires_name` `illegal_argument_exception`.++### Added++- **OpenSearch client auto-sensing.** XERJ answers the identity endpoints+ (`GET /`, `GET /_nodes`) per request based on the caller's User-Agent, so+ one running instance serves an `opensearch-py`/`opensearch-js`/OSD client+ and an Elasticsearch client simultaneously, each seeing a+ version/distribution block its own compatibility gate accepts. Explicit+ `--compat-distribution` / `--compat-version` (and the matching env vars)+ still pin the identity for every client. The `x-elastic-product` header+ is no longer sent to a detected OpenSearch caller.+- **Opt-in ingest memory attribution (developer feature).** Setting+ `XERJ_INGEST_MEMORY_TRACE=summary` emits a bounded `xerj.ingest_memory.v1`+ NDJSON ledger — per-owner logical bytes across HTTP bodies, raw and+ parsed semantic sources, prepared docs and vectors, active memtables and+ drained flush snapshots — alongside jemalloc allocated/active/resident,+ RSS, CPU time, and accounting/dropped-event counters, plus a separate+ `/proc`-derived `xerj.process_sample.v1` stream. Default is fully off.+ Merge and read-cache owners are reported `unavailable`, not zero. This+ makes ingest retention *measurable*; it is not itself a memory bound.+ A deterministic bounded diagnostic suite+ (`demo/usecases/autoindex/scale/bounded/`) drives ingest → refresh →+ flush → force-merge → restart with exact count and sentinel checks.+- **Private pprof debugging toolkit.** A Linux-only `debug-profiling`+ feature compiles in bounded CPU and jemalloc heap profiling with no+ network endpoint; artifacts are written mode-0600 to an operator-created+ directory, with `capture.py`/`inspect.py` wrappers that hash the binary+ and every artifact. Not in any shipped feature set; no runtime overhead+ or speedup is claimed.+- **Experimental ONNX embedding backend**, wired through `autoindex` behind+ the `onnx-experimental` feature. Its lazy session initialization is+ cancellation-safe: model loading moved to one process-owned thread whose+ single terminal result is shared, so a cancelled first request can no+ longer strand every later one.+- The ES-YAML conformance runner now exits non-zero when any case fails+ (it previously reported failures but exited 0).++### Changed — performance (concurrent dashboard bursts)++- **Full-corpus aggregations no longer deep-clone the memtable under the+ shard lock.** The `need_full_corpus` path (any request with `aggs` that+ the columnar fast path can't serve) cloned every buffered document's JSON+ tree while holding the per-shard read guard, serialising concurrent+ panels behind O(docs) work each. It now Arc-shares out under the lock and+ clones after releasing it. Measured: 15 concurrent `date_histogram`+ queries against a real ~14k-doc memtable-resident index, 1.2–2.7 s →+ ~150–170 ms (commit `6a54cf5`).+- **Full-corpus aggregations no longer re-decode segments per query.** The+ same path did an unconditional open + decompress + full `serde_json`+ parse of a segment's entire stored section on every request once data had+ flushed. Segments are immutable, so it now uses the existing+ single-flight stored-value cache. Measured: 20 concurrent requests+ against a real flushed eCommerce segment, 15–26 s → 1–4 ms including the+ cold first hit (commit `f6daf70`). A segment read failure during+ full-corpus assembly is now a hard error rather than a silent skip that+ undercounted every bucket.+- **The scan path no longer stalls on cold, concurrent bursts.** Two+ compounding costs: every scanned doc's `_source` was deep-cloned to+ splice `_id` in even though only a deeply-nested `ids` clause needs it+ (now computed once per scan from the query shape), and the raw-fallback+ decode had no single-flight protection, so N concurrent requests against+ a cold segment each paid the full open+decompress. Measured against real+ Kibana OSS 7.10.2 and eCommerce sample data: a 24-concurrent+ `query_string`+`match_phrase` burst immediately after restart (worst-case+ cold caches) went from ~4.5 s per request to 100–165 ms, with the warm+ repeat at 1–5 ms (commit `7e963ff`).+- **Semantic and vector work is bounded by the request.** One absolute+ deadline computed at search entry is carried through single-flight+ waiting, admission, embedding, hybrid/multi-kNN recursion, and exact+ scanning; partials set `timed_out` and `hits.total.relation: gte`, and+ the ES handler aborts its child task when the request is dropped.+ Previously a cancelled semantic search could drain for minutes past its+ timeout. Cold vector segment loads also moved to bounded `spawn_blocking`+ producers so they can no longer pin every async worker.++### Fixed — storage, durability & ingest++- **Raw ingest validates before it publishes.** Caller bytes were appended+ to the WAL before being proven to be complete JSON, so malformed input+ could reserve sequence numbers and become durable, with the later parse+ path silently substituting `{}`. Whole-batch UTF-8/JSON/nesting+ validation now completes before any sequence, WAL, version-map, memtable,+ or schema mutation, and malformed bulk sources return per-item+ `document_parsing_exception`. The parsed `Value` is now the single+ authority for turbo ingest (live indexing and WAL replay could previously+ disagree), and `copy_to` is applied before WAL publication so GET,+ search, flush, replay, and restart all observe the same source.+- **Per-shard WAL buffers bounded to 64 KiB** (was 8 MiB). Every index+ eagerly opens one writer per ingest shard, so a 16×16 default reserved+ ~2 GiB of allocator capacity before a single document was written;+ capacity above a frame cannot batch across requests because every+ acknowledged append already drains the buffer. Measured on 256 empty+ writers: jemalloc allocated 2,147,503,496 → 16,797,064 bytes (−99.2%),+ with ingest throughput ratios of 0.976 (single shard) and 1.020+ (eight shards) and exact replay preserved (commit `761b915`). Cost:+ records larger than the 64 KiB buffer issue more write syscalls+ (measured 1,372 → 3,372 on 131 KiB × 1000), with throughput unchanged.+- **Same-ID writes are serialized.** A keyed publication coordinator now+ spans the current-state/CAS check through WAL, version map, FTS, and HNSW+ publication for single-document paths, closing races that could admit two+ creates, lose an update patch, or let a delete overtake a write. Scripted+ `_update` and `_update_by_query` run through the same boundary, so two+ concurrent `ctx._source.n += 1` requests can no longer both read `n=0`.+ Distinct IDs do not contend. (Turbo batch paths remain follow-up work.)+- **Semantic vectors stay out of full-text indexes.** The dynamic-field+ walkers treated pooled embeddings and their `_chunks` companions as+ ordinary text and fed every float into FTS as a decimal token. In one+ failed diagnostic run that produced 2.338 GiB of vector FSTs inside a+ 3.449 GiB partial index. Embedding outputs are now excluded by schema+ designation (not a name heuristic) across pre-analysis, flush, and merge.+ Existing vector FST sidecars are not rewritten on upgrade — reindexing is+ the safe way to reclaim them.+- **The admin API key persists across restarts.** The server wrote+ `data_dir/admin.key` but never read it back, minting a fresh key on every+ start and locking out every client configured against the previous one.+ An existing well-formed key is now reused; anything missing or malformed+ still falls through to fresh generation.++### Fixed — autoindex++- **Content deduplication with crash-safe replacement.** Byte-identical+ paths were indexed independently (one reproduction journaled 2,442+ records for 1,221 live documents). Sources are now hashed with streaming+ XXH3-128 and digest peers byte-verified before one canonical path is+ chosen; every current name is preserved in `ax_paths`. Publication became+ an explicit durable generation transaction (synced `file_replace_start`+ → staged, source-verified extraction → synced `file_done` commit), and+ journal replay repairs torn tails and rolls back failed appends.+ Ambiguous legacy prefix collisions fail closed before any backend+ mutation instead of guessing.+- **Follow-on fixes to that rework:** resume now guarantees exclusive+ plan-key ownership (two files resolving to one key each ran the+ replacement transaction and deleted the other's documents);+ `delete_by_query` repeats until a pass deletes nothing, because the+ server implements it as a single `size:10000` pass and >10k-doc+ generations left permanent ghost documents; and both recoverable error+ paths now name the offending files or journal offset instead of+ advising a blanket `--fresh` re-extract.+- **PDFs are parsed by `pdf_oxide` in isolated same-binary workers.** The+ previous byte scanner interpreted font character codes without resolving+ page resources or ToUnicode maps, producing NUL-separated and shifted+ text that schema inference (correctly) refused to elect as+ `semantic_text`. Workers get process groups, a 1536 MiB `RLIMIT_AS`, and+ descendant kill/reap on timeout — documented as resource isolation, not+ a security sandbox, with no OS memory cap on non-Unix. New+ `--pdf-workers` and `--pdf-timeout-secs` flags. Cost: the no-default+ release binary grows 34,160,328 → 39,018,184 bytes (+4.6 MiB).+- **Text files are classified by sentence density, not line length.** The+ old `avg_len > 60` rule split markdown across two datasets with two+ different field names — adding `## headings` to a document made it *more*+ likely to be treated as a record stream, which split BM25 statistics+ across two corpora. Terminal-punctuation density separates prose+ (0.43–0.57 on the measured corpus) from logs and source (0.00–0.20);+ CSV rows fall on the record-stream side of the same threshold.++### Changed — autoindex (what it produces)++- **Line-oriented text is chunked into overlapping windows** (40 lines, 10+ overlap) instead of one document per line, and prose sections dropped+ from 32 KB to 2 KB with overlap, because BM25 scores per document and a+ single line rarely contains the caller's own wording. Measured on this+ repository (234 files, 170k LOC + docs + 460 commit messages) with 8+ "where/why is X" questions: 3/8 → 7/8 answered, 162,883 → 5,508 records,+ indexing 234 s → 1.9 s (commit `4510189`). Every chunk carries+ `start_line`/`end_line`.+- **`semantic_text` election is by language, not length.** The rule was+ "largest `text` field with `avg_len >= 200`", which elected 300-character+ base64 and concatenated-id columns while skipping a genuinely semantic+ 150-character summary. A field must now actually look like natural+ language — `word_ratio >= 0.55` (tokens matching `[A-Za-z]{3,}`) **and**+ `mean_tokens >= 3`. Measured `word_ratio` is 0.00 for+ trace_id/user_id/order_id/numeric columns and 0.78–1.00 for prose, log+ messages and source. This changes which field gets embedded, so it+ affects both semantic-search quality and ingest cost (the built-in neural+ backend measures ~2.8 docs/s). The election note records the numbers that+ drove the decision.++### Added — autoindex++- **`--bulk-timeout-secs`** (default 300, range 1–3600) applies only to+ `POST /_bulk`, so a legitimately slow neural bulk can be accommodated+ without loosening the deadline on control requests. Retries are bounded+ at six attempts with the identical body and deterministic document IDs.++### Docs++- **WordPress core security audit case study**+ (`docs/case-studies/wordpress-security-audit/`): a reproducible record of+ an AI agent auditing real WordPress core (1,492 PHP files, ~619k lines)+ with XERJ as the retrieval substrate — sink census, interprocedural taint+ analysis, authorization graph, POP-gadget hunt — plus a copyable Claude+ Code skill and a step-by-step playbook. The honest result is that **core+ came back hardened**: the documented findings are three Medium-severity,+ known-class items (an incomplete SSRF deny-list missing `169.254.0.0/16`,+ an ImageMagick parse surface, and an unguarded `role` sink in+ `user-new.php`), a fourth downgraded to not-reachable-in-core, and+ verified negatives for IDOR, SQL de-escaping, and sanitizer composition.+ None of it is claimed as a novel 0-day. The published grep-wins+ counter-examples are shown alongside the wins, and the XERJ bug the audit+ surfaced (the keyword-array `term` defect partially fixed above — the+ memtable half) is disclosed.+ Site use case 06 (`/use-cases/code-security-audit.html`) is built from it.+- **Token-usage guidebook** (`docs/TOKEN_USAGE.md`): a measured decomposition+ of what a XERJ answer costs an agent in tokens — envelope overhead,+ answer, and materialized intermediate data — instead of hand-waving.+- **AST + graph + FTS vulnerability research**+ (`docs/research/ast-graph-vuln-detection.md`, `docs/examples/ast-vuln-graph/`),+ including a multi-language tree-sitter taint scanner tested at real+ WordPress scale.+- Verified embedding examples: XERJ with Google AI (EmbeddingGemma, Gemini+ API, ADK) and with any OpenAI-compatible `/v1/embeddings` endpoint.++### Known limitations++Known-open items found while validating this release and **not** fixed in+it. The first three are zero-hit defects — a query returns no documents+that ES would return:++- **`multi_match` looks inverted.** On a 6,022-document index a long query+ returned 0 hits with `operator: "or"` (the ES default) and 2 hits with+ `operator: "and"`. OR must be a superset of AND.+- **`match` against a `semantic_text` field returns 0 hits** for a long+ natural-language string, while the same query against a plain `text`+ field returns thousands.+- **`term`/`terms` on a multi-valued keyword field can still miss after a+ flush** — only the memtable half of that fix landed (see the first bullet+ of "Fixed — query semantics").+- **`autoindex` dataset clustering merges same-shape files** with different+ subjects: 213 source files sharing the schema `{text}` collapse to one+ index, and embedding centroids separate the subject groups only weakly.+- 3 of 1,363 ES-YAML conformance cases are skipped.+## [1.0.0-rc.4] - 2026-07-22Fourth release candidate: the **production-hardening release**. A 9-review@@ -251,5 +633,9 @@ for evaluation, but may still change before the final 1.0.0.[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) and `engine/CLAUDE.md` for thecurrent supported surface.-[Unreleased]: https://github.com/xerj-org/xerj/compare/v1.0.0-rc.1...HEAD+[Unreleased]: https://github.com/xerj-org/xerj/compare/v1.0.0-rc.5...HEAD+[1.0.0-rc.5]: https://github.com/xerj-org/xerj/releases/tag/v1.0.0-rc.5+[1.0.0-rc.4]: https://github.com/xerj-org/xerj/releases/tag/v1.0.0-rc.4+[1.0.0-rc.3]: https://github.com/xerj-org/xerj/releases/tag/v1.0.0-rc.3+[1.0.0-rc.2]: https://github.com/xerj-org/xerj/releases/tag/v1.0.0-rc.2[1.0.0-rc.1]: https://github.com/xerj-org/xerj/releases/tag/v1.0.0-rc.1diff --git a/engine/Cargo.lock b/engine/Cargo.lockindex 59b1e5064..cd412f7cc 100644--- a/engine/Cargo.lock+++ b/engine/Cargo.lock@@ -6170,7 +6170,7 @@ dependencies = [[[package]]name = "xerj-ai"-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"dependencies = ["anyhow","candle-core",@@ -6191,7 +6191,7 @@ dependencies = [[[package]]name = "xerj-api"-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"dependencies = ["anyhow","async-trait",@@ -6220,7 +6220,7 @@ dependencies = [[[package]]name = "xerj-autoindex"-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"dependencies = ["anyhow","chrono",@@ -6247,7 +6247,7 @@ dependencies = [[[package]]name = "xerj-cluster"-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"dependencies = ["anyhow","async-trait",@@ -6269,7 +6269,7 @@ dependencies = [[[package]]name = "xerj-common"-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"dependencies = ["anyhow","byteorder",@@ -6287,7 +6287,7 @@ dependencies = [[[package]]name = "xerj-compress"-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"dependencies = ["byteorder","bytes",@@ -6300,7 +6300,7 @@ dependencies = [[[package]]name = "xerj-console-api"-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"dependencies = ["anyhow","arc-swap",@@ -6336,7 +6336,7 @@ dependencies = [[[package]]name = "xerj-engine"-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"dependencies = ["anyhow","arc-swap",@@ -6375,7 +6375,7 @@ dependencies = [[[package]]name = "xerj-fts"-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"dependencies = ["anyhow","bitpacking",@@ -6401,7 +6401,7 @@ dependencies = [[[package]]name = "xerj-logs"-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"dependencies = ["anyhow","byteorder",@@ -6419,7 +6419,7 @@ dependencies = [[[package]]name = "xerj-mcp"-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"dependencies = ["anyhow","reqwest",@@ -6429,7 +6429,7 @@ dependencies = [[[package]]name = "xerj-query"-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"dependencies = ["anyhow","base64 0.22.1",@@ -6444,7 +6444,7 @@ dependencies = [[[package]]name = "xerj-server"-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"dependencies = ["anyhow","axum",@@ -6490,7 +6490,7 @@ dependencies = [[[package]]name = "xerj-storage"-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"dependencies = ["anyhow","arc-swap",@@ -6520,7 +6520,7 @@ dependencies = [[[package]]name = "xerj-vector"-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"dependencies = ["anyhow","byteorder",@@ -6538,7 +6538,7 @@ dependencies = [[[package]]name = "xerj-wasm"-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"dependencies = ["anyhow","bytes",diff --git a/engine/Cargo.toml b/engine/Cargo.tomlindex d8cc59723..6fc16761f 100644--- a/engine/Cargo.toml+++ b/engine/Cargo.toml@@ -21,7 +21,7 @@ members = [][workspace.package]-version = "1.0.0-rc.4"+version = "1.0.0-rc.5"edition = "2021"license = "Apache-2.0"authors = ["xerj Contributors"]diff --git a/engine/crates/xerj-api/src/stub.rs b/engine/crates/xerj-api/src/stub.rsindex b1006d562..89c97a418 100644--- a/engine/crates/xerj-api/src/stub.rs+++ b/engine/crates/xerj-api/src/stub.rs@@ -59,7 +59,7 @@ pub fn not_implemented_yet("reason": format!("{feature} is not implemented in this xerj build. {note}"),"feature": feature,"planned_milestone": milestone,- "roadmap": "https://github.com/xerj-ai/xerj/blob/main/engine/reports/PATH_TO_100_PCT_v0.6.0_to_v1.0.md",+ "roadmap": "https://github.com/xerj-org/xerj/blob/main/engine/reports/PATH_TO_100_PCT_v0.6.0_to_v1.0.md",},"status": 501,});diff --git a/engine/crates/xerj-server/tests/pdf_worker.rs b/engine/crates/xerj-server/tests/pdf_worker.rsindex 5c53085ff..32a462aa1 100644--- a/engine/crates/xerj-server/tests/pdf_worker.rs+++ b/engine/crates/xerj-server/tests/pdf_worker.rs@@ -96,7 +96,7 @@ fn pdf_worker_is_fresh_process_deterministic_and_preserves_page_provenance() {assert_eq!(first.stdout, second.stdout);let response: Value = serde_json::from_slice(&first.stdout).unwrap();- assert_eq!(response["extractor"], "xerj-autoindex/1.0.0-rc.4");+ assert_eq!(response["extractor"], "xerj-autoindex/1.0.0-rc.5");assert_eq!(response["parser"], "pdf_oxide/0.3.75");assert!(response["containment"].as_str()diff --git a/engine/reports/SECURITY_DEPLOYMENT_MODEL.md b/engine/reports/SECURITY_DEPLOYMENT_MODEL.mdindex 7d07adb5c..1b55f67d0 100644--- a/engine/reports/SECURITY_DEPLOYMENT_MODEL.md+++ b/engine/reports/SECURITY_DEPLOYMENT_MODEL.md@@ -173,7 +173,7 @@ gap during procurement review.| Stack overflow on deep query | Query-depth cap = 64 (enforced) | Low — covered |If you have a threat we haven't considered, file an issue at-[github.com/xerj-ai/xerj/issues](https://github.com/xerj-ai/xerj/issues)+[github.com/xerj-org/xerj/issues](https://github.com/xerj-org/xerj/issues)or email [email protected] (use PGP key from [/security/pgp.txt]).---diff --git a/landing/demo/index.html b/landing/demo/index.htmlindex 02bed2145..e9737e857 100644--- a/landing/demo/index.html+++ b/landing/demo/index.html@@ -245,7 +245,7 @@ footer .links { display: flex; gap: 28px; flex-wrap: wrap; }<a href="/">Home</a><a href="/architecture/">Architecture</a><a href="/pricing/">Pricing</a>- <a href="https://github.com/xerj-ai/xerj">GitHub</a>+ <a href="https://github.com/xerj-org/xerj">GitHub</a></span></nav>@@ -256,7 +256,7 @@ footer .links { display: flex; gap: 28px; flex-wrap: wrap; }Pull a real production log from <a href="https://github.com/logpai/loghub">loghub</a>.Ingest 12 GiB into Xerj with one CLI command. Query it from Xerj Console.Each step shows the Elasticsearch alternative — resources, time, config —- sourced from <a href="https://github.com/xerj-ai/xerj/tree/main/user-feedback">collected community feedback</a>+ sourced from <a href="https://github.com/xerj-org/xerj/tree/main/user-feedback">collected community feedback</a>(G2, Gartner, TrustRadius, GitHub issues, Reddit, HN, Jepsen, CVE).</p><div class="stats">@@ -271,7 +271,7 @@ footer .links { display: flex; gap: 28px; flex-wrap: wrap; }Every number above + every CLI block on this page was captured live on thismachine on <strong style="color: var(--y);">2026-04-27</strong> against the<code>v1.0.0-rc.1</code> binary (commit <code>78a9bcc</code>). Full- transcript: <a href="https://github.com/xerj-ai/xerj/blob/main/demo/CLI_VALIDATED_2026-04-27.md"><code>demo/CLI_VALIDATED_2026-04-27.md</code></a>.+ transcript: <a href="https://github.com/xerj-org/xerj/blob/main/demo/CLI_VALIDATED_2026-04-27.md"><code>demo/CLI_VALIDATED_2026-04-27.md</code></a>.</p></header>@@ -325,7 +325,7 @@ footer .links { display: flex; gap: 28px; flex-wrap: wrap; }because no real LLM-traffic capture ships with the repo, and theAI / vector / agent / anomaly / ingest dashboards need that shapeto render meaningfully. Generator:- <code><a href="https://github.com/xerj-ai/xerj/blob/main/demo/data/extras/generate_demo_corpus.py">demo/data/extras/generate_demo_corpus.py</a></code>.+ <code><a href="https://github.com/xerj-org/xerj/blob/main/demo/data/extras/generate_demo_corpus.py">demo/data/extras/generate_demo_corpus.py</a></code>.</div></div></div>@@ -349,7 +349,7 @@ footer .links { display: flex; gap: 28px; flex-wrap: wrap; }is already running on port 9200.</p>- <pre class="cmd"><span class="prompt">$ </span>curl -L -o xerj.tar.gz https://github.com/xerj-ai/xerj/releases/download/v1.0.0-rc.1/xerj-1.0.0-rc.1-x86_64-unknown-linux-gnu.tar.gz+ <pre class="cmd"><span class="prompt">$ </span>curl -L -o xerj.tar.gz https://github.com/xerj-org/xerj/releases/download/v1.0.0-rc.1/xerj-1.0.0-rc.1-x86_64-unknown-linux-gnu.tar.gz<span class="prompt">$ </span>tar xzf xerj.tar.gz<span class="prompt">$ </span>./xerj --insecure --data-dir ./data<span class="out">2026-04-27T19:11:41.6Z INFO xerj: xerj v1.0.0-rc.1 starting@@ -428,7 +428,7 @@ Ready: /home/claude/ai/xerj/engine/demo-data/ssh_big.ndjson</span></pre><code>@timestamp · host · proc · pid · event · user · src_ip · src_port · message</code>.Event tags (<code>auth_failure_invalid_user</code>, <code>possible_break_in</code>,<code>auth_success</code>, …) come from a small regex parser- (<a href="https://github.com/xerj-ai/xerj/blob/main/engine/demo-data/build_corpus.py"><code>build_corpus.py</code></a>).+ (<a href="https://github.com/xerj-org/xerj/blob/main/engine/demo-data/build_corpus.py"><code>build_corpus.py</code></a>).</p><div class="se-says">@@ -476,7 +476,7 @@ Ready: /home/claude/ai/xerj/engine/demo-data/ssh_big.ndjson</span></pre><h3>Sustained scale — full 60.9 M / 12 GiB</h3><p style="font-size: 13px; color: var(--dim); margin: 4px 0 12px;">- Output below is from the <a href="https://github.com/xerj-ai/xerj/blob/main/engine/reports/2026-04-25T22-50-00_xerj_vs_elasticsearch_rerun.md">2026-04-25 head-to-head bench</a>+ Output below is from the <a href="https://github.com/xerj-org/xerj/blob/main/engine/reports/2026-04-25T22-50-00_xerj_vs_elasticsearch_rerun.md">2026-04-25 head-to-head bench</a>(commit <code>73c6367</code>, pre-rc). We did <em>not</em> re-run the 22-minute 60 M ingeston 2026-04-27 — burst rate above is the only ingest measurementre-validated against rc.1. Reproduce with@@ -656,7 +656,7 @@ node demo/scripts/full-demo-flow.jssonnet / opus / gpt-5 numbers come from?". Honest answer: the demoflow ingests six NDJSON corpora into the engine before the SPAopens. The corpora are generated reproducibly (random.seed=42)- from <code><a href="https://github.com/xerj-ai/xerj/blob/main/demo/data/extras/generate_demo_corpus.py">demo/data/extras/generate_demo_corpus.py</a></code>+ from <code><a href="https://github.com/xerj-org/xerj/blob/main/demo/data/extras/generate_demo_corpus.py">demo/data/extras/generate_demo_corpus.py</a></code>and pushed in via the <strong>same ES-compat <code>/_bulk</code> endpoint</strong>that Logstash, Filebeat, every ES SDK and your existing collectorsalready speak — there's no Xerj-specific ingest API, no SDK to@@ -759,7 +759,7 @@ node demo/scripts/full-demo-flow.jsthe corpus, the dashboard changes. Delete the index, the SPA'slive adapter returns <code>null</code> and the panel collapses toits mock skeleton (visible difference, no silent fallback). Code- path: <code><a href="https://github.com/xerj-ai/xerj/blob/main/xerj-ux/src/data/backends/xerj.js">xerj-ux/src/data/backends/xerj.js</a></code>+ path: <code><a href="https://github.com/xerj-org/xerj/blob/main/xerj-ux/src/data/backends/xerj.js">xerj-ux/src/data/backends/xerj.js</a></code>function <code>liveAiOverview()</code> at line ~378.</p>@@ -781,7 +781,7 @@ node demo/scripts/full-demo-flow.js(4 000 · VECTOR · INDEX), <code>ingest-events</code> (12 000 · INGEST · PIPELINE)— <strong>25 648 documents total</strong>. Every dashboard belowqueries one or more of these indices. Full transcript:- <a href="https://github.com/xerj-ai/xerj/blob/main/demo/INGEST_FLOW_2026-04-27.md"><code>demo/INGEST_FLOW_2026-04-27.md</code></a>.+ <a href="https://github.com/xerj-org/xerj/blob/main/demo/INGEST_FLOW_2026-04-27.md"><code>demo/INGEST_FLOW_2026-04-27.md</code></a>.</p><h3>3.5.6 — Every AI dashboard reads from those indices</h3>@@ -988,7 +988,7 @@ node demo/scripts/full-demo-flow.js<div class="small">Direct doc-values column scan + topK heap. Samelatency profile on 60 M docs as on 100 K (block-skip indices).No JVM cache warmup, no per-query GC pressure spike.</div>- <div class="source">measured: <a href="https://github.com/xerj-ai/xerj/blob/main/engine/reports/2026-04-25T03-30-00_xerj_vs_elasticsearch_bench.md">2026-04-25 head-to-head bench</a> (terms agg p50)</div>+ <div class="source">measured: <a href="https://github.com/xerj-org/xerj/blob/main/engine/reports/2026-04-25T03-30-00_xerj_vs_elasticsearch_bench.md">2026-04-25 head-to-head bench</a> (terms agg p50)</div></div><div class="es-side"><div class="label">Elasticsearch · terms agg p50</div>@@ -1103,10 +1103,10 @@ node demo/scripts/full-demo-flow.js<strong style="color: var(--y);">Source-of-truth:</strong> the three Xerj numbers below werere-captured on <code>v1.0.0-rc.1</code> (commit <code>78a9bcc</code>) on<strong>2026-04-27</strong> — see- <a href="https://github.com/xerj-ai/xerj/blob/main/demo/CLI_VALIDATED_2026-04-27.md"><code>demo/CLI_VALIDATED_2026-04-27.md</code></a>+ <a href="https://github.com/xerj-org/xerj/blob/main/demo/CLI_VALIDATED_2026-04-27.md"><code>demo/CLI_VALIDATED_2026-04-27.md</code></a>§9 / §11. ES comparator numbers are unchanged from the 2026-04-25head-to-head bench- (<a href="https://github.com/xerj-ai/xerj/blob/main/engine/reports/2026-04-25T22-50-00_xerj_vs_elasticsearch_rerun.md"><code>2026-04-25T22-50-00_xerj_vs_elasticsearch_rerun.md</code></a>);+ (<a href="https://github.com/xerj-org/xerj/blob/main/engine/reports/2026-04-25T22-50-00_xerj_vs_elasticsearch_rerun.md"><code>2026-04-25T22-50-00_xerj_vs_elasticsearch_rerun.md</code></a>);if a prospect wants methodology, send that file.</p>@@ -1194,7 +1194,7 @@ node demo/scripts/full-demo-flow.js<p class="why">The migration story. Xerj implements the Elasticsearch wire protocol —1305 of 1329 official ES YAML conformance tests pass on- <a href="https://github.com/xerj-ai/xerj/tree/main/engine/tests/es-compat-yaml">the ES 8.13 test suite</a>.+ <a href="https://github.com/xerj-org/xerj/tree/main/engine/tests/es-compat-yaml">the ES 8.13 test suite</a>.That means existing curl, official ES SDKs (Python, Go, Java, JS), andeven Kibana itself can point at port 9200 unchanged.</p>@@ -1213,7 +1213,7 @@ node demo/scripts/full-demo-flow.js<div class="num">1305 / 1329</div><div class="small">98.2 % of ES 8.13 official YAML conformance tests pass.The remaining 24 are documented in- <a href="https://github.com/xerj-ai/xerj/blob/main/engine/releases/v0.1.0/reports/PATH_TO_100_PCT_v0.6.0_to_v1.0.md"><code>PATH_TO_100_PCT</code></a>.+ <a href="https://github.com/xerj-org/xerj/blob/main/engine/reports/PATH_TO_100_PCT_v0.6.0_to_v1.0.md"><code>PATH_TO_100_PCT</code></a>.This includes search DSL, all 38 query types, all aggs, bulk, scroll,index templates, aliases, knn, hybrid, semantic.</div><div class="source">measured: <code>cargo run -p es-yaml-runner</code></div>@@ -1241,10 +1241,10 @@ node demo/scripts/full-demo-flow.js<!-- ─────────────────────────────────────────────────────────── --><footer><div>- <div>Xerj · v1.0.0-rc.1 · <a href="https://github.com/xerj-ai/xerj">github.com/xerj-ai/xerj</a></div>+ <div>Xerj · <a href="https://github.com/xerj-org/xerj">github.com/xerj-org/xerj</a></div><div style="margin-top: 8px;">All numbers on this page are real, captured live ona 32-core box with a single NVMe SSD. Reproduce in < 5 min with the- <a href="https://github.com/xerj-ai/xerj/tree/main/engine/demo-data">demo-data scripts</a>:+ <a href="https://github.com/xerj-org/xerj/tree/main/engine/demo-data">demo-data scripts</a>:<code>build_corpus.py</code> + <code>xerj index</code> + the curls above.Source dataset: <a href="https://github.com/logpai/loghub">loghub OpenSSH</a>(LabSZ, 2017, 71 MB / 655 K real lines).</div>@@ -1253,7 +1253,7 @@ node demo/scripts/full-demo-flow.js<a href="/architecture/">Architecture</a><a href="/pricing/">Pricing</a><a href="/security/">Security</a>- <a href="https://github.com/xerj-ai/xerj/tree/main/user-feedback">ES feedback corpus</a>+ <a href="https://github.com/xerj-org/xerj/tree/main/user-feedback">ES feedback corpus</a></div></footer>diff --git a/landing/docs/install.html b/landing/docs/install.htmlindex a3f8a8186..e4ba0d919 100644--- a/landing/docs/install.html+++ b/landing/docs/install.html@@ -188,11 +188,11 @@ $ curl -s -H "Authorization: ApiKey $(sudo cat /var/lib/xerj/admin.key)" \<h2>Docker</h2><p>No published image yet — build it locally from <code class="inline">engine/Dockerfile</code> (the image defaults to <code class="inline">--insecure --data-dir /data</code>, right for CI and local dev):</p>-<pre class="code">$ docker build -t xerj:1.0.0-rc.1 engine/+<pre class="code">$ docker build -t xerj:local engine/$ docker run -d --name xerj \-p 8080:8080 -p 9200:9200 \-v xerj-data:/data \- xerj:1.0.0-rc.1</pre>+ xerj:local</pre><p class="source-note">Source · engine/README.md</p>