Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

release: cut v1.0.0-rc.5

xerj-org16d6df0e9da2
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index c6fb1cd93..b716fe184 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -11,11 +11,11 @@ on:
# workflow's single linux-gnu build cannot, so a real tag never
# discovers a broken target for the first time at release time.
branches: [main]
+ # Manual runs take no inputs: the version and the publish decision are both
+ # derived from the ref you dispatch against (GITHUB_REF_NAME / `refs/tags/v*`).
+ # Dispatch against the TAG ref to rebuild and republish a release; dispatch
+ # against a branch for an artifacts-only cross-compile check.
workflow_dispatch:
- inputs:
- tag:
- description: 'Tag to build (e.g. v0.5.9)'
- required: true
permissions:
contents: write
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 054dc4ca5..85a8cfeba 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,388 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
+## [1.0.0-rc.5] - 2026-07-27
+
+Fifth release candidate: the **real-client compatibility release**. Much of
+it came from pointing real Elasticsearch and OpenSearch tooling at
+XERJ — Kibana 8.13, Kibana OSS 7.10.2, OpenSearch Dashboards 2.11.1/3.6,
+and their own shipped sample datasets (flights, eCommerce, logs) — and
+fixing the wrong answers, 500s, and stalls those clients produced. The rest
+is storage, ingest and `autoindex` work that did not come from client
+testing. Headline for users: the query classes that silently matched
+**zero** documents (booleans, `.keyword` multi-fields, `match_phrase` on
+arrays — and keyword arrays before a flush, see the qualification below)
+now return what ES returns; a real Kibana/OSD instance boots,
+logs in, and saves objects against XERJ end to end; and a dashboard firing
+several panel queries at once no longer stalls the node. ES-YAML
+conformance holds at 1360 passed / 0 failed / 3 skipped. Zero-hit defects
+found during this cycle that are **not** fixed here are listed under Known
+limitations.
+
+### Fixed — query semantics (classes that silently matched zero docs)
+
+- **Keyword arrays (partial — memtable half only):** a `term`/`terms` query
+ on a multi-valued keyword field only ever compared element `[0]` — the
+ memtable stored just the first value and the segment keyword column is
+ one ordinal per doc — so exact lookups returned silent false negatives
+ while `terms` aggregations and `match` (which join all elements) looked
+ correct. The memtable path is fixed: those doc-values lookups now bail
+ for array fields and fall through to the array-aware source scan, so a
+ `term` on a multi-valued keyword field is correct **before** a flush.
+ Flushed segments are **not** fixed — the segment keyword column is still
+ one ordinal per document, so after a flush a `term`/`terms` on a
+ non-first array element can still silently miss. The complete fix needs
+ multi-valued segment keyword columns (a storage-format change); the
+ regression test
+ (`xerj-engine` `test_term_matches_non_first_array_element`) is committed
+ but `#[ignore]`d until that lands.
+- **Booleans:** `term`/`match_phrase` on a boolean field undercounted to 0
+ while a `terms` aggregation on the *same* field bucketed `true`/`false`
+ correctly. The trigger was the count shortcut over memtable-resident
+ data — i.e. any bulk import before an explicit `_flush`, which is what
+ every real importer (OSD's own sample-data loader included) produces.
+- **`.keyword` multi-fields:** the brute-force scan's field resolver had no
+ multi-field fallback, so `term`/`match_phrase` on `category.keyword`,
+ `manufacturer.keyword` and friends matched nothing — `_source` never
+ contains a literal `"category.keyword"` key. It now strips the trailing
+ segment and retries against the parent when the parent is a leaf value
+ (guarded, so a genuinely absent nested/object field is still absent).
+- **`match_phrase` / `match_phrase_prefix` on arrays:** both arms only
+ handled scalar values, so an array-valued field never matched any
+ phrase. ES semantics restored: the doc matches if any element does.
+- **`match_phrase` with a non-string query value:** `{"query": true}` was a
+ hard parse error surfaced as `search_phase_execution_exception` — the
+ exact shape OSD's filter bar sends for a boolean filter pill, which
+ broke every panel on the dashboard at once. Scalars are now coerced the
+ way `match` already coerced them.
+- **Empty query strings:** `match`/`match_phrase`/`match_phrase_prefix`
+ with `""` returned 400; ES treats an empty analyzed query as zero terms.
+ They now resolve to `match_none` (200, no hits) — this is what Kibana's
+ saved-objects `_find?search=*` builds, so that endpoint works again.
+- **`geo_point`:** aggregations and `geo_distance`/`geo_bounding_box`
+ rejected string-encoded `{"lat": "50.03", "lon": "8.57"}` coordinates
+ (the shape the flights sample data ships), and `geo_centroid`/
+ `geo_bounds` used a flat lookup that never found a nested geo field.
+ Both now behave like ES.
+- **Aggregation scripts:** `terms` required a top-level `field` and
+ returned `{"buckets": []}` whenever it was absent — any script-based
+ terms aggregation silently produced no buckets. `script` is now a real
+ key source alongside `field`.
+- **Painless date accessors:** `doc['t'].value` returned a plain string, so
+ `.getHour()`, `.getDayOfWeek()`, `.getYear()` and the rest of the common
+ accessor set failed the whole script. They now parse and extract the
+ requested UTC component.
+- **Dynamic date mapping:** ISO-8601 strings are inferred as `date` instead
+ of `text`; `date_detection: false` was accepted-but-ignored and is now
+ honored; a non-date value written into an inferred date field returns
+ `mapper_parsing_exception` instead of being silently stored and then
+ being invisible to range queries, sorts, and time filters; and
+ `date_detection` survives the `PUT /_mapping` merge path.
+- **Segment-path parity:** projected FTS bool queries dropped
+ `minimum_should_match` (so `match_bool_prefix` with `mm=3` matched every
+ single-term hit after flush) and doc-values wildcards did not case-fold;
+ nested kNN returned zero hits for any parent doc living in a segment
+ because the reassembled `{_id, _seq_no, _source}` shape was not
+ unwrapped. `_count` now resolves `terms` lookups.
+
+### Fixed — API surface (what real clients call)
+
+- **Aliases:** single-index endpoints (`GET /{alias}`, `/_mapping`,
+ `/_settings`, …) never resolved an alias to its backing index, and
+ aliases were never persisted at all — index data survived a restart but
+ the alias pointing at it did not. Aliases now resolve everywhere ES
+ accepts them and are written to `aliases.json` (atomic temp+rename).
+ Together this unsticks a fresh OSD container looping on "Another
+ OpenSearch Dashboards instance appears to be migrating the index".
+- **`_field_caps`:** never listed declared multi-fields (so `fields=*`
+ omitted every `.keyword` entry and index-pattern refresh reported
+ "field not found"), and only special-cased the literal `*`/`_all`
+ wildcards — any real glob such as `wiki-test*` returned empty caps.
+ Both fixed; wildcard/comma index specs also now resolve on 9 further
+ endpoints that previously 404'd, 400'd, or — for `_refresh` and
+ `_cat/count` — silently reported success / zero documents while doing
+ nothing (`_refresh` answered `{"_shards":{"successful":1,…}}` without
+ refreshing; `_cat/count` returned 0, indistinguishable from a genuinely
+ empty index).
+- **`POST /{index}/_update`** never returned a `get` block, so every Kibana
+ saved-object write crashed client-side reading `body.get._source`. It is
+ now returned exactly when the caller passes `_source`/`_source_includes`/
+ `_source_excludes`, matching ES (absent otherwise, so `_doc` responses
+ are byte-identical to before).
+- **`_source` vs `stored_fields`:** the implicit `_source` suppression that
+ `stored_fields` triggers is now only a default — an explicit top-level
+ `_source` in the same request wins, as in ES. This is what starved
+ almost every column in OSD's Discover.
+- **`_bulk`** reports each item's real `_seq_no` and `_version` instead of
+ a wall-clock microsecond timestamp and a hardcoded `1`.
+- **`_update_by_query` / `_delete_by_query`** honor
+ `wait_for_completion=false` and return the async `{"task": …}` form;
+ `GET /_tasks/{id}` reports completion with the final response.
+- **`/_xpack`** respects `--compat-version` (it had its own hardcoded
+ `8.13.0`, which Kibana OSS 7.10.2 surfaced as an opaque "license not
+ available" refusal to start), and `/_xpack` + `/_xpack/usage` report the
+ real auth state instead of `security.enabled: true` — so `--insecure`
+ no longer makes Kibana render a login screen for auth that is off.
+- **Missing HTTP verbs:** `_refresh`, `_analyze`, `_msearch` accept GET and
+ `_clone`/`_shrink`/`_split` accept PUT, as ES does — the PUT `_clone`
+ 405 was a fatal error during Kibana's saved-object migration.
+- **Login-path endpoints:** HTTP Basic auth (Kibana's interactive realm),
+ `POST /_security/profile/_activate`, `GET /_security/profile/{uid}`,
+ `GET`/`POST /_security/user/_has_privileges`, and a real
+ `GET`/`PUT`/`DELETE /_security/privilege` store with ES-shaped 404s —
+ each of these was a 500 or a hang on the Kibana login and home pages.
+- **`GET /_cat/templates/{pattern}`** (a 404 that crashed OSD) and
+ **`POST /_index_template/_simulate`** (body-only template preview) added.
+
+### Changed — index resolution (can break existing callers)
+
+These three align index addressing with ES 8 and turn some
+previously-succeeding requests into errors. Review any caller that relies
+on the old behavior.
+
+- **A comma-separated index spec now 404s when a concrete name is
+ missing**, implementing ES's default `ignore_unavailable=false`. The
+ missing name used to be dropped silently — `POST /real,typo/_refresh`
+ answered `200` with `_shards.total: 1`. Wildcard/`_all` specs that match
+ nothing are still a valid empty result (ES's `allow_no_indices` default).
+- **Wildcard and `_all` expansion no longer sweeps in hidden
+ (dot-prefixed) indices**, so an operation like `POST /*/_close` can no
+ longer hit `.xerj_users`-class system indices. A dot-prefixed pattern
+ opts back in, as ES's hidden addressing does.
+- **`POST /_close` refuses a wildcard or `_all` target outright** with ES
+ 8's `action.destructive_requires_name` `illegal_argument_exception`.
+
+### Added
+
+- **OpenSearch client auto-sensing.** XERJ answers the identity endpoints
+ (`GET /`, `GET /_nodes`) per request based on the caller's User-Agent, so
+ one running instance serves an `opensearch-py`/`opensearch-js`/OSD client
+ and an Elasticsearch client simultaneously, each seeing a
+ version/distribution block its own compatibility gate accepts. Explicit
+ `--compat-distribution` / `--compat-version` (and the matching env vars)
+ still pin the identity for every client. The `x-elastic-product` header
+ is no longer sent to a detected OpenSearch caller.
+- **Opt-in ingest memory attribution (developer feature).** Setting
+ `XERJ_INGEST_MEMORY_TRACE=summary` emits a bounded `xerj.ingest_memory.v1`
+ NDJSON ledger — per-owner logical bytes across HTTP bodies, raw and
+ parsed semantic sources, prepared docs and vectors, active memtables and
+ drained flush snapshots — alongside jemalloc allocated/active/resident,
+ RSS, CPU time, and accounting/dropped-event counters, plus a separate
+ `/proc`-derived `xerj.process_sample.v1` stream. Default is fully off.
+ Merge and read-cache owners are reported `unavailable`, not zero. This
+ makes ingest retention *measurable*; it is not itself a memory bound.
+ A deterministic bounded diagnostic suite
+ (`demo/usecases/autoindex/scale/bounded/`) drives ingest → refresh →
+ flush → force-merge → restart with exact count and sentinel checks.
+- **Private pprof debugging toolkit.** A Linux-only `debug-profiling`
+ feature compiles in bounded CPU and jemalloc heap profiling with no
+ network endpoint; artifacts are written mode-0600 to an operator-created
+ directory, with `capture.py`/`inspect.py` wrappers that hash the binary
+ and every artifact. Not in any shipped feature set; no runtime overhead
+ or speedup is claimed.
+- **Experimental ONNX embedding backend**, wired through `autoindex` behind
+ the `onnx-experimental` feature. Its lazy session initialization is
+ cancellation-safe: model loading moved to one process-owned thread whose
+ single terminal result is shared, so a cancelled first request can no
+ longer strand every later one.
+- The ES-YAML conformance runner now exits non-zero when any case fails
+ (it previously reported failures but exited 0).
+
+### Changed — performance (concurrent dashboard bursts)
+
+- **Full-corpus aggregations no longer deep-clone the memtable under the
+ shard lock.** The `need_full_corpus` path (any request with `aggs` that
+ the columnar fast path can't serve) cloned every buffered document's JSON
+ tree while holding the per-shard read guard, serialising concurrent
+ panels behind O(docs) work each. It now Arc-shares out under the lock and
+ clones after releasing it. Measured: 15 concurrent `date_histogram`
+ queries against a real ~14k-doc memtable-resident index, 1.2–2.7 s →
+ ~150–170 ms (commit `6a54cf5`).
+- **Full-corpus aggregations no longer re-decode segments per query.** The
+ same path did an unconditional open + decompress + full `serde_json`
+ parse of a segment's entire stored section on every request once data had
+ flushed. Segments are immutable, so it now uses the existing
+ single-flight stored-value cache. Measured: 20 concurrent requests
+ against a real flushed eCommerce segment, 15–26 s → 1–4 ms including the
+ cold first hit (commit `f6daf70`). A segment read failure during
+ full-corpus assembly is now a hard error rather than a silent skip that
+ undercounted every bucket.
+- **The scan path no longer stalls on cold, concurrent bursts.** Two
+ compounding costs: every scanned doc's `_source` was deep-cloned to
+ splice `_id` in even though only a deeply-nested `ids` clause needs it
+ (now computed once per scan from the query shape), and the raw-fallback
+ decode had no single-flight protection, so N concurrent requests against
+ a cold segment each paid the full open+decompress. Measured against real
+ Kibana OSS 7.10.2 and eCommerce sample data: a 24-concurrent
+ `query_string`+`match_phrase` burst immediately after restart (worst-case
+ cold caches) went from ~4.5 s per request to 100–165 ms, with the warm
+ repeat at 1–5 ms (commit `7e963ff`).
+- **Semantic and vector work is bounded by the request.** One absolute
+ deadline computed at search entry is carried through single-flight
+ waiting, admission, embedding, hybrid/multi-kNN recursion, and exact
+ scanning; partials set `timed_out` and `hits.total.relation: gte`, and
+ the ES handler aborts its child task when the request is dropped.
+ Previously a cancelled semantic search could drain for minutes past its
+ timeout. Cold vector segment loads also moved to bounded `spawn_blocking`
+ producers so they can no longer pin every async worker.
+
+### Fixed — storage, durability & ingest
+
+- **Raw ingest validates before it publishes.** Caller bytes were appended
+ to the WAL before being proven to be complete JSON, so malformed input
+ could reserve sequence numbers and become durable, with the later parse
+ path silently substituting `{}`. Whole-batch UTF-8/JSON/nesting
+ validation now completes before any sequence, WAL, version-map, memtable,
+ or schema mutation, and malformed bulk sources return per-item
+ `document_parsing_exception`. The parsed `Value` is now the single
+ authority for turbo ingest (live indexing and WAL replay could previously
+ disagree), and `copy_to` is applied before WAL publication so GET,
+ search, flush, replay, and restart all observe the same source.
+- **Per-shard WAL buffers bounded to 64 KiB** (was 8 MiB). Every index
+ eagerly opens one writer per ingest shard, so a 16×16 default reserved
+ ~2 GiB of allocator capacity before a single document was written;
+ capacity above a frame cannot batch across requests because every
+ acknowledged append already drains the buffer. Measured on 256 empty
+ writers: jemalloc allocated 2,147,503,496 → 16,797,064 bytes (−99.2%),
+ with ingest throughput ratios of 0.976 (single shard) and 1.020
+ (eight shards) and exact replay preserved (commit `761b915`). Cost:
+ records larger than the 64 KiB buffer issue more write syscalls
+ (measured 1,372 → 3,372 on 131 KiB × 1000), with throughput unchanged.
+- **Same-ID writes are serialized.** A keyed publication coordinator now
+ spans the current-state/CAS check through WAL, version map, FTS, and HNSW
+ publication for single-document paths, closing races that could admit two
+ creates, lose an update patch, or let a delete overtake a write. Scripted
+ `_update` and `_update_by_query` run through the same boundary, so two
+ concurrent `ctx._source.n += 1` requests can no longer both read `n=0`.
+ Distinct IDs do not contend. (Turbo batch paths remain follow-up work.)
+- **Semantic vectors stay out of full-text indexes.** The dynamic-field
+ walkers treated pooled embeddings and their `_chunks` companions as
+ ordinary text and fed every float into FTS as a decimal token. In one
+ failed diagnostic run that produced 2.338 GiB of vector FSTs inside a
+ 3.449 GiB partial index. Embedding outputs are now excluded by schema
+ designation (not a name heuristic) across pre-analysis, flush, and merge.
+ Existing vector FST sidecars are not rewritten on upgrade — reindexing is
+ the safe way to reclaim them.
+- **The admin API key persists across restarts.** The server wrote
+ `data_dir/admin.key` but never read it back, minting a fresh key on every
+ start and locking out every client configured against the previous one.
+ An existing well-formed key is now reused; anything missing or malformed
+ still falls through to fresh generation.
+
+### Fixed — autoindex
+
+- **Content deduplication with crash-safe replacement.** Byte-identical
+ paths were indexed independently (one reproduction journaled 2,442
+ records for 1,221 live documents). Sources are now hashed with streaming
+ XXH3-128 and digest peers byte-verified before one canonical path is
+ chosen; every current name is preserved in `ax_paths`. Publication became
+ an explicit durable generation transaction (synced `file_replace_start`
+ → staged, source-verified extraction → synced `file_done` commit), and
+ journal replay repairs torn tails and rolls back failed appends.
+ Ambiguous legacy prefix collisions fail closed before any backend
+ mutation instead of guessing.
+- **Follow-on fixes to that rework:** resume now guarantees exclusive
+ plan-key ownership (two files resolving to one key each ran the
+ replacement transaction and deleted the other's documents);
+ `delete_by_query` repeats until a pass deletes nothing, because the
+ server implements it as a single `size:10000` pass and >10k-doc
+ generations left permanent ghost documents; and both recoverable error
+ paths now name the offending files or journal offset instead of
+ advising a blanket `--fresh` re-extract.
+- **PDFs are parsed by `pdf_oxide` in isolated same-binary workers.** The
+ previous byte scanner interpreted font character codes without resolving
+ page resources or ToUnicode maps, producing NUL-separated and shifted
+ text that schema inference (correctly) refused to elect as
+ `semantic_text`. Workers get process groups, a 1536 MiB `RLIMIT_AS`, and
+ descendant kill/reap on timeout — documented as resource isolation, not
+ a security sandbox, with no OS memory cap on non-Unix. New
+ `--pdf-workers` and `--pdf-timeout-secs` flags. Cost: the no-default
+ release binary grows 34,160,328 → 39,018,184 bytes (+4.6 MiB).
+- **Text files are classified by sentence density, not line length.** The
+ old `avg_len > 60` rule split markdown across two datasets with two
+ different field names — adding `## headings` to a document made it *more*
+ likely to be treated as a record stream, which split BM25 statistics
+ across two corpora. Terminal-punctuation density separates prose
+ (0.43–0.57 on the measured corpus) from logs and source (0.00–0.20);
+ CSV rows fall on the record-stream side of the same threshold.
+
+### Changed — autoindex (what it produces)
+
+- **Line-oriented text is chunked into overlapping windows** (40 lines, 10
+ overlap) instead of one document per line, and prose sections dropped
+ from 32 KB to 2 KB with overlap, because BM25 scores per document and a
+ single line rarely contains the caller's own wording. Measured on this
+ repository (234 files, 170k LOC + docs + 460 commit messages) with 8
+ "where/why is X" questions: 3/8 → 7/8 answered, 162,883 → 5,508 records,
+ indexing 234 s → 1.9 s (commit `4510189`). Every chunk carries
+ `start_line`/`end_line`.
+- **`semantic_text` election is by language, not length.** The rule was
+ "largest `text` field with `avg_len >= 200`", which elected 300-character
+ base64 and concatenated-id columns while skipping a genuinely semantic
+ 150-character summary. A field must now actually look like natural
+ language — `word_ratio >= 0.55` (tokens matching `[A-Za-z]{3,}`) **and**
+ `mean_tokens >= 3`. Measured `word_ratio` is 0.00 for
+ trace_id/user_id/order_id/numeric columns and 0.78–1.00 for prose, log
+ messages and source. This changes which field gets embedded, so it
+ affects both semantic-search quality and ingest cost (the built-in neural
+ backend measures ~2.8 docs/s). The election note records the numbers that
+ drove the decision.
+
+### Added — autoindex
+
+- **`--bulk-timeout-secs`** (default 300, range 1–3600) applies only to
+ `POST /_bulk`, so a legitimately slow neural bulk can be accommodated
+ without loosening the deadline on control requests. Retries are bounded
+ at six attempts with the identical body and deterministic document IDs.
+
+### Docs
+
+- **WordPress core security audit case study**
+ (`docs/case-studies/wordpress-security-audit/`): a reproducible record of
+ an AI agent auditing real WordPress core (1,492 PHP files, ~619k lines)
+ with XERJ as the retrieval substrate — sink census, interprocedural taint
+ analysis, authorization graph, POP-gadget hunt — plus a copyable Claude
+ Code skill and a step-by-step playbook. The honest result is that **core
+ came back hardened**: the documented findings are three Medium-severity,
+ known-class items (an incomplete SSRF deny-list missing `169.254.0.0/16`,
+ an ImageMagick parse surface, and an unguarded `role` sink in
+ `user-new.php`), a fourth downgraded to not-reachable-in-core, and
+ verified negatives for IDOR, SQL de-escaping, and sanitizer composition.
+ None of it is claimed as a novel 0-day. The published grep-wins
+ counter-examples are shown alongside the wins, and the XERJ bug the audit
+ surfaced (the keyword-array `term` defect partially fixed above — the
+ memtable half) is disclosed.
+ Site use case 06 (`/use-cases/code-security-audit.html`) is built from it.
+- **Token-usage guidebook** (`docs/TOKEN_USAGE.md`): a measured decomposition
+ of what a XERJ answer costs an agent in tokens — envelope overhead,
+ answer, and materialized intermediate data — instead of hand-waving.
+- **AST + graph + FTS vulnerability research**
+ (`docs/research/ast-graph-vuln-detection.md`, `docs/examples/ast-vuln-graph/`),
+ including a multi-language tree-sitter taint scanner tested at real
+ WordPress scale.
+- Verified embedding examples: XERJ with Google AI (EmbeddingGemma, Gemini
+ API, ADK) and with any OpenAI-compatible `/v1/embeddings` endpoint.
+
+### Known limitations
+
+Known-open items found while validating this release and **not** fixed in
+it. The first three are zero-hit defects — a query returns no documents
+that ES would return:
+
+- **`multi_match` looks inverted.** On a 6,022-document index a long query
+ returned 0 hits with `operator: "or"` (the ES default) and 2 hits with
+ `operator: "and"`. OR must be a superset of AND.
+- **`match` against a `semantic_text` field returns 0 hits** for a long
+ natural-language string, while the same query against a plain `text`
+ field returns thousands.
+- **`term`/`terms` on a multi-valued keyword field can still miss after a
+ flush** — only the memtable half of that fix landed (see the first bullet
+ of "Fixed — query semantics").
+- **`autoindex` dataset clustering merges same-shape files** with different
+ subjects: 213 source files sharing the schema `{text}` collapse to one
+ index, and embedding centroids separate the subject groups only weakly.
+- 3 of 1,363 ES-YAML conformance cases are skipped.
+
## [1.0.0-rc.4] - 2026-07-22
Fourth release candidate: the **production-hardening release**. A 9-review
@@ -251,5 +633,9 @@ for evaluation, but may still change before the final 1.0.0.
[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) and `engine/CLAUDE.md` for the
current supported surface.
-[Unreleased]: https://github.com/xerj-org/xerj/compare/v1.0.0-rc.1...HEAD
+[Unreleased]: https://github.com/xerj-org/xerj/compare/v1.0.0-rc.5...HEAD
+[1.0.0-rc.5]: https://github.com/xerj-org/xerj/releases/tag/v1.0.0-rc.5
+[1.0.0-rc.4]: https://github.com/xerj-org/xerj/releases/tag/v1.0.0-rc.4
+[1.0.0-rc.3]: https://github.com/xerj-org/xerj/releases/tag/v1.0.0-rc.3
+[1.0.0-rc.2]: https://github.com/xerj-org/xerj/releases/tag/v1.0.0-rc.2
[1.0.0-rc.1]: https://github.com/xerj-org/xerj/releases/tag/v1.0.0-rc.1
diff --git a/engine/Cargo.lock b/engine/Cargo.lock
index 59b1e5064..cd412f7cc 100644
--- a/engine/Cargo.lock
+++ b/engine/Cargo.lock
@@ -6170,7 +6170,7 @@ dependencies = [
[[package]]
name = "xerj-ai"
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
dependencies = [
"anyhow",
"candle-core",
@@ -6191,7 +6191,7 @@ dependencies = [
[[package]]
name = "xerj-api"
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
dependencies = [
"anyhow",
"async-trait",
@@ -6220,7 +6220,7 @@ dependencies = [
[[package]]
name = "xerj-autoindex"
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
dependencies = [
"anyhow",
"chrono",
@@ -6247,7 +6247,7 @@ dependencies = [
[[package]]
name = "xerj-cluster"
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
dependencies = [
"anyhow",
"async-trait",
@@ -6269,7 +6269,7 @@ dependencies = [
[[package]]
name = "xerj-common"
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
dependencies = [
"anyhow",
"byteorder",
@@ -6287,7 +6287,7 @@ dependencies = [
[[package]]
name = "xerj-compress"
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
dependencies = [
"byteorder",
"bytes",
@@ -6300,7 +6300,7 @@ dependencies = [
[[package]]
name = "xerj-console-api"
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
dependencies = [
"anyhow",
"arc-swap",
@@ -6336,7 +6336,7 @@ dependencies = [
[[package]]
name = "xerj-engine"
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
dependencies = [
"anyhow",
"arc-swap",
@@ -6375,7 +6375,7 @@ dependencies = [
[[package]]
name = "xerj-fts"
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
dependencies = [
"anyhow",
"bitpacking",
@@ -6401,7 +6401,7 @@ dependencies = [
[[package]]
name = "xerj-logs"
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
dependencies = [
"anyhow",
"byteorder",
@@ -6419,7 +6419,7 @@ dependencies = [
[[package]]
name = "xerj-mcp"
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
dependencies = [
"anyhow",
"reqwest",
@@ -6429,7 +6429,7 @@ dependencies = [
[[package]]
name = "xerj-query"
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
dependencies = [
"anyhow",
"base64 0.22.1",
@@ -6444,7 +6444,7 @@ dependencies = [
[[package]]
name = "xerj-server"
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
dependencies = [
"anyhow",
"axum",
@@ -6490,7 +6490,7 @@ dependencies = [
[[package]]
name = "xerj-storage"
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
dependencies = [
"anyhow",
"arc-swap",
@@ -6520,7 +6520,7 @@ dependencies = [
[[package]]
name = "xerj-vector"
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
dependencies = [
"anyhow",
"byteorder",
@@ -6538,7 +6538,7 @@ dependencies = [
[[package]]
name = "xerj-wasm"
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
dependencies = [
"anyhow",
"bytes",
diff --git a/engine/Cargo.toml b/engine/Cargo.toml
index d8cc59723..6fc16761f 100644
--- a/engine/Cargo.toml
+++ b/engine/Cargo.toml
@@ -21,7 +21,7 @@ members = [
]
[workspace.package]
-version = "1.0.0-rc.4"
+version = "1.0.0-rc.5"
edition = "2021"
license = "Apache-2.0"
authors = ["xerj Contributors"]
diff --git a/engine/crates/xerj-api/src/stub.rs b/engine/crates/xerj-api/src/stub.rs
index b1006d562..89c97a418 100644
--- a/engine/crates/xerj-api/src/stub.rs
+++ b/engine/crates/xerj-api/src/stub.rs
@@ -59,7 +59,7 @@ pub fn not_implemented_yet(
"reason": format!("{feature} is not implemented in this xerj build. {note}"),
"feature": feature,
"planned_milestone": milestone,
- "roadmap": "https://github.com/xerj-ai/xerj/blob/main/engine/reports/PATH_TO_100_PCT_v0.6.0_to_v1.0.md",
+ "roadmap": "https://github.com/xerj-org/xerj/blob/main/engine/reports/PATH_TO_100_PCT_v0.6.0_to_v1.0.md",
},
"status": 501,
});
diff --git a/engine/crates/xerj-server/tests/pdf_worker.rs b/engine/crates/xerj-server/tests/pdf_worker.rs
index 5c53085ff..32a462aa1 100644
--- a/engine/crates/xerj-server/tests/pdf_worker.rs
+++ b/engine/crates/xerj-server/tests/pdf_worker.rs
@@ -96,7 +96,7 @@ fn pdf_worker_is_fresh_process_deterministic_and_preserves_page_provenance() {
assert_eq!(first.stdout, second.stdout);
let response: Value = serde_json::from_slice(&first.stdout).unwrap();
- assert_eq!(response["extractor"], "xerj-autoindex/1.0.0-rc.4");
+ assert_eq!(response["extractor"], "xerj-autoindex/1.0.0-rc.5");
assert_eq!(response["parser"], "pdf_oxide/0.3.75");
assert!(response["containment"]
.as_str()
diff --git a/engine/reports/SECURITY_DEPLOYMENT_MODEL.md b/engine/reports/SECURITY_DEPLOYMENT_MODEL.md
index 7d07adb5c..1b55f67d0 100644
--- a/engine/reports/SECURITY_DEPLOYMENT_MODEL.md
+++ b/engine/reports/SECURITY_DEPLOYMENT_MODEL.md
@@ -173,7 +173,7 @@ gap during procurement review.
| Stack overflow on deep query | Query-depth cap = 64 (enforced) | Low — covered |
If you have a threat we haven't considered, file an issue at
-[github.com/xerj-ai/xerj/issues](https://github.com/xerj-ai/xerj/issues)
+[github.com/xerj-org/xerj/issues](https://github.com/xerj-org/xerj/issues)
or email [email protected] (use PGP key from [/security/pgp.txt]).
---
diff --git a/landing/demo/index.html b/landing/demo/index.html
index 02bed2145..e9737e857 100644
--- a/landing/demo/index.html
+++ b/landing/demo/index.html
@@ -245,7 +245,7 @@ footer .links { display: flex; gap: 28px; flex-wrap: wrap; }
<a href="/">Home</a>
<a href="/architecture/">Architecture</a>
<a href="/pricing/">Pricing</a>
- <a href="https://github.com/xerj-ai/xerj">GitHub</a>
+ <a href="https://github.com/xerj-org/xerj">GitHub</a>
</span>
</nav>
@@ -256,7 +256,7 @@ footer .links { display: flex; gap: 28px; flex-wrap: wrap; }
Pull a real production log from <a href="https://github.com/logpai/loghub">loghub</a>.
Ingest 12&nbsp;GiB into Xerj with one CLI command. Query it from Xerj Console.
Each step shows the Elasticsearch alternative — resources, time, config —
- sourced from <a href="https://github.com/xerj-ai/xerj/tree/main/user-feedback">collected community feedback</a>
+ sourced from <a href="https://github.com/xerj-org/xerj/tree/main/user-feedback">collected community feedback</a>
(G2, Gartner, TrustRadius, GitHub issues, Reddit, HN, Jepsen, CVE).
</p>
<div class="stats">
@@ -271,7 +271,7 @@ footer .links { display: flex; gap: 28px; flex-wrap: wrap; }
Every number above + every CLI block on this page was captured live on this
machine on <strong style="color: var(--y);">2026-04-27</strong> against the
<code>v1.0.0-rc.1</code> binary (commit <code>78a9bcc</code>). Full
- transcript: <a href="https://github.com/xerj-ai/xerj/blob/main/demo/CLI_VALIDATED_2026-04-27.md"><code>demo/CLI_VALIDATED_2026-04-27.md</code></a>.
+ transcript: <a href="https://github.com/xerj-org/xerj/blob/main/demo/CLI_VALIDATED_2026-04-27.md"><code>demo/CLI_VALIDATED_2026-04-27.md</code></a>.
</p>
</header>
@@ -325,7 +325,7 @@ footer .links { display: flex; gap: 28px; flex-wrap: wrap; }
because no real LLM-traffic capture ships with the repo, and the
AI / vector / agent / anomaly / ingest dashboards need that shape
to render meaningfully. Generator:
- <code><a href="https://github.com/xerj-ai/xerj/blob/main/demo/data/extras/generate_demo_corpus.py">demo/data/extras/generate_demo_corpus.py</a></code>.
+ <code><a href="https://github.com/xerj-org/xerj/blob/main/demo/data/extras/generate_demo_corpus.py">demo/data/extras/generate_demo_corpus.py</a></code>.
</div>
</div>
</div>
@@ -349,7 +349,7 @@ footer .links { display: flex; gap: 28px; flex-wrap: wrap; }
is already running on port 9200.
</p>
- <pre class="cmd"><span class="prompt">$ </span>curl -L -o xerj.tar.gz https://github.com/xerj-ai/xerj/releases/download/v1.0.0-rc.1/xerj-1.0.0-rc.1-x86_64-unknown-linux-gnu.tar.gz
+ <pre class="cmd"><span class="prompt">$ </span>curl -L -o xerj.tar.gz https://github.com/xerj-org/xerj/releases/download/v1.0.0-rc.1/xerj-1.0.0-rc.1-x86_64-unknown-linux-gnu.tar.gz
<span class="prompt">$ </span>tar xzf xerj.tar.gz
<span class="prompt">$ </span>./xerj --insecure --data-dir ./data
<span class="out">2026-04-27T19:11:41.6Z INFO xerj: xerj v1.0.0-rc.1 starting
@@ -428,7 +428,7 @@ Ready: /home/claude/ai/xerj/engine/demo-data/ssh_big.ndjson</span></pre>
<code>@timestamp · host · proc · pid · event · user · src_ip · src_port · message</code>.
Event tags (<code>auth_failure_invalid_user</code>, <code>possible_break_in</code>,
<code>auth_success</code>, …) come from a small regex parser
- (<a href="https://github.com/xerj-ai/xerj/blob/main/engine/demo-data/build_corpus.py"><code>build_corpus.py</code></a>).
+ (<a href="https://github.com/xerj-org/xerj/blob/main/engine/demo-data/build_corpus.py"><code>build_corpus.py</code></a>).
</p>
<div class="se-says">
@@ -476,7 +476,7 @@ Ready: /home/claude/ai/xerj/engine/demo-data/ssh_big.ndjson</span></pre>
<h3>Sustained scale — full 60.9 M / 12 GiB</h3>
<p style="font-size: 13px; color: var(--dim); margin: 4px 0 12px;">
- Output below is from the <a href="https://github.com/xerj-ai/xerj/blob/main/engine/reports/2026-04-25T22-50-00_xerj_vs_elasticsearch_rerun.md">2026-04-25 head-to-head bench</a>
+ Output below is from the <a href="https://github.com/xerj-org/xerj/blob/main/engine/reports/2026-04-25T22-50-00_xerj_vs_elasticsearch_rerun.md">2026-04-25 head-to-head bench</a>
(commit <code>73c6367</code>, pre-rc). We did <em>not</em> re-run the 22-minute 60 M ingest
on 2026-04-27 — burst rate above is the only ingest measurement
re-validated against rc.1. Reproduce with
@@ -656,7 +656,7 @@ node demo/scripts/full-demo-flow.js
sonnet / opus / gpt-5 numbers come from?". Honest answer: the demo
flow ingests six NDJSON corpora into the engine before the SPA
opens. The corpora are generated reproducibly (random.seed=42)
- from <code><a href="https://github.com/xerj-ai/xerj/blob/main/demo/data/extras/generate_demo_corpus.py">demo/data/extras/generate_demo_corpus.py</a></code>
+ from <code><a href="https://github.com/xerj-org/xerj/blob/main/demo/data/extras/generate_demo_corpus.py">demo/data/extras/generate_demo_corpus.py</a></code>
and pushed in via the <strong>same ES-compat <code>/_bulk</code> endpoint</strong>
that Logstash, Filebeat, every ES SDK and your existing collectors
already speak — there's no Xerj-specific ingest API, no SDK to
@@ -759,7 +759,7 @@ node demo/scripts/full-demo-flow.js
the corpus, the dashboard changes. Delete the index, the SPA's
live adapter returns <code>null</code> and the panel collapses to
its mock skeleton (visible difference, no silent fallback). Code
- path: <code><a href="https://github.com/xerj-ai/xerj/blob/main/xerj-ux/src/data/backends/xerj.js">xerj-ux/src/data/backends/xerj.js</a></code>
+ path: <code><a href="https://github.com/xerj-org/xerj/blob/main/xerj-ux/src/data/backends/xerj.js">xerj-ux/src/data/backends/xerj.js</a></code>
function <code>liveAiOverview()</code> at line ~378.
</p>
@@ -781,7 +781,7 @@ node demo/scripts/full-demo-flow.js
(4 000 · VECTOR · INDEX), <code>ingest-events</code> (12 000 · INGEST · PIPELINE)
— <strong>25 648 documents total</strong>. Every dashboard below
queries one or more of these indices. Full transcript:
- <a href="https://github.com/xerj-ai/xerj/blob/main/demo/INGEST_FLOW_2026-04-27.md"><code>demo/INGEST_FLOW_2026-04-27.md</code></a>.
+ <a href="https://github.com/xerj-org/xerj/blob/main/demo/INGEST_FLOW_2026-04-27.md"><code>demo/INGEST_FLOW_2026-04-27.md</code></a>.
</p>
<h3>3.5.6 — Every AI dashboard reads from those indices</h3>
@@ -988,7 +988,7 @@ node demo/scripts/full-demo-flow.js
<div class="small">Direct doc-values column scan + topK heap. Same
latency profile on 60 M docs as on 100 K (block-skip indices).
No JVM cache warmup, no per-query GC pressure spike.</div>
- <div class="source">measured: <a href="https://github.com/xerj-ai/xerj/blob/main/engine/reports/2026-04-25T03-30-00_xerj_vs_elasticsearch_bench.md">2026-04-25 head-to-head bench</a> (terms agg p50)</div>
+ <div class="source">measured: <a href="https://github.com/xerj-org/xerj/blob/main/engine/reports/2026-04-25T03-30-00_xerj_vs_elasticsearch_bench.md">2026-04-25 head-to-head bench</a> (terms agg p50)</div>
</div>
<div class="es-side">
<div class="label">Elasticsearch · terms agg p50</div>
@@ -1103,10 +1103,10 @@ node demo/scripts/full-demo-flow.js
<strong style="color: var(--y);">Source-of-truth:</strong> the three Xerj numbers below were
re-captured on <code>v1.0.0-rc.1</code> (commit <code>78a9bcc</code>) on
<strong>2026-04-27</strong> — see
- <a href="https://github.com/xerj-ai/xerj/blob/main/demo/CLI_VALIDATED_2026-04-27.md"><code>demo/CLI_VALIDATED_2026-04-27.md</code></a>
+ <a href="https://github.com/xerj-org/xerj/blob/main/demo/CLI_VALIDATED_2026-04-27.md"><code>demo/CLI_VALIDATED_2026-04-27.md</code></a>
§9 / §11. ES comparator numbers are unchanged from the 2026-04-25
head-to-head bench
- (<a href="https://github.com/xerj-ai/xerj/blob/main/engine/reports/2026-04-25T22-50-00_xerj_vs_elasticsearch_rerun.md"><code>2026-04-25T22-50-00_xerj_vs_elasticsearch_rerun.md</code></a>);
+ (<a href="https://github.com/xerj-org/xerj/blob/main/engine/reports/2026-04-25T22-50-00_xerj_vs_elasticsearch_rerun.md"><code>2026-04-25T22-50-00_xerj_vs_elasticsearch_rerun.md</code></a>);
if a prospect wants methodology, send that file.
</p>
@@ -1194,7 +1194,7 @@ node demo/scripts/full-demo-flow.js
<p class="why">
The migration story. Xerj implements the Elasticsearch wire protocol —
1305 of 1329 official ES YAML conformance tests pass on
- <a href="https://github.com/xerj-ai/xerj/tree/main/engine/tests/es-compat-yaml">the ES 8.13 test suite</a>.
+ <a href="https://github.com/xerj-org/xerj/tree/main/engine/tests/es-compat-yaml">the ES 8.13 test suite</a>.
That means existing curl, official ES SDKs (Python, Go, Java, JS), and
even Kibana itself can point at port 9200 unchanged.
</p>
@@ -1213,7 +1213,7 @@ node demo/scripts/full-demo-flow.js
<div class="num">1305 / 1329</div>
<div class="small">98.2 % of ES 8.13 official YAML conformance tests pass.
The remaining 24 are documented in
- <a href="https://github.com/xerj-ai/xerj/blob/main/engine/releases/v0.1.0/reports/PATH_TO_100_PCT_v0.6.0_to_v1.0.md"><code>PATH_TO_100_PCT</code></a>.
+ <a href="https://github.com/xerj-org/xerj/blob/main/engine/reports/PATH_TO_100_PCT_v0.6.0_to_v1.0.md"><code>PATH_TO_100_PCT</code></a>.
This includes search DSL, all 38 query types, all aggs, bulk, scroll,
index templates, aliases, knn, hybrid, semantic.</div>
<div class="source">measured: <code>cargo run -p es-yaml-runner</code></div>
@@ -1241,10 +1241,10 @@ node demo/scripts/full-demo-flow.js
<!-- ─────────────────────────────────────────────────────────── -->
<footer>
<div>
- <div>Xerj · v1.0.0-rc.1 · <a href="https://github.com/xerj-ai/xerj">github.com/xerj-ai/xerj</a></div>
+ <div>Xerj · <a href="https://github.com/xerj-org/xerj">github.com/xerj-org/xerj</a></div>
<div style="margin-top: 8px;">All numbers on this page are real, captured live on
a 32-core box with a single NVMe SSD. Reproduce in &lt; 5 min with the
- <a href="https://github.com/xerj-ai/xerj/tree/main/engine/demo-data">demo-data scripts</a>:
+ <a href="https://github.com/xerj-org/xerj/tree/main/engine/demo-data">demo-data scripts</a>:
<code>build_corpus.py</code> + <code>xerj index</code> + the curls above.
Source dataset: <a href="https://github.com/logpai/loghub">loghub OpenSSH</a>
(LabSZ, 2017, 71 MB / 655 K real lines).</div>
@@ -1253,7 +1253,7 @@ node demo/scripts/full-demo-flow.js
<a href="/architecture/">Architecture</a>
<a href="/pricing/">Pricing</a>
<a href="/security/">Security</a>
- <a href="https://github.com/xerj-ai/xerj/tree/main/user-feedback">ES feedback corpus</a>
+ <a href="https://github.com/xerj-org/xerj/tree/main/user-feedback">ES feedback corpus</a>
</div>
</footer>
diff --git a/landing/docs/install.html b/landing/docs/install.html
index a3f8a8186..e4ba0d919 100644
--- a/landing/docs/install.html
+++ b/landing/docs/install.html
@@ -188,11 +188,11 @@ $ curl -s -H "Authorization: ApiKey $(sudo cat /var/lib/xerj/admin.key)" \
<h2>Docker</h2>
<p>No published image yet — build it locally from <code class="inline">engine/Dockerfile</code> (the image defaults to <code class="inline">--insecure --data-dir /data</code>, right for CI and local dev):</p>
-<pre class="code">$ docker build -t xerj:1.0.0-rc.1 engine/
+<pre class="code">$ docker build -t xerj:local engine/
$ docker run -d --name xerj \
-p 8080:8080 -p 9200:9200 \
-v xerj-data:/data \
- xerj:1.0.0-rc.1</pre>
+ xerj:local</pre>
<p class="source-note">Source · engine/README.md</p>