Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

feat(security): make the cargo-audit and cargo-fuzz claims real, and fix what they found (#207)

xerj-orgc7fde02f6926
diff --git a/.github/scripts/fuzz-smoke.sh b/.github/scripts/fuzz-smoke.sh
new file mode 100755
index 000000000..ba6ed9f1c
--- /dev/null
+++ b/.github/scripts/fuzz-smoke.sh
@@ -0,0 +1,86 @@
+#!/usr/bin/env bash
+# Build and exercise every libFuzzer harness in engine/fuzz.
+#
+# This is the script CI runs, so a contributor can reproduce the gate exactly:
+#
+# rustup toolchain install nightly
+# cargo install cargo-fuzz --locked
+# bash .github/scripts/fuzz-smoke.sh
+#
+# It does two things, and the second is the one that matters on a pull request:
+#
+# 1. builds every target with AddressSanitizer + libFuzzer coverage, so a
+# harness cannot silently rot into "does not compile";
+# 2. runs every target for FUZZ_SECONDS over its checked-in seeds, which both
+# replays known-interesting inputs as a regression suite and spends the
+# remaining budget on new mutations.
+#
+# Seeds live in engine/fuzz/seeds/<target>/ and are tracked. The working corpus
+# in engine/fuzz/corpus/<target>/ is NOT: libFuzzer writes every input it finds
+# interesting there, and committing a few hundred opaque blobs after each run is
+# how a seed corpus stops being reviewable. Seeds are copied in before each run;
+# promote a discovered input by copying it into seeds/ under a name that says
+# what it covers.
+#
+# A crash, a sanitizer report, an OOM, or a single input slower than
+# FUZZ_TIMEOUT fails the job, and the offending input is written to
+# engine/fuzz/artifacts/<target>/ for CI to upload.
+set -euo pipefail
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
+FUZZ_DIR="$REPO_ROOT/engine/fuzz"
+
+# Per-target wall-clock budget. Short on a pull request (the point there is the
+# seed replay plus a little exploration); override for a real campaign:
+# FUZZ_SECONDS=3600 bash .github/scripts/fuzz-smoke.sh
+FUZZ_SECONDS="${FUZZ_SECONDS:-20}"
+# A single input allowed to take this long. Anything slower is a
+# denial-of-service bug in a parser reachable from an unauthenticated request,
+# which is exactly what these harnesses exist to find.
+FUZZ_TIMEOUT="${FUZZ_TIMEOUT:-25}"
+FUZZ_RSS_MB="${FUZZ_RSS_MB:-4096}"
+
+if ! cargo +nightly fuzz --version >/dev/null 2>&1; then
+ echo "fuzz-smoke: needs a nightly toolchain and cargo-fuzz." >&2
+ echo " rustup toolchain install nightly" >&2
+ echo " cargo install cargo-fuzz --locked" >&2
+ exit 1
+fi
+
+cd "$REPO_ROOT/engine"
+
+echo "==> building fuzz targets (ASan + libFuzzer)"
+cargo +nightly fuzz build
+
+# The target list comes from the crate, never from a list pasted into this
+# script: a target that is added and not wired up here would otherwise be
+# claimed-but-not-run, which is the whole reason this file exists.
+mapfile -t TARGETS < <(cargo +nightly fuzz list)
+if [ "${#TARGETS[@]}" -eq 0 ]; then
+ echo "fuzz-smoke: no fuzz targets found in engine/fuzz — refusing to pass." >&2
+ exit 1
+fi
+
+echo "==> ${#TARGETS[@]} targets, ${FUZZ_SECONDS}s each"
+for target in "${TARGETS[@]}"; do
+ seeds="$FUZZ_DIR/seeds/$target"
+ if [ ! -d "$seeds" ] || [ -z "$(ls -A "$seeds" 2>/dev/null)" ]; then
+ echo "fuzz-smoke: $target has no seeds at $seeds — refusing to pass." >&2
+ echo " A target with no seeds explores from the empty input and proves" >&2
+ echo " almost nothing; add real inputs under that directory." >&2
+ exit 1
+ fi
+
+ corpus="$FUZZ_DIR/corpus/$target"
+ mkdir -p "$corpus"
+ cp -f "$seeds"/* "$corpus"/
+
+ echo "--- $target ($(ls -1 "$seeds" | wc -l) seeds)"
+ cargo +nightly fuzz run "$target" -- \
+ -max_total_time="$FUZZ_SECONDS" \
+ -timeout="$FUZZ_TIMEOUT" \
+ -rss_limit_mb="$FUZZ_RSS_MB" \
+ -print_final_stats=1
+done
+
+echo "==> all ${#TARGETS[@]} fuzz targets survived ${FUZZ_SECONDS}s each"
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 6d348de27..1b34dd732 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -64,6 +64,100 @@ jobs:
working-directory: engine
run: cargo clippy --workspace --all-targets -- -D warnings
+ # XERJ's security pitch is "no JVM, so none of the Log4Shell class of supply
+ # chain CVEs" — which is only worth anything if somebody is actually watching
+ # the 660-odd crates we do depend on. This job is that somebody. It was
+ # documented as existing long before it did (#207); a claimed gate that no
+ # runner executes is worse than no gate, because readers price it in.
+ #
+ # Failure policy: RUSTSEC *vulnerabilities* fail the build. Informational
+ # warnings (unmaintained / unsound / yanked) are printed and do not, because
+ # most of ours sit in transitive crates we cannot upgrade unilaterally and a
+ # gate that is red for reasons nobody can act on gets ignored, then disabled.
+ # They are still in the log on every run, which is the point.
+ security-audit:
+ name: Dependency advisory audit (cargo-audit)
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+
+ - name: Install Rust
+ uses: dtolnay/rust-toolchain@stable
+
+ # Cache the binary, not the advisory database — the database is the part
+ # that must be fetched fresh on every run. Bump the `-vN` suffix to force
+ # a cargo-audit upgrade (a very old binary can fail to read a newer
+ # advisory-db schema).
+ - name: Cache cargo-audit
+ uses: actions/cache@v4
+ with:
+ path: ~/.cargo/bin/cargo-audit
+ key: ci-cargo-audit-bin-v1
+
+ - name: Install cargo-audit
+ run: command -v cargo-audit >/dev/null || cargo install cargo-audit --locked
+
+ - name: cargo audit
+ working-directory: engine
+ run: cargo audit
+
+ # Real fuzzing, for the same reason: `user-feedback/09-security/` promised
+ # "fuzz testing (cargo-fuzz) on all input parsing paths" while the tree held
+ # no fuzz target at all (#207). The harnesses live in engine/fuzz and cover
+ # the parsers an unauthenticated request reaches: the ES query DSL, the
+ # Lucene query_string grammar, date math and date formats, SQL, and Painless.
+ #
+ # On a pull request this is mostly a *regression* run — every checked-in seed
+ # is replayed through every target, and the leftover budget explores. For a
+ # real campaign, run the same script with a bigger budget:
+ # FUZZ_SECONDS=3600 bash .github/scripts/fuzz-smoke.sh
+ fuzz:
+ name: Fuzz harnesses (cargo-fuzz)
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+
+ # libFuzzer + the sanitizer runtimes are nightly-only.
+ - name: Install Rust (nightly)
+ uses: dtolnay/rust-toolchain@nightly
+ with:
+ components: rustfmt
+
+ - name: Cache
+ uses: actions/cache@v4
+ with:
+ path: |
+ ~/.cargo/registry/cache
+ ~/.cargo/registry/index
+ ~/.cargo/git/db
+ ~/.cargo/bin/cargo-fuzz
+ engine/fuzz/target
+ key: ci-fuzz-${{ hashFiles('engine/Cargo.lock', 'engine/fuzz/Cargo.toml') }}
+ restore-keys: ci-fuzz-
+
+ - name: Install cargo-fuzz
+ run: command -v cargo-fuzz >/dev/null || cargo install cargo-fuzz --locked
+
+ # The fuzz crate is its own workspace, so the `lint` job's
+ # `cargo fmt --all` never sees it. No `--all` here: it would follow the
+ # path dependencies back into the engine and re-check what `lint` covers.
+ - name: rustfmt the fuzz crate
+ working-directory: engine
+ run: cargo fmt --manifest-path fuzz/Cargo.toml --check
+
+ - name: Build + run every fuzz target
+ run: bash .github/scripts/fuzz-smoke.sh
+
+ # A crashing input is the entire product of this job; losing it would
+ # mean the failure is unreproducible.
+ - name: Upload crashing inputs
+ if: failure()
+ uses: actions/upload-artifact@v4
+ with:
+ name: fuzz-artifacts
+ path: engine/fuzz/artifacts/
+ if-no-files-found: ignore
+
build-test:
name: Build + Test
runs-on: ubuntu-latest
diff --git a/.gitignore b/.gitignore
index 0417af8e6..4e37b13fd 100644
--- a/.gitignore
+++ b/.gitignore
@@ -51,6 +51,18 @@ node_modules/
# Rust build output
target/
+# cargo-fuzz working state. The harnesses and their SEEDS are tracked
+# (engine/fuzz/fuzz_targets/, engine/fuzz/seeds/) — the seeds are the
+# regression suite and every one of them is named after what it covers. The
+# working corpus is not: `cargo fuzz run` writes every input it finds
+# interesting into it, and a few hundred opaque blobs per run is how a seed
+# corpus stops being reviewable. Promote a discovered input by copying it into
+# seeds/ under a descriptive name.
+engine/fuzz/corpus/
+engine/fuzz/artifacts/
+engine/fuzz/coverage/
+engine/fuzz/Cargo.lock
+
# Upstream Elasticsearch reference clone — not part of this repo
es-reference/
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 00ff4fc62..d866b90a0 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,8 +7,157 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
+### Security
+
+- **`cargo-audit` and `cargo-fuzz` now actually run in CI — they were only ever
+ documented** ([#207](https://github.com/xerj-org/xerj/issues/207)).
+ `user-feedback/09-security/cves-and-vulnerabilities.md` listed "`cargo-audit`
+ for dependency vulnerability scanning" and "fuzz testing (cargo-fuzz) on all
+ input parsing paths" as part of XERJ's answer to Elasticsearch's CVE history.
+ Neither existed: no audit job, no fuzz job, no fuzz target anywhere in the
+ tree. A reader comparing the two engines priced in a fuzzed parser surface
+ that was not there.
+ - New CI job `security-audit` runs `cargo audit` on every push and pull
+ request; a RUSTSEC vulnerability advisory fails the build.
+ - New CI job `fuzz` builds and runs six libFuzzer harnesses
+ (`engine/fuzz/`) over checked-in seeds: the Elasticsearch query DSL, the
+ Lucene `query_string` grammar, date math and date-format patterns,
+ index-name date math, SQL, and Painless scripts. `.github/scripts/fuzz-smoke.sh` is the same entry
+ point locally and in CI.
+ - `xerj-engine/tests/security_tooling_claims.rs` fails if a documented tool
+ stops running, if a fuzz target ships without a seed corpus, or if the
+ "all input parsing paths" overclaim reappears in any prose file.
+ - The claim itself is now specific about what is and is not covered.
+
+- **Fixed two high-severity advisories in the document-ingest XML parser.**
+ Turning the audit gate on found `quick-xml` 0.36.2 in `xerj-autoindex`, which
+ parses `.docx`, `.xlsx` and `.xml` files supplied by whoever owns the folder
+ being indexed: RUSTSEC-2026-0195 (unbounded namespace-declaration allocation →
+ memory-exhaustion DoS, CVSS 7.5) and RUSTSEC-2026-0194 (quadratic run time on
+ duplicate attribute names, CVSS 7.5). Upgraded to 0.41, which also
+ de-duplicates the crate — 0.41 was already in the tree via `pdf_oxide`.
+ `crossbeam-epoch` moved 0.9.18 → 0.9.20 for RUSTSEC-2026-0204.
+
+### Changed
+
+- **`merge.strategy = "log_structured"` is now refused at startup instead of
+ silently running the size-tiered policy**
+ ([#207](https://github.com/xerj-org/xerj/issues/207)). Nothing in the tree
+ ever read `merge.strategy`; `run_merge_once` always builds a
+ `SizeTieredMergePolicy`. An operator who chose a levelled policy for its read
+ amplification got the other one, quietly. This follows the `storage.backend`
+ and `vector.default_quantization` guards already in `Config::validate`.
+
+- **The three dormant `[merge]` settings now say so, in the code, in the shipped
+ TOML, on the docs site, and in the log at startup**
+ ([#207](https://github.com/xerj-org/xerj/issues/207)). The issue reported
+ merge I/O rate limiting as "claimed but dormant" and it is: the `RateLimiter`
+ that honours `io_rate_mb_per_sec` is wired only into `xerj-storage`'s legacy
+ `MergeExecutor`, which the engine never constructs. `min_segments` and
+ `max_concurrent` are unread too — the real per-tier trigger is
+ `min_merge_count`, and merge parallelism comes from `XERJ_MERGE_PARALLELISM`.
+ `MergeConfig::dormant_overrides` names any of the three the operator has moved
+ off its default and `xerj-server` logs each at WARN, so the operator throttling
+ merges to protect query latency finds out instead of guessing. They stay
+ accepted rather than becoming hard errors because the wrong value costs
+ latency, not data, and `io_rate_mb_per_sec = 100` has shipped in
+ `xerj.default.toml` since v0.1. Wiring a real throttle into `run_merge_once`
+ is the fix that makes this list shorter.
+
+- **`engine/xerj.default.toml` disagreed with the defaults it claims to
+ document, in four places.** `max_segment_mb` said 5120 against a real 8192,
+ `wal_max_size_mb` 512 against 1024, `flush_size_mb` 256 against 512, and
+ `default_quantization` `"scalar8"` against `"none"` — so `cp xerj.default.toml
+ xerj.toml`, the documented first step, silently changed four engine behaviours
+ including turning on 8-bit vector quantization. The file is corrected and
+ `shipped_default_config_documents_the_real_defaults` now diffs every leaf key
+ against `Config::default()`, so it cannot drift again. The docs site carried
+ the same four stale defaults and is corrected too.
+
+- **Four other claims from [#207](https://github.com/xerj-org/xerj/issues/207)
+ now match the source.**
+ - *Settings count.* `config.rs` said 38 in its header and 56 twenty lines
+ down while the count test asserted 60 — and the test asserted it by
+ re-adding a hardcoded sum (`5 + 2 + 3 + … - 1 == 38`), an identity that
+ could not fail whatever `Config` held. There was a second copy of the same
+ non-test in `config.rs` itself (`count_user_facing_settings`, `61 == 61`).
+ Both now serialise `Config::default()` and count leaf keys, section by
+ section. The measured figure is **103**, and 38 / 50 / 56 / 60 / 61 / "<50"
+ are corrected in `config.rs`, `xerj-common/src/lib.rs`, `engine/README.md`,
+ `xerj.default.toml` and the feedback responses, along with the stale
+ per-sub-config annotations (limits 3 → 13, storage 5 → 10, embedding 4 → 19,
+ merge 5 → 8, cluster 4 → 5, tls 3 → 4, auth 2 → 3). This closes item 10 of
+ `user-feedback/ROADMAP-TO-GA.md`. 103 versus 3,000+ is still the winning
+ story, told truthfully.
+ - *Wire-compatibility test framing.* `journey_es_migration` was described as
+ proving "the same curl commands" work. It calls the Rust engine API
+ directly and never makes an HTTP request. The response-shape assertions are
+ real and stay; the comment now says what the test does and points at the
+ ES-compat YAML conformance suite, which is what actually covers the wire.
+ - *OpenAPI spec.* `landing/openapi.json` declared `version: 1.0` while
+ specifying 7 routes of roughly 210. It is now titled and versioned as a
+ partial spec, and says so.
+
### Fixed
+- **A `query_string` containing one non-ASCII character aborted the process.**
+ `{"query_string":{"query":"_\u0660"}}` — or the same text in `?q=` — took the
+ server down. The Lucene tokeniser walks bytes but slices `&q[start..i]` as a
+ `&str`, and it broke on `is_whitespace()`, which is true for Latin-1 NBSP
+ (0xA0). In valid UTF-8 that byte is only ever a *continuation* byte, so the
+ scan stopped between the two bytes of U+0660 and the slice panicked on a
+ character boundary. Every delimiter test in that scanner is now `is_ascii_*`,
+ which makes "the scan can only stop on a character boundary" true by
+ construction. Nothing is lost: the only non-ASCII bytes the old test accepted
+ were 0x85 and 0xA0, neither of which is a standalone character in UTF-8.
+
+- **An index name of `<{}{}>` aborted the process.** Index-name date math finds
+ the first `{` and the last `}` inside the braces; when the `}` comes first,
+ `date_part[2..0]` is a panic, not an empty slice. Reachable from any request
+ that names an index. The closing brace is now required to come after the
+ opening one.
+
+- **A Painless script containing one non-ASCII character aborted the process.**
+ The second thing the new fuzz harnesses found, inside a second. The tokeniser
+ walks *bytes* (`bytes[i] as char` reinterprets each byte as Latin-1) but
+ slices `&src[start..i]` as a `&str`. Every continuation byte of a multi-byte
+ character reads as an alphabetic Latin-1 char, so an identifier scan could
+ start or stop inside one and the slice panicked on a char boundary —
+ `end byte index 24 is not a char boundary; it is inside 'ʋ'`. Scripts arrive
+ in ordinary search and update bodies and the release profile sets
+ `panic = "abort"`, so this was not a 400, it was the whole server. The
+ identifier scan is now ASCII-only, which keeps both ends on a character
+ boundary by construction; a non-ASCII byte reaches the tokeniser's
+ `unexpected char` error instead. String literals still carry any Unicode.
+
+- **A search body could abort the whole process: `{"range":{"ts":{"gte":"now+33333333333333H"}}}`.**
+ The first thing the new `date_math` fuzz target found, 20 seconds into its
+ first run. `chrono`'s `Duration::hours` and its siblings **panic** when the
+ count does not fit a `TimeDelta`, and they run before the
+ `checked_add_signed` that was supposed to make
+ `xerj_query::dates::add_unit` total — so the careful checking around them
+ bought nothing. A range bound goes straight from the request body into this
+ code (`parser.rs:931`), the release profile sets `panic = "abort"`, and the
+ result is an unauthenticated remote denial of service on a released binary.
+ Fixed with the fallible `try_*` constructors. Grepping for the same pattern
+ found two more copies in the index-name date-math resolvers
+ (`xerj-engine/src/index.rs`, `xerj-api/src/es_compat.rs`), reachable from a
+ request URI as `<logs-{now+9999999999999d}>`; both are fixed too, and their
+ `n * 30` / `n * 365` multiplications are now checked.
+
+- **An XML entity nobody declared no longer discards the text around it.**
+ Fallout from the `quick-xml` upgrade above, which is a breaking change to how
+ character data arrives: 0.38 stopped folding `&amp;` into the surrounding
+ `Event::Text` and began emitting each reference as its own
+ `Event::GeneralRef`. Entity handling is preserved across that change (a
+ reader that ignored the new event would have silently dropped every `&`, `<`,
+ `>` and `&#233;` while the words around them still arrived), and one case
+ gets better: 0.36's `BytesText::unescape()` failed the whole text node on an
+ entity no DTD declared for us, and `unwrap_or_default()` then discarded it —
+ now only the reference itself is dropped. Character data is also reassembled
+ from its fragments before being emitted, so an entity-bearing field stays one
+ value instead of becoming an array of pieces.
+
- **`autoindex` no longer leaves immortal catalog entries for skipped files**
([#238](https://github.com/xerj-org/xerj/issues/238)). A file added after the
resume plan was frozen is skipped and reported in the catalog — but that
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index ac19292de..dd19cfd16 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -59,6 +59,28 @@ cargo run -p es-yaml-runner -- --file tests/es-compat-yaml/yaml/bulk/10_basic.ym
Run the full suite before opening a PR. A test that was passing yesterday and fails today is a P0 regression.
+### Security gates (CI runs both on every PR)
+
+```bash
+# Dependency advisories. A RUSTSEC *vulnerability* fails CI; unmaintained /
+# unsound / yanked warnings are printed and do not.
+cargo install cargo-audit --locked
+cd engine && cargo audit
+
+# Fuzz harnesses over the parsers an unauthenticated request reaches
+# (ES query DSL, query_string, date math, SQL, Painless).
+rustup toolchain install nightly
+cargo install cargo-fuzz --locked
+bash .github/scripts/fuzz-smoke.sh # what CI runs, ~20s per target
+FUZZ_SECONDS=3600 bash .github/scripts/fuzz-smoke.sh # a real campaign
+```
+
+If you touch a parser that takes bytes off the wire, run the fuzz script before
+opening the PR — it is cheap, and the first thing it ever ran found an
+unauthenticated crash. See [`engine/fuzz/README.md`](./engine/fuzz/README.md)
+for how to add a target; seeds go in `engine/fuzz/seeds/<target>/` and CI picks
+new targets up automatically.
+
## Code style
- Format with `cargo fmt` and keep `cargo clippy` clean (`cargo clippy --all-targets -- -D warnings`). CI enforces both.
diff --git a/SECURITY.md b/SECURITY.md
index f0c8f8372..1df655077 100644
--- a/SECURITY.md
+++ b/SECURITY.md
@@ -70,6 +70,30 @@ We ask that reporters give us a reasonable opportunity to remediate an issue
before any public disclosure. We aim to resolve critical issues promptly and
will work with you to agree on a disclosure timeline.
+## Automated checks that run on every change
+
+Both of these run in [`.github/workflows/ci.yml`](.github/workflows/ci.yml) on
+every push and pull request. This section describes what the CI actually does —
+if you find a gap between this text and the workflow file, that is itself a bug
+worth reporting.
+
+- **`security-audit`** — [`cargo-audit`](https://github.com/rustsec/rustsec)
+ against `engine/Cargo.lock`. A RUSTSEC *vulnerability* advisory fails the
+ build. Informational advisories (unmaintained, unsound, yanked) are printed in
+ the job log but do not fail it, because they usually name transitive crates we
+ cannot upgrade on our own.
+- **`fuzz`** — [`cargo-fuzz`](https://github.com/rust-fuzz/cargo-fuzz)
+ harnesses in [`engine/fuzz/`](engine/fuzz/), covering the parsers an
+ unauthenticated request reaches: the Elasticsearch query DSL, the Lucene
+ `query_string` grammar, date math and date-format patterns, index-name date
+ math, SQL, and Painless scripts. Each target replays a checked-in seed corpus
+ and then fuzzes for a bounded budget. That list is the coverage — it is not
+ every parser in the engine.
+
+Neither is a substitute for a report. Fuzzing finds crashes and hangs, not logic
+flaws, and an advisory database only knows about vulnerabilities somebody has
+already published.
+
## Scope
This policy applies to the XERJ engine and its official crates in the
diff --git a/demo/playbooks/KIBANA_FEEDBACK_SCORECARD_RC4.md b/demo/playbooks/KIBANA_FEEDBACK_SCORECARD_RC4.md
index ed47c6f80..5da256d9a 100644
--- a/demo/playbooks/KIBANA_FEEDBACK_SCORECARD_RC4.md
+++ b/demo/playbooks/KIBANA_FEEDBACK_SCORECARD_RC4.md
@@ -204,7 +204,7 @@ Two very different halves. The dominant pain — the ELK observability stack is
Where the pain is "the SIEM stack is heavy, license-poisoned, slow, and a CVE treadmill," XERJ genuinely lands. Where the pain is the actual SIEM job — detection rules that fire actions, least-privilege analyst access, a complete audit trail — **XERJ is scaffolding wearing real clothes**: store-only Watcher, mock Alerts page, explicitly unenforced RBAC, and an audit log with exactly one call site.
- **DELIVERED — all-in-one self-hosted SIEM box:** single binary + bundled Console (`spa.rs:12-14,41-46`); Apache-2.0; in-process rustls TLS with auto self-signed cert + 0600 key tightening (`main.rs:343-385,204`); auth-on posture with exempt health probes (`auth.rs:23-44`).
-- **PARTIAL — CVE treadmill:** structurally smaller surface (no JVM, no plugins, vanilla-JS console); passkey auth with anti-enumeration fake challenges (`auth/login.rs:1-11`); rate limiting; constant-time compares. LIMITATION: no cargo-audit/deny gate, no fuzz harnesses (→ ES-scorecard Wave 4), and no CSP header on the served SPA.
+- **PARTIAL — CVE treadmill:** structurally smaller surface (no JVM, no plugins, vanilla-JS console); passkey auth with anti-enumeration fake challenges (`auth/login.rs:1-11`); rate limiting; constant-time compares. LIMITATION: no `cargo-deny` gate and no CSP header on the served SPA. (`cargo-audit` and the `cargo-fuzz` harnesses landed 2026-08-09 with #207 — CI jobs `security-audit` and `fuzz`.)
- **PARTIAL — detection rules that actually fire:** real half — ML anomaly detection with continuous datafeeds (`es_compat.rs:24956-24987,25767-25818`). Missing half — Watcher is a store-only stub (`es_compat.rs:20608-20629`); `.xerj_alert_rules`/`.xerj_alert_fires` are seeded but nothing ever writes fires (`indices.rs:32-33`); the Console Alerts page renders hardcoded mock numbers (`alerts.js:38-41`).
- **PARTIAL — slow at SIEM volumes:** doc-values prefilters + prefilter cache give single-digit-ms uncached segment reads (`index.rs:601-609,7893,8351`); slow-query capture built in (`es_compat.rs:8042-8053`). LIMITATION: read-latency tails under heavy writes (the ingest-plus-triage pattern) are the open mixed-p99 defect. → ES-scorecard Wave 1.
- **PARTIAL — least-privilege analyst access:** auth is real (admin key + minted API keys, HTTP+gRPC identical, `auth.rs:87-107`) but authorization is binary — rbac.rs's own header says "NOT ENFORCED… every authenticated caller is superuser" (`rbac.rs:1-12`). A SOC cannot hand an analyst a read-only credential today. → ES-scorecard Wave 4.
diff --git a/demo/playbooks/USER_FEEDBACK_SCORECARD_RC4.md b/demo/playbooks/USER_FEEDBACK_SCORECARD_RC4.md
index 1d9f9c444..9a168d40e 100644
--- a/demo/playbooks/USER_FEEDBACK_SCORECARD_RC4.md
+++ b/demo/playbooks/USER_FEEDBACK_SCORECARD_RC4.md
@@ -90,7 +90,8 @@ These are the strategic decisions for rc5, stated without spin.
- ILM/retention is store-only (DashMap round-trip); no background executor ever deletes expired data.
- Dynamic-mapping `strict` + field-limit accepted but not enforced on the ingest path — mapping explosion not prevented.
- `role_descriptors` accepted then ignored — every API key is effectively superuser.
-- No `cargo-audit`/`cargo-deny` gate (528 unscanned dep crates) and no `cargo-fuzz` harnesses.
+- ~~No `cargo-audit`/`cargo-deny` gate (528 unscanned dep crates) and no `cargo-fuzz` harnesses.~~
+ **CLOSED 2026-08-09 (#207):** CI job `security-audit` runs `cargo audit` on every push and PR, and job `fuzz` builds and runs the libFuzzer harnesses in `engine/fuzz/` (ES DSL, `query_string`, date math, index-name date math, SQL, Painless). `cargo-deny` is still absent.
- Binary/RaBitQ (BBQ) quantization unimplemented; `bbq_*`/`int4_*` mappings silently keep full f32 (no memory saving) — HNSW RAM wall only 4× mitigated by SQ8, no DiskANN.
**Marketing numbers to reconcile with measured reality (honesty-posture debt):**
@@ -138,8 +139,8 @@ Ranking = severity × reach (how many pain points / clusters it closes) × inver
| Candidate | Sev | Effort | Closes |
|---|---|---|---|
-| **Add `cargo-audit` + `cargo-deny` advisory gate to CI** | MED | S | security dependency-scanning GAP (528 unscanned crates) |
-| **Ship `cargo-fuzz` harnesses** for ES-DSL / query_string / bulk parsers | MED | M | security fuzz-testing GAP (the untrusted-input surfaces) |
+| ~~**Add `cargo-audit` + `cargo-deny` advisory gate to CI**~~ — `cargo-audit` DONE 2026-08-09 (#207), `cargo-deny` still open | MED | S | security dependency-scanning GAP (528 unscanned crates) |
+| ~~**Ship `cargo-fuzz` harnesses** for ES-DSL / query_string / bulk parsers~~ — DONE 2026-08-09 (#207): `engine/fuzz/`, 6 targets, run by CI. Bulk NDJSON is not among them | MED | M | security fuzz-testing GAP (the untrusted-input surfaces) |
| **TLS on-by-default via auto self-signed cert** (or first-run prompt) | MED | M | resolves the "secure/TLS by default" overclaim across resource-cost + security |
| **Enforce API-key `role_descriptors`** (real per-index/per-field RBAC) | MED | L | security RBAC PARTIAL — every key is superuser today; turns ES's paid-gate into an XERJ win |
| **Tamper-evident (WORM/append-only) audit log** | LOW | L | security audit PARTIAL (roadmap v0.9) — ES gates real audit behind Enterprise |
diff --git a/engine/Cargo.lock b/engine/Cargo.lock
index 9f38df4ca..abce508d5 100644
--- a/engine/Cargo.lock
+++ b/engine/Cargo.lock
@@ -1019,9 +1019,9 @@ dependencies = [
[[package]]
name = "crossbeam-epoch"
-version = "0.9.18"
+version = "0.9.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "5b82ac4a3c2ca9c3460964f020e1402edd5753411d7737aa39c3714ad1b5420e"
+checksum = "2d6914041f254d6e9176c01941b21115dcfb7089e55135a35411081bd106ef3f"
dependencies = [
"crossbeam-utils",
]
@@ -3086,7 +3086,7 @@ dependencies = [
"fast-float2",
"libc",
"log",
- "quick-xml 0.41.0",
+ "quick-xml",
"serde",
"serde_json",
"thiserror 2.0.18",
@@ -3275,7 +3275,7 @@ dependencies = [
"nom 8.0.0",
"office_oxide",
"phf 0.14.0",
- "quick-xml 0.41.0",
+ "quick-xml",
"regex",
"serde",
"serde_json",
@@ -3806,15 +3806,6 @@ version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a993555f31e5a609f617c12db6250dedcac1b0a85076912c436e6fc9b2c8e6a3"
-[[package]]
-name = "quick-xml"
-version = "0.36.2"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "f7649a7b4df05aed9ea7ec6f628c67c9953a43869b8bc50929569b2999d443fe"
-dependencies = [
- "memchr",
-]
-
[[package]]
name = "quick-xml"
version = "0.41.0"
@@ -6440,7 +6431,7 @@ dependencies = [
"libc",
"memchr",
"pdf_oxide",
- "quick-xml 0.36.2",
+ "quick-xml",
"rayon",
"regex",
"reqwest",
diff --git a/engine/README.md b/engine/README.md
index fde49a58f..d654a89ef 100644
--- a/engine/README.md
+++ b/engine/README.md
@@ -53,7 +53,7 @@ HTTP wire payloads, single-node, tmpfs data dir, both freshly started
| Per-batch (5K) bulk p99 | **34.8 ms** | 80.3 ms | ES 2.3× |
| Data loss on restart | 0% | 0% | tied |
| GC pauses | 20–40 ms (JVM) | **0** (no GC) | **xerj** |
-| Config surface | 3,000+ knobs | **38** | **xerj** |
+| Config surface | 3,000+ knobs | **103**, all optional | **xerj** |
xerj wins on every operational and read-path metric. ES still wins
bulk-ingest throughput on warm Lucene `IndexWriter` (perf-backlog item
@@ -81,7 +81,9 @@ The resulting binary is at `target/release/xerj`.
## Configuration
-See [`xerj.default.toml`](xerj.default.toml) for all 38 settings with documentation.
+See [`xerj.default.toml`](xerj.default.toml) for the commonly-tuned settings, and
+[`crates/xerj-common/src/config.rs`](crates/xerj-common/src/config.rs) for all 103
+— every one of them optional.
The minimal config to get started:
diff --git a/engine/crates/xerj-api/src/es_compat.rs b/engine/crates/xerj-api/src/es_compat.rs
index 91c93f51d..3279cd622 100644
--- a/engine/crates/xerj-api/src/es_compat.rs
+++ b/engine/crates/xerj-api/src/es_compat.rs
@@ -14653,8 +14653,14 @@ fn resolve_date_math_expr(expr: &str) -> String {
None => return expr.to_string(),
};
let brace_end = match expr.rfind('}') {
- Some(i) => i,
- None => return expr.to_string(),
+ // The closing brace must come AFTER the opening one, or the slice below
+ // is `expr[4..0]` — a panic, not an empty string. `<}<}<>{>{}>` is
+ // enough, and `panic = "abort"` makes it the process rather than a 400
+ // (#207). Braces in the wrong order are not date math, so the name
+ // passes through. Second copy of the same defect in
+ // `xerj-engine::index::resolve_date_math_inner`.
+ Some(i) if i > brace_start => i,
+ _ => return expr.to_string(),
};
let prefix = &expr[..brace_start];
@@ -14662,7 +14668,11 @@ fn resolve_date_math_expr(expr: &str) -> String {
// Split on an inner `{` for optional format: `now/d{yyyy.MM.dd}`.
let (math_expr, fmt) = if let Some(inner_brace) = date_part.find('{') {
- let inner_end = date_part.rfind('}').unwrap_or(date_part.len());
+ // Same ordering requirement as the outer pair, for the same reason.
+ let inner_end = date_part
+ .rfind('}')
+ .filter(|end| *end > inner_brace)
+ .unwrap_or(date_part.len());
(
&date_part[..inner_brace],
&date_part[inner_brace + 1..inner_end],
@@ -14807,23 +14817,37 @@ fn resolve_now_expr(
.unwrap_or(rest.len()),
);
let n: i64 = num_str.parse().unwrap_or(0) * sign;
- // Parse unit.
- let (unit, rest) = if !rest.is_empty() {
- (&rest[..1], &rest[1..])
- } else {
- ("", rest)
+ // Parse unit — one CHARACTER, not one byte. The digit scan stops at
+ // the first non-digit, which may be multi-byte, and `rest[..1]` would
+ // then split it in half and panic. `panic = "abort"` makes that the
+ // process, from an index name in a request URI. Second copy of what the
+ // `index_name_date_math` fuzz target found in `index.rs` (#207).
+ let (unit, rest) = match rest.chars().next() {
+ Some(c) => rest.split_at(c.len_utf8()),
+ None => ("", rest),
};
+ // `Duration::days` and friends PANIC on a count that does not fit a
+ // `TimeDelta`, and `n * 30` overflows first. `expr` is the date math
+ // inside an index name (`<logs-{now-9999999999999d}>`) taken from a
+ // request URI, so a panic here is an unauthenticated crash. Third copy
+ // of the defect the `date_math` fuzz target found in
+ // `xerj_query::dates::add_unit`.
+ //
+ // An unrepresentable offset degrades to "no offset" — the same thing
+ // this resolver already does with an unrecognised unit, and it has no
+ // error channel to report into.
let offset = match unit {
- "d" => Duration::days(n),
- "h" => Duration::hours(n),
- "m" => Duration::minutes(n),
- "s" => Duration::seconds(n),
- "w" => Duration::weeks(n),
- "M" => Duration::days(n * 30),
- "y" => Duration::days(n * 365),
- _ => Duration::zero(),
- };
- (base + offset, rest)
+ "d" => Duration::try_days(n),
+ "h" => Duration::try_hours(n),
+ "m" => Duration::try_minutes(n),
+ "s" => Duration::try_seconds(n),
+ "w" => Duration::try_weeks(n),
+ "M" => n.checked_mul(30).and_then(Duration::try_days),
+ "y" => n.checked_mul(365).and_then(Duration::try_days),
+ _ => Some(Duration::zero()),
+ }
+ .unwrap_or_else(Duration::zero);
+ (base.checked_add_signed(offset).unwrap_or(base), rest)
} else {
(base, rest)
};
diff --git a/engine/crates/xerj-autoindex/Cargo.toml b/engine/crates/xerj-autoindex/Cargo.toml
index 55283c9e9..9abfe6067 100644
--- a/engine/crates/xerj-autoindex/Cargo.toml
+++ b/engine/crates/xerj-autoindex/Cargo.toml
@@ -46,7 +46,7 @@ streaming-iterator = "0.1"
flate2 = "1" # gzip streaming (rust backend)
pdf_oxide = { version = "=0.3.75", default-features = false, features = ["legacy-crypto"] }
csv = "1" # streaming CSV with sniffed dialect
-quick-xml = "0.36" # XML pull parser + DOCX internals
+quick-xml = "0.41" # XML pull parser + DOCX internals
zip = { version = "2", default-features = false, features = ["deflate"] } # DOCX container
serde_yaml = "0.9" # YAML multi-doc
# The one C dependency (bundled SQLite), isolated to this crate.
diff --git a/engine/crates/xerj-autoindex/src/extract/docx.rs b/engine/crates/xerj-autoindex/src/extract/docx.rs
index f048641bf..a461ca5a7 100644
--- a/engine/crates/xerj-autoindex/src/extract/docx.rs
+++ b/engine/crates/xerj-autoindex/src/extract/docx.rs
@@ -83,7 +83,18 @@ fn extract_bounded(
// Stop appending once one paragraph reaches the body cap; the
// per-paragraph text a real document holds is far below it.
if para.len() < max_body_bytes {
- para.push_str(&t.unescape().unwrap_or_default());
+ para.push_str(&t.xml10_content().unwrap_or_default());
+ }
+ }
+ // quick-xml >= 0.38 emits `&amp;` and friends as their own event
+ // instead of folding them into the neighbouring text, so a
+ // paragraph containing an entity loses its `&`/`<`/`>` unless this
+ // arm puts it back.
+ Ok(Event::GeneralRef(r)) => {
+ if para.len() < max_body_bytes {
+ if let Some(resolved) = super::xml_x::resolve_general_ref(&r) {
+ para.push_str(&resolved);
+ }
}
}
Ok(Event::End(e)) => {
@@ -186,6 +197,31 @@ mod tests {
format!("m{i:06}")
}
+ /// quick-xml >= 0.38 emits `&amp;` as its own `Event::GeneralRef` instead
+ /// of folding it into the neighbouring `Event::Text`, so a reader that only
+ /// handles `Event::Text` drops every ampersand, angle bracket and numeric
+ /// character reference in the document — silently, since the surrounding
+ /// words still arrive. This pins that they come back.
+ #[test]
+ fn entity_and_character_references_survive_extraction() {
+ let xml = format!(
+ "{XML_HEAD}{}{XML_TAIL}",
+ r#"<w:p><w:r><w:t>Ben &amp; Jerry &lt;tag&gt; caf&#233;</w:t></w:r></w:p>"#
+ );
+ let dir = tempfile::tempdir().unwrap();
+ let path = dir.path().join("entities.docx");
+ write_docx(&path, &xml);
+
+ let (sections, stats) = extract_sections(&path, 1 << 20, 1 << 20);
+ assert_eq!(stats.junk, 0);
+ assert_eq!(
+ sections.concat().trim(),
+ "Ben & Jerry <tag> café",
+ "entity and numeric character references must resolve, and the text \
+ around them must not be re-split or lose its spaces"
+ );
+ }
+
#[test]
fn a_document_inflating_past_the_decompression_cap_is_truncated_at_the_byte_boundary() {
const PARAS: usize = 20_000;
diff --git a/engine/crates/xerj-autoindex/src/extract/xml_x.rs b/engine/crates/xerj-autoindex/src/extract/xml_x.rs
index defb78155..a99dd9c6e 100644
--- a/engine/crates/xerj-autoindex/src/extract/xml_x.rs
+++ b/engine/crates/xerj-autoindex/src/extract/xml_x.rs
@@ -6,14 +6,38 @@
use super::{
sanitize_field_name, ExtractStats, FieldOrigin, RawRecord, Sink, MAX_FIELDS_PER_RECORD,
+ MAX_LINE,
};
use anyhow::Result;
-use quick_xml::events::{BytesStart, Event};
+use quick_xml::events::{BytesRef, BytesStart, Event};
use quick_xml::Reader;
use serde_json::{Map, Value};
use std::collections::HashMap;
use std::path::Path;
+/// Resolve one `&…;` general reference to the text it stands for.
+///
+/// quick-xml >= 0.38 no longer folds entity references into the surrounding
+/// `Event::Text`: it emits each one as its own `Event::GeneralRef`, so a
+/// consumer that wants `5 &lt; 10` to read as `5 < 10` has to reassemble the
+/// text itself. Predefined XML entities and numeric character references
+/// resolve here; anything else (an entity a DTD we never read would have
+/// declared) yields `None` and the reference is dropped.
+///
+/// Dropping just the reference is deliberately *better* than what 0.36 did:
+/// there, one unknown entity failed `BytesText::unescape()` for the whole node
+/// and `unwrap_or_default()` threw the entire text away.
+pub(super) fn resolve_general_ref(r: &BytesRef) -> Option<String> {
+ match r.resolve_char_ref() {
+ Ok(Some(c)) => return Some(c.to_string()),
+ Ok(None) => {}
+ // Out-of-range or malformed `&#…;` — not text, so contribute nothing.
+ Err(_) => return None,
+ }
+ let name = r.decode().ok()?;
+ quick_xml::escape::resolve_predefined_entity(&name).map(str::to_string)
+}
+
struct State {
record_tag: Option<String>,
capture: Option<Map<String, Value>>,
@@ -29,8 +53,17 @@ pub fn extract(path: &Path, gzip: bool, sink: Sink) -> Result<ExtractStats> {
let r = super::open_reader(path, gzip, None)?;
let mut reader = Reader::from_reader(r);
- reader.config_mut().trim_text(true);
+ // Deliberately NOT `trim_text(true)`: the reader trims each fragment
+ // independently, and `5 &lt; 10` arrives as three fragments, so trimming
+ // per fragment would splice it back together as `5<10`. Trim once, at the
+ // element boundary, over the reassembled text instead.
+ reader.config_mut().trim_text(false);
let mut buf = Vec::new();
+ // Character data accumulates here and is flushed as ONE value when the
+ // next structural event arrives. Text, CDATA and entity references are all
+ // fragments of the same logical value; emitting them separately would turn
+ // every entity-bearing field into an array of pieces.
+ let mut pending = String::new();
let mut st = State {
record_tag,
capture: None,
@@ -43,9 +76,11 @@ pub fn extract(path: &Path, gzip: bool, sink: Sink) -> Result<ExtractStats> {
loop {
match reader.read_event_into(&mut buf) {
Ok(Event::Start(e)) => {
+ flush_text(&mut pending, &mut st);
handle_open(&e, false, &mut st);
}
Ok(Event::Empty(e)) => {
+ flush_text(&mut pending, &mut st);
if let Some(fields) = handle_open(&e, true, &mut st) {
// `<record …/>` self-closing: complete record, no End event.
if !fields.is_empty() {
@@ -64,6 +99,7 @@ pub fn extract(path: &Path, gzip: bool, sink: Sink) -> Result<ExtractStats> {
}
}
Ok(Event::End(e)) => {
+ flush_text(&mut pending, &mut st);
let name = String::from_utf8_lossy(e.name().as_ref()).to_string();
if let Some(m) = st.capture.as_mut() {
if st.stack.is_empty() && st.record_tag.as_deref() == Some(name.as_str()) {
@@ -90,20 +126,25 @@ pub fn extract(path: &Path, gzip: bool, sink: Sink) -> Result<ExtractStats> {
}
}
Ok(Event::Text(t)) => {
- let txt = t.unescape().unwrap_or_default().trim().to_string();
- if !txt.is_empty() {
- handle_text(&txt, &mut st);
+ push_text(&mut pending, &t.xml10_content().unwrap_or_default());
+ }
+ Ok(Event::GeneralRef(entity)) => {
+ if let Some(resolved) = resolve_general_ref(&entity) {
+ push_text(&mut pending, &resolved);
}
}
Ok(Event::CData(t)) => {
- let txt = String::from_utf8_lossy(&t).trim().to_string();
- if !txt.is_empty() {
- handle_text(&txt, &mut st);
- }
+ push_text(&mut pending, &String::from_utf8_lossy(&t));
+ }
+ Ok(Event::Eof) => {
+ flush_text(&mut pending, &mut st);
+ break;
}
- Ok(Event::Eof) => break,
Ok(_) => {}
Err(_) => {
+ // Everything read before the break is still emitted, so the
+ // text already buffered has to land before we stop.
+ flush_text(&mut pending, &mut st);
stats.junk += 1;
break;
}
@@ -190,6 +231,31 @@ fn handle_open(e: &BytesStart, empty: bool, st: &mut State) -> Option<Map<String
None
}
+/// Append one character-data fragment to the pending value.
+///
+/// SECURITY: the fragments of a single value are now concatenated rather than
+/// emitted one at a time, so a document that never closes an element could
+/// otherwise grow `pending` for the whole file. Stop appending at the same
+/// 16 MB bound the line-oriented extractors use; no real XML text node is
+/// anywhere near it.
+fn push_text(pending: &mut String, fragment: &str) {
+ if pending.len() < MAX_LINE {
+ pending.push_str(fragment);
+ }
+}
+
+/// Emit whatever character data has accumulated as one value, keyed by the
+/// element path that was open while it was read — so this must run *before*
+/// the structural event that triggered it mutates the stack.
+fn flush_text(pending: &mut String, st: &mut State) {
+ let txt = pending.trim();
+ if !txt.is_empty() {
+ let txt = txt.to_string();
+ handle_text(&txt, st);
+ }
+ pending.clear();
+}
+
fn handle_text(txt: &str, st: &mut State) {
if let Some(m) = st.capture.as_mut() {
let key = if st.stack.is_empty() {
@@ -403,6 +469,35 @@ mod tests {
assert!(p.contains(&serde_json::json!("<b>raw</b>")));
}
+ /// quick-xml >= 0.38 splits `a &amp; b` into Text / GeneralRef / Text, so a
+ /// reader that emits each fragment as it arrives turns one field into an
+ /// array of pieces — and, with the reader's own `trim_text`, one that has
+ /// lost the spaces that separated them. Both failures are silent: the words
+ /// are all still there, in the wrong shape. Character references have to
+ /// come through the same path.
+ #[test]
+ fn text_split_around_a_reference_is_reassembled_into_one_value() {
+ let (_, recs) = run(
+ r#"<doc><row><v>Ben &amp; Jerry &lt;ice&gt; caf&#233; &#x21;</v></row>
+ <row><v>plain</v></row><row><v>plain</v></row></doc>"#,
+ );
+ assert_eq!(
+ recs[0].fields["v"],
+ serde_json::json!("Ben & Jerry <ice> café !"),
+ "one text node is one value, spaces intact, references resolved"
+ );
+ }
+
+ /// An entity no DTD declared for us resolves to nothing. It must cost only
+ /// itself: quick-xml 0.36's `unescape()` failed the whole node, and
+ /// `unwrap_or_default()` then threw away every word around it.
+ #[test]
+ fn an_unresolvable_reference_costs_only_itself() {
+ let (_, recs) = run(r#"<doc><row><v>before &nosuchentity; after</v></row>
+ <row><v>plain</v></row><row><v>plain</v></row></doc>"#);
+ assert_eq!(recs[0].fields["v"], serde_json::json!("before after"));
+ }
+
#[test]
fn a_truncated_document_is_junk_filed_and_keeps_what_it_parsed() {
let (stats, recs) =
diff --git a/engine/crates/xerj-common/src/config.rs b/engine/crates/xerj-common/src/config.rs
index 8eb487731..f02362052 100644
--- a/engine/crates/xerj-common/src/config.rs
+++ b/engine/crates/xerj-common/src/config.rs
@@ -1,8 +1,14 @@
//! xerj configuration system.
//!
-//! Configuration is intentionally minimal: **38 settings** versus Elasticsearch's
-//! 3000+. Every option is named, documented, and has a sensible production-ready
-//! default. The format is TOML, loaded from a single file.
+//! Configuration is intentionally minimal: **103 settings** versus
+//! Elasticsearch's 3000+. Every option is named, documented, and has a sensible
+//! production-ready default. The format is TOML, loaded from a single file.
+//!
+//! That number is measured, not asserted: `journey_zero_config` in
+//! `xerj-engine/tests/product_experience.rs` counts the leaf keys of a default
+//! `Config` and fails when it drifts. It used to say 38 here, 56 twenty lines
+//! down and 60 in the test — three figures, none of them the truth, because the
+//! test re-added a hardcoded sum instead of counting anything (#207).
//!
//! ## Quick start
//!
@@ -48,15 +54,15 @@ use crate::error::XerjError;
pub struct Config {
/// Network and data directory settings — 6 settings.
pub server: ServerConfig,
- /// Authentication — 2 settings.
+ /// Authentication — 3 settings.
pub auth: AuthConfig,
/// CORS (cross-origin browser access) — 2 settings. Default restrictive.
pub cors: CorsConfig,
- /// TLS — 3 settings.
+ /// TLS — 4 settings.
pub tls: TlsConfig,
- /// Write-ahead log and flush behaviour — 5 settings.
+ /// Write-ahead log and flush behaviour — 10 settings.
pub storage: StorageConfig,
- /// Segment merging — 5 settings.
+ /// Segment merging — 8 settings.
pub merge: MergeConfig,
/// Data compression — 3 settings.
pub compression: CompressionConfig,
@@ -66,15 +72,15 @@ pub struct Config {
pub vector: VectorConfig,
/// Log (time-series) retention — 2 settings.
pub logs: LogsConfig,
- /// External embedding service — 4 settings.
+ /// External embedding service — 19 settings.
pub embedding: EmbeddingConfig,
- /// Resource limits — 3 settings.
+ /// Resource limits — 13 settings.
pub limits: LimitsConfig,
/// High-throughput turbo indexing — 3 settings.
pub indexing: IndexingConfig,
/// Engine parallelism — 4 settings.
pub engine: EngineConfig,
- /// Cluster / Raft settings — 4 settings.
+ /// Cluster / Raft settings — 5 settings.
pub cluster: ClusterConfig,
/// Point-in-time TTL + sweep cadence — 3 settings.
pub pit: PitConfig,
@@ -86,8 +92,10 @@ pub struct Config {
pub compat: CompatConfig,
}
-// Total: 5+3+2+3+10+5+3+1+6+2+4+3+4+3+2 = 56 fields (incl. cors: 2, auth: 3,
-// logging: 2). `Default` is derived — every field is a sub-config that
+// 19 sub-configs, 103 leaf settings in total. Do not maintain that sum by hand
+// — `journey_zero_config` in xerj-engine/tests/product_experience.rs counts a
+// serialised `Config::default()` and fails if this comment and the module
+// header stop matching. `Default` is derived: every field is a sub-config that
// implements `Default`, so the derive produces exactly the same all-defaults
// value the manual impl used to build by hand.
@@ -213,6 +221,19 @@ impl Config {
));
}
+ // Merge: only the size-tiered policy is implemented. `run_merge_once`
+ // (xerj-engine/src/index.rs) always builds a `SizeTieredMergePolicy`;
+ // nothing anywhere reads `merge.strategy`. Accepting `log_structured`
+ // would run size-tiered merging for an operator who chose a levelled
+ // policy for its read amplification — the same silent substitution the
+ // `storage.backend` guard above exists to prevent (#207).
+ if self.merge.strategy != MergeStrategy::SizeTiered {
+ return Err(XerjError::config(
+ "merge.strategy: the log_structured merge policy is not implemented in this \
+ build; only \"size_tiered\" is supported",
+ ));
+ }
+
// Vector: `scalar8` (SQ8) quantization is now wired into the kNN
// serving path (`Index::run_knn_brute_force`): a `scalar8` dense_vector
// field keeps a per-field u8 code store (1 byte/dim vs 4) and scores
@@ -308,7 +329,8 @@ impl Config {
}
// ═════════════════════════════════════════════════════════════════════════════
-// Sub-configs (38 user-facing settings total)
+// Sub-configs (103 user-facing settings total; counted by
+// `journey_zero_config`, not by hand)
// ═════════════════════════════════════════════════════════════════════════════
/// Network and data-directory settings.
@@ -654,25 +676,45 @@ pub enum WalSync {
/// Segment merge settings.
///
-/// **5 settings.**
+/// **8 settings**, of which three are **not read by the merge path that runs**
+/// — see [`MergeConfig::dormant_overrides`]. `run_merge_once`
+/// (`xerj-engine/src/index.rs`) reads `max_segment_mb`, `tier_floor_mb`,
+/// `min_merge_count` and `max_merge_count`, and nothing else from this struct.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(default, deny_unknown_fields)]
pub struct MergeConfig {
- /// Merge strategy: `"size_tiered"` or `"log_structured"` (default: `"size_tiered"`).
+ /// Merge strategy (default: `"size_tiered"`).
///
- /// - `"size_tiered"` — groups similarly-sized segments. Best for write-heavy.
- /// - `"log_structured"` — LSMT-style tiered levels. Best for mixed workloads.
+ /// Only `"size_tiered"` is implemented: `run_merge_once` always builds a
+ /// `SizeTieredMergePolicy`. `"log_structured"` is a documented variant with
+ /// no implementation behind it, so [`Config::validate`] rejects it rather
+ /// than run size-tiered merging while the operator believes otherwise.
pub strategy: MergeStrategy,
/// Minimum number of segments to trigger a merge (default: `10`).
+ ///
+ /// **DORMANT — accepted and validated, but nothing reads it.** The real
+ /// trigger is per-tier and comes from `min_merge_count`. Setting this away
+ /// from its default logs a warning at startup (see
+ /// [`MergeConfig::dormant_overrides`]).
pub min_segments: u32,
/// Maximum merged segment size in MiB (default: `8192` = 8 GiB).
/// Segments at or above this size are excluded from further merges.
pub max_segment_mb: u64,
/// I/O rate cap for merge operations in MiB/s (default: `100`).
///
- /// Throttle this to prevent merges from saturating I/O on shared storage.
+ /// **DORMANT — accepted, but merges are not throttled.** The `RateLimiter`
+ /// that honours this value is wired only into `xerj-storage`'s legacy
+ /// `MergeExecutor`, which the engine does not use; the merge that actually
+ /// runs (`run_merge_once`) never consults it. Setting it away from its
+ /// default logs a warning at startup (see
+ /// [`MergeConfig::dormant_overrides`]).
pub io_rate_mb_per_sec: u64,
/// Maximum number of concurrent merge operations (default: `1`).
+ ///
+ /// **DORMANT — accepted, but nothing reads it.** Merge parallelism comes
+ /// from the `XERJ_MERGE_PARALLELISM` environment variable, which also
+ /// defaults to 1. Setting this away from its default logs a warning at
+ /// startup (see [`MergeConfig::dormant_overrides`]).
pub max_concurrent: u32,
/// Tier boundary base for size-tiered merge policy in MiB (default: `4`).
/// Segments group into tiers by `floor(log2(size / tier_floor_mb))`.
@@ -700,6 +742,54 @@ impl Default for MergeConfig {
}
}
+impl MergeConfig {
+ /// Merge settings this build accepts but does not act on, reported as
+ /// `("merge.io_rate_mb_per_sec", "what actually happens")` — and only when
+ /// the operator has moved one away from its default, because leaving a
+ /// dormant knob alone asks for nothing.
+ ///
+ /// Issue #207 found `io_rate_mb_per_sec` documented, validated and inert:
+ /// the `RateLimiter` that honours it lives in `xerj-storage`'s legacy
+ /// `MergeExecutor`, which the engine never constructs. `min_segments` and
+ /// `max_concurrent` are in the same state. That is the accepted-and-ignored
+ /// pattern tracked by #204, and the operator it hurts is the one throttling
+ /// merges to protect query latency — they get no throttle and no signal.
+ ///
+ /// These stay accepted rather than becoming hard errors like
+ /// `storage.backend`: the wrong value costs latency, not data, and
+ /// `io_rate_mb_per_sec = 100` has shipped in `xerj.default.toml` since v0.1,
+ /// so rejecting it would refuse to start on a config the project itself
+ /// handed out. The caller (`xerj-server/src/main.rs`) logs each line at
+ /// WARN. Wiring a real throttle into `run_merge_once` is the fix that makes
+ /// this list shorter.
+ pub fn dormant_overrides(&self) -> Vec<(&'static str, &'static str)> {
+ let d = MergeConfig::default();
+ let mut out = Vec::new();
+ if self.io_rate_mb_per_sec != d.io_rate_mb_per_sec {
+ out.push((
+ "merge.io_rate_mb_per_sec",
+ "merge I/O is not throttled in this build — the rate limiter is \
+ wired only into xerj-storage's unused MergeExecutor",
+ ));
+ }
+ if self.min_segments != d.min_segments {
+ out.push((
+ "merge.min_segments",
+ "nothing reads it — the merge trigger is per-tier and comes from \
+ merge.min_merge_count",
+ ));
+ }
+ if self.max_concurrent != d.max_concurrent {
+ out.push((
+ "merge.max_concurrent",
+ "nothing reads it — merge parallelism comes from the \
+ XERJ_MERGE_PARALLELISM environment variable",
+ ));
+ }
+ out
+ }
+}
+
/// Segment merge strategy.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
@@ -899,7 +989,7 @@ impl Default for LogsConfig {
/// * `"auto"` (default) — use the proxy when [`default_endpoint`] is set,
/// otherwise lexical. This preserves the historical behavior exactly.
///
-/// **8 settings.**
+/// **19 settings.**
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(default, deny_unknown_fields)]
pub struct EmbeddingConfig {
@@ -1001,7 +1091,7 @@ impl Default for EmbeddingConfig {
/// Resource limits.
///
-/// **11 settings.**
+/// **13 settings.**
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(default, deny_unknown_fields)]
pub struct LimitsConfig {
@@ -1580,6 +1670,131 @@ mod tests {
.expect("shipped xerj.default.toml must pass Config::validate");
}
+ /// Flatten a serialised config to `("merge.max_segment_mb", 8192)` pairs.
+ fn flatten(prefix: &str, value: &serde_json::Value, out: &mut Vec<(String, String)>) {
+ match value {
+ serde_json::Value::Object(fields) => {
+ for (name, v) in fields {
+ let path = if prefix.is_empty() {
+ name.clone()
+ } else {
+ format!("{prefix}.{name}")
+ };
+ flatten(&path, v, out);
+ }
+ }
+ other => out.push((prefix.to_string(), other.to_string())),
+ }
+ }
+
+ /// The shipped file opens with "documents … their production-ready
+ /// defaults", so every value it sets must BE the default — otherwise
+ /// `cp xerj.default.toml xerj.toml`, the documented first step, silently
+ /// changes the engine's behaviour and the file becomes a second, divergent
+ /// source of truth for every number it quotes.
+ ///
+ /// It had already drifted when #207 was filed: the file said
+ /// `max_segment_mb = 5120` against a real default of 8192, so copying it
+ /// shrank the largest mergeable segment by 37% without saying so.
+ #[test]
+ fn shipped_default_config_documents_the_real_defaults() {
+ let toml_src = include_str!("../../../xerj.default.toml");
+ let from_file = serde_json::to_value(Config::from_toml_str(toml_src).unwrap()).unwrap();
+ let defaults = serde_json::to_value(Config::default()).unwrap();
+
+ let (mut a, mut b) = (Vec::new(), Vec::new());
+ flatten("", &from_file, &mut a);
+ flatten("", &defaults, &mut b);
+
+ let drift: Vec<String> = a
+ .iter()
+ .zip(b.iter())
+ .filter(|((_, file), (_, code))| file != code)
+ .map(|((key, file), (_, code))| format!("{key}: file says {file}, code says {code}"))
+ .collect();
+ assert!(
+ drift.is_empty(),
+ "engine/xerj.default.toml claims to document the shipped defaults but \
+ disagrees with Config::default() on {} setting(s):\n {}",
+ drift.len(),
+ drift.join("\n ")
+ );
+
+ // …and the file's own header quotes how many of the 103 it sets. That
+ // number was 38, then 56, and never once the truth (#207), so count the
+ // assignments instead of trusting the sentence.
+ let set_here = toml_src
+ .lines()
+ .filter(|l| {
+ let l = l.trim_start();
+ !l.starts_with('#')
+ && l.split_once('=')
+ .is_some_and(|(k, _)| !k.trim().is_empty() && !k.contains('['))
+ })
+ .count();
+ let total: usize = SETTINGS_BY_SECTION.iter().map(|(_, c)| c).sum();
+ let claim = format!("sets {set_here} of XERJ's {total} user-facing settings");
+ assert!(
+ toml_src.contains(&claim),
+ "xerj.default.toml's header must say {claim:?} — it sets {set_here} keys"
+ );
+ }
+
+ /// `merge.strategy = "log_structured"` used to parse, validate and then run
+ /// size-tiered merging anyway — nothing in the tree reads the field (#207).
+ /// An operator who picks a levelled policy for its read amplification must
+ /// hear that it does not exist, not get the other one silently.
+ #[test]
+ fn the_unimplemented_merge_strategy_is_rejected_not_silently_substituted() {
+ let err = Config::from_toml_str("[merge]\nstrategy = \"log_structured\"\n")
+ .expect_err("log_structured must be refused while it is unimplemented")
+ .to_string();
+ assert!(
+ err.contains("merge.strategy") && err.contains("not implemented"),
+ "the error must name the setting and say why: {err}"
+ );
+
+ Config::from_toml_str("[merge]\nstrategy = \"size_tiered\"\n")
+ .expect("the implemented policy must still be accepted");
+ }
+
+ /// The three merge knobs this build accepts without acting on. Silence was
+ /// the bug (#207): the operator throttling merges to protect query latency
+ /// got no throttle and no signal.
+ #[test]
+ fn dormant_merge_settings_are_named_only_when_an_operator_sets_them() {
+ assert!(
+ MergeConfig::default().dormant_overrides().is_empty(),
+ "an untouched default asks for nothing, so it must not warn"
+ );
+
+ for (toml, expected) in [
+ ("io_rate_mb_per_sec = 250", "merge.io_rate_mb_per_sec"),
+ ("min_segments = 4", "merge.min_segments"),
+ ("max_concurrent = 4", "merge.max_concurrent"),
+ ] {
+ let cfg = Config::from_toml_str(&format!("[merge]\n{toml}\n")).unwrap();
+ let named: Vec<&str> = cfg
+ .merge
+ .dormant_overrides()
+ .into_iter()
+ .map(|(key, _)| key)
+ .collect();
+ assert_eq!(
+ named,
+ vec![expected],
+ "setting {toml} must be reported, and nothing else"
+ );
+ }
+
+ // A setting that IS read by `run_merge_once` must never be reported.
+ let cfg = Config::from_toml_str("[merge]\nmax_segment_mb = 4096\n").unwrap();
+ assert!(
+ cfg.merge.dormant_overrides().is_empty(),
+ "max_segment_mb reaches the merge policy, so it is not dormant"
+ );
+ }
+
#[test]
fn onnx_throughput_controls_preserve_single_session_defaults() {
let cfg = Config::from_toml_str("[embedding]\n").unwrap();
@@ -1646,33 +1861,87 @@ mod tests {
);
}
+ /// Leaf keys in a serialised value — one per setting a user can set.
+ /// Sub-configs are grouping, not settings, so only leaves count.
+ fn count_settings(value: &serde_json::Value) -> usize {
+ match value {
+ serde_json::Value::Object(fields) => fields.values().map(count_settings).sum(),
+ _ => 1,
+ }
+ }
+
+ /// Settings per section, and the total. Every one of these numbers is
+ /// quoted in a doc comment above the matching struct (and the total in the
+ /// module header, in `xerj-common/src/lib.rs`, in `engine/README.md` and in
+ /// `xerj.default.toml`) — change the table and the prose together.
+ const SETTINGS_BY_SECTION: &[(&str, usize)] = &[
+ ("server", 6),
+ ("auth", 3),
+ ("cors", 2),
+ ("tls", 4),
+ ("storage", 10),
+ ("merge", 8),
+ ("compression", 3),
+ ("fts", 1),
+ ("vector", 6),
+ ("logs", 2),
+ ("embedding", 19),
+ ("limits", 13),
+ ("indexing", 3),
+ ("engine", 4),
+ ("cluster", 5),
+ ("pit", 3),
+ ("search_context", 7),
+ ("logging", 2),
+ ("compat", 2),
+ ];
+
+ /// Count the settings by *counting them*.
+ ///
+ /// This test used to read
+ ///
+ /// ```ignore
+ /// let total = 6 + 3 + 4 + 10 + 5 + 3 + 1 + 6 + 2 + 4 + 12 + 3 + 2;
+ /// assert_eq!(total, 61);
+ /// ```
+ ///
+ /// — the identity `61 == 61`, which could not fail whatever `Config`
+ /// actually held. By the time #207 was filed it had drifted so far that the
+ /// tree quoted 38, 50, 56, 60 and 61 in five places, none of them right.
+ /// `Config` is `Serialize` and every field defaults, so the honest count is
+ /// the number of leaf keys in a default config.
#[test]
fn count_user_facing_settings() {
- // 50 user-facing settings:
- // server: 6 (rest_port, grpc_port, es_compat_port, data_dir,
- // bind_address, trusted_proxies)
- // auth: 3 (enabled, admin_api_key, metrics_token) ← RC4-W4 item 4
- // tls: 4 (enabled, cert_path, key_path,
- // allow_insecure_grpc_h2c) ← issue #229
- // storage: 10 (wal_sync, wal_batch_ms, wal_max_size_mb, flush_size_mb,
- // flush_interval_secs, backend, s3_bucket, s3_prefix,
- // s3_region, local_cache_dir)
- // merge: 5 (strategy, min_segments, max_segment_mb, io_rate_mb_per_sec, max_concurrent)
- // compression: 3 (enabled, level, block_size_docs)
- // fts: 1 (default_analyzer)
- // vector: 6 (default_metric, hnsw_m, hnsw_ef_construction, hnsw_ef_search,
- // default_quantization, max_dimensions)
- // logs: 2 (retention_days, time_partition)
- // embedding: 4 (default_endpoint, default_model, batch_size, timeout_ms)
- // limits: 12 (query/concurrency/mapping/bulk-actions/body/result/mget/bucket
- // limits, snapshot repo allowlist, total memtable,
- // segment hydration cache, RSS, disk)
- // indexing: 3 (turbo_batch_size, turbo_parallel, turbo_fast_analyzer)
- // logging: 2 (format, access_log) ← RC4-W4 item 6
- // ─────────
- // total: 61 fields, minus 1 auto-generated (admin_api_key) = 60 meaningful user settings
- let total: usize = 6 + 3 + 4 + 10 + 5 + 3 + 1 + 6 + 2 + 4 + 12 + 3 + 2;
- assert_eq!(total, 61);
+ let cfg = serde_json::to_value(Config::default()).expect("Config serialises");
+ let sections = cfg.as_object().expect("Config is a table");
+
+ let measured: Vec<(String, usize)> = sections
+ .iter()
+ .map(|(name, v)| (name.clone(), count_settings(v)))
+ .collect();
+ let expected: Vec<(String, usize)> = SETTINGS_BY_SECTION
+ .iter()
+ .map(|(n, c)| ((*n).to_string(), *c))
+ .collect();
+ assert_eq!(
+ measured, expected,
+ "the per-section settings counts changed. That is fine — but each is \
+ quoted in the doc comment above its struct, so update both"
+ );
+
+ let total: usize = SETTINGS_BY_SECTION.iter().map(|(_, c)| c).sum();
+ assert_eq!(
+ total,
+ count_settings(&cfg),
+ "the section table must sum to the whole config"
+ );
+ assert_eq!(
+ total, 103,
+ "the total settings count changed. It is quoted in this module's \
+ header, in xerj-common/src/lib.rs, in engine/README.md, in \
+ xerj.default.toml and in EXPECTED_SETTINGS in \
+ xerj-engine/tests/product_experience.rs — update all of them"
+ );
}
// ── gRPC h2c exposure: fail closed (issue #229) ──────────────────────────
diff --git a/engine/crates/xerj-common/src/lib.rs b/engine/crates/xerj-common/src/lib.rs
index 7a94e6a1e..3afbbb258 100644
--- a/engine/crates/xerj-common/src/lib.rs
+++ b/engine/crates/xerj-common/src/lib.rs
@@ -7,13 +7,13 @@
//! ## Design philosophy
//!
//! Unlike Elasticsearch's 3000+ configuration knobs, xerj deliberately exposes
-//! exactly **38 settings**, each meaningful and production-tested. Every default
+//! **103 settings**, each meaningful and production-tested. Every default
//! is chosen so that a fresh deployment with zero configuration changes performs
//! well for the majority of workloads.
//!
//! ## Modules
//!
-//! - [`config`] — TOML-based configuration (38 settings)
+//! - [`config`] — TOML-based configuration (103 settings)
//! - [`error`] — Unified error type ([`XerjError`])
//! - [`types`] — Core domain types (documents, fields, IDs)
//! - [`schema`] — Index schema management and mapping evolution
diff --git a/engine/crates/xerj-engine/src/index.rs b/engine/crates/xerj-engine/src/index.rs
index 66a2a0ccf..7b52ab29d 100644
--- a/engine/crates/xerj-engine/src/index.rs
+++ b/engine/crates/xerj-engine/src/index.rs
@@ -25521,15 +25521,28 @@ fn resolve_date_math_inner(expr: &str) -> String {
None => return expr.to_string(),
};
let brace_end = match expr.rfind('}') {
- Some(i) => i,
- None => return expr.to_string(),
+ // The closing brace must come AFTER the opening one, or the slice
+ // below is `expr[4..0]` — a panic, not an empty string. `<}<}<>{>{}>`
+ // is enough, and `panic = "abort"` makes it the process rather than a
+ // 400 (#207, found by the `index_name_date_math` fuzz target). Braces
+ // in the wrong order are not date math, so the name passes through.
+ Some(i) if i > brace_start => i,
+ _ => return expr.to_string(),
};
let prefix = &expr[..brace_start];
let date_part = &expr[brace_start + 1..brace_end];
let (math_expr, fmt) = if let Some(inner_brace) = date_part.find('{') {
- let inner_end = date_part.rfind('}').unwrap_or(date_part.len());
+ // The closing brace has to come AFTER the opening one. `<{}{}>` gives a
+ // `date_part` of `}{`, where `rfind('}')` is 0 and `inner_brace` is 1 —
+ // and `date_part[2..0]` is a panic, not an error. With
+ // `panic = "abort"` that is the process, from an index name in a
+ // request URI. Same class as the two date-math offsets below.
+ let inner_end = date_part
+ .rfind('}')
+ .filter(|end| *end > inner_brace)
+ .unwrap_or(date_part.len());
(
&date_part[..inner_brace],
&date_part[inner_brace + 1..inner_end],
@@ -25544,6 +25557,73 @@ fn resolve_date_math_inner(expr: &str) -> String {
format!("{}{}", prefix, formatted)
}
+#[cfg(test)]
+mod index_name_date_math_tests {
+ use super::resolve_date_math;
+
+ /// Two regressions in one, both reachable from an index name in a request
+ /// URI, and both aborts rather than errors because the release profile sets
+ /// `panic = "abort"` (#207).
+ ///
+ /// * `<{}{}>` gives an inner `date_part` of `}{`, where the closing brace
+ /// is found *before* the opening one — and `date_part[2..0]` is a panic,
+ /// not an empty slice.
+ /// * `<logs-{now+9999999999999d}>` reached `chrono::Duration::days`, which
+ /// panics on a count that does not fit a `TimeDelta`. Same defect the
+ /// `date_math` fuzz target found in `xerj_query::dates::add_unit`.
+ ///
+ /// Before the fix each of these aborts the test process.
+ #[test]
+ fn a_malformed_or_unrepresentable_index_name_resolves_instead_of_aborting() {
+ for name in [
+ "<{}{}>",
+ "<}{>",
+ "<a{}{}z>",
+ "<{{}}>",
+ "<logs-{now+9999999999999d}>",
+ "<logs-{now-9999999999999d}>",
+ "<logs-{now+9223372036854775807y}>",
+ "<logs-{now+9223372036854775807M}>",
+ "<logs-{now+9223372036854775807w}>",
+ "<logs-{now+9223372036854775807h}>",
+ "<logs-{now/d",
+ "<>",
+ // The unit position holding a multi-byte character — the digit
+ // scan stops there and the old code sliced one byte out of it.
+ "<logs-{now+9\u{1be}999d}>",
+ "<logs-{now+1\u{4e2d}}>",
+ "<logs-{now-\u{e9}}>",
+ "<logs-{now+1d/\u{4e2d}}>",
+ "<logs-{now\u{1be}}>",
+ // Braces in the wrong order, outer pair and inner pair.
+ "<}<}<>{>{}>",
+ "}{",
+ "<a}b{c>",
+ "<a{b}c{d>",
+ ] {
+ // Any string is allowed to come back. None may take the process.
+ let _ = resolve_date_math(name);
+ }
+ }
+
+ /// The ordinary forms still resolve, so the guards above did not turn the
+ /// feature off to make the crash go away.
+ #[test]
+ fn ordinary_index_name_date_math_still_resolves() {
+ let today = chrono::Utc::now().format("%Y.%m.%d").to_string();
+ assert_eq!(resolve_date_math("<logs-{now/d}>"), format!("logs-{today}"));
+ assert_eq!(
+ resolve_date_math("<logs-{now/d{yyyy.MM.dd}}>"),
+ format!("logs-{today}")
+ );
+ assert_eq!(
+ resolve_date_math("plain-index"),
+ "plain-index",
+ "a name with no date math is returned unchanged"
+ );
+ }
+}
+
fn resolve_now_date(
expr: &str,
base: chrono::DateTime<chrono::Utc>,
@@ -25563,20 +25643,35 @@ fn resolve_now_date(
.find(|c: char| !c.is_ascii_digit())
.unwrap_or(rest.len());
let n: i64 = rest[..end].parse().unwrap_or(0) * sign;
- let (unit, rest) = if end < rest.len() {
- (&rest[end..end + 1], &rest[end + 1..])
- } else {
- ("", &rest[end..])
- };
+ // The unit is one CHARACTER, not one byte. `end` lands on the first
+ // non-digit, which may be multi-byte: `<logs-{now+9ƾ999d}>` made
+ // `rest[end..end + 1]` split U+01BE in half and panic. With
+ // `panic = "abort"` that is the process, from an index name in a
+ // request URI. Found by the `index_name_date_math` fuzz target (#207).
+ let (unit, rest) = match rest[end..].chars().next() {
+ Some(c) => rest[end..].split_at(c.len_utf8()),
+ None => ("", &rest[end..]),
+ };
+ // `Duration::days` and friends PANIC when the count does not fit a
+ // `TimeDelta`, and `n * 30` overflows before that. `expr` is the
+ // date-math inside an index name (`<logs-{now+9999999999999d}>`), which
+ // arrives in a request URI — so the panicking constructors are an
+ // unauthenticated crash. Same defect the `date_math` fuzz target found
+ // in `xerj_query::dates::add_unit`; this is the second copy.
+ //
+ // An offset that cannot be represented degrades to "no offset", which
+ // is what this resolver already does for an unrecognised unit. It has
+ // no error channel to report into.
let offset = match unit {
- "d" => Duration::days(n),
- "h" => Duration::hours(n),
- "w" => Duration::weeks(n),
- "M" => Duration::days(n * 30),
- "y" => Duration::days(n * 365),
- _ => Duration::zero(),
- };
- (base + offset, rest)
+ "d" => Duration::try_days(n),
+ "h" => Duration::try_hours(n),
+ "w" => Duration::try_weeks(n),
+ "M" => n.checked_mul(30).and_then(Duration::try_days),
+ "y" => n.checked_mul(365).and_then(Duration::try_days),
+ _ => Some(Duration::zero()),
+ }
+ .unwrap_or_else(Duration::zero);
+ (base.checked_add_signed(offset).unwrap_or(base), rest)
} else {
(base, rest)
};
diff --git a/engine/crates/xerj-engine/src/painless.rs b/engine/crates/xerj-engine/src/painless.rs
index 07c82f4a1..2616e0f64 100644
--- a/engine/crates/xerj-engine/src/painless.rs
+++ b/engine/crates/xerj-engine/src/painless.rs
@@ -348,8 +348,11 @@ fn tokenize(src: &str) -> Result<Vec<Tok>, String> {
let mut i = 0;
while i < bytes.len() {
let c = bytes[i] as char;
- // Skip whitespace
- if c.is_whitespace() {
+ // Skip whitespace. ASCII-only for the same reason the identifier scan
+ // below is: `is_whitespace()` is true for Latin-1 NBSP (0xA0), which in
+ // valid UTF-8 is only ever a continuation byte, so accepting it lets
+ // the scanner step into the middle of a character.
+ if c.is_ascii_whitespace() {
i += 1;
continue;
}
@@ -454,11 +457,28 @@ fn tokenize(src: &str) -> Result<Vec<Tok>, String> {
continue;
}
// Identifier / keyword
- if c.is_alphabetic() || c == '_' || c == '$' {
+ //
+ // ASCII-only, and that is load-bearing rather than a limitation of the
+ // subset. This tokeniser walks BYTES (`bytes[i] as char` reinterprets
+ // each byte as Latin-1) but slices `&src[start..i]` as a `&str`. Every
+ // continuation byte of a multi-byte character reads as an alphabetic
+ // Latin-1 char, so the old `is_alphabetic()` test let the scan start or
+ // stop inside one — and the slice then panicked on a char boundary.
+ // With `panic = "abort"` in the release profile that is not a 400, it
+ // is the process: `{"script":{"source":"doc[\"title\"].valueʋ..."}}`
+ // took the whole server down. Found by the `painless` fuzz target in
+ // under a second.
+ //
+ // Restricting the scan to ASCII keeps both ends on a character
+ // boundary by construction. A non-ASCII byte now falls through to the
+ // punctuation arms and ends at the `unexpected char` error below,
+ // which is the correct answer for an identifier this subset cannot
+ // evaluate anyway.
+ if c.is_ascii_alphabetic() || c == '_' || c == '$' {
let start = i;
while i < bytes.len() {
let cc = bytes[i] as char;
- if cc.is_alphanumeric() || cc == '_' || cc == '$' {
+ if cc.is_ascii_alphanumeric() || cc == '_' || cc == '$' {
i += 1;
} else {
break;
@@ -3920,4 +3940,55 @@ mod tests {
c.work_units()
);
}
+
+ /// Regression for the crash the `painless` fuzz target found in under a
+ /// second (issue #207). The tokeniser walks bytes but slices `&src[..]` as
+ /// a `&str`, and every continuation byte of a multi-byte character reads as
+ /// an alphabetic Latin-1 char — so an identifier scan could stop inside one
+ /// and the slice panicked on a char boundary. Scripts arrive in ordinary
+ /// search and update bodies and the release profile sets `panic = "abort"`,
+ /// so this was not a 400, it was the process.
+ ///
+ /// Before the fix each of these aborts the test process rather than
+ /// failing an assertion.
+ #[test]
+ fn a_non_ascii_character_in_a_script_is_an_error_not_a_panic() {
+ let doc = json!({ "title": "the quick brown fox" });
+ let params = json!({});
+
+ for src in [
+ // The input libFuzzer minimised to: an identifier scan that runs
+ // into U+028B.
+ "emit(doc[\"title\"].value\u{28b}oUppe",
+ "\u{28b}",
+ "value\u{28b}",
+ "\u{28b}value",
+ "doc[\"title\"].\u{4e2d}\u{6587}",
+ // Multi-byte inside a string literal must still be fine — the
+ // fix must not have made ordinary Unicode text unusable.
+ "\"caf\u{e9} \u{4e2d}\u{6587} \u{28b}\"",
+ // Non-ASCII at the very end, where the scan runs off the input.
+ "a\u{e9}",
+ "\u{e9}\u{e9}\u{e9}",
+ ] {
+ // Neither entry point may panic. Both are allowed to fail.
+ let _ = check_script_limits(src);
+ let _ = eval_painless(src, &ctx(&doc, &params, 0.0));
+ }
+
+ // A string literal carrying non-ASCII still evaluates to itself: the
+ // fix narrows the *identifier* scan, not the language.
+ let v = eval_painless("\"caf\u{e9}\"", &ctx(&doc, &params, 0.0)).expect("evaluate");
+ assert!(
+ matches!(&v, PainlessValue::String(s) if s == "caf\u{e9}"),
+ "{v:?}"
+ );
+
+ // ASCII identifiers are untouched by the narrowed scan.
+ let v = eval_painless("doc['title'].value", &ctx(&doc, &params, 0.0)).expect("evaluate");
+ assert!(
+ matches!(&v, PainlessValue::String(s) if s == "the quick brown fox"),
+ "{v:?}"
+ );
+ }
}
diff --git a/engine/crates/xerj-engine/tests/product_experience.rs b/engine/crates/xerj-engine/tests/product_experience.rs
index 78e00a588..d3e6cb6fd 100644
--- a/engine/crates/xerj-engine/tests/product_experience.rs
+++ b/engine/crates/xerj-engine/tests/product_experience.rs
@@ -427,9 +427,18 @@ async fn journey_log_analytics() {
#[tokio::test]
async fn journey_es_migration() {
- // Verify ES API contracts work unchanged.
- // Same curl commands. Same response fields. Drop-in replacement.
- // Verify: PUT index, index doc, search, bulk, count.
+ // Verify that the ES-shaped *response bodies* the migration story depends on
+ // are produced: `_index`/`_id`/`result`/`_shards` on a write, `hits.total`
+ // as an object on a search, per-item results on a bulk.
+ //
+ // Scope, stated honestly because this test used to claim more than it does
+ // (#207): it drives the Rust engine API directly — `create_index`,
+ // `index_document`, `process_bulk` — and never makes an HTTP request. It is
+ // NOT evidence that "the same curl commands" work; nothing here exercises
+ // routing, URI parameters, headers, status codes or content negotiation.
+ // The HTTP wire contract is covered by the ES-compat YAML conformance suite
+ // (`engine/tests/es-compat-yaml`, run by the `conformance` CI job), which
+ // replays real Elasticsearch REST test cases over the network.
let dir = TempDir::new().unwrap();
let engine = make_engine(&dir);
@@ -730,6 +739,22 @@ async fn journey_upgrade() {
// Journey 8: "How many settings do I need to configure?"
// ═════════════════════════════════════════════════════════════════════════════
+/// Leaf keys in a serialised config — one per setting a user can set.
+///
+/// Sub-configs are grouping, not settings, so only leaves count. An empty map
+/// contributes nothing, which is correct: a map-valued setting with no entries
+/// is not a knob anybody has turned.
+fn count_settings(value: &serde_json::Value) -> usize {
+ match value {
+ serde_json::Value::Object(fields) => fields.values().map(count_settings).sum(),
+ _ => 1,
+ }
+}
+
+/// Settings in a default `Config`, measured by `count_settings`. Quoted in
+/// `xerj-common/src/config.rs`; keep the two in step.
+const EXPECTED_SETTINGS: usize = 103;
+
#[tokio::test]
async fn journey_zero_config() {
// Start with ZERO configuration.
@@ -774,13 +799,26 @@ async fn journey_zero_config() {
"Search must work with zero configuration"
);
- // Count user-facing settings.
- // xerj: 38 settings (5+2+3+5+5+3+1+6+2+4+3 - 1 auto-generated).
- // ES: 3,000+ settings across elasticsearch.yml, jvm.options, log4j2.properties.
- let xerj_settings: usize = 5 + 2 + 3 + 5 + 5 + 3 + 1 + 6 + 2 + 4 + 3 - 1; // = 38
+ // Count user-facing settings — by *counting them*, not by re-adding a
+ // hardcoded sum. This assertion used to read
+ // let xerj_settings = 5 + 2 + 3 + 5 + 5 + 3 + 1 + 6 + 2 + 4 + 3 - 1;
+ // assert_eq!(xerj_settings, 38);
+ // which is the identity 38 == 38: it could not fail whatever `Config`
+ // actually held, and by the time #207 was filed the real number was nowhere
+ // near 38 while three places in the tree quoted three different figures.
+ //
+ // `Config` is `Serialize` and every field defaults, so the honest count is
+ // the number of leaf keys in a default config. (`xerj.default.toml` sets
+ // the commonly tuned 53 of them and describes the rest in comments; its own
+ // agreement with `Config::default()` is pinned by
+ // `shipped_default_config_documents_the_real_defaults` in xerj-common.)
+ let xerj_settings = count_settings(&serde_json::to_value(Config::default()).unwrap());
assert_eq!(
- xerj_settings, 38,
- "xerj must have exactly 38 user-facing settings"
+ xerj_settings, EXPECTED_SETTINGS,
+ "the settings count changed. That is fine — but it is quoted in prose, \
+ so update xerj-common/src/config.rs (module header and the total under \
+ `struct Config`) and EXPECTED_SETTINGS together, and re-check any doc \
+ that cites a number"
);
print_result(
diff --git a/engine/crates/xerj-engine/tests/security_tooling_claims.rs b/engine/crates/xerj-engine/tests/security_tooling_claims.rs
new file mode 100644
index 000000000..0b0319dd5
--- /dev/null
+++ b/engine/crates/xerj-engine/tests/security_tooling_claims.rs
@@ -0,0 +1,245 @@
+//! The security tooling this repo documents must be tooling that actually runs.
+//!
+//! Issue #207: `user-feedback/09-security/cves-and-vulnerabilities.md` advertised
+//! `cargo-audit` "for dependency vulnerability scanning" and "fuzz testing
+//! (cargo-fuzz) on all input parsing paths". Neither existed — no audit job, no
+//! fuzz job, no fuzz target anywhere in the tree. A reader comparing XERJ against
+//! the incumbent's CVE history priced in a fuzzed parser surface that was not
+//! there, which is a worse failure than admitting the gap.
+//!
+//! Prose cannot be unit-tested, but *wiring* can, and the wiring is what the
+//! prose is a claim about. These tests read the workflow file, the fuzz crate and
+//! the doc off disk and fail if they disagree. All four fail at the commit that
+//! filed #207.
+
+use std::collections::BTreeSet;
+use std::path::{Path, PathBuf};
+
+/// Walk up from this crate to the repository root.
+///
+/// Deliberately panics rather than skipping: a check that quietly turns into a
+/// no-op when it cannot find its inputs is exactly the tautological pass #207
+/// complains about elsewhere in this repo's test suite.
+fn repo_root() -> PathBuf {
+ let mut dir = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
+ loop {
+ if dir.join(".github/workflows/ci.yml").is_file() {
+ return dir;
+ }
+ assert!(
+ dir.pop(),
+ "no .github/workflows/ci.yml above {} — this test must run from a \
+ repository checkout, and must not pass by failing to look",
+ env!("CARGO_MANIFEST_DIR"),
+ );
+ }
+}
+
+fn read(path: &Path) -> String {
+ std::fs::read_to_string(path).unwrap_or_else(|e| panic!("read {}: {e}", path.display()))
+}
+
+/// `[[bin]] name = "…"` entries in the fuzz crate — the definitive list of
+/// harnesses that exist, taken from the manifest rather than from a list pasted
+/// into this test.
+fn declared_fuzz_targets(root: &Path) -> BTreeSet<String> {
+ let manifest = root.join("engine/fuzz/Cargo.toml");
+ assert!(
+ manifest.is_file(),
+ "engine/fuzz/Cargo.toml is missing: the repo documents cargo-fuzz harnesses, \
+ so the crate that holds them has to exist"
+ );
+ let text = read(&manifest);
+ let mut in_bin = false;
+ let mut names = BTreeSet::new();
+ for line in text.lines() {
+ let line = line.trim();
+ if line.starts_with('[') {
+ in_bin = line == "[[bin]]";
+ continue;
+ }
+ if in_bin {
+ if let Some(rest) = line.strip_prefix("name") {
+ if let Some(value) = rest.split('=').nth(1) {
+ names.insert(value.trim().trim_matches('"').to_string());
+ }
+ }
+ }
+ }
+ names
+}
+
+#[test]
+fn cargo_audit_runs_in_ci() {
+ let root = repo_root();
+ let ci = read(&root.join(".github/workflows/ci.yml"));
+
+ assert!(
+ ci.contains("security-audit:"),
+ "no `security-audit` job in ci.yml, but the security docs say cargo-audit \
+ scans our dependencies"
+ );
+ assert!(
+ ci.contains("cargo audit"),
+ "the `security-audit` job never invokes `cargo audit`"
+ );
+ assert!(
+ ci.contains("cargo install cargo-audit"),
+ "the audit job does not install cargo-audit, so `cargo audit` would be a \
+ command-not-found — which some runners report as a skipped step rather \
+ than a failure"
+ );
+}
+
+#[test]
+fn every_fuzz_target_has_a_harness_and_seeds() {
+ let root = repo_root();
+ let targets = declared_fuzz_targets(&root);
+
+ assert!(
+ !targets.is_empty(),
+ "engine/fuzz declares no [[bin]] targets — the cargo-fuzz claim would be \
+ satisfied by an empty crate"
+ );
+
+ for target in &targets {
+ let harness = root.join(format!("engine/fuzz/fuzz_targets/{target}.rs"));
+ assert!(
+ harness.is_file(),
+ "fuzz target `{target}` is declared in Cargo.toml but {} does not exist",
+ harness.display()
+ );
+
+ let seed_dir = root.join(format!("engine/fuzz/seeds/{target}"));
+ let seeds = std::fs::read_dir(&seed_dir)
+ .map(|d| d.flatten().count())
+ .unwrap_or(0);
+ assert!(
+ seeds > 0,
+ "fuzz target `{target}` has no seeds at {}. Starting from the empty \
+ input, a bounded CI run reaches almost none of the parser, so the \
+ target would pass without proving anything",
+ seed_dir.display()
+ );
+ }
+}
+
+#[test]
+fn ci_executes_the_fuzz_harnesses() {
+ let root = repo_root();
+ let ci = read(&root.join(".github/workflows/ci.yml"));
+ let script_rel = ".github/scripts/fuzz-smoke.sh";
+
+ assert!(
+ ci.contains("fuzz:"),
+ "no `fuzz` job in ci.yml — harnesses that exist but never run are the same \
+ false claim in a different place"
+ );
+ assert!(
+ ci.contains(script_rel),
+ "the fuzz job does not run {script_rel}"
+ );
+
+ let script = root.join(script_rel);
+ assert!(script.is_file(), "{script_rel} is missing");
+ let body = read(&script);
+
+ // The script must discover targets from the crate. A hardcoded list is how
+ // a new harness ends up shipped-but-never-executed.
+ assert!(
+ body.contains("fuzz list"),
+ "{script_rel} must enumerate targets with `cargo fuzz list` so a newly \
+ added harness cannot be silently left out of the run"
+ );
+}
+
+#[test]
+fn the_security_docs_claim_only_what_ci_does() {
+ let root = repo_root();
+ let ci = read(&root.join(".github/workflows/ci.yml"));
+
+ for doc_rel in [
+ "user-feedback/09-security/cves-and-vulnerabilities.md",
+ "SECURITY.md",
+ ] {
+ let doc = read(&root.join(doc_rel));
+ if doc.contains("cargo-audit") {
+ assert!(
+ ci.contains("cargo audit"),
+ "{doc_rel} claims cargo-audit; CI does not run it"
+ );
+ }
+ if doc.contains("cargo-fuzz") {
+ assert!(
+ !declared_fuzz_targets(&root).is_empty(),
+ "{doc_rel} claims cargo-fuzz; the tree has no fuzz targets"
+ );
+ }
+ }
+
+ // The specific overclaim #207 was filed about, hunted across every prose
+ // file rather than just this one — the point of the issue is that the claim
+ // must not survive anywhere, and copy gets copied.
+ //
+ // Coverage is five named parsers, not "all" of them, and the difference is
+ // the whole point: a reader is deciding whether to trust a parser surface.
+ let mut offenders = Vec::new();
+ scan_prose(&root, &mut |path, text| {
+ // CHANGELOG.md quotes the retired claim to say it was retired. A
+ // changelog is a record of what happened, not a statement about what
+ // the software does now, and it is the one file where the sentence
+ // appearing is correct.
+ if path.file_name().is_some_and(|n| n == "CHANGELOG.md") {
+ return;
+ }
+ if text.contains("all input parsing paths") {
+ offenders.push(path.to_path_buf());
+ }
+ });
+ assert!(
+ offenders.is_empty(),
+ "these files still claim fuzz coverage of *all* input parsing paths; list \
+ the parsers actually covered instead: {offenders:#?}"
+ );
+}
+
+/// Visit every prose file (`.md`, `.html`, `.txt`) that ships in this repo.
+///
+/// Skips VCS, build and dependency directories, and the fuzz inputs — those
+/// hold adversarial data, not claims about the product.
+fn scan_prose(dir: &Path, visit: &mut impl FnMut(&Path, &str)) {
+ const SKIP: [&str; 9] = [
+ ".git",
+ "target",
+ "node_modules",
+ "seeds",
+ "corpus",
+ "artifacts",
+ ".venv",
+ "vendor",
+ // Gitignored local clone of upstream Elasticsearch. Not our prose, and
+ // large enough to make this test feel broken on a dev machine.
+ "es-reference",
+ ];
+ let Ok(entries) = std::fs::read_dir(dir) else {
+ return;
+ };
+ for entry in entries.flatten() {
+ let path = entry.path();
+ let name = entry.file_name();
+ let name = name.to_string_lossy();
+ if SKIP.contains(&name.as_ref()) {
+ continue;
+ }
+ if path.is_dir() {
+ scan_prose(&path, visit);
+ } else if matches!(
+ path.extension().and_then(|e| e.to_str()),
+ Some("md" | "html" | "txt")
+ ) {
+ if let Ok(text) = std::fs::read_to_string(&path) {
+ visit(&path, &text);
+ }
+ }
+ }
+}
diff --git a/engine/crates/xerj-query/src/dates.rs b/engine/crates/xerj-query/src/dates.rs
index 53ed53257..5424b4d14 100644
--- a/engine/crates/xerj-query/src/dates.rs
+++ b/engine/crates/xerj-query/src/dates.rs
@@ -1201,15 +1201,25 @@ fn ms_to_naive(ms: i64) -> Option<NaiveDateTime> {
chrono::DateTime::from_timestamp_millis(ms).map(|dt| dt.naive_utc())
}
+/// `n` is a caller-supplied count straight out of a range bound
+/// (`"gte": "now+33333333333333H"`), so every arithmetic step here has to be
+/// fallible.
+///
+/// The `try_*` constructors are load-bearing: `Duration::hours` and its
+/// siblings **panic** when the count does not fit the underlying `TimeDelta`,
+/// and they panic *before* `checked_add_signed` ever sees the value — so the
+/// careful checking on the outside bought nothing. Found by the `date_math`
+/// fuzz target within 20 seconds of it first existing; `now+33333333333333H`
+/// panicked a search worker on an unauthenticated request.
fn add_unit(dt: NaiveDateTime, n: i64, unit: char) -> Option<NaiveDateTime> {
match unit {
'y' => shift_months(dt, n.checked_mul(12)?),
'M' => shift_months(dt, n),
- 'w' => dt.checked_add_signed(Duration::weeks(n)),
- 'd' => dt.checked_add_signed(Duration::days(n)),
- 'h' | 'H' => dt.checked_add_signed(Duration::hours(n)),
- 'm' => dt.checked_add_signed(Duration::minutes(n)),
- 's' => dt.checked_add_signed(Duration::seconds(n)),
+ 'w' => dt.checked_add_signed(Duration::try_weeks(n)?),
+ 'd' => dt.checked_add_signed(Duration::try_days(n)?),
+ 'h' | 'H' => dt.checked_add_signed(Duration::try_hours(n)?),
+ 'm' => dt.checked_add_signed(Duration::try_minutes(n)?),
+ 's' => dt.checked_add_signed(Duration::try_seconds(n)?),
_ => None,
}
}
@@ -2390,4 +2400,56 @@ mod ignored_metadata_field_oracle {
);
}
}
+ /// Regression for the crash the `date_math` fuzz target found within 20
+ /// seconds of first existing (issue #207): `chrono`'s `Duration::hours`
+ /// and its siblings **panic** on a count that does not fit a `TimeDelta`,
+ /// and they run before the `checked_add_signed` that was supposed to make
+ /// this arithmetic total. `{"range":{"ts":{"gte":"now+33333333333333H"}}}`
+ /// therefore panicked a search worker on an unauthenticated request.
+ ///
+ /// Before the fix every case below aborts the test process rather than
+ /// failing an assertion.
+ #[test]
+ fn an_out_of_range_date_math_offset_is_an_error_not_a_panic() {
+ // The exact input libFuzzer minimised to, plus one per unit and both
+ // signs — the panic is in the unit constructors, so each has its own.
+ for math in [
+ "+33333333333333H",
+ "+9223372036854775807H",
+ "-9223372036854775807H",
+ "+9223372036854775807h",
+ "+9223372036854775807m",
+ "+9223372036854775807s",
+ "+9223372036854775807d",
+ "-9223372036854775807d",
+ "+9223372036854775807w",
+ "+9223372036854775807M",
+ "+9223372036854775807y",
+ ] {
+ for round_up in [false, true] {
+ assert!(
+ apply_date_math(1_700_000_000_000, math, round_up).is_err(),
+ "`now{math}` must be rejected, not panic"
+ );
+ }
+ }
+
+ // The in-range neighbours still work, so the guard is not just
+ // rejecting everything.
+ assert!(apply_date_math(1_700_000_000_000, "+1H", false).is_ok());
+ assert!(apply_date_math(1_700_000_000_000, "-24h/d", true).is_ok());
+
+ // …and the whole way in from a request body, which is how it is
+ // reached: `resolve_range_bound` hands any string bound straight to
+ // `resolve_date_bound_str`. The workspace builds with
+ // `panic = "abort"`, so before the fix this body did not fail a
+ // request — it took the process down.
+ let body = serde_json::json!({
+ "query": { "range": { "ts": { "gte": "now+33333333333333H" } } }
+ });
+ assert!(
+ crate::parse_request(&body).is_err(),
+ "an unrepresentable range bound must be a 400, not an abort"
+ );
+ }
}
diff --git a/engine/crates/xerj-query/src/parser.rs b/engine/crates/xerj-query/src/parser.rs
index 7a6200035..12cebb5f8 100644
--- a/engine/crates/xerj-query/src/parser.rs
+++ b/engine/crates/xerj-query/src/parser.rs
@@ -1393,7 +1393,12 @@ fn qs_parse_cmp_range(q: &str, field: &str, rest: &str, mut i: usize) -> Result<
};
while i < bytes.len() {
let c = bytes[i] as char;
- if c.is_whitespace() || c == '(' || c == ')' {
+ // ASCII-only: this returns `i` to `tokenize_query_string`, which
+ // resumes scanning there and slices `&q[..]` from it. Stopping on a
+ // Latin-1-looking continuation byte (0xA0) would hand back an offset in
+ // the middle of a character and panic the caller's slice — see the
+ // invariant documented on `tokenize_query_string`.
+ if c.is_ascii_whitespace() || c == '(' || c == ')' {
break;
}
val.push(c);
@@ -1427,13 +1432,32 @@ fn qs_parse_cmp_range(q: &str, field: &str, rest: &str, mut i: usize) -> Result<
/// Tokenize a Lucene query string. `Ok(None)` means "can't tokenize — fall
/// back to the opaque QueryString path"; `Err` is a hard parse error
/// (malformed range syntax must not degrade to a term match).
+/// Tokenise a Lucene `query_string`.
+///
+/// SAFETY-OF-SLICING INVARIANT: this scanner walks BYTES — `bytes[i] as char`
+/// reinterprets each byte as Latin-1 — but slices `&q[start..i]` as a `&str`.
+/// That is only sound if every position it can stop at is a character
+/// boundary, which is why every delimiter test below is `is_ascii_*`: the bytes
+/// of a multi-byte UTF-8 character are all >= 0x80, so an ASCII test can never
+/// match one, and the scan can never stop inside a character.
+///
+/// It used to test `is_whitespace()`, which is true for Latin-1 NBSP (0xA0) —
+/// a *continuation byte*. `query_string: "…_٠…"` (U+0660, encoded D9 A0) broke
+/// the token scan between the two bytes and panicked on the slice. The release
+/// profile sets `panic = "abort"`, so that was not a 400 on an unauthenticated
+/// `?q=`, it was the process. Found by the `query_string` fuzz target (#207).
+///
+/// Nothing is lost by the narrowing: the only non-ASCII bytes `is_whitespace()`
+/// accepted were 0x85 and 0xA0, and both are continuation bytes, never a
+/// standalone character in valid UTF-8. Real Unicode whitespace (U+3000 and
+/// friends) was already token content here.
fn tokenize_query_string(q: &str) -> Result<Option<Vec<QsTok>>> {
let bytes = q.as_bytes();
let mut out: Vec<QsTok> = Vec::new();
let mut i = 0;
while i < bytes.len() {
let c = bytes[i] as char;
- if c.is_whitespace() {
+ if c.is_ascii_whitespace() {
i += 1;
continue;
}
@@ -1492,7 +1516,8 @@ fn tokenize_query_string(q: &str) -> Result<Option<Vec<QsTok>>> {
let start = i;
while i < bytes.len() {
let ch = bytes[i] as char;
- if ch.is_whitespace() || ch == '(' || ch == ')' {
+ // ASCII-only, per the slicing invariant on this function.
+ if ch.is_ascii_whitespace() || ch == '(' || ch == ')' {
break;
}
i += 1;
@@ -5458,4 +5483,52 @@ mod tests {
assert!(err.to_string().contains("too_many_clauses"), "got {err}");
}
}
+
+ /// Regression for the crash the `query_string` fuzz target found (#207).
+ ///
+ /// The tokeniser walks bytes and slices `&q[start..i]` as a `&str`, and it
+ /// used to break on `is_whitespace()` — which is true for Latin-1 NBSP
+ /// (0xA0). In valid UTF-8 that byte is only ever a *continuation* byte, so
+ /// `_٠` (U+0660, encoded D9 A0) stopped the scan between the two bytes and
+ /// the slice panicked on a character boundary. `query_string` is reachable
+ /// unauthenticated from a request body and from `?q=`, and the release
+ /// profile sets `panic = "abort"` — so this was the process, not a 400.
+ ///
+ /// Before the fix these abort the test process rather than failing.
+ #[test]
+ fn a_non_ascii_query_string_is_parsed_or_refused_but_never_panics() {
+ // The character libFuzzer landed on, plus other multi-byte widths and
+ // the two bytes `is_whitespace()` used to accept (0x85, 0xA0).
+ for query in [
+ "?:\"i(`:\"||+(l0O)_\u{660}\u{556}_ti(?:\"kk:))",
+ "_\u{660}",
+ "\u{660}",
+ "a\u{660}b c",
+ "field:\u{660}",
+ "field:>\u{660}",
+ "field:[\u{660} TO \u{661}]",
+ "\u{4e2d}\u{6587} AND \u{e9}t\u{e9}",
+ "caf\u{e9}\u{85}bar",
+ "caf\u{e9}\u{a0}bar",
+ "\u{1f600}",
+ ] {
+ for body in [
+ json!({ "query_string": { "query": query, "default_field": "body" } }),
+ json!({ "query_string": { "query": query } }),
+ json!({ "simple_query_string": { "query": query } }),
+ ] {
+ // Parsing may fail. It may not take the process with it.
+ let _ = parse_query(&body);
+ }
+ }
+
+ // ASCII query strings still parse to the same shapes.
+ let node = q(json!({
+ "query_string": { "query": "title:quick AND body:fox", "default_field": "body" }
+ }));
+ assert!(
+ matches!(node, QueryNode::Bool { .. }),
+ "an ordinary fielded query_string must still lower to a bool: {node:?}"
+ );
+ }
}
diff --git a/engine/crates/xerj-server/src/main.rs b/engine/crates/xerj-server/src/main.rs
index 69d208de9..5f36d6b01 100644
--- a/engine/crates/xerj-server/src/main.rs
+++ b/engine/crates/xerj-server/src/main.rs
@@ -364,6 +364,15 @@ fn load_config(args: &CliArgs) -> Result<Config> {
cfg.server.data_dir = dir.clone();
}
+ // Merge settings this build accepts but does not act on. An operator who
+ // throttles merges to protect query latency was getting no throttle and no
+ // signal (#207); the signal is the minimum. See
+ // `MergeConfig::dormant_overrides` for why these are a warning rather than
+ // the hard startup error `storage.backend` gets.
+ for (key, effect) in cfg.merge.dormant_overrides() {
+ warn!("{key} is set but has no effect in this build: {effect}");
+ }
+
if args.insecure {
warn!("--insecure: TLS and auth disabled");
cfg.tls.enabled = false;
diff --git a/engine/fuzz/Cargo.toml b/engine/fuzz/Cargo.toml
new file mode 100644
index 000000000..e1be09fa0
--- /dev/null
+++ b/engine/fuzz/Cargo.toml
@@ -0,0 +1,66 @@
+[package]
+name = "xerj-fuzz"
+version = "0.0.0"
+publish = false
+edition = "2021"
+license = "Apache-2.0"
+description = "libFuzzer harnesses for xerj's untrusted-input parsers"
+
+[package.metadata]
+cargo-fuzz = true
+
+[dependencies]
+libfuzzer-sys = "0.4"
+serde_json = "1"
+xerj-query = { path = "../crates/xerj-query" }
+xerj-engine = { path = "../crates/xerj-engine" }
+
+# Keep the harnesses out of the engine workspace: they only build under a
+# nightly toolchain with sanitizers, so a `cargo build --workspace` on stable
+# must not try to compile them.
+[workspace]
+
+[profile.release]
+debug = 1
+
+[[bin]]
+name = "query_dsl"
+path = "fuzz_targets/query_dsl.rs"
+test = false
+doc = false
+bench = false
+
+[[bin]]
+name = "query_string"
+path = "fuzz_targets/query_string.rs"
+test = false
+doc = false
+bench = false
+
+[[bin]]
+name = "date_math"
+path = "fuzz_targets/date_math.rs"
+test = false
+doc = false
+bench = false
+
+[[bin]]
+name = "sql"
+path = "fuzz_targets/sql.rs"
+test = false
+doc = false
+bench = false
+
+[[bin]]
+name = "painless"
+path = "fuzz_targets/painless.rs"
+test = false
+doc = false
+bench = false
+
+[[bin]]
+name = "index_name_date_math"
+path = "fuzz_targets/index_name_date_math.rs"
+test = false
+doc = false
+bench = false
diff --git a/engine/fuzz/README.md b/engine/fuzz/README.md
new file mode 100644
index 000000000..364c78e39
--- /dev/null
+++ b/engine/fuzz/README.md
@@ -0,0 +1,75 @@
+# Fuzz harnesses
+
+libFuzzer targets for the parsers an untrusted request reaches. CI builds and
+runs all of them on every push and pull request — see the `fuzz` job in
+[`.github/workflows/ci.yml`](../../.github/workflows/ci.yml), which is just a
+wrapper around [`.github/scripts/fuzz-smoke.sh`](../../.github/scripts/fuzz-smoke.sh).
+
+## Running them
+
+```sh
+rustup toolchain install nightly
+cargo install cargo-fuzz --locked
+
+# exactly what CI does: build every target, replay every seed, explore for 20s each
+bash .github/scripts/fuzz-smoke.sh
+
+# a real campaign
+FUZZ_SECONDS=3600 bash .github/scripts/fuzz-smoke.sh
+
+# one target, interactively
+cd engine && cargo +nightly fuzz run query_string
+```
+
+Crashing inputs land in `fuzz/artifacts/<target>/`; re-run one with
+`cargo +nightly fuzz run <target> fuzz/artifacts/<target>/<file>`.
+
+## seeds/ vs corpus/
+
+`fuzz/seeds/<target>/` is tracked; every file is named after what it covers and
+is a regression case. `fuzz/corpus/<target>/` is the working corpus and is
+**gitignored** — `cargo fuzz run` writes every input it finds interesting there,
+and a few hundred opaque blobs per run is how a seed corpus stops being
+reviewable. `fuzz-smoke.sh` copies the seeds in before each run.
+
+To promote a discovered input (a crash, or a case that reached something new),
+copy it into `seeds/<target>/` under a descriptive name.
+`seeds/date_math/regression-207-overflow-hours` is the first one: it is the
+input libFuzzer minimised the `TimeDelta::hours out of bounds` panic down to.
+
+## The targets
+
+| target | entry point | why it is untrusted |
+|---|---|---|
+| `query_dsl` | `xerj_query::parse_request`, `rewrite` | the whole `_search` body: nesting depth, clause count, sort and `_source` specs |
+| `query_string` | `xerj_query::parse_query` on `query_string` / `simple_query_string` | the hand-written Lucene grammar, also reachable via `?q=` — ES's CVE-2023-31419 was a stack overflow here |
+| `date_math` | `xerj_query::dates` | `"gte": "now-1d/d"` and `"format": "…"` are caller-supplied; both compile through hand-written tokenisers |
+| `sql` | `xerj_engine::sql::parse_sql` | `/_sql` takes a statement from the wire; ES's CVE-2024-43709 was an OOM from a crafted SQL query |
+| `painless` | `xerj_engine::painless::{check_script_limits, eval_painless}` | attacker-supplied *code* inside a query body, plus the depth/op/deadline budgets that are supposed to bound it |
+| `index_name_date_math` | `xerj_engine::index::resolve_date_math` | `<logs-{now/d}>` arrives in a request URI — a second date-math implementation with its own brace scanner |
+
+This list is the coverage. It is not every parser in the engine, and the
+security docs must name these rather than claim the parser surface generally —
+that overstatement is what [#207](https://github.com/xerj-org/xerj/issues/207)
+was filed about.
+
+## Adding a target
+
+1. Write `fuzz_targets/<name>.rs` and add the matching `[[bin]]` to `Cargo.toml`.
+2. Seed `seeds/<name>/` with real, *valid* inputs. A target with no seeds
+ starts from the empty input and reaches almost nothing in a bounded run.
+3. Nothing else — `fuzz-smoke.sh` discovers targets with `cargo fuzz list`, so
+ CI picks it up automatically, and
+ `xerj-engine/tests/security_tooling_claims.rs` fails if the harness or its
+ seeds are missing.
+
+## Conventions
+
+- Harnesses cap their input length. The engine's own caps are larger; the
+ smaller bound here keeps the fuzzer's budget on grammar states instead of on
+ the length check that rejects everything past it.
+- Prefer the uncached entry point when a memoising wrapper exists. Driving
+ unique keys through a bounded cache measures its eviction policy, not the
+ parser.
+- Harnesses must stay deterministic and free of I/O: no files, no sockets, no
+ clocks that change the control flow.
diff --git a/engine/fuzz/fuzz_targets/date_math.rs b/engine/fuzz/fuzz_targets/date_math.rs
new file mode 100644
index 000000000..497e74c7b
--- /dev/null
+++ b/engine/fuzz/fuzz_targets/date_math.rs
@@ -0,0 +1,50 @@
+//! Fuzz date-math expressions and date-format patterns.
+//!
+//! Both key spaces are attacker-influenced: `"gte": "now-1d/d"` and
+//! `"format": "yyyy-MM-dd'T'HH:mm:ss.SSSZ||epoch_millis"` arrive in a range
+//! query body, and the pattern compiler is a hand-written tokeniser over
+//! quoted literals, repeated field letters and `||`-separated alternatives.
+//!
+//! Deliberately uses the *uncached* `compile_formats`: the cached wrapper is
+//! keyed on the format string, and driving unique keys through it at fuzzing
+//! rates measures the cache's eviction policy instead of the parser.
+#![no_main]
+
+use libfuzzer_sys::fuzz_target;
+
+fuzz_target!(|data: &[u8]| {
+ let Ok(text) = std::str::from_utf8(data) else {
+ return;
+ };
+ if text.len() > 1024 {
+ return;
+ }
+
+ // Date math applied to both epoch 0 and a realistic base, rounding both
+ // ways — the rounding paths do the calendar arithmetic that can overflow.
+ for base in [0i64, 1_700_000_000_000] {
+ for round_up in [false, true] {
+ let _ = xerj_query::dates::apply_date_math(base, text, round_up);
+ }
+ }
+
+ // Pattern compilation, then parsing real values with whatever compiled.
+ if let Ok(formats) = xerj_query::dates::compile_formats(text) {
+ for probe in [
+ serde_json::Value::String("2026-01-31T23:59:59.999Z".to_string()),
+ serde_json::Value::String(text.to_string()),
+ serde_json::Value::from(1_700_000_000_000i64),
+ ] {
+ let _ = xerj_query::dates::date_value_matches_formats(&probe, &formats);
+ }
+ for round_up in [false, true] {
+ let _ = xerj_query::dates::resolve_date_bound_str(text, round_up, Some(&formats));
+ }
+ }
+
+ // Bound resolution with no declared format — the `now`-anchored and
+ // `<anchor>||<math>` branches.
+ for round_up in [false, true] {
+ let _ = xerj_query::dates::resolve_date_bound_str(text, round_up, None);
+ }
+});
diff --git a/engine/fuzz/fuzz_targets/index_name_date_math.rs b/engine/fuzz/fuzz_targets/index_name_date_math.rs
new file mode 100644
index 000000000..178729d89
--- /dev/null
+++ b/engine/fuzz/fuzz_targets/index_name_date_math.rs
@@ -0,0 +1,20 @@
+//! Fuzz Elasticsearch index-name date math: `<logs-{now/d}>`.
+//!
+//! This one is reached from a request *URI*, not a body — every write and read
+//! that names an index runs the name through it — and it is a second,
+//! independent implementation of date math with its own hand-written brace
+//! scanner on top. Both halves are attacker-controlled: the arithmetic
+//! (`now+9999999999999d`) and the brace/format structure (`<a{b{c}}>`).
+#![no_main]
+
+use libfuzzer_sys::fuzz_target;
+
+fuzz_target!(|data: &[u8]| {
+ let Ok(name) = std::str::from_utf8(data) else {
+ return;
+ };
+ if name.len() > 1024 {
+ return;
+ }
+ let _ = xerj_engine::index::resolve_date_math(name);
+});
diff --git a/engine/fuzz/fuzz_targets/painless.rs b/engine/fuzz/fuzz_targets/painless.rs
new file mode 100644
index 000000000..1b593788e
--- /dev/null
+++ b/engine/fuzz/fuzz_targets/painless.rs
@@ -0,0 +1,38 @@
+//! Fuzz the Painless script tokeniser, parser and evaluator.
+//!
+//! Scripts arrive inside ordinary search and update bodies, so this is
+//! attacker-supplied *code* — the highest-value parser in the engine and the
+//! historical source of the worst ES CVEs (CVE-2015-1427, Groovy sandbox
+//! escape). The harness runs both the admission check (`check_script_limits`,
+//! which must reject rather than hang) and a full evaluation against a
+//! realistic document, so the depth, op-count and deadline budgets are
+//! exercised rather than just the grammar.
+#![no_main]
+
+use libfuzzer_sys::fuzz_target;
+use xerj_engine::painless::{check_script_limits, eval_painless, PainlessCtx};
+
+fuzz_target!(|data: &[u8]| {
+ let Ok(src) = std::str::from_utf8(data) else {
+ return;
+ };
+ // The engine's own cap is larger; a smaller one here keeps every input the
+ // fuzzer generates inside the interesting (accepted) region.
+ if src.len() > 4096 {
+ return;
+ }
+
+ let _ = check_script_limits(src);
+
+ let doc = serde_json::json!({
+ "title": "the quick brown fox",
+ "price": 12.5,
+ "count": 7,
+ "tags": ["a", "b", "c"],
+ "nested": { "inner": [1, 2, 3] },
+ });
+ let params = serde_json::json!({ "factor": 2, "name": "x" });
+ let ctx = PainlessCtx::new(&doc, &params, 1.0);
+ let _ = eval_painless(src, &ctx);
+ let _ = ctx.take_emits();
+});
diff --git a/engine/fuzz/fuzz_targets/query_dsl.rs b/engine/fuzz/fuzz_targets/query_dsl.rs
new file mode 100644
index 000000000..b6e28e1de
--- /dev/null
+++ b/engine/fuzz/fuzz_targets/query_dsl.rs
@@ -0,0 +1,32 @@
+//! Fuzz the Elasticsearch-compatible search DSL parser.
+//!
+//! Reachable by any client that can POST to `/{index}/_search`, so the whole
+//! body is untrusted: nesting depth, clause count, field names, numeric
+//! ranges, `_source` filters, sort specs and aggregation shapes all come
+//! straight off the wire.
+//!
+//! The harness runs parse *and* rewrite, because the rewriter walks the tree
+//! the parser produced — a parser that accepts a pathological shape and a
+//! rewriter that then recurses on it are one bug, not two.
+#![no_main]
+
+use libfuzzer_sys::fuzz_target;
+
+fuzz_target!(|data: &[u8]| {
+ // serde_json is a third-party parser with its own fuzzing; this harness
+ // is about what xerj does with the value once it is JSON.
+ let Ok(value) = serde_json::from_slice::<serde_json::Value>(data) else {
+ return;
+ };
+
+ if let Ok(request) = xerj_query::parse_request(&value) {
+ // The parser's depth and clause budgets are the only thing standing
+ // between an untrusted body and unbounded recursion here.
+ let _ = xerj_query::rewrite(request.query);
+ }
+
+ // `parse_query` is also called directly (binary protocol, percolate-style
+ // paths), so exercise it on the same input rather than only through a
+ // well-formed request envelope.
+ let _ = xerj_query::parse_query(&value);
+});
diff --git a/engine/fuzz/fuzz_targets/query_string.rs b/engine/fuzz/fuzz_targets/query_string.rs
new file mode 100644
index 000000000..3ef64dc64
--- /dev/null
+++ b/engine/fuzz/fuzz_targets/query_string.rs
@@ -0,0 +1,41 @@
+//! Fuzz the Lucene `query_string` / `simple_query_string` mini-language.
+//!
+//! This is the one hand-written recursive-descent grammar on the read path:
+//! it tokenises quotes, escapes, field prefixes, boolean operators, grouping
+//! parens, wildcards and `[a TO b]` / `field:>10` range syntax. It is reachable
+//! unauthenticated wherever search is, both from a request body and from the
+//! `?q=` URI parameter, which is exactly the surface ES burned itself on in
+//! CVE-2023-31419 (stack overflow via crafted query strings).
+#![no_main]
+
+use libfuzzer_sys::fuzz_target;
+
+fuzz_target!(|data: &[u8]| {
+ let Ok(query) = std::str::from_utf8(data) else {
+ return;
+ };
+ // The parser enforces its own length cap; staying under it keeps the
+ // fuzzer's budget on grammar states rather than on the length check.
+ if query.len() > 4096 {
+ return;
+ }
+
+ for default_operator in ["OR", "AND"] {
+ let body = serde_json::json!({
+ "query_string": {
+ "query": query,
+ "default_field": "body",
+ "default_operator": default_operator,
+ }
+ });
+ let _ = xerj_query::parse_query(&body);
+ }
+
+ // No `default_field`: takes the `*`/multi-field lowering branch, which has
+ // different range-syntax handling from the single-field one.
+ let body = serde_json::json!({ "query_string": { "query": query } });
+ let _ = xerj_query::parse_query(&body);
+
+ let body = serde_json::json!({ "simple_query_string": { "query": query } });
+ let _ = xerj_query::parse_query(&body);
+});
diff --git a/engine/fuzz/fuzz_targets/sql.rs b/engine/fuzz/fuzz_targets/sql.rs
new file mode 100644
index 000000000..241aa97fb
--- /dev/null
+++ b/engine/fuzz/fuzz_targets/sql.rs
@@ -0,0 +1,19 @@
+//! Fuzz the SQL parser behind `/_sql`.
+//!
+//! A second hand-written tokeniser and recursive-descent parser, reachable
+//! with a single POST. ES's own equivalent surface produced CVE-2024-43709
+//! (crafted SQL query → OutOfMemoryError → node crash), so "does a crafted
+//! statement panic or allocate without bound" is the exact question here.
+#![no_main]
+
+use libfuzzer_sys::fuzz_target;
+
+fuzz_target!(|data: &[u8]| {
+ let Ok(sql) = std::str::from_utf8(data) else {
+ return;
+ };
+ if sql.len() > 8192 {
+ return;
+ }
+ let _ = xerj_engine::sql::parse_sql(sql);
+});
diff --git a/engine/fuzz/seeds/date_math/anchor b/engine/fuzz/seeds/date_math/anchor
new file mode 100644
index 000000000..fe947d35e
--- /dev/null
+++ b/engine/fuzz/seeds/date_math/anchor
@@ -0,0 +1 @@
+2026-01-31T23:59:59.999Z||+1M/M
\ No newline at end of file
diff --git a/engine/fuzz/seeds/date_math/iso_fmt b/engine/fuzz/seeds/date_math/iso_fmt
new file mode 100644
index 000000000..2549bda90
--- /dev/null
+++ b/engine/fuzz/seeds/date_math/iso_fmt
@@ -0,0 +1 @@
+yyyy-MM-dd'T'HH:mm:ss.SSSZ
\ No newline at end of file
diff --git a/engine/fuzz/seeds/date_math/literal b/engine/fuzz/seeds/date_math/literal
new file mode 100644
index 000000000..f688b1ad0
--- /dev/null
+++ b/engine/fuzz/seeds/date_math/literal
@@ -0,0 +1 @@
+'literal 'yyyy
\ No newline at end of file
diff --git a/engine/fuzz/seeds/date_math/multi b/engine/fuzz/seeds/date_math/multi
new file mode 100644
index 000000000..b5690702c
--- /dev/null
+++ b/engine/fuzz/seeds/date_math/multi
@@ -0,0 +1 @@
+yyyy/MM/dd||yyyy-MM-dd||epoch_second
\ No newline at end of file
diff --git a/engine/fuzz/seeds/date_math/named b/engine/fuzz/seeds/date_math/named
new file mode 100644
index 000000000..18797fe34
--- /dev/null
+++ b/engine/fuzz/seeds/date_math/named
@@ -0,0 +1 @@
+strict_date_optional_time||epoch_millis
\ No newline at end of file
diff --git a/engine/fuzz/seeds/date_math/now b/engine/fuzz/seeds/date_math/now
new file mode 100644
index 000000000..ceac75e01
--- /dev/null
+++ b/engine/fuzz/seeds/date_math/now
@@ -0,0 +1 @@
+now-1d/d
\ No newline at end of file
diff --git a/engine/fuzz/seeds/date_math/now_complex b/engine/fuzz/seeds/date_math/now_complex
new file mode 100644
index 000000000..8285ecfc1
--- /dev/null
+++ b/engine/fuzz/seeds/date_math/now_complex
@@ -0,0 +1 @@
+now+1M-3h/w
\ No newline at end of file
diff --git a/engine/fuzz/seeds/date_math/regression-207-overflow-hours b/engine/fuzz/seeds/date_math/regression-207-overflow-hours
new file mode 100644
index 000000000..3cb0700e1
Binary files /dev/null and b/engine/fuzz/seeds/date_math/regression-207-overflow-hours differ
diff --git a/engine/fuzz/seeds/date_math/round_all b/engine/fuzz/seeds/date_math/round_all
new file mode 100644
index 000000000..bc67ec6e0
--- /dev/null
+++ b/engine/fuzz/seeds/date_math/round_all
@@ -0,0 +1 @@
+now/y
\ No newline at end of file
diff --git a/engine/fuzz/seeds/date_math/weekyear b/engine/fuzz/seeds/date_math/weekyear
new file mode 100644
index 000000000..ce5b93f55
--- /dev/null
+++ b/engine/fuzz/seeds/date_math/weekyear
@@ -0,0 +1 @@
+YYYY-ww-e
\ No newline at end of file
diff --git a/engine/fuzz/seeds/index_name_date_math/hourly b/engine/fuzz/seeds/index_name_date_math/hourly
new file mode 100644
index 000000000..4b291b438
--- /dev/null
+++ b/engine/fuzz/seeds/index_name_date_math/hourly
@@ -0,0 +1 @@
+<metrics-{now/h{yyyy.MM.dd.HH}}>
\ No newline at end of file
diff --git a/engine/fuzz/seeds/index_name_date_math/no_math b/engine/fuzz/seeds/index_name_date_math/no_math
new file mode 100644
index 000000000..83991ca13
--- /dev/null
+++ b/engine/fuzz/seeds/index_name_date_math/no_math
@@ -0,0 +1 @@
+ordinary-index-name
\ No newline at end of file
diff --git a/engine/fuzz/seeds/index_name_date_math/offset b/engine/fuzz/seeds/index_name_date_math/offset
new file mode 100644
index 000000000..e5ca210e5
--- /dev/null
+++ b/engine/fuzz/seeds/index_name_date_math/offset
@@ -0,0 +1 @@
+<logs-{now-1d/d}>
\ No newline at end of file
diff --git a/engine/fuzz/seeds/index_name_date_math/plain b/engine/fuzz/seeds/index_name_date_math/plain
new file mode 100644
index 000000000..e888c0ac1
--- /dev/null
+++ b/engine/fuzz/seeds/index_name_date_math/plain
@@ -0,0 +1 @@
+<logs-{now/d}>
\ No newline at end of file
diff --git a/engine/fuzz/seeds/index_name_date_math/regression-207-inverted-braces b/engine/fuzz/seeds/index_name_date_math/regression-207-inverted-braces
new file mode 100644
index 000000000..91c35c42a
--- /dev/null
+++ b/engine/fuzz/seeds/index_name_date_math/regression-207-inverted-braces
@@ -0,0 +1 @@
+<{}{}>
\ No newline at end of file
diff --git a/engine/fuzz/seeds/index_name_date_math/regression-207-inverted-outer-braces b/engine/fuzz/seeds/index_name_date_math/regression-207-inverted-outer-braces
new file mode 100644
index 000000000..6459d2714
--- /dev/null
+++ b/engine/fuzz/seeds/index_name_date_math/regression-207-inverted-outer-braces
@@ -0,0 +1 @@
+}<}<>{>{}
\ No newline at end of file
diff --git a/engine/fuzz/seeds/index_name_date_math/regression-207-multibyte-unit b/engine/fuzz/seeds/index_name_date_math/regression-207-multibyte-unit
new file mode 100644
index 000000000..8723929e6
--- /dev/null
+++ b/engine/fuzz/seeds/index_name_date_math/regression-207-multibyte-unit
@@ -0,0 +1 @@
+<logs-{now+9ƾ999d}>
\ No newline at end of file
diff --git a/engine/fuzz/seeds/index_name_date_math/regression-207-offset-overflow b/engine/fuzz/seeds/index_name_date_math/regression-207-offset-overflow
new file mode 100644
index 000000000..803cb01fd
--- /dev/null
+++ b/engine/fuzz/seeds/index_name_date_math/regression-207-offset-overflow
@@ -0,0 +1 @@
+<logs-{now+9999999999999d}>
\ No newline at end of file
diff --git a/engine/fuzz/seeds/index_name_date_math/suffix_prefix b/engine/fuzz/seeds/index_name_date_math/suffix_prefix
new file mode 100644
index 000000000..03c279439
--- /dev/null
+++ b/engine/fuzz/seeds/index_name_date_math/suffix_prefix
@@ -0,0 +1 @@
+prefix-<logs-{now/M{yyyy.MM}}>-suffix
\ No newline at end of file
diff --git a/engine/fuzz/seeds/index_name_date_math/unbalanced b/engine/fuzz/seeds/index_name_date_math/unbalanced
new file mode 100644
index 000000000..7ca915f50
--- /dev/null
+++ b/engine/fuzz/seeds/index_name_date_math/unbalanced
@@ -0,0 +1 @@
+<logs-{now/d
\ No newline at end of file
diff --git a/engine/fuzz/seeds/index_name_date_math/with_format b/engine/fuzz/seeds/index_name_date_math/with_format
new file mode 100644
index 000000000..b94bc679d
--- /dev/null
+++ b/engine/fuzz/seeds/index_name_date_math/with_format
@@ -0,0 +1 @@
+<logs-{now/d{yyyy.MM.dd}}>
\ No newline at end of file
diff --git a/engine/fuzz/seeds/painless/ctx_update b/engine/fuzz/seeds/painless/ctx_update
new file mode 100644
index 000000000..f3f55a704
--- /dev/null
+++ b/engine/fuzz/seeds/painless/ctx_update
@@ -0,0 +1 @@
+ctx._source.count = ctx._source.count + 1;
\ No newline at end of file
diff --git a/engine/fuzz/seeds/painless/deep b/engine/fuzz/seeds/painless/deep
new file mode 100644
index 000000000..3efd8cbed
--- /dev/null
+++ b/engine/fuzz/seeds/painless/deep
@@ -0,0 +1 @@
+return ((((((1 + 2) * 3) - 4) / 5) + 6) * 7);
\ No newline at end of file
diff --git a/engine/fuzz/seeds/painless/doc_value b/engine/fuzz/seeds/painless/doc_value
new file mode 100644
index 000000000..280dbc7ca
--- /dev/null
+++ b/engine/fuzz/seeds/painless/doc_value
@@ -0,0 +1 @@
+doc["price"].value * params.factor
\ No newline at end of file
diff --git a/engine/fuzz/seeds/painless/emit b/engine/fuzz/seeds/painless/emit
new file mode 100644
index 000000000..0225f18ed
--- /dev/null
+++ b/engine/fuzz/seeds/painless/emit
@@ -0,0 +1 @@
+emit(doc["title"].value.toUpperCase());
\ No newline at end of file
diff --git a/engine/fuzz/seeds/painless/if_else b/engine/fuzz/seeds/painless/if_else
new file mode 100644
index 000000000..bf9071671
--- /dev/null
+++ b/engine/fuzz/seeds/painless/if_else
@@ -0,0 +1 @@
+if (doc["count"].value > 3) { return 1; } else { return 0; }
\ No newline at end of file
diff --git a/engine/fuzz/seeds/painless/list b/engine/fuzz/seeds/painless/list
new file mode 100644
index 000000000..724bbcde4
--- /dev/null
+++ b/engine/fuzz/seeds/painless/list
@@ -0,0 +1 @@
+def l = [1,2,3]; return l.size();
\ No newline at end of file
diff --git a/engine/fuzz/seeds/painless/loop b/engine/fuzz/seeds/painless/loop
new file mode 100644
index 000000000..b2631c685
--- /dev/null
+++ b/engine/fuzz/seeds/painless/loop
@@ -0,0 +1 @@
+int s = 0; for (int i = 0; i < 10; i++) { s += i; } return s;
\ No newline at end of file
diff --git a/engine/fuzz/seeds/painless/regression-207-non-ascii-identifier b/engine/fuzz/seeds/painless/regression-207-non-ascii-identifier
new file mode 100644
index 000000000..5e91dd9bf
--- /dev/null
+++ b/engine/fuzz/seeds/painless/regression-207-non-ascii-identifier
@@ -0,0 +1 @@
+emit(doc["title"].valueʋoUppe
\ No newline at end of file
diff --git a/engine/fuzz/seeds/painless/string b/engine/fuzz/seeds/painless/string
new file mode 100644
index 000000000..43d314b6d
--- /dev/null
+++ b/engine/fuzz/seeds/painless/string
@@ -0,0 +1 @@
+return doc["title"].value.length() + params.name.length();
\ No newline at end of file
diff --git a/engine/fuzz/seeds/painless/ternary b/engine/fuzz/seeds/painless/ternary
new file mode 100644
index 000000000..24cd0dc0c
--- /dev/null
+++ b/engine/fuzz/seeds/painless/ternary
@@ -0,0 +1 @@
+return doc["price"].value > 10 ? "hi" : "lo";
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_dsl/aggs.json b/engine/fuzz/seeds/query_dsl/aggs.json
new file mode 100644
index 000000000..3e1458fe8
--- /dev/null
+++ b/engine/fuzz/seeds/query_dsl/aggs.json
@@ -0,0 +1 @@
+{"size":0,"aggs":{"by_day":{"date_histogram":{"field":"ts","calendar_interval":"1d"},"aggs":{"avg_price":{"avg":{"field":"price"}}}}}}
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_dsl/bool.json b/engine/fuzz/seeds/query_dsl/bool.json
new file mode 100644
index 000000000..012994edf
--- /dev/null
+++ b/engine/fuzz/seeds/query_dsl/bool.json
@@ -0,0 +1 @@
+{"query":{"bool":{"must":[{"term":{"status":"ok"}}],"should":[{"match":{"body":"rust"}}],"must_not":[{"exists":{"field":"deleted"}}],"filter":[{"range":{"price":{"gte":10,"lt":100}}}],"minimum_should_match":1}}}
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_dsl/highlight.json b/engine/fuzz/seeds/query_dsl/highlight.json
new file mode 100644
index 000000000..a2180f1a7
--- /dev/null
+++ b/engine/fuzz/seeds/query_dsl/highlight.json
@@ -0,0 +1 @@
+{"query":{"query_string":{"query":"title:(quick OR brown) AND price:>10"}},"highlight":{"fields":{"title":{}}},"explain":true,"profile":true}
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_dsl/ids_terms.json b/engine/fuzz/seeds/query_dsl/ids_terms.json
new file mode 100644
index 000000000..f1557dd2a
--- /dev/null
+++ b/engine/fuzz/seeds/query_dsl/ids_terms.json
@@ -0,0 +1 @@
+{"query":{"bool":{"filter":[{"ids":{"values":["1","2","3"]}},{"terms":{"tag":["a","b"]}},{"wildcard":{"path":"/a/*/c"}}]}},"search_after":[1,"x"]}
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_dsl/knn.json b/engine/fuzz/seeds/query_dsl/knn.json
new file mode 100644
index 000000000..5541fd991
--- /dev/null
+++ b/engine/fuzz/seeds/query_dsl/knn.json
@@ -0,0 +1 @@
+{"knn":{"field":"vec","query_vector":[0.1,0.2,0.3],"k":10,"num_candidates":100},"size":10}
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_dsl/match.json b/engine/fuzz/seeds/query_dsl/match.json
new file mode 100644
index 000000000..489744609
--- /dev/null
+++ b/engine/fuzz/seeds/query_dsl/match.json
@@ -0,0 +1 @@
+{"query":{"match":{"title":"quick brown fox"}},"size":10}
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_dsl/multi_match.json b/engine/fuzz/seeds/query_dsl/multi_match.json
new file mode 100644
index 000000000..32f2aca22
--- /dev/null
+++ b/engine/fuzz/seeds/query_dsl/multi_match.json
@@ -0,0 +1 @@
+{"query":{"multi_match":{"query":"hello","fields":["title^2","body"],"type":"best_fields","fuzziness":"AUTO"}}}
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_dsl/nested_bool.json b/engine/fuzz/seeds/query_dsl/nested_bool.json
new file mode 100644
index 000000000..e095dda10
--- /dev/null
+++ b/engine/fuzz/seeds/query_dsl/nested_bool.json
@@ -0,0 +1 @@
+{"query":{"bool":{"must":[{"bool":{"should":[{"bool":{"must":[{"prefix":{"a":"b"}}]}}]}}]}}}
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_dsl/range_datemath.json b/engine/fuzz/seeds/query_dsl/range_datemath.json
new file mode 100644
index 000000000..76472856a
--- /dev/null
+++ b/engine/fuzz/seeds/query_dsl/range_datemath.json
@@ -0,0 +1 @@
+{"query":{"range":{"ts":{"gte":"now-7d/d","lte":"now","format":"epoch_millis"}}}}
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_dsl/script_fields.json b/engine/fuzz/seeds/query_dsl/script_fields.json
new file mode 100644
index 000000000..d520d2ea2
--- /dev/null
+++ b/engine/fuzz/seeds/query_dsl/script_fields.json
@@ -0,0 +1 @@
+{"query":{"match_all":{}},"script_fields":{"double":{"script":{"source":"doc[\"price\"].value * 2"}}},"track_total_hits":true}
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_dsl/sort_source.json b/engine/fuzz/seeds/query_dsl/sort_source.json
new file mode 100644
index 000000000..c2a400d1b
--- /dev/null
+++ b/engine/fuzz/seeds/query_dsl/sort_source.json
@@ -0,0 +1 @@
+{"query":{"match_all":{}},"sort":[{"date":"desc"},"_score"],"_source":{"includes":["a","b"],"excludes":["c"]},"from":0,"size":25}
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_string/boost b/engine/fuzz/seeds/query_string/boost
new file mode 100644
index 000000000..219a423e8
--- /dev/null
+++ b/engine/fuzz/seeds/query_string/boost
@@ -0,0 +1 @@
+title:rust^3 body:search^0.5
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_string/date_range b/engine/fuzz/seeds/query_string/date_range
new file mode 100644
index 000000000..d75b2d875
--- /dev/null
+++ b/engine/fuzz/seeds/query_string/date_range
@@ -0,0 +1 @@
+ts:[2026-01-01 TO 2026-12-31] AND ts:{now-1d/d TO now}
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_string/escapes b/engine/fuzz/seeds/query_string/escapes
new file mode 100644
index 000000000..c448215a2
--- /dev/null
+++ b/engine/fuzz/seeds/query_string/escapes
@@ -0,0 +1 @@
+model:claude\-haiku\-4\-5 AND status:ok
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_string/fielded b/engine/fuzz/seeds/query_string/fielded
new file mode 100644
index 000000000..1801ffc2f
--- /dev/null
+++ b/engine/fuzz/seeds/query_string/fielded
@@ -0,0 +1 @@
+title:quick AND body:fox
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_string/grouping b/engine/fuzz/seeds/query_string/grouping
new file mode 100644
index 000000000..dead81e76
--- /dev/null
+++ b/engine/fuzz/seeds/query_string/grouping
@@ -0,0 +1 @@
+(a OR b) AND NOT (c AND d)
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_string/phrase b/engine/fuzz/seeds/query_string/phrase
new file mode 100644
index 000000000..7dbfc53b6
--- /dev/null
+++ b/engine/fuzz/seeds/query_string/phrase
@@ -0,0 +1 @@
+"exact phrase here"~2
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_string/range_bracket b/engine/fuzz/seeds/query_string/range_bracket
new file mode 100644
index 000000000..0d39aaac7
--- /dev/null
+++ b/engine/fuzz/seeds/query_string/range_bracket
@@ -0,0 +1 @@
+price:[10 TO 100] AND ts:[* TO now]
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_string/range_op b/engine/fuzz/seeds/query_string/range_op
new file mode 100644
index 000000000..f235147a8
--- /dev/null
+++ b/engine/fuzz/seeds/query_string/range_op
@@ -0,0 +1 @@
+price:>=10 OR count:<5
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_string/regression-207-nbsp-continuation-byte b/engine/fuzz/seeds/query_string/regression-207-nbsp-continuation-byte
new file mode 100644
index 000000000..b89cc2877
Binary files /dev/null and b/engine/fuzz/seeds/query_string/regression-207-nbsp-continuation-byte differ
diff --git a/engine/fuzz/seeds/query_string/simple b/engine/fuzz/seeds/query_string/simple
new file mode 100644
index 000000000..1c12a544b
--- /dev/null
+++ b/engine/fuzz/seeds/query_string/simple
@@ -0,0 +1 @@
+quick brown fox
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_string/unbalanced b/engine/fuzz/seeds/query_string/unbalanced
new file mode 100644
index 000000000..b126e131f
--- /dev/null
+++ b/engine/fuzz/seeds/query_string/unbalanced
@@ -0,0 +1 @@
+((((a
\ No newline at end of file
diff --git a/engine/fuzz/seeds/query_string/wildcard b/engine/fuzz/seeds/query_string/wildcard
new file mode 100644
index 000000000..3f43f474a
--- /dev/null
+++ b/engine/fuzz/seeds/query_string/wildcard
@@ -0,0 +1 @@
+ti?le:qu*ck AND body:/reg.*ex/
\ No newline at end of file
diff --git a/engine/fuzz/seeds/sql/in_between b/engine/fuzz/seeds/sql/in_between
new file mode 100644
index 000000000..1a42e8c36
--- /dev/null
+++ b/engine/fuzz/seeds/sql/in_between
@@ -0,0 +1 @@
+SELECT * FROM idx WHERE code IN (1,2,3) AND price BETWEEN 5 AND 10
\ No newline at end of file
diff --git a/engine/fuzz/seeds/sql/like b/engine/fuzz/seeds/sql/like
new file mode 100644
index 000000000..8535ab473
--- /dev/null
+++ b/engine/fuzz/seeds/sql/like
@@ -0,0 +1 @@
+SELECT * FROM idx WHERE msg LIKE '%timeout%' OR msg IS NOT NULL
\ No newline at end of file
diff --git a/engine/fuzz/seeds/sql/nested_parens b/engine/fuzz/seeds/sql/nested_parens
new file mode 100644
index 000000000..5d234210a
--- /dev/null
+++ b/engine/fuzz/seeds/sql/nested_parens
@@ -0,0 +1 @@
+SELECT * FROM t WHERE ((((a = 1))))
\ No newline at end of file
diff --git a/engine/fuzz/seeds/sql/order_limit b/engine/fuzz/seeds/sql/order_limit
new file mode 100644
index 000000000..b2685061d
--- /dev/null
+++ b/engine/fuzz/seeds/sql/order_limit
@@ -0,0 +1 @@
+SELECT ts, msg FROM logs WHERE ts > 100 ORDER BY ts DESC LIMIT 50
\ No newline at end of file
diff --git a/engine/fuzz/seeds/sql/select_star b/engine/fuzz/seeds/sql/select_star
new file mode 100644
index 000000000..ef460fd1c
--- /dev/null
+++ b/engine/fuzz/seeds/sql/select_star
@@ -0,0 +1 @@
+SELECT * FROM logs
\ No newline at end of file
diff --git a/engine/fuzz/seeds/sql/truncated b/engine/fuzz/seeds/sql/truncated
new file mode 100644
index 000000000..755e95aac
--- /dev/null
+++ b/engine/fuzz/seeds/sql/truncated
@@ -0,0 +1 @@
+SELECT a FROM
\ No newline at end of file
diff --git a/engine/fuzz/seeds/sql/where b/engine/fuzz/seeds/sql/where
new file mode 100644
index 000000000..121767473
--- /dev/null
+++ b/engine/fuzz/seeds/sql/where
@@ -0,0 +1 @@
+SELECT a, b FROM logs WHERE status = 200 AND level = 'error'
\ No newline at end of file
diff --git a/engine/reports/FEATURE_FAIRNESS_REVIEW_v0.6.0_2026-04-25.md b/engine/reports/FEATURE_FAIRNESS_REVIEW_v0.6.0_2026-04-25.md
index 13d1581d3..ac8a910f6 100644
--- a/engine/reports/FEATURE_FAIRNESS_REVIEW_v0.6.0_2026-04-25.md
+++ b/engine/reports/FEATURE_FAIRNESS_REVIEW_v0.6.0_2026-04-25.md
@@ -172,6 +172,8 @@ The painless-execute endpoint hardening that landed in v0.6.0 (input limits, no
- Graceful SIGTERM shutdown (fixed in v0.5.9)
- 187 ES API endpoints exposed
- Single static binary, single config file (TOML), 38 knobs vs ES 3 000+
+ [CORRECTION 2026-08-09 (#207): "38" was never measured. The counted figure is
+ 103 settings, all optional — still ~29x fewer than ES.]
**Missing:**
- Distributed tracing (OTLP log ingest works; **xerj does not emit spans**)
diff --git a/engine/tests/battle_sla.sh b/engine/tests/battle_sla.sh
index 78a3e725e..8b2ca29b4 100755
--- a/engine/tests/battle_sla.sh
+++ b/engine/tests/battle_sla.sh
@@ -624,11 +624,17 @@ log "Scenario: count user-facing config settings in xerj.default.toml"
log "ES comparison: elasticsearch.yml has 3000+ documented properties"
log ""
+# The shipped file sets the commonly tuned subset; the engine's full surface is
+# 103 settings, every one optional — measured by `count_user_facing_settings`
+# (xerj-common/src/config.rs), which counts leaf keys of a serialised
+# `Config::default()`. This bound is on the FILE, and it is deliberately not the
+# published headline: quoting a number nobody counted is what issue #207 was
+# filed about.
USER_SETTINGS=$(grep -v "^#\|^$\|^\[" "$DEFAULT_TOML" | grep "=" | wc -l || echo 0)
-info "Total key=value lines in xerj.default.toml: $USER_SETTINGS"
-[ "$USER_SETTINGS" -le 50 ] \
- && pass "Config has $USER_SETTINGS settings (Elasticsearch has 3000+)" \
- || fail "Config has $USER_SETTINGS settings (expected ≤ 50)"
+info "Key=value lines in xerj.default.toml: $USER_SETTINGS (of 103 settings in total)"
+[ "$USER_SETTINGS" -le 60 ] \
+ && pass "The shipped config sets $USER_SETTINGS keys; XERJ has 103 settings in total, all optional (Elasticsearch has 3000+)" \
+ || fail "The shipped config sets $USER_SETTINGS keys (expected ≤ 60)"
# Verify every setting has a comment above it
COMMENT_RATIO=$(grep -c "^#" "$DEFAULT_TOML" || echo 0)
diff --git a/engine/xerj.default.toml b/engine/xerj.default.toml
index 952ce0b1f..9e47527ca 100644
--- a/engine/xerj.default.toml
+++ b/engine/xerj.default.toml
@@ -1,8 +1,11 @@
# xerj Default Configuration
# ============================================================
-# This file documents all 38 user-facing settings with their
-# production-ready defaults. Uncomment and edit any section
-# you want to override.
+# This file sets 53 of XERJ's 103 user-facing settings to their
+# production-ready defaults, and describes the rest in comments.
+# Every value here IS the default, so copying this file changes
+# nothing until you edit it (a test enforces that). The complete
+# set — every one of them optional — is documented in
+# crates/xerj-common/src/config.rs.
#
# To use: copy this file to xerj.toml and start the server:
# xerj --config /path/to/xerj.toml
@@ -50,7 +53,7 @@ bind_address = "0.0.0.0"
# trusted_proxies = ["10.0.0.7", "10.1.0.0/16", "::1"]
trusted_proxies = []
-# ── Authentication (2 settings) ───────────────────────────────────────────────
+# ── Authentication (3 settings) ───────────────────────────────────────────────
[auth]
@@ -107,7 +110,7 @@ key_path = ""
# opt-out. Ignored entirely when enabled = false.
allow_insecure_grpc_h2c = false
-# ── Storage & WAL (5 settings) ────────────────────────────────────────────────
+# ── Storage & WAL (5 of 10 settings) ──────────────────────────────────────────
[storage]
@@ -135,48 +138,67 @@ wal_batch_ms = 100
# After a rollover, the old WAL file is retained until a flush checkpoint
# advances past it. Larger values reduce rollover overhead at the cost of
# longer recovery times after a crash.
-wal_max_size_mb = 512
+wal_max_size_mb = 1024
# In-memory buffer size that triggers a segment flush to disk (MiB).
# When the memtable for an index reaches this size, it is flushed to a
# durable on-disk segment. Larger values improve write throughput but
# increase recovery time and memory usage.
-flush_size_mb = 256
+flush_size_mb = 512
# Maximum time between flushes, regardless of buffer size (seconds).
# Ensures data is durable even during low-write periods.
flush_interval_secs = 30
-# ── Segment Merging (5 settings) ──────────────────────────────────────────────
+# ── Segment Merging (8 settings) ──────────────────────────────────────────────
+#
+# Three of these are DORMANT: accepted and validated, but no code path in this
+# build reads them. Setting one away from its default logs a warning at startup.
+# They are marked below. The merge that runs reads max_segment_mb, tier_floor_mb,
+# min_merge_count and max_merge_count, and nothing else.
[merge]
# Merge strategy for combining small segments into larger ones.
#
-# "size_tiered" — merges segments of similar size. Best for write-heavy
-# workloads with infrequent reads.
-# "log_structured" — LSMT-style levelled merging. Better read performance
-# for mixed read/write workloads.
+# "size_tiered" — merges segments of similar size. Best for write-heavy
+# workloads with infrequent reads.
+#
+# Only size_tiered is implemented. "log_structured" is refused at startup rather
+# than silently running size-tiered merging.
strategy = "size_tiered"
+# DORMANT — nothing reads this. The merge trigger is per-tier and comes from
+# min_merge_count below.
# Minimum number of segments that must exist before a merge is triggered.
-# Higher values delay merging (more write throughput, slower reads).
# Must be >= 2. Default: 10.
min_segments = 10
-# Maximum size of a merged segment (MiB). Segments larger than this are
-# never merged. 5120 MiB = 5 GiB.
-max_segment_mb = 5120
+# Maximum size of a merged segment (MiB). Segments at or above this size are
+# never merged again. 8192 MiB = 8 GiB.
+max_segment_mb = 8192
+# DORMANT — merge I/O is NOT throttled in this build. The rate limiter that
+# honours this value is wired only into an unused storage-crate merge executor,
+# so setting it buys nothing today.
# I/O rate cap for merge operations (MiB/s).
-# Throttle merges to prevent them from saturating disk I/O on shared storage.
-# Set to 0 to disable throttling (not recommended in production).
io_rate_mb_per_sec = 100
+# DORMANT — nothing reads this. Merge parallelism comes from the
+# XERJ_MERGE_PARALLELISM environment variable, which also defaults to 1.
# Maximum number of concurrent merge operations.
-# Increase on systems with fast NVMe arrays. Lower on spinning disks.
max_concurrent = 1
+# Tier boundary base for the size-tiered policy (MiB). Segments group into tiers
+# by floor(log2(size / tier_floor_mb)).
+tier_floor_mb = 4
+
+# Minimum segments in one tier before that tier is merged.
+min_merge_count = 4
+
+# Maximum segments merged per batch. Caps peak merge RAM.
+max_merge_count = 16
+
# ── Compression (3 settings) ──────────────────────────────────────────────────
[compression]
@@ -216,7 +238,7 @@ block_size_docs = 128
# creation time via the mappings API.
default_analyzer = "standard"
-# ── Vector Search / HNSW (6 settings) ────────────────────────────────────────
+# ── Vector Search / HNSW (6 settings) ─────────────────────────────────────────
[vector]
@@ -255,7 +277,7 @@ hnsw_ef_search = 100
# vectors while claiming a 32× saving. 4-bit ("scalar4") quantization and
# automatic quantization downgrade are not implemented either and have no
# settings here.
-default_quantization = "scalar8"
+default_quantization = "none"
# Maximum supported vector dimensionality.
# Vectors with more dimensions than this are rejected at ingest time.
@@ -280,7 +302,7 @@ retention_days = 90
# Use "1d" for lower-volume application logs.
time_partition = "1h"
-# ── External Embedding Service (4 settings) ────────────────────────────────────
+# ── External Embedding Service (4 of 19 settings) ─────────────────────────────
#
# NOTE: Token limits are model-specific, not xerj-specific.
#
@@ -317,7 +339,7 @@ batch_size = 64
# request fails with a timeout error.
timeout_ms = 5000
-# ── Resource Limits (11 settings; omitted keys use documented defaults) ───────
+# ── Resource Limits (6 of 13 settings) ────────────────────────────────────────
[limits]
@@ -383,7 +405,7 @@ turbo_parallel = true
# (e.g. "running" will not match "run").
turbo_fast_analyzer = false
-# ── Cluster (5 settings) ──────────────────────────────────────────────────────
+# ── Cluster (5 settings, all commented out below) ─────────────────────────────
#
# Multi-node cluster mode is OFF by default and is experimental. The whole
# block is commented out; uncomment it only if you are evaluating clustering.
diff --git a/landing/docs/config.html b/landing/docs/config.html
index d6e9b7363..ac05248d9 100644
--- a/landing/docs/config.html
+++ b/landing/docs/config.html
@@ -124,7 +124,7 @@
<p>Paths for the config are, in precedence order: <code class="inline">--config /path/to/xerj.toml</code> on the command line, then <code class="inline">/etc/xerj/xerj.toml</code>, then <code class="inline">./xerj.toml</code> in the working directory. If none exist, the full default is used.</p>
<h2>Full key table</h2>
- <div class="cfg"><div class="head">KEY</div><div class="head">TYPE</div><div class="head">DEFAULT</div><div class="head">DESCRIPTION</div><div class="group" id="server">[server]</div><div class="cell k">rest_port</div><div class="cell t">u16</div><div class="cell d">8080</div><div class="cell dsc">Native REST API listener port. All native endpoints (/v1/*) live here.</div><div class="cell k">grpc_port</div><div class="cell t">u16</div><div class="cell d">8081</div><div class="cell dsc">Reserved for a future gRPC API. Not wired in v0.1 — leave at default.</div><div class="cell k">es_compat_port</div><div class="cell t">u16</div><div class="cell d">9200</div><div class="cell dsc">Elasticsearch-compatible wire port. Point Kibana, Logstash, or the ES client here. Set to 0 to disable.</div><div class="cell k">bind_address</div><div class="cell t">string</div><div class="cell d">"0.0.0.0"</div><div class="cell dsc">Interface to bind every listener to. Use "127.0.0.1" for dev or a private address in production.</div><div class="cell k">data_dir</div><div class="cell t">path</div><div class="cell d">"./data"</div><div class="cell dsc">Root for indices, WAL files, and segments. Needs fast I/O and enough free space.</div><div class="group" id="auth">[auth]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">true</div><div class="cell dsc">Require an API key on every request. Set to false only on a trusted network.</div><div class="cell k">admin_api_key</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Static admin key. Left empty, a 256-bit key is generated on first run and written to <data_dir>/admin.key.</div><div class="group" id="tls">[tls]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">false</div><div class="cell dsc">Terminate TLS at the server. Requires cert_path + key_path. If you terminate at a proxy, leave false.</div><div class="cell k">cert_path</div><div class="cell t">path</div><div class="cell d">""</div><div class="cell dsc">PEM X.509 certificate. Use a CA-signed cert in production.</div><div class="cell k">key_path</div><div class="cell t">path</div><div class="cell d">""</div><div class="cell dsc">PEM private key. Permissions should be 0600.</div><div class="group" id="storage">[storage]</div><div class="cell k">wal_sync</div><div class="cell t">enum</div><div class="cell d">"batched"</div><div class="cell dsc">"sync" · "batched" · "async". Durability vs throughput. Batched is the recommended default.</div><div class="cell k">wal_batch_ms</div><div class="cell t">u32</div><div class="cell d">100</div><div class="cell dsc">Fsync cadence when wal_sync="batched" (ms). Range 1–10000. Lower = smaller loss window.</div><div class="cell k">wal_max_size_mb</div><div class="cell t">u32</div><div class="cell d">512</div><div class="cell dsc">WAL rollover threshold (MiB). Larger means fewer rollovers, longer crash recovery.</div><div class="cell k">flush_size_mb</div><div class="cell t">u32</div><div class="cell d">256</div><div class="cell dsc">Memtable buffer size that triggers a segment flush (MiB).</div><div class="cell k">flush_interval_secs</div><div class="cell t">u32</div><div class="cell d">30</div><div class="cell dsc">Maximum wall-clock interval between flushes even if the buffer is not full.</div><div class="group" id="merge">[merge]</div><div class="cell k">strategy</div><div class="cell t">enum</div><div class="cell d">"size_tiered"</div><div class="cell dsc">"size_tiered" or "log_structured". Size-tiered for write-heavy, log-structured for mixed workloads.</div><div class="cell k">min_segments</div><div class="cell t">u32</div><div class="cell d">10</div><div class="cell dsc">Minimum live segments before a merge pass is scheduled. Must be ≥ 2.</div><div class="cell k">max_segment_mb</div><div class="cell t">u32</div><div class="cell d">5120</div><div class="cell dsc">Upper bound on a mergeable segment (MiB). Segments larger than this are never merged.</div><div class="cell k">io_rate_mb_per_sec</div><div class="cell t">u32</div><div class="cell d">100</div><div class="cell dsc">Merge I/O throttle (MiB/s). Set 0 to disable — not recommended in production.</div><div class="cell k">max_concurrent</div><div class="cell t">u8</div><div class="cell d">1</div><div class="cell dsc">Concurrent merge workers per index. Bump to 2–4 on fast NVMe.</div><div class="group" id="compression">[compression]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">true</div><div class="cell dsc">Block-level compression. Disabling increases disk usage and lowers read CPU.</div><div class="cell k">level</div><div class="cell t">enum</div><div class="cell d">"balanced"</div><div class="cell dsc">"fast" (LZ4), "balanced" (Zstd L3, default), "best" (Zstd L19, cold storage).</div><div class="cell k">block_size_docs</div><div class="cell t">u32</div><div class="cell d">128</div><div class="cell dsc">Docs per compressed block. Range 16–4096. Larger = better ratio, higher fetch cost.</div><div class="group" id="fts">[fts]</div><div class="cell k">default_analyzer</div><div class="cell t">enum</div><div class="cell d">"standard"</div><div class="cell dsc">"standard" · "whitespace" · "simple" · "english". Override per-field in the mapping.</div><div class="group" id="vector">[vector]</div><div class="cell k">default_metric</div><div class="cell t">enum</div><div class="cell d">"cosine"</div><div class="cell dsc">"cosine" · "dot_product" · "euclidean". Cosine for text embeddings is the usual choice.</div><div class="cell k">hnsw_m</div><div class="cell t">u32</div><div class="cell d">16</div><div class="cell dsc">Accepted and validated; the built-in HNSW graph currently builds with fixed M=16 — this key is not yet wired to the build.</div><div class="cell k">hnsw_ef_construction</div><div class="cell t">u32</div><div class="cell d">200</div><div class="cell dsc">Accepted; must be ≥ hnsw_m. The graph currently builds with fixed ef_construction=200 — this key is not yet wired to the build.</div><div class="cell k">hnsw_ef_search</div><div class="cell t">u32</div><div class="cell d">100</div><div class="cell dsc">Accepted but not read by the serving path — the ANN beam width comes from the request's num_candidates (floored at 800).</div><div class="cell k">default_quantization</div><div class="cell t">enum</div><div class="cell d">"scalar8"</div><div class="cell dsc">"none" · "scalar8" · "scalar4" · "binary". scalar8 = 4× RAM saving, 1–2% recall loss.</div><div class="cell k">max_dimensions</div><div class="cell t">u32</div><div class="cell d">16384</div><div class="cell dsc">Upper bound on vector dimensionality. 4× the Elasticsearch limit of 4096.</div><div class="group" id="logs">[logs]</div><div class="cell k">retention_days</div><div class="cell t">u32</div><div class="cell d">90</div><div class="cell dsc">Auto-delete log docs older than N days. 0 disables.</div><div class="cell k">time_partition</div><div class="cell t">enum</div><div class="cell d">"1h"</div><div class="cell dsc">"1m" · "5m" · "15m" · "1h" · "6h" · "1d". Time-slice granularity for retention pruning.</div><div class="group" id="embedding">[embedding]</div><div class="cell k">default_endpoint</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">OpenAI-compatible embeddings URL. Empty disables auto-embedding on ingest. Example: "https://api.openai.com/v1/embeddings".</div><div class="cell k">default_model</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Model name passed to the endpoint. Example: "text-embedding-3-small" or "nomic-embed-text".</div><div class="cell k">batch_size</div><div class="cell t">u32</div><div class="cell d">64</div><div class="cell dsc">Docs per embedding API call. Range 1–2048. Bigger batches amortise round-trip cost.</div><div class="cell k">timeout_ms</div><div class="cell t">u32</div><div class="cell d">5000</div><div class="cell dsc">HTTP timeout for embedding calls (ms). Ingest fails with a timeout error if exceeded.</div><div class="group" id="limits">[limits]</div><div class="cell k">max_query_memory_mb</div><div class="cell t">u32</div><div class="cell d">512</div><div class="cell dsc">Per-query memory cap (MiB). Queries that exceed it are cancelled.</div><div class="cell k">max_concurrent_searches</div><div class="cell t">u32</div><div class="cell d">64</div><div class="cell dsc">Global in-flight search ceiling. Extras are queued.</div><div class="cell k">max_fields_per_index</div><div class="cell t">u32</div><div class="cell d">500</div><div class="cell dsc">Field-explosion protection. ES default is 1000; 500 is intentionally stricter.</div><div class="group" id="indexing">[indexing]</div><div class="cell k">turbo_batch_size</div><div class="cell t">u32</div><div class="cell d">1000</div><div class="cell dsc">Docs per batch in turbo mode. Range 500–5000. Larger = higher throughput, slightly higher latency.</div><div class="cell k">turbo_parallel</div><div class="cell t">bool</div><div class="cell d">true</div><div class="cell dsc">Parallel tokenisation on Rayon threads. Disable only for debugging.</div><div class="cell k">turbo_fast_analyzer</div><div class="cell t">bool</div><div class="cell d">false</div><div class="cell dsc">Skip stemming and stop-word removal in turbo mode. Trades recall for speed.</div><div class="group" id="cluster">[cluster]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">false</div><div class="cell dsc">Enable multi-node cluster mode. When true, the Raft state machine and cluster transport are started on port.</div><div class="cell k">port</div><div class="cell t">u16</div><div class="cell d">9300</div><div class="cell dsc">TCP port for intra-cluster Raft and search messages. Must be reachable from every peer.</div><div class="cell k">peers</div><div class="cell t">array</div><div class="cell d">[]</div><div class="cell dsc">Peer list in "node_id=host:port" format. The local node identifies itself from the entry matching bind_address:port. Example: ["a=10.0.0.1:9300","b=10.0.0.2:9300","c=10.0.0.3:9300"].</div><div class="cell k">tick_ms</div><div class="cell t">u64</div><div class="cell d">50</div><div class="cell dsc">Raft tick interval (ms). Lower = faster leader election at the cost of CPU.</div><div class="cell k">auth_secret</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Cluster-wide shared secret authenticating every control frame on port (HMAC-SHA256 over a per-connection challenge). Mandatory when enabled = true — with cluster mode on and no secret here or in XERJ_CLUSTER_AUTH_SECRET the node refuses to start. Same value on every node; minimum 16 chars; generate with `openssl rand -hex 32`. Authenticates, does not encrypt.</div></div>
+ <div class="cfg"><div class="head">KEY</div><div class="head">TYPE</div><div class="head">DEFAULT</div><div class="head">DESCRIPTION</div><div class="group" id="server">[server]</div><div class="cell k">rest_port</div><div class="cell t">u16</div><div class="cell d">8080</div><div class="cell dsc">Native REST API listener port. All native endpoints (/v1/*) live here.</div><div class="cell k">grpc_port</div><div class="cell t">u16</div><div class="cell d">8081</div><div class="cell dsc">Reserved for a future gRPC API. Not wired in v0.1 — leave at default.</div><div class="cell k">es_compat_port</div><div class="cell t">u16</div><div class="cell d">9200</div><div class="cell dsc">Elasticsearch-compatible wire port. Point Kibana, Logstash, or the ES client here. Set to 0 to disable.</div><div class="cell k">bind_address</div><div class="cell t">string</div><div class="cell d">"0.0.0.0"</div><div class="cell dsc">Interface to bind every listener to. Use "127.0.0.1" for dev or a private address in production.</div><div class="cell k">data_dir</div><div class="cell t">path</div><div class="cell d">"./data"</div><div class="cell dsc">Root for indices, WAL files, and segments. Needs fast I/O and enough free space.</div><div class="group" id="auth">[auth]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">true</div><div class="cell dsc">Require an API key on every request. Set to false only on a trusted network.</div><div class="cell k">admin_api_key</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Static admin key. Left empty, a 256-bit key is generated on first run and written to <data_dir>/admin.key.</div><div class="group" id="tls">[tls]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">false</div><div class="cell dsc">Terminate TLS at the server. Requires cert_path + key_path. If you terminate at a proxy, leave false.</div><div class="cell k">cert_path</div><div class="cell t">path</div><div class="cell d">""</div><div class="cell dsc">PEM X.509 certificate. Use a CA-signed cert in production.</div><div class="cell k">key_path</div><div class="cell t">path</div><div class="cell d">""</div><div class="cell dsc">PEM private key. Permissions should be 0600.</div><div class="group" id="storage">[storage]</div><div class="cell k">wal_sync</div><div class="cell t">enum</div><div class="cell d">"batched"</div><div class="cell dsc">"sync" · "batched" · "async". Durability vs throughput. Batched is the recommended default.</div><div class="cell k">wal_batch_ms</div><div class="cell t">u32</div><div class="cell d">100</div><div class="cell dsc">Fsync cadence when wal_sync="batched" (ms). Range 1–10000. Lower = smaller loss window.</div><div class="cell k">wal_max_size_mb</div><div class="cell t">u32</div><div class="cell d">1024</div><div class="cell dsc">WAL rollover threshold (MiB). Larger means fewer rollovers, longer crash recovery.</div><div class="cell k">flush_size_mb</div><div class="cell t">u32</div><div class="cell d">512</div><div class="cell dsc">Memtable buffer size that triggers a segment flush (MiB).</div><div class="cell k">flush_interval_secs</div><div class="cell t">u32</div><div class="cell d">30</div><div class="cell dsc">Maximum wall-clock interval between flushes even if the buffer is not full.</div><div class="group" id="merge">[merge]</div><div class="cell k">strategy</div><div class="cell t">enum</div><div class="cell d">"size_tiered"</div><div class="cell dsc">Only "size_tiered" is implemented; "log_structured" is refused at startup rather than silently running the other policy.</div><div class="cell k">min_segments</div><div class="cell t">u32</div><div class="cell d">10</div><div class="cell dsc">Accepted and validated (must be ≥ 2) but <strong>not wired</strong>: the merge trigger is per-tier and comes from min_merge_count. Setting it away from the default logs a warning at startup.</div><div class="cell k">max_segment_mb</div><div class="cell t">u32</div><div class="cell d">8192</div><div class="cell dsc">Upper bound on a mergeable segment (MiB). Segments at or above this size are never merged again.</div><div class="cell k">io_rate_mb_per_sec</div><div class="cell t">u32</div><div class="cell d">100</div><div class="cell dsc">Accepted but <strong>not wired</strong>: merge I/O is not throttled in this build — the rate limiter that honours it sits in an unused storage-crate merge executor. Setting it away from the default logs a warning at startup.</div><div class="cell k">max_concurrent</div><div class="cell t">u8</div><div class="cell d">1</div><div class="cell dsc">Accepted but <strong>not wired</strong>: merge parallelism comes from the XERJ_MERGE_PARALLELISM environment variable, which also defaults to 1. Setting it away from the default logs a warning at startup.</div><div class="group" id="compression">[compression]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">true</div><div class="cell dsc">Block-level compression. Disabling increases disk usage and lowers read CPU.</div><div class="cell k">level</div><div class="cell t">enum</div><div class="cell d">"balanced"</div><div class="cell dsc">"fast" (LZ4), "balanced" (Zstd L3, default), "best" (Zstd L19, cold storage).</div><div class="cell k">block_size_docs</div><div class="cell t">u32</div><div class="cell d">128</div><div class="cell dsc">Docs per compressed block. Range 16–4096. Larger = better ratio, higher fetch cost.</div><div class="group" id="fts">[fts]</div><div class="cell k">default_analyzer</div><div class="cell t">enum</div><div class="cell d">"standard"</div><div class="cell dsc">"standard" · "whitespace" · "simple" · "english". Override per-field in the mapping.</div><div class="group" id="vector">[vector]</div><div class="cell k">default_metric</div><div class="cell t">enum</div><div class="cell d">"cosine"</div><div class="cell dsc">"cosine" · "dot_product" · "euclidean". Cosine for text embeddings is the usual choice.</div><div class="cell k">hnsw_m</div><div class="cell t">u32</div><div class="cell d">16</div><div class="cell dsc">Accepted and validated; the built-in HNSW graph currently builds with fixed M=16 — this key is not yet wired to the build.</div><div class="cell k">hnsw_ef_construction</div><div class="cell t">u32</div><div class="cell d">200</div><div class="cell dsc">Accepted; must be ≥ hnsw_m. The graph currently builds with fixed ef_construction=200 — this key is not yet wired to the build.</div><div class="cell k">hnsw_ef_search</div><div class="cell t">u32</div><div class="cell d">100</div><div class="cell dsc">Accepted but not read by the serving path — the ANN beam width comes from the request's num_candidates (floored at 800).</div><div class="cell k">default_quantization</div><div class="cell t">enum</div><div class="cell d">"none"</div><div class="cell dsc">"none" (the default) or "scalar8" = 4× RAM saving, 1–2% recall loss. "scalar4" and "binary" have no quantizer behind them and are refused at startup rather than silently stored at full precision.</div><div class="cell k">max_dimensions</div><div class="cell t">u32</div><div class="cell d">16384</div><div class="cell dsc">Upper bound on vector dimensionality. 4× the Elasticsearch limit of 4096.</div><div class="group" id="logs">[logs]</div><div class="cell k">retention_days</div><div class="cell t">u32</div><div class="cell d">90</div><div class="cell dsc">Auto-delete log docs older than N days. 0 disables.</div><div class="cell k">time_partition</div><div class="cell t">enum</div><div class="cell d">"1h"</div><div class="cell dsc">"1m" · "5m" · "15m" · "1h" · "6h" · "1d". Time-slice granularity for retention pruning.</div><div class="group" id="embedding">[embedding]</div><div class="cell k">default_endpoint</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">OpenAI-compatible embeddings URL. Empty disables auto-embedding on ingest. Example: "https://api.openai.com/v1/embeddings".</div><div class="cell k">default_model</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Model name passed to the endpoint. Example: "text-embedding-3-small" or "nomic-embed-text".</div><div class="cell k">batch_size</div><div class="cell t">u32</div><div class="cell d">64</div><div class="cell dsc">Docs per embedding API call. Range 1–2048. Bigger batches amortise round-trip cost.</div><div class="cell k">timeout_ms</div><div class="cell t">u32</div><div class="cell d">5000</div><div class="cell dsc">HTTP timeout for embedding calls (ms). Ingest fails with a timeout error if exceeded.</div><div class="group" id="limits">[limits]</div><div class="cell k">max_query_memory_mb</div><div class="cell t">u32</div><div class="cell d">512</div><div class="cell dsc">Per-query memory cap (MiB). Queries that exceed it are cancelled.</div><div class="cell k">max_concurrent_searches</div><div class="cell t">u32</div><div class="cell d">64</div><div class="cell dsc">Global in-flight search ceiling. Extras are queued.</div><div class="cell k">max_fields_per_index</div><div class="cell t">u32</div><div class="cell d">500</div><div class="cell dsc">Field-explosion protection. ES default is 1000; 500 is intentionally stricter.</div><div class="group" id="indexing">[indexing]</div><div class="cell k">turbo_batch_size</div><div class="cell t">u32</div><div class="cell d">1000</div><div class="cell dsc">Docs per batch in turbo mode. Range 500–5000. Larger = higher throughput, slightly higher latency.</div><div class="cell k">turbo_parallel</div><div class="cell t">bool</div><div class="cell d">true</div><div class="cell dsc">Parallel tokenisation on Rayon threads. Disable only for debugging.</div><div class="cell k">turbo_fast_analyzer</div><div class="cell t">bool</div><div class="cell d">false</div><div class="cell dsc">Skip stemming and stop-word removal in turbo mode. Trades recall for speed.</div><div class="group" id="cluster">[cluster]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">false</div><div class="cell dsc">Enable multi-node cluster mode. When true, the Raft state machine and cluster transport are started on port.</div><div class="cell k">port</div><div class="cell t">u16</div><div class="cell d">9300</div><div class="cell dsc">TCP port for intra-cluster Raft and search messages. Must be reachable from every peer.</div><div class="cell k">peers</div><div class="cell t">array</div><div class="cell d">[]</div><div class="cell dsc">Peer list in "node_id=host:port" format. The local node identifies itself from the entry matching bind_address:port. Example: ["a=10.0.0.1:9300","b=10.0.0.2:9300","c=10.0.0.3:9300"].</div><div class="cell k">tick_ms</div><div class="cell t">u64</div><div class="cell d">50</div><div class="cell dsc">Raft tick interval (ms). Lower = faster leader election at the cost of CPU.</div><div class="cell k">auth_secret</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Cluster-wide shared secret authenticating every control frame on port (HMAC-SHA256 over a per-connection challenge). Mandatory when enabled = true — with cluster mode on and no secret here or in XERJ_CLUSTER_AUTH_SECRET the node refuses to start. Same value on every node; minimum 16 chars; generate with `openssl rand -hex 32`. Authenticates, does not encrypt.</div></div>
<h2 id="server-section">[server]</h2>
@@ -165,10 +165,13 @@ flush_interval_secs = 30</pre>
<p>Segment compaction. <code class="inline">size_tiered</code> is the right default: it merges same-size segments, which is cheap and write-optimal. Switch to <code class="inline">log_structured</code> if your reads are doing a lot of full-range scans.</p>
<pre class="code">[merge]
strategy = "size_tiered"
-min_segments = 10
-max_segment_mb = 5120 # 5 GiB cap
-io_rate_mb_per_sec = 100 # don't starve queries
-max_concurrent = 2 # bump on NVMe</pre>
+min_segments = 10 # accepted, not wired
+max_segment_mb = 8192 # 8 GiB cap
+io_rate_mb_per_sec = 100 # accepted, not wired — merges are not throttled
+max_concurrent = 1 # accepted, not wired (see XERJ_MERGE_PARALLELISM)
+tier_floor_mb = 4
+min_merge_count = 4 # the real per-tier merge trigger
+max_merge_count = 16 # caps peak merge RAM</pre>
<h2 id="compression-section">[compression]</h2>
<p>See <a href="/docs/compression.html">Compression</a> for the encoding catalog. This section picks the outer <em>block</em> codec only — the inner per-column encodings are chosen automatically at write time.</p>
diff --git a/landing/docs/operations.html b/landing/docs/operations.html
index 66677acde..44e1cf030 100644
--- a/landing/docs/operations.html
+++ b/landing/docs/operations.html
@@ -178,7 +178,7 @@ OK</pre>
<p>Most tunables pick up on <code class="inline">SIGHUP</code> without a restart. A few (ports, TLS certs, data_dir, cluster.enabled) require a full restart — those reject a reload with a clear error in the log.</p>
<pre class="code">$ sudo systemctl reload xerj # sends SIGHUP
$ journalctl -u xerj -n 10 --no-pager
-... config reloaded: merge.io_rate_mb_per_sec 100 → 250
+... config reloaded: merge.max_merge_count 16 → 24
... config reloaded: limits.max_concurrent_searches 64 → 128</pre>
<h2>Log levels</h2>
diff --git a/landing/docs/storage.html b/landing/docs/storage.html
index 5f13d7ae5..091ee2f54 100644
--- a/landing/docs/storage.html
+++ b/landing/docs/storage.html
@@ -159,7 +159,8 @@
<p>Segments are <strong>immutable once written</strong>. Updates and deletes work by writing a new segment and a tombstone; merges rewrite surviving documents into a larger segment.</p>
<h2>Merges</h2>
- <p><code class="inline">[merge] strategy</code> picks between <code class="inline">size_tiered</code> (default) and <code class="inline">log_structured</code> (LSMT-style levelled). <code class="inline">min_segments</code> sets the trigger (default 10). <code class="inline">io_rate_mb_per_sec</code> throttles the merger to leave headroom for queries (default 100 MiB/s; set 0 to disable). <code class="inline">max_concurrent</code> caps parallel merge workers (default 1 — bump to 2–4 on fast NVMe).</p>
+ <p><code class="inline">[merge] max_segment_mb</code> excludes anything that big from further merging (default 8192). <code class="inline">tier_floor_mb</code> (default 4) sets the tier boundaries, <code class="inline">min_merge_count</code> (default 4) is the per-tier trigger, and <code class="inline">max_merge_count</code> (default 16) caps how many segments one batch merges — and so caps peak merge RAM. Those four are what the merge path reads.</p>
+ <p><strong>Three keys in this section are accepted but not wired</strong> and setting them logs a warning at startup: <code class="inline">strategy</code> (only <code class="inline">size_tiered</code> exists — <code class="inline">log_structured</code> is refused rather than silently substituted), <code class="inline">min_segments</code> (superseded by <code class="inline">min_merge_count</code>), and <code class="inline">io_rate_mb_per_sec</code> — <strong>merge I/O is not throttled in this build</strong>. <code class="inline">max_concurrent</code> is likewise unread; merge parallelism comes from the <code class="inline">XERJ_MERGE_PARALLELISM</code> environment variable (default 1).</p>
<h2>Cluster metadata</h2>
<p>In single-node mode there is no <code class="inline">cluster/</code> directory — everything the engine needs is right next to the index data. When the server is started with a cluster config, a sibling <code class="inline">cluster/</code> directory holds the embedded Raft log and snapshots. Index <em>data</em> is never in Raft — only the <strong>metadata</strong> (index schemas, shard assignments, node roster). See <a href="/docs/clustering.html">Clustering</a>.</p>
diff --git a/landing/docs/troubleshooting.html b/landing/docs/troubleshooting.html
index 66240dca6..097205d1e 100644
--- a/landing/docs/troubleshooting.html
+++ b/landing/docs/troubleshooting.html
@@ -128,7 +128,7 @@
xerj_segment_count{index="logs"} 2847
xerj_merge_duration_seconds_count{index="logs"} 183
xerj_merge_duration_seconds_sum{index="logs"} 1847.2</pre>
- <p><strong>Fix:</strong> raise <code class="inline">[merge] max_concurrent</code> (1 → 2–4), raise <code class="inline">io_rate_mb_per_sec</code> (100 → 250–500 on NVMe), and raise <code class="inline">[storage] flush_size_mb</code> so flushes produce bigger starter segments.</p>
+ <p><strong>Fix:</strong> raise <code class="inline">[storage] flush_size_mb</code> so flushes produce bigger starter segments, and raise <code class="inline">[merge] max_merge_count</code> so each pass consumes more of them (watch RSS — peak merge memory scales with it). Set <code class="inline">XERJ_MERGE_PARALLELISM</code> above 1 for more concurrent merges on fast NVMe. <code class="inline">[merge] max_concurrent</code> and <code class="inline">io_rate_mb_per_sec</code> are accepted but not wired in this build, so neither will change anything here.</p>
<h2>Symptom: queries time out under load</h2>
<p><strong>Likely cause:</strong> too many concurrent queries or a single query that grew too large.</p>
diff --git a/landing/for-agents.html b/landing/for-agents.html
index 287d053a7..7be6caaa3 100644
--- a/landing/for-agents.html
+++ b/landing/for-agents.html
@@ -331,7 +331,7 @@ $ curl -sXPOST localhost:9200/docfolder/_search \
</p>
<ul class="pitch" style="max-width:760px;">
<li><a href="/llms.txt"><code class="inline">/llms.txt</code></a> — a plain-text manifest of the base URL, the seven agent operations (autoindex included), and the honesty caveats, written for an LLM to read directly.</li>
- <li><a href="/openapi.json"><code class="inline">/openapi.json</code></a> — an OpenAPI description of the endpoints, for tool/codegen pipelines.</li>
+ <li><a href="/openapi.json"><code class="inline">/openapi.json</code></a> — an OpenAPI description of the six canonical agent operations (7 routes), for tool/codegen pipelines. It is a deliberately partial spec, not the whole ~210-route REST surface: an endpoint missing from it is "not written up yet", never "not supported".</li>
</ul>
</section>
diff --git a/landing/openapi.json b/landing/openapi.json
index b52f68a01..1905c1a5c 100644
--- a/landing/openapi.json
+++ b/landing/openapi.json
@@ -1,10 +1,10 @@
{
"openapi": "3.1.0",
"info": {
- "title": "XERJ REST API (Elasticsearch-compatible)",
- "version": "1.0",
+ "title": "XERJ REST API \u2014 agent core (partial spec)",
+ "version": "0.1-partial",
"summary": "Core index, ingest, search, and agent-memory endpoints for XERJ.",
- "description": "XERJ is an Elasticsearch-compatible search engine written in Rust. This document covers the core REST surface that backs the six canonical agent operations: index creation, document ingest (single and bulk), search (lexical, semantic, vector/kNN, and hybrid), and namespaced agent memory (store + recall). The search endpoint is a single verb whose behaviour is selected by the request body: a `semantic` query for meaning-based recall, a top-level `knn` clause for vector search, or a `hybrid` query for score fusion. Semantic and semantic-memory features use XERJ's built-in embedder and require no external embedding provider or API key. Unfiltered vector (kNN) search is served by a persisted HNSW graph (approximate) with exact rescoring of candidates — measured recall@10 1.00 on the official bench query, 100-probe mean 0.976; num_candidates sets the beam width (floored at 800). Filtered kNN and other ineligible shapes (non-cosine, SQ8, indexes under 1,024 docs) run an exact brute-force scan (recall 1.00).",
+ "description": "XERJ is an Elasticsearch-compatible search engine written in Rust. This document covers the core REST surface that backs the six canonical agent operations: index creation, document ingest (single and bulk), search (lexical, semantic, vector/kNN, and hybrid), and namespaced agent memory (store + recall). The search endpoint is a single verb whose behaviour is selected by the request body: a `semantic` query for meaning-based recall, a top-level `knn` clause for vector search, or a `hybrid` query for score fusion. Semantic and semantic-memory features use XERJ's built-in embedder and require no external embedding provider or API key. Unfiltered vector (kNN) search is served by a persisted HNSW graph (approximate) with exact rescoring of candidates — measured recall@10 1.00 on the official bench query, 100-probe mean 0.976; num_candidates sets the beam width (floored at 800). Filtered kNN and other ineligible shapes (non-cosine, SQ8, indexes under 1,024 docs) run an exact brute-force scan (recall 1.00). SCOPE: this document is deliberately partial. It specifies the 7 routes behind those six operations, not XERJ's REST surface, which is roughly 210 routes wide. The version above describes this document, not the engine \u2014 read an endpoint's absence here as \"not written up yet\", never as \"not supported\". The complete wire contract is the one the ES-compat YAML conformance suite exercises (engine/tests/es-compat-yaml).",
"contact": { "name": "XERJ", "url": "https://xerj.org" },
"license": { "name": "Apache-2.0", "identifier": "Apache-2.0" }
},
diff --git a/user-feedback/01-operational-complexity/cluster-management.md b/user-feedback/01-operational-complexity/cluster-management.md
index 41cbf2d73..41804b8a4 100644
--- a/user-feedback/01-operational-complexity/cluster-management.md
+++ b/user-feedback/01-operational-complexity/cluster-management.md
@@ -60,7 +60,7 @@ The #1 complaint across all review platforms.
- Single-node architecture in M1 eliminates all cluster coordination
- No master election, no shard allocation, no cluster state broadcast
- Single binary deployment: `./xerj --config config.toml`
-- < 50 configuration settings (ES has 3,000+)
+- 103 configuration settings, every one optional (ES has 3,000+)
- No dedicated specialists needed for basic operation
## Sources
diff --git a/user-feedback/01-operational-complexity/community-horror-stories.md b/user-feedback/01-operational-complexity/community-horror-stories.md
index a69f8886e..30c00b2da 100644
--- a/user-feedback/01-operational-complexity/community-horror-stories.md
+++ b/user-feedback/01-operational-complexity/community-horror-stories.md
@@ -79,7 +79,7 @@ A team spent 14 hours manually relocating shards, only to discover the real issu
Every one of these complaints traces to one or more of:
1. JVM/GC → Rust eliminates entirely
2. Cluster coordination → single-node M1 eliminates entirely
-3. Configuration sprawl → <50 settings with sensible defaults
+3. Configuration sprawl → 103 settings, all optional, with sensible defaults
4. Shard management → no shards, automatic segment management
5. Upgrade complexity → replace binary + restart
diff --git a/user-feedback/01-operational-complexity/learning-curve.md b/user-feedback/01-operational-complexity/learning-curve.md
index 8fa374b7a..4d07b7ca1 100644
--- a/user-feedback/01-operational-complexity/learning-curve.md
+++ b/user-feedback/01-operational-complexity/learning-curve.md
@@ -48,7 +48,7 @@ Appears in nearly every negative review across all platforms.
- Minimal API surface: 10 REST endpoints, not hundreds
- Native REST API with intuitive JSON (not deeply nested DSL)
- ES-compat layer on port 9200 for teams already fluent in ES DSL
-- < 50 config settings with sensible defaults
+- 103 config settings, every one optional, all with sensible defaults
- Single TOML config file (not elasticsearch.yml + jvm.options + log4j2.properties)
- No shard/replica/JVM concepts to learn for M1
diff --git a/user-feedback/09-security/cves-and-vulnerabilities.md b/user-feedback/09-security/cves-and-vulnerabilities.md
index 94d86d76b..69e6ff5d2 100644
--- a/user-feedback/09-security/cves-and-vulnerabilities.md
+++ b/user-feedback/09-security/cves-and-vulnerabilities.md
@@ -64,8 +64,19 @@
- Rust memory safety eliminates entire classes of CVEs (buffer overflows, use-after-free)
- Minimal dependency tree: Rust crates are statically linked, auditable
- Single binary: smaller attack surface than ES + Kibana + Logstash + plugins
-- `cargo-audit` for dependency vulnerability scanning
-- Fuzz testing (cargo-fuzz) on all input parsing paths
+- `cargo-audit` runs on every push and pull request (`.github/workflows/ci.yml`,
+ job `security-audit`). A RUSTSEC **vulnerability** advisory against anything in
+ `engine/Cargo.lock` fails the build; informational advisories (unmaintained,
+ unsound, yanked) are printed in the job log and do not
+- `cargo-fuzz` harnesses live in `engine/fuzz/` and run on every push and pull
+ request (job `fuzz`, script `.github/scripts/fuzz-smoke.sh`). They cover the
+ parsers an unauthenticated request can reach: the Elasticsearch query DSL, the
+ Lucene `query_string` / `simple_query_string` grammar, date math and date-format
+ patterns, index-name date math (`<logs-{now/d}>`), SQL, and Painless scripts.
+ Each target replays a checked-in seed corpus and then fuzzes for a bounded
+ budget. **That list is the coverage** —
+ it is not every parser in the engine, and this line should be edited, not
+ broadened, when targets are added or removed
## Sources
- Elastic Security Advisory ESA-2021-31
diff --git a/user-feedback/15-clustering-durability-expectations-2025-2026/03-user-expectations-2025-2026.md b/user-feedback/15-clustering-durability-expectations-2025-2026/03-user-expectations-2025-2026.md
index 15548f137..48cee7116 100644
--- a/user-feedback/15-clustering-durability-expectations-2025-2026/03-user-expectations-2025-2026.md
+++ b/user-feedback/15-clustering-durability-expectations-2025-2026/03-user-expectations-2025-2026.md
@@ -104,7 +104,7 @@ Based on all user feedback collected:
5. **Instant write visibility** — no refresh delay
6. **AI-native** — text + vector + hybrid in one query
7. **WASM pipelines** — user-defined transforms at ingest speed
-8. **38 settings, not 3,000** — works out of the box
+8. **103 settings, not 3,000** — every one optional; works out of the box
9. **Serverless pricing** — pay per query, not per node
10. **Zero-downtime everything** — upgrades, scaling, schema changes
diff --git a/user-feedback/ROADMAP-TO-GA.md b/user-feedback/ROADMAP-TO-GA.md
index 54da8951b..aa7e7d4b8 100644
--- a/user-feedback/ROADMAP-TO-GA.md
+++ b/user-feedback/ROADMAP-TO-GA.md
@@ -111,9 +111,10 @@ destructive step (engine/crates/xerj-storage/src/index_store.rs:595-608, 772-778
**Config sprawl — "3,000+ settings" (01-operational-complexity/learning-curve.md:18-20).** An empty
or absent config file is valid; every setting has a production default; a typo'd key fails startup
(`deny_unknown_fields`); schema-present-but-unwired features are rejected loudly at startup
-(config.rs:42-47, 94-108, 198-229). The true count is 102 serde-visible settings (field-by-field
-verified) — the published "38"/"<50" figures are wrong and are blocker 10. 102 vs 3,000+ is ~29x
-fewer (PROJECTED: 3000/102), told truthfully.
+(config.rs:42-47, 94-108, 198-229). The true count is 103 serde-visible settings — measured by
+`count_user_facing_settings`, which serialises `Config::default()` and counts leaf keys, not by
+hand. The published "38"/"<50" figures were wrong; they were corrected with blocker 10
+(2026-08-09, #207). 103 vs 3,000+ is ~29x fewer (PROJECTED: 3000/103), told truthfully.
**Shard sizing expertise (04-scaling-and-shards, cluster-management.md:11).** No up-front shard
decision exists: ES `number_of_shards` is stored and echoed for wire-compat but never used as a
@@ -428,11 +429,18 @@ Files: engine/crates/xerj-api/src/es_compat.rs, engine/crates/xerj-common/src/me
migrating users' dashboards silently lose their breaker signal, and the engine's strongest anti-OOM
feature is experienced as a regression.
-### 10. Correct the settings-count claim and make the count drift-proof (small)
+### 10. Correct the settings-count claim and make the count drift-proof (small) — DONE 2026-08-09 (#207)
+
+> Landed as described: the count test now serialises `Config::default()` and counts leaf
+> keys (`journey_zero_config`), the measured total is **103**, and the 38/56/60/`<50`
+> figures are corrected across config.rs, xerj-common/src/lib.rs, engine/README.md,
+> xerj.default.toml and the feedback responses. Per-sub-config annotations are corrected
+> too (limits 3 → 13, storage 5 → 10, cluster 4 → 5, embedding 4 → 19, merge 5 → 8,
+> tls 3 → 4, auth 2 → 3). The adversarial pass counted 102; the measured figure is 103.
**Pain.** The "<50 configuration settings (ES has 3,000+)" figure is published in the shipped
feedback responses (user-feedback/01-operational-complexity/cluster-management.md:63).
-**Today.** config.rs simultaneously claims 38 settings (lines 1-5 and 274-276), 56 fields (89-92),
+**Was (before the fix).** config.rs simultaneously claimed 38 settings (lines 1-5 and 274-276), 56 fields (89-92),
and 60 (the count test, 1592-1617 — whose own comment two lines earlier says 50, and which sums
hardcoded integers referencing no struct); per-section doc comments are also stale (config.rs:49-86:
limits "3" vs actual 13, storage "5" vs 10, cluster "4" vs 5). The true count is 102 serde-visible
@@ -442,8 +450,8 @@ verifier disputed it to blocking under the repo's honest-claims rules — a publ
doing it before GA, not waiving it. This roadmap adopts the verifier's position.]
**Build.** Fix all inconsistent counts; replace the hand-written count test with introspection
(serialize `Config::default()` to `toml::Value`, count leaf keys) so the number can never drift;
-sweep public docs and the feedback responses for the stale "<50"/"38". 102 vs 3,000+ is still the
-winning story, told truthfully. Files: engine/crates/xerj-common/src/config.rs, docs, user-feedback
+sweep public docs and the feedback responses for the stale "<50"/"38". 103 vs 3,000+ is still the
+winning story, told truthfully. [DONE 2026-08-09, #207 — the measured figure is 103.] Files: engine/crates/xerj-common/src/config.rs, docs, user-feedback
response sections.
**If shipped without.** A reviewer counts the TOML keys, finds 2.7x the claimed number, and every
other XERJ number takes the credibility hit.