Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

fix(config): the settings count is 105 after #247 — move the prose with it

xerj-orgf079d73eaa39
diff --git a/CHANGELOG.md b/CHANGELOG.md
index b740efe24..288ef507c 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -126,14 +126,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
could not fail whatever `Config` held. There was a second copy of the same
non-test in `config.rs` itself (`count_user_facing_settings`, `61 == 61`).
Both now serialise `Config::default()` and count leaf keys, section by
- section. The measured figure is **104** (103 when this work was done; the
- rc.14 `server.allow_insecure_network_bind` key makes it 104), and 38 / 50 /
+ section. The measured figure is **105** (103 when this work was done; the
+ rc.14 `server.allow_insecure_network_bind` key made it 104 and #247's
+ `lifecycle.tick_interval_secs` makes it 105 — which is the point: it is
+ measured, so it moves with the code), and 38 / 50 /
56 / 60 / 61 / "<50"
are corrected in `config.rs`, `xerj-common/src/lib.rs`, `engine/README.md`,
`xerj.default.toml` and the feedback responses, along with the stale
per-sub-config annotations (limits 3 → 13, storage 5 → 10, embedding 4 → 19,
merge 5 → 8, cluster 4 → 5, tls 3 → 4, auth 2 → 3). This closes item 10 of
- `user-feedback/ROADMAP-TO-GA.md`. 104 versus 3,000+ is still the winning
+ `user-feedback/ROADMAP-TO-GA.md`. 105 versus 3,000+ is still the winning
story, told truthfully.
- *Wire-compatibility test framing.* `journey_es_migration` was described as
proving "the same curl commands" work. It calls the Rust engine API
diff --git a/demo/playbooks/USER_FEEDBACK_SCORECARD_RC4.md b/demo/playbooks/USER_FEEDBACK_SCORECARD_RC4.md
index 9a168d40e..403ef44d1 100644
--- a/demo/playbooks/USER_FEEDBACK_SCORECARD_RC4.md
+++ b/demo/playbooks/USER_FEEDBACK_SCORECARD_RC4.md
@@ -91,7 +91,7 @@ These are the strategic decisions for rc5, stated without spin.
- Dynamic-mapping `strict` + field-limit accepted but not enforced on the ingest path — mapping explosion not prevented.
- `role_descriptors` accepted then ignored — every API key is effectively superuser.
- ~~No `cargo-audit`/`cargo-deny` gate (528 unscanned dep crates) and no `cargo-fuzz` harnesses.~~
- **CLOSED 2026-08-09 (#207):** CI job `security-audit` runs `cargo audit` on every push and PR, and job `fuzz` builds and runs the libFuzzer harnesses in `engine/fuzz/` (ES DSL, `query_string`, date math, index-name date math, SQL, Painless). `cargo-deny` is still absent.
+ **CLOSED 2026-08-09 (#207):** CI job `security-audit` runs `cargo audit` on every push and PR, and job `fuzz` builds and runs the libFuzzer harnesses in `engine/fuzz/` (ES DSL, `query_string`, date math, index-name date math, SQL, Painless, and the aggregation script tokenisers). `cargo-deny` is still absent.
- Binary/RaBitQ (BBQ) quantization unimplemented; `bbq_*`/`int4_*` mappings silently keep full f32 (no memory saving) — HNSW RAM wall only 4× mitigated by SQ8, no DiskANN.
**Marketing numbers to reconcile with measured reality (honesty-posture debt):**
@@ -140,7 +140,7 @@ Ranking = severity × reach (how many pain points / clusters it closes) × inver
| Candidate | Sev | Effort | Closes |
|---|---|---|---|
| ~~**Add `cargo-audit` + `cargo-deny` advisory gate to CI**~~ — `cargo-audit` DONE 2026-08-09 (#207), `cargo-deny` still open | MED | S | security dependency-scanning GAP (528 unscanned crates) |
-| ~~**Ship `cargo-fuzz` harnesses** for ES-DSL / query_string / bulk parsers~~ — DONE 2026-08-09 (#207): `engine/fuzz/`, 6 targets, run by CI. Bulk NDJSON is not among them | MED | M | security fuzz-testing GAP (the untrusted-input surfaces) |
+| ~~**Ship `cargo-fuzz` harnesses** for ES-DSL / query_string / bulk parsers~~ — DONE 2026-08-09 (#207): `engine/fuzz/`, 7 targets, run by CI. Bulk NDJSON is not among them | MED | M | security fuzz-testing GAP (the untrusted-input surfaces) |
| **TLS on-by-default via auto self-signed cert** (or first-run prompt) | MED | M | resolves the "secure/TLS by default" overclaim across resource-cost + security |
| **Enforce API-key `role_descriptors`** (real per-index/per-field RBAC) | MED | L | security RBAC PARTIAL — every key is superuser today; turns ES's paid-gate into an XERJ win |
| **Tamper-evident (WORM/append-only) audit log** | LOW | L | security audit PARTIAL (roadmap v0.9) — ES gates real audit behind Enterprise |
diff --git a/engine/README.md b/engine/README.md
index c0fb90597..84dc397de 100644
--- a/engine/README.md
+++ b/engine/README.md
@@ -53,7 +53,7 @@ HTTP wire payloads, single-node, tmpfs data dir, both freshly started
| Per-batch (5K) bulk p99 | **34.8 ms** | 80.3 ms | ES 2.3× |
| Data loss on restart | 0% | 0% | tied |
| GC pauses | 20–40 ms (JVM) | **0** (no GC) | **xerj** |
-| Config surface | 3,000+ knobs | **104**, all optional | **xerj** |
+| Config surface | 3,000+ knobs | **105**, all optional | **xerj** |
xerj wins on every operational and read-path metric. ES still wins
bulk-ingest throughput on warm Lucene `IndexWriter` (perf-backlog item
@@ -86,7 +86,7 @@ The resulting binary is at `target/release/xerj`.
## Configuration
See [`xerj.default.toml`](xerj.default.toml) for the commonly-tuned settings, and
-[`crates/xerj-common/src/config.rs`](crates/xerj-common/src/config.rs) for all 104
+[`crates/xerj-common/src/config.rs`](crates/xerj-common/src/config.rs) for all 105
— every one of them optional.
The minimal config to get started:
diff --git a/engine/crates/xerj-common/src/config.rs b/engine/crates/xerj-common/src/config.rs
index 2155c4bd2..7ced3a626 100644
--- a/engine/crates/xerj-common/src/config.rs
+++ b/engine/crates/xerj-common/src/config.rs
@@ -1,6 +1,6 @@
//! xerj configuration system.
//!
-//! Configuration is intentionally minimal: **104 settings** versus
+//! Configuration is intentionally minimal: **105 settings** versus
//! Elasticsearch's 3000+. Every option is named, documented, and has a sensible
//! production-ready default. The format is TOML, loaded from a single file.
//!
@@ -52,7 +52,7 @@ use crate::error::XerjError;
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
#[serde(default, deny_unknown_fields)]
pub struct Config {
- /// Network and data directory settings — 6 settings.
+ /// Network and data directory settings — 7 settings.
pub server: ServerConfig,
/// Authentication — 3 settings.
pub auth: AuthConfig,
@@ -94,7 +94,7 @@ pub struct Config {
pub lifecycle: LifecycleConfig,
}
-// 19 sub-configs, 104 leaf settings in total. Do not maintain that sum by hand
+// 20 sub-configs, 105 leaf settings in total. Do not maintain that sum by hand
// — `journey_zero_config` in xerj-engine/tests/product_experience.rs counts a
// serialised `Config::default()` and fails if this comment and the module
// header stop matching. `Default` is derived: every field is a sub-config that
@@ -394,7 +394,7 @@ impl Config {
}
// ═════════════════════════════════════════════════════════════════════════════
-// Sub-configs (104 user-facing settings total; counted by
+// Sub-configs (105 user-facing settings total; counted by
// `journey_zero_config`, not by hand)
// ═════════════════════════════════════════════════════════════════════════════
@@ -1861,7 +1861,7 @@ mod tests {
drift.join("\n ")
);
- // …and the file's own header quotes how many of the 104 it sets. That
+ // …and the file's own header quotes how many of the 105 it sets. That
// number was 38, then 56, and never once the truth (#207), so count the
// assignments instead of trusting the sentence.
let set_here = toml_src
@@ -2225,6 +2225,7 @@ mod tests {
("search_context", 7),
("logging", 2),
("compat", 2),
+ ("lifecycle", 1),
];
/// Count the settings by *counting them*.
@@ -2267,7 +2268,7 @@ mod tests {
"the section table must sum to the whole config"
);
assert_eq!(
- total, 104,
+ total, 105,
"the total settings count changed. It is quoted in this module's \
header, in xerj-common/src/lib.rs, in engine/README.md, in \
xerj.default.toml and in EXPECTED_SETTINGS in \
diff --git a/engine/crates/xerj-common/src/lib.rs b/engine/crates/xerj-common/src/lib.rs
index 5fbaa904b..c21b38555 100644
--- a/engine/crates/xerj-common/src/lib.rs
+++ b/engine/crates/xerj-common/src/lib.rs
@@ -7,13 +7,13 @@
//! ## Design philosophy
//!
//! Unlike Elasticsearch's 3000+ configuration knobs, xerj deliberately exposes
-//! **104 settings**, each meaningful and production-tested. Every default
+//! **105 settings**, each meaningful and production-tested. Every default
//! is chosen so that a fresh deployment with zero configuration changes performs
//! well for the majority of workloads.
//!
//! ## Modules
//!
-//! - [`config`] — TOML-based configuration (104 settings)
+//! - [`config`] — TOML-based configuration (105 settings)
//! - [`error`] — Unified error type ([`XerjError`])
//! - [`types`] — Core domain types (documents, fields, IDs)
//! - [`schema`] — Index schema management and mapping evolution
diff --git a/engine/crates/xerj-engine/tests/product_experience.rs b/engine/crates/xerj-engine/tests/product_experience.rs
index 53c464f22..04e3c27d1 100644
--- a/engine/crates/xerj-engine/tests/product_experience.rs
+++ b/engine/crates/xerj-engine/tests/product_experience.rs
@@ -753,7 +753,7 @@ fn count_settings(value: &serde_json::Value) -> usize {
/// Settings in a default `Config`, measured by `count_settings`. Quoted in
/// `xerj-common/src/config.rs`; keep the two in step.
-const EXPECTED_SETTINGS: usize = 104;
+const EXPECTED_SETTINGS: usize = 105;
#[tokio::test]
async fn journey_zero_config() {
diff --git a/engine/tests/battle_sla.sh b/engine/tests/battle_sla.sh
index e4c80a082..2408ac86d 100755
--- a/engine/tests/battle_sla.sh
+++ b/engine/tests/battle_sla.sh
@@ -625,15 +625,15 @@ log "ES comparison: elasticsearch.yml has 3000+ documented properties"
log ""
# The shipped file sets the commonly tuned subset; the engine's full surface is
-# 104 settings, every one optional — measured by `count_user_facing_settings`
+# 105 settings, every one optional — measured by `count_user_facing_settings`
# (xerj-common/src/config.rs), which counts leaf keys of a serialised
# `Config::default()`. This bound is on the FILE, and it is deliberately not the
# published headline: quoting a number nobody counted is what issue #207 was
# filed about.
USER_SETTINGS=$(grep -v "^#\|^$\|^\[" "$DEFAULT_TOML" | grep "=" | wc -l || echo 0)
-info "Key=value lines in xerj.default.toml: $USER_SETTINGS (of 104 settings in total)"
+info "Key=value lines in xerj.default.toml: $USER_SETTINGS (of 105 settings in total)"
[ "$USER_SETTINGS" -le 60 ] \
- && pass "The shipped config sets $USER_SETTINGS keys; XERJ has 104 settings in total, all optional (Elasticsearch has 3000+)" \
+ && pass "The shipped config sets $USER_SETTINGS keys; XERJ has 105 settings in total, all optional (Elasticsearch has 3000+)" \
|| fail "The shipped config sets $USER_SETTINGS keys (expected ≤ 60)"
# Verify every setting has a comment above it
diff --git a/engine/xerj.default.toml b/engine/xerj.default.toml
index e499ce40e..f4f667f7a 100644
--- a/engine/xerj.default.toml
+++ b/engine/xerj.default.toml
@@ -1,6 +1,6 @@
# xerj Default Configuration
# ============================================================
-# This file sets 54 of XERJ's 104 user-facing settings to their
+# This file sets 54 of XERJ's 105 user-facing settings to their
# production-ready defaults, and describes the rest in comments.
# Every value here IS the default, so copying this file changes
# nothing until you edit it (a test enforces that). The complete
diff --git a/user-feedback/01-operational-complexity/cluster-management.md b/user-feedback/01-operational-complexity/cluster-management.md
index 381a487d1..154c20ee6 100644
--- a/user-feedback/01-operational-complexity/cluster-management.md
+++ b/user-feedback/01-operational-complexity/cluster-management.md
@@ -60,7 +60,7 @@ The #1 complaint across all review platforms.
- Single-node architecture in M1 eliminates all cluster coordination
- No master election, no shard allocation, no cluster state broadcast
- Single binary deployment: `./xerj --config config.toml`
-- 104 configuration settings, every one optional (ES has 3,000+)
+- 105 configuration settings, every one optional (ES has 3,000+)
- No dedicated specialists needed for basic operation
## Sources
diff --git a/user-feedback/01-operational-complexity/community-horror-stories.md b/user-feedback/01-operational-complexity/community-horror-stories.md
index 16d9664ee..c0486a18d 100644
--- a/user-feedback/01-operational-complexity/community-horror-stories.md
+++ b/user-feedback/01-operational-complexity/community-horror-stories.md
@@ -79,7 +79,7 @@ A team spent 14 hours manually relocating shards, only to discover the real issu
Every one of these complaints traces to one or more of:
1. JVM/GC → Rust eliminates entirely
2. Cluster coordination → single-node M1 eliminates entirely
-3. Configuration sprawl → 104 settings, all optional, with sensible defaults
+3. Configuration sprawl → 105 settings, all optional, with sensible defaults
4. Shard management → no shards, automatic segment management
5. Upgrade complexity → replace binary + restart
diff --git a/user-feedback/01-operational-complexity/learning-curve.md b/user-feedback/01-operational-complexity/learning-curve.md
index d556b0aa9..d71fd967f 100644
--- a/user-feedback/01-operational-complexity/learning-curve.md
+++ b/user-feedback/01-operational-complexity/learning-curve.md
@@ -48,7 +48,7 @@ Appears in nearly every negative review across all platforms.
- Minimal API surface: 10 REST endpoints, not hundreds
- Native REST API with intuitive JSON (not deeply nested DSL)
- ES-compat layer on port 9200 for teams already fluent in ES DSL
-- 104 config settings, every one optional, all with sensible defaults
+- 105 config settings, every one optional, all with sensible defaults
- Single TOML config file (not elasticsearch.yml + jvm.options + log4j2.properties)
- No shard/replica/JVM concepts to learn for M1
diff --git a/user-feedback/15-clustering-durability-expectations-2025-2026/03-user-expectations-2025-2026.md b/user-feedback/15-clustering-durability-expectations-2025-2026/03-user-expectations-2025-2026.md
index 1385afe91..573e8a828 100644
--- a/user-feedback/15-clustering-durability-expectations-2025-2026/03-user-expectations-2025-2026.md
+++ b/user-feedback/15-clustering-durability-expectations-2025-2026/03-user-expectations-2025-2026.md
@@ -104,7 +104,7 @@ Based on all user feedback collected:
5. **Instant write visibility** — no refresh delay
6. **AI-native** — text + vector + hybrid in one query
7. **WASM pipelines** — user-defined transforms at ingest speed
-8. **104 settings, not 3,000** — every one optional; works out of the box
+8. **105 settings, not 3,000** — every one optional; works out of the box
9. **Serverless pricing** — pay per query, not per node
10. **Zero-downtime everything** — upgrades, scaling, schema changes
diff --git a/user-feedback/ROADMAP-TO-GA.md b/user-feedback/ROADMAP-TO-GA.md
index dcce32bb9..531be6048 100644
--- a/user-feedback/ROADMAP-TO-GA.md
+++ b/user-feedback/ROADMAP-TO-GA.md
@@ -111,10 +111,10 @@ destructive step (engine/crates/xerj-storage/src/index_store.rs:595-608, 772-778
**Config sprawl — "3,000+ settings" (01-operational-complexity/learning-curve.md:18-20).** An empty
or absent config file is valid; every setting has a production default; a typo'd key fails startup
(`deny_unknown_fields`); schema-present-but-unwired features are rejected loudly at startup
-(config.rs:42-47, 94-108, 198-229). The true count is 104 serde-visible settings — measured by
+(config.rs:42-47, 94-108, 198-229). The true count is 105 serde-visible settings — measured by
`count_user_facing_settings`, which serialises `Config::default()` and counts leaf keys, not by
hand. The published "38"/"<50" figures were wrong; they were corrected with blocker 10
-(2026-08-09, #207). 104 vs 3,000+ is ~29x fewer (PROJECTED: 3000/104), told truthfully.
+(2026-08-09, #207). 105 vs 3,000+ is ~29x fewer (PROJECTED: 3000/105), told truthfully.
**Shard sizing expertise (04-scaling-and-shards, cluster-management.md:11).** No up-front shard
decision exists: ES `number_of_shards` is stored and echoed for wire-compat but never used as a
@@ -432,8 +432,9 @@ feature is experienced as a regression.
### 10. Correct the settings-count claim and make the count drift-proof (small) — DONE 2026-08-09 (#207)
> Landed as described: the count test now serialises `Config::default()` and counts leaf
-> keys (`journey_zero_config`), the measured total was **103** at the time (104 since
-> rc.14 added `server.allow_insecure_network_bind`), and the 38/56/60/`<50`
+> keys (`journey_zero_config`), the measured total was **103** at the time (104 after
+> rc.14 added `server.allow_insecure_network_bind`, 105 after #247 added
+> `lifecycle.tick_interval_secs`), and the 38/56/60/`<50`
> figures are corrected across config.rs, xerj-common/src/lib.rs, engine/README.md,
> xerj.default.toml and the feedback responses. Per-sub-config annotations are corrected
> too (limits 3 → 13, storage 5 → 10, cluster 4 → 5, embedding 4 → 19, merge 5 → 8,
@@ -451,8 +452,9 @@ verifier disputed it to blocking under the repo's honest-claims rules — a publ
doing it before GA, not waiving it. This roadmap adopts the verifier's position.]
**Build.** Fix all inconsistent counts; replace the hand-written count test with introspection
(serialize `Config::default()` to `toml::Value`, count leaf keys) so the number can never drift;
-sweep public docs and the feedback responses for the stale "<50"/"38". 104 vs 3,000+ is still the
-winning story, told truthfully. [DONE 2026-08-09, #207 — the measured figure is 103.] Files: engine/crates/xerj-common/src/config.rs, docs, user-feedback
+sweep public docs and the feedback responses for the stale "<50"/"38". 105 vs 3,000+ is still the
+winning story, told truthfully. [DONE 2026-08-09, #207 — the measured figure was 103 then and is
+105 now; the point of the fix is that it is measured, so it moves with the code.] Files: engine/crates/xerj-common/src/config.rs, docs, user-feedback
response sections.
**If shipped without.** A reviewer counts the TOML keys, finds 2.7x the claimed number, and every
other XERJ number takes the credibility hit.