Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

Record independent catalog validation and gate hybrid pagination in ES-YAML

xerj-orgbd8e4582fbfa
diff --git a/CHANGELOG.md b/CHANGELOG.md
index b0ce4447b..031595f2d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,6 +9,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
+- Empty autoindex maps now explain that managed corpus slugs and autoindex
+ dataset/index names belong to separate discovery surfaces. CLI diagnostics,
+ MCP empty responses and the `xerj_map` description point to corpus discovery
+ without claiming a corpus exists. Missing catalogs remain errors; auth
+ refusals and successful dataset maps retain their behavior (#2412).
+
- Public Agent Skills Discovery v0.2.0 index and a documentation-routing skill
at `/.well-known/agent-skills/`. The index is generated from the published
skill's metadata and exact-byte SHA-256, with a CI drift check.
@@ -100,12 +106,6 @@ continues in #2427 and #1122, which stay open.
### Added
-- Empty autoindex maps now explain that managed corpus slugs and autoindex
- dataset/index names belong to separate discovery surfaces. CLI diagnostics,
- MCP empty responses and the `xerj_map` description point to corpus discovery
- without claiming a corpus exists. Missing catalogs remain errors; auth
- refusals and successful dataset maps retain their behavior (#2412).
-
- `scripts/bench/neural-bulk-ab.sh CONTROL_SHA CANDIDATE_SHA [N]` is an
interleaved A/B harness for neural-embedding `_bulk` ingest. It builds each
SHA in a worktree, runs the real `xerj autoindex` client against a frozen
diff --git a/engine/tests/es-compat-yaml/yaml/search/773_hybrid_page_size.yml b/engine/tests/es-compat-yaml/yaml/search/773_hybrid_page_size.yml
new file mode 100644
index 000000000..dd5235c5e
--- /dev/null
+++ b/engine/tests/es-compat-yaml/yaml/search/773_hybrid_page_size.yml
@@ -0,0 +1,51 @@
+---
+setup:
+ - do:
+ indices.create:
+ index: hybrid_page
+ body:
+ mappings:
+ properties:
+ label: { type: keyword }
+ kind: { type: keyword }
+ v: { type: dense_vector, dims: 2 }
+ - do:
+ bulk:
+ refresh: true
+ body:
+ - '{"index":{"_index":"hybrid_page","_id":"a"}}'
+ - '{"label":"a","kind":"vector","v":[1,0]}'
+ - '{"index":{"_index":"hybrid_page","_id":"b"}}'
+ - '{"label":"b","kind":"vector","v":[0.8,0.2]}'
+ - '{"index":{"_index":"hybrid_page","_id":"c"}}'
+ - '{"label":"c","kind":"vector","v":[0.6,0.4]}'
+ - '{"index":{"_index":"hybrid_page","_id":"d"}}'
+ - '{"label":"d","kind":"lexical"}'
+ - match: { errors: false }
+
+---
+"hybrid size and from page the union without expanding size to knn k (#2561)":
+ - do:
+ search:
+ index: hybrid_page
+ body:
+ query: { term: { kind: lexical } }
+ knn: { field: v, query_vector: [1, 0], k: 3, num_candidates: 4 }
+ sort: [{ label: asc }]
+ from: 1
+ size: 1
+ - match: { hits.total.value: 4 }
+ - length: { hits.hits: 1 }
+ - match: { hits.hits.0._id: b }
+
+---
+"hybrid size zero keeps the union count and returns no hits (#2561)":
+ - do:
+ search:
+ index: hybrid_page
+ body:
+ query: { term: { kind: lexical } }
+ knn: { field: v, query_vector: [1, 0], k: 3, num_candidates: 4 }
+ size: 0
+ - match: { hits.total.value: 4 }
+ - length: { hits.hits: 0 }
diff --git a/tools/catalog-map/VERIFICATION.md b/tools/catalog-map/VERIFICATION.md
index 7cee50c9e..c9f4f54c0 100644
--- a/tools/catalog-map/VERIFICATION.md
+++ b/tools/catalog-map/VERIFICATION.md
@@ -116,3 +116,46 @@ applicable to this read-only hint change. No speed/quality claim is made.
Not tested: Linux/Windows execution, real managed corpus availability,
production registry configurations or fixes to ranking/identity discovery.
The known proxy fixture failure remains unresolved; hosted CI is separate.
+
+
+## Independent maintainer review — 2026-10-29
+
+Agent-assisted review on Linux x86_64, composed with the hybrid page-size fix
+from #2572 and current main. Original contributor commits remain ancestors.
+The source-built `xerj v1.0.0-rc.100` executable has SHA-256 `6de2bc0c54174b79c9bffdb94049e70630b8b029ec547c4b8d08eccdb58e2606`.
+The release-tested engine tree is `26f5586be30c715d99bff86fece77616b3f033a4`;
+subsequent changes add the YAML cases and this receipt, not runtime Rust.
+
+- Negative catalog transport control: the previous binary failed exactly the
+ four empty/missing namespace cases; the six populated/401/403 controls passed.
+ The candidate passed all ten cases using `python3 tools/catalog-map/smoke.py
+ --binary engine/target/release/xerj`.
+- `RUST_MIN_STACK=16777216 cargo test --profile ci-test --locked -j 16
+ -p xerj-autoindex -p xerj-mcp -- --test-threads=2`: 2,064 passed, zero failed,
+ five existing ignored. No platform filters or skipped test names.
+- The new CLI parsing comparison initially exposed concurrent cwd/key discovery
+ changes. Holding the existing sandbox `ENV_LOCK` across both parses fixes
+ that test race without changing authentication behavior.
+- A separate initial default-parallel run failed the existing PDF-worker timeout
+ assertion (30.01 seconds versus its ten-second ceiling). An unchanged repeat
+ of the complete autoindex library at default parallelism passed all 1,849
+ tests, with two existing ignored. That intermittent timeout remains a known
+ test limitation; its assertion and the production timeout were not changed.
+- The API crate passed 1,009 tests, zero failed, two existing ignored. Restoring
+ the old unconditional page widening made the strengthened hybrid regression
+ fail; the fix passed size-zero, pagination and complete-union controls.
+- Strict Clippy passed for `xerj-api`, `xerj-autoindex`, and `xerj-mcp`, all
+ targets, with `-D warnings` and no added allowances. Scoped release build of
+ those crates plus `xerj-server` and `es-yaml-runner` passed.
+- Live schema generation and checking passed with 22 tools. Only the map
+ description changes relative to main; no tool schema is hand-written.
+- Fresh private-node ES-YAML: **231 files, 1,449 passed, zero failed, ten skipped**,
+ 75 existing catch-lenient cases. This includes two new hybrid page-size YAML
+ cases. An earlier harness launch hit an occupied adjacent listener port;
+ reserving a free three-port range resolved that harness setup failure.
+- All 19 local site/guard commands passed: the SEO workflow's tests and generator
+ checks, factcheck self-test/fixture/error checks, landing constants, and
+ workflow timeouts.
+
+No performance improvement, Windows/macOS rerun, live corpus availability,
+production authentication setup, or hosted CI completion is claimed here.