Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

merge: bring in main (#254 + the autoindex rerun-refusal wave)

xerj-org252857162b05
diff --cc CHANGELOG.md
index 76e5feed1,ef625a381..1a32cf870
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@@ -29,84 -29,55 +29,133 @@@ and this project adheres to [Semantic V
`pdf_extraction_reuse` block so the behaviour is observable rather than
inferred from timing. Original work by Leonid Bugaev (@buger).
+ ### Changed — autoindex refuses reruns that would strand documents
+
+ - **Deleting an indexed file and rerunning `xerj autoindex` now fails instead
+ of exiting 0.** Nothing in the pipeline removes the documents, aliases,
+ graph edges or catalog entries that the deleted file published, so a rerun
+ that ignored the deletion left them live and searchable with no source file
+ behind them. The rerun is now refused before any remote call other than the
+ endpoint-readiness ping: no mapping, delete-by-query, bulk, refresh, graph
+ or catalog write is attempted, and the journal is not appended to. The error
+ names the removed files and their content keys — the first ten, then an
+ `… and N more` tail, since a whole unmounted subtree can vanish at once —
+ and gives three recovery routes: restore the files and rerun, rebuild in
+ place by deleting the named indices and the state directory, or rebuild
+ under a new state directory, prefix and brain. `--fresh` is refused for the
+ same case, because it does not delete those documents either. `--json` emits
+ the same facts — every removed entry, uncapped — as
+ `xerj.autoindex.unsupported_sync_delta.v1` on stdout with exit 1. Deleting a
+ file that was only ever *skipped* is not refused: a junk file publishes no
+ documents, and the stale junk-catalog sweep removes its one catalog row, so
+ nothing is stranded.
+ - **Files added after a plan was frozen are now called out on stderr.** They
+ are still not indexed by a rerun that resumes an existing plan — the plan is
+ a crash-resume boundary, not a folder-sync generation — but the run now
+ lists them, records them as skipped (exit 3, completed-with-junk) and points
+ at `--fresh`, which rebuilds the plan in place and picks them up. Adding or
+ changing files and rerunning keeps working; only removals are refused.
+ - **`xerj brain` no longer turns an absent or zero node-count probe into an
+ automatic reset.** Journal/server disagreement now fails with the journal,
+ URL, prefix, and brain identity plus recovery instructions, including the
+ explicit `--fresh` rerun for a genuinely wiped data directory. Probe
+ transport and malformed-response failures remain errors instead of being
+ reported as an empty destination. A nodes index that was deleted out from
+ under a surviving brain meta doc reads as absence and reaches that recovery
+ text, rather than surfacing as a raw HTTP 404.
+ - **`--fresh` still recovers a state directory whose journal cannot be
+ parsed.** The rerun gate reads the durable plan before the run starts, and a
+ plan that is malformed, or recorded for a different root/URL/prefix, is fatal
+ to a resume — but not to `--fresh`, which deletes that journal unread. The
+ preflight is now no more fatal than the open it precedes: under `--fresh` the
+ unreadable plan is reported on stderr and rebuilt from the current folder.
+ Without `--fresh` the refusal is unchanged. Because the removal gate has no
+ comparison basis in that case, the warning says so: documents already
+ published for files that are now gone cannot be identified from an unreadable
+ plan and are not deleted.
+ - **Refusal and skip listings are capped at ten entries plus an "and N more"
+ tail.** Unmounting a bind mount under an indexed root vanishes every content
+ group at once, so the uncapped listing was one rendered entry per journalled
+ file. `--json` still carries every entry.
+
+### Fixed
+
+- **An index whose metadata will not parse is refused instead of silently
+ reopened with an empty mapping**
+ ([#202](https://github.com/xerj-org/xerj/issues/202)). `Index::open` treated
+ "this file is not there" and "this file does not parse" as the same thing:
+ `load_schema`/`load_settings` mapped ENOENT, EACCES, EIO and malformed JSON
+ alike to one anonymous error, and the caller answered all of them with a
+ fresh dynamic mapping. Measured on a field deliberately mapped `keyword`:
+ after `schema.json` was truncated to half its bytes the index still opened
+ `Ok` with `field_count = 0`, the field came back `None`, and one further
+ document re-inferred it as `long` — a mapping silently replaced by a
+ different one, with nothing in any response saying so. Absent stays absent
+ (indices predating create-time schema persistence legitimately have no
+ `schema.json`); present-but-unparseable now fails the open with an error
+ naming the file. `es_mapping.json` — the full-fidelity mapping behind
+ `GET /_mapping` — was logged-and-ignored on a parse failure and now fails the
+ index too, on the boot scan, on snapshot restore and on a retry.
+
+ **Visible on upgrade:** a node whose data dir already holds a corrupt sidecar
+ now boots **red** with that index unserved, where it previously came up green
+ with an empty mapping. It joins the failed set
+ [#206](https://github.com/xerj-org/xerj/issues/206) introduced, so every
+ surface that set feeds already reports it — measured on a node holding two
+ user indices, `victim` with a truncated `schema.json`:
+
+ ```
+ GET /_cluster/health -> red, unassigned_primary_shards: 1
+ GET /_cluster/health?level=indices
+ -> victim red, unassigned_info.details = the
+ absolute path of schema.json and the parse error
+ GET /_cat/indices -> "red open victim <uuid> 1 0 0 0 0b 0b 0b"
+ GET /_cluster/indices/failed -> the reason, plus the retry and delete calls
+ PUT /victim, POST /victim/_doc, POST /victim/_search
+ -> 503 no_shard_available_action_exception
+ carrying that same reason
+ GET /health/ready -> 200 "ready (degraded): 1 of 2 indices
+ serving, 1 failed to open; see GET
+ /_cluster/indices/failed"
+ POST /healthy/_search -> 200
+ ```
+
+ The readiness counts above are the two-index test router's. A running node
+ also holds its internal `.xerj_*` indices, so the same condition on the shipped
+ binary reads `ready (degraded): 15 of 16 indices serving, 1 failed to open` —
+ the 200 and its meaning are unchanged, only the totals move with what else the
+ node holds.
+
+ Recovery is the three doors #206 added. Two were measured end to end on the
+ shipped binary: repair the file and
+ `POST /_cluster/indices/failed/{index}/_retry` (503 while it is still torn,
+ `200 {"reopened": true}` once it is not, health back to green), and
+ `DELETE /{index}` (200, directory removed, health back to green). The third —
+ restore the index from a snapshot — is the path this change touches (a
+ successful restore now clears the recorded failure, which it has to or health
+ would stay red after the repair worked); it is covered by an engine-level
+ test rather than measured over HTTP. The node stays in kubelet rotation as
+ long as any index is still serving.
+
+- **Concurrent sidecar writes can no longer manufacture the torn file the reader
+ now refuses.** `write_file_atomic` staged every write in
+ `path.with_extension("tmp")` — one shared name for all writers of that file —
+ so two concurrent writers of one sidecar (`PUT /{index}/_settings` racing
+ another settings update, two API-key mints for `api_keys.json`) both opened it
+ `O_TRUNC`, interleaved their bytes, and each renamed it into place. The rename
+ is atomic; the content being renamed was not one writer's. Measured with the
+ old shared name, 4 threads × two different-length settings bodies × 200
+ rounds, over four runs: **86–294 of 800 writes failed with ENOENT** (the loser
+ renaming a file the winner had already moved) and, with those errors swallowed
+ as the callers did, **1–16 of 200 rounds left an unparseable `settings.json`**.
+ The counts are race-dependent and vary with machine load; what does not vary is
+ that both are non-zero on every run with the shared name and **0 and 0** on
+ every run with a unique staging name. Each write now stages
+ in its own sibling file (`<file>.tmp.<pid>.<seq>`) in the target's directory
+ and removes its own debris on failure — which for `api_keys.json` is key
+ material. `update_settings` also writes `settings.json` through
+ `write_file_atomic` rather than a plain `fs::write`; it was the last
+ non-atomic sidecar writer left.
## [1.0.0-rc.14] - 2026-08-10