Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

docs(#463): refresh docstrings to approach B (skeptic should-fix)

xerj-orgf9657cca4e98
diff --git a/engine/crates/xerj-api/tests/close_frees_memory.rs b/engine/crates/xerj-api/tests/close_frees_memory.rs
index 33fe4364c..d07b64e4e 100644
--- a/engine/crates/xerj-api/tests/close_frees_memory.rs
+++ b/engine/crates/xerj-api/tests/close_frees_memory.rs
@@ -1,11 +1,13 @@
-//! Issue #463: `POST /{index}/_close` marks an index unqueryable but frees ~no
-//! memory — `close_index` only sets the `closed_indices` flag; the `Arc<Index>`
-//! (memtable + per-segment caches + hydration budget) stays in `engine.indices`.
+//! Issue #463: `POST /{index}/_close` must actually reclaim RAM. The fix frees
+//! the index's rebuildable caches while keeping the `Arc<Index>` loaded, so the
+//! closed index stays visible and a later `_open` re-hydrates from disk.
//!
-//! This probe establishes the fail-before observable: after `_close`, the global
-//! `segment_hydration.current_in_bytes` gauge does NOT drop (memory retained),
-//! and a subsequent `_open` + query must still return every doc (the fix must
-//! flush before releasing, so no data is lost).
+//! These are the HTTP-level guards: the `_close`→`_open`→query round-trip is
+//! lossless, and a closed index stays listed in `_cat/indices` (it did not
+//! vanish). The actual memory-reclaim assertion lives in the engine test
+//! (`close_releases_index`: `total_cache_entries` drops to 0) — the global
+//! `segment_hydration` gauge is not initialised in-process, so it cannot be an
+//! in-test observable here.
use axum::body::Body;
use axum::http::{Request, StatusCode};
diff --git a/engine/crates/xerj-engine/src/engine.rs b/engine/crates/xerj-engine/src/engine.rs
index 3c796055e..45ad40228 100644
--- a/engine/crates/xerj-engine/src/engine.rs
+++ b/engine/crates/xerj-engine/src/engine.rs
@@ -3188,24 +3188,27 @@ impl Engine {
}
/// Whether the concrete index `name` is currently held in memory (its
- /// `Arc<Index>` — memtable, per-segment caches, hydration budget — is live
- /// in `self.indices`). A `_close` that actually reclaims RAM drops the handle
- /// here (#463); a still-loaded name after close means the close was a no-op.
- /// Not alias-resolving — callers pass a concrete index name.
+ /// `Arc<Index>` is live in `self.indices`). A closed index stays loaded —
+ /// `_close` frees the index's rebuildable caches but keeps the handle here
+ /// (#463), so this returns `true` for a closed index too; it is `false` only
+ /// for a name that was never opened or has been deleted. Not alias-resolving
+ /// — callers pass a concrete index name.
pub fn is_index_loaded(&self, name: &str) -> bool {
self.indices.contains_key(name)
}
/// `_close` that actually reclaims RAM (#463). Marks the index unqueryable
- /// AND releases its in-memory handle — the memtable, the per-segment caches,
- /// and the hydration budget deallocate once no in-flight request holds a
- /// clone of the `Arc<Index>`. The on-disk bytes remain untouched, so
- /// [`reopen_index`](Self::reopen_index) reconstructs the index losslessly.
+ /// AND frees its rebuildable in-memory caches via [`Index::release_memory`]
+ /// (the ~15 per-segment cache DashMaps; reads re-hydrate from disk). The
+ /// `Arc<Index>` handle STAYS in `self.indices`, so every loaded-index view
+ /// (`_cat/indices`, `_cluster/health`, the `expand_wildcards` filtering) is
+ /// unchanged — a closed index is loaded-and-flagged, matching the rest of the
+ /// codebase's model. [`reopen_index`](Self::reopen_index) just clears the flag.
///
- /// The memtable is flushed to disk FIRST: dropping the handle without
- /// flushing would discard every not-yet-published document — turning a memory
- /// win into silent data loss. A flush failure aborts the close and leaves the
- /// index fully in service.
+ /// The memtable is flushed to disk FIRST: freeing the caches without flushing
+ /// would be harmless (caches only mirror durable data), but the flush also
+ /// drains the memtable so the closed index holds no unpublished writes. A
+ /// flush failure aborts the close and leaves the index fully in service.
pub async fn close_index(&self, name: &str) -> Result<()> {
self.ensure_cluster_state_writable()?;
if let Some(idx) = self.indices.get(name).map(|r| Arc::clone(r.value())) {
diff --git a/engine/crates/xerj-engine/tests/close_releases_index.rs b/engine/crates/xerj-engine/tests/close_releases_index.rs
index 7e8560da2..bf2ff7d46 100644
--- a/engine/crates/xerj-engine/tests/close_releases_index.rs
+++ b/engine/crates/xerj-engine/tests/close_releases_index.rs
@@ -1,10 +1,10 @@
//! Issue #463: `_close` must actually reclaim RAM, not just flip a flag.
//!
//! Before the fix, `close_index` only set the `closed_indices` gate; the
-//! `Arc<Index>` (memtable, per-segment caches, hydration budget) stayed in
-//! `Engine::indices`, so `_close` freed ~0.03% of an index's memory. The fix
-//! releases the in-memory handle on close and reconstructs it from disk on
-//! reopen — and must flush first so no not-yet-published document is lost.
+//! `Arc<Index>`'s per-segment caches stayed resident, so `_close` freed ~0.03%
+//! of an index's memory. The fix flushes the memtable then frees the index's
+//! rebuildable caches while KEEPING the `Arc<Index>` loaded (so `_cat`/`_cluster`
+//! views are unchanged); the next read re-hydrates from disk.
use serde_json::json;
use tempfile::TempDir;