Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

fix(embed): refuse startup when the neural model cannot load (#1877)

xerj-org23b5f1216dd4
diff --git a/docs/recipes/air-gapped-deployment.md b/docs/recipes/air-gapped-deployment.md
index 75c5410a4..d1adabd61 100644
--- a/docs/recipes/air-gapped-deployment.md
+++ b/docs/recipes/air-gapped-deployment.md
@@ -9,10 +9,14 @@ an optional, separately staged path.
- The default embedding mode is **lexical feature hashing**. It needs no model,
no embedding service, and no network connection.
-- Neural mode is optional. Without `embedding.local_model_dir`, the first
- operation that needs an embedding downloads `config.json`, `tokenizer.json`,
- and `model.safetensors` for `sentence-transformers/all-MiniLM-L6-v2` from the
- Hugging Face Hub. With that directory set, XERJ reads those files locally.
+- Neural mode is optional. The model is loaded when the process starts, not
+ on the first embed. Without `embedding.local_model_dir`, startup downloads
+ `config.json`, `tokenizer.json`, and `model.safetensors` for
+ `sentence-transformers/all-MiniLM-L6-v2` from the Hugging Face Hub
+ (`HF_ENDPOINT` selects a mirror). If that download fails, the process
+ refuses to start rather than indexing with the lexical embedder. With
+ `local_model_dir` set, XERJ reads those files locally and does not contact
+ the Hub.
- `proxy` calls only the configured OpenAI-compatible
`embedding.default_endpoint`. The experimental `onnx-experimental` backend
needs an ONNX-enabled build and explicit local assets; neither is the base
diff --git a/docs/recipes/production-deployment.md b/docs/recipes/production-deployment.md
index 9bfa1c8b4..4a1d16d81 100644
--- a/docs/recipes/production-deployment.md
+++ b/docs/recipes/production-deployment.md
@@ -467,9 +467,14 @@ in-process BERT sentence encoder (default `all-MiniLM-L6-v2`, 384-dim, ~90 MB),
no Python, no external service. Select it per node with `[embedding] mode =
"neural"` (or `--embed-mode neural`, or `XERJ_EMBED_MODE=neural`).
-On a connected host the model downloads itself **once, on first use**, from the
-HuggingFace Hub and is cached for every later start. On an **air-gapped** host
-there is no Hub to reach, so stage the model files yourself.
+On a connected host the model is downloaded from the HuggingFace Hub and loaded
+**at startup**, then cached for every later start. If that load fails the
+process **refuses to start** with one error that names the cause, the command
+`xerj --embed-mode lexical`, `embedding.local_model_dir`, and `HF_ENDPOINT`
+(a Hub mirror). It does not stay up and fail every ingest, and it does not
+substitute lexical vectors — those are not comparable with neural vectors.
+Explicit `proxy` already fails closed the same way. On an **air-gapped** host
+there is no Hub to reach, so stage the model files yourself before starting.
### Step 1 — download the three files on a connected machine
@@ -505,26 +510,21 @@ local_model_dir = "/opt/xerj/models/all-MiniLM-L6-v2" # holds the 3 files abov
```
When `local_model_dir` is set it takes precedence over the Hub — XERJ makes
-**no outbound network call** for the model. Selecting the neural backend logs a
-line when the semantic index is built:
+**no outbound network call** for the model. The weights load at startup, before
+the data directory is opened. A correct staging just works and makes no network
+call. A **missing or wrong file refuses startup** with the offending path:
```
-INFO embedding backend: built-in neural BERT (loads on first use)
+embedding.mode=neural failed to load model sentence-transformers/all-MiniLM-L6-v2: local model dir /opt/xerj/models/... is missing tokenizer.json
+… has no model.safetensors (candle requires safetensors, not pytorch_model.bin).
+Refusing to start. …
+To index without a model, restart with: xerj --embed-mode lexical
```
-The weights themselves load lazily, from your staged directory, on the first
-`semantic_text` ingest (not at boot). A correct staging just works — the local
-path is silent on success and makes no network call. A **missing or wrong file
-fails loud** with the offending path, which is your signal to fix the staging:
-
-```
-local model dir /opt/xerj/models/... is missing tokenizer.json
-… has no model.safetensors (candle requires safetensors, not pytorch_model.bin)
-```
-
-(The `neural embedder: model ready` log line you may have seen elsewhere is the
-*Hub-download* path's completion message; the offline `local_model_dir` path
-does not print it.)
+(The `neural embedder: model ready` log line is the *Hub-download* path's
+completion message; the offline `local_model_dir` path does not print it.
+After a successful load, opening a semantic index logs
+`embedding backend: built-in neural BERT (loaded)`.)
> **Slim builds.** A `--no-default-features` build omits the neural backend;
> `mode = "neural"` on such a binary logs a warning and falls back to the
diff --git a/engine/crates/xerj-ai/src/embedder.rs b/engine/crates/xerj-ai/src/embedder.rs
index 812909c17..29c51c0e9 100644
--- a/engine/crates/xerj-ai/src/embedder.rs
+++ b/engine/crates/xerj-ai/src/embedder.rs
@@ -12,8 +12,10 @@
//! service ([`crate::embed::EmbeddingProxy`]). Bring any model/provider.
//! * [`Embedder::Neural`] — the built-in BERT sentence encoder
//! ([`crate::neural`]), running in-process via `candle`. Compiled only
-//! under the `neural` cargo feature; the model is loaded lazily on first
-//! use (download-on-first-run) so startup stays instant.
+//! under the `neural` cargo feature. The server loads it at startup
+//! (`Embedder::preload_neural`); a blocked download refuses the process
+//! rather than failing every ingest. The handle still shares one
+//! process-wide model.
//! * [`Embedder::Onnx`] — the experimental in-process ONNX Runtime backend.
//! It is compiled only under `onnx-experimental`, requires explicit model
//! and tokenizer paths, and loads the model lazily on first use.
@@ -215,7 +217,8 @@ pub enum Embedder {
Lexical,
/// External OpenAI-compatible embedding service.
Proxy(EmbeddingProxy),
- /// Built-in neural BERT embedder (candle). Lazily loaded on first use.
+ /// Built-in neural BERT embedder (candle). The server preloads it at
+ /// startup; a handle built without that preload still loads on first embed.
#[cfg(feature = "neural")]
Neural(NeuralHandle),
/// Experimental local ONNX Runtime sentence encoder.
@@ -234,12 +237,36 @@ impl Embedder {
Embedder::Proxy(proxy)
}
- /// A lazily-loaded built-in neural embedder.
+ /// A built-in neural embedder. Call [`preload_neural`] first when a
+ /// failed download must surface before the first embed.
#[cfg(feature = "neural")]
pub fn neural(cfg: crate::neural::NeuralConfig) -> Self {
Embedder::Neural(NeuralHandle::new(cfg))
}
+ /// Load the neural model into the process-wide cache.
+ ///
+ /// The server calls this at startup (#1840). On success, every later
+ /// [`Self::neural`] handle with the same config reuses the loaded model
+ /// instead of downloading again. On failure the cell stays empty — this
+ /// never installs the lexical embedder.
+ #[cfg(feature = "neural")]
+ pub fn preload_neural(
+ cfg: &crate::neural::NeuralConfig,
+ ) -> Result<std::sync::Arc<crate::neural::NeuralEmbedder>> {
+ let cell = shared_neural_cell(cfg);
+ if let Some(ready) = cell.get() {
+ return Ok(std::sync::Arc::clone(ready));
+ }
+ let loaded = std::sync::Arc::new(crate::neural::NeuralEmbedder::load(cfg)?);
+ // A concurrent preload may win `set`. Keep whichever model landed in
+ // the cell so every handle shares one copy.
+ match cell.set(std::sync::Arc::clone(&loaded)) {
+ Ok(()) => Ok(loaded),
+ Err(_) => Ok(cell.get().cloned().unwrap_or(loaded)),
+ }
+ }
+
#[cfg(feature = "onnx-experimental")]
pub fn onnx(cfg: OnnxConfig) -> Self {
Embedder::Onnx(OnnxHandle::new(cfg))
@@ -870,6 +897,7 @@ mod neural_handle_tests {
model_id: "test/model-shared".into(),
cache_dir: Some(PathBuf::from("/tmp/xerj-neural-shared-cache")),
local_dir: None,
+ hub_endpoint: None,
};
let first = NeuralHandle::new(cfg.clone());
let second = NeuralHandle::new(cfg);
@@ -884,11 +912,13 @@ mod neural_handle_tests {
model_id: "test/model-a".into(),
cache_dir: None,
local_dir: None,
+ hub_endpoint: None,
});
let second = NeuralHandle::new(NeuralConfig {
model_id: "test/model-b".into(),
cache_dir: None,
local_dir: None,
+ hub_endpoint: None,
});
assert!(!Arc::ptr_eq(&first.cell, &second.cell));
@@ -900,6 +930,7 @@ mod neural_handle_tests {
model_id: "test/model-reclaimable".into(),
cache_dir: None,
local_dir: None,
+ hub_endpoint: None,
};
let weak = {
let handle = NeuralHandle::new(cfg.clone());
diff --git a/engine/crates/xerj-ai/src/neural.rs b/engine/crates/xerj-ai/src/neural.rs
index c2dd5a31d..c3c3d45af 100644
--- a/engine/crates/xerj-ai/src/neural.rs
+++ b/engine/crates/xerj-ai/src/neural.rs
@@ -10,10 +10,13 @@
//! * [`crate::embed::EmbeddingProxy`] — any external OpenAI-compatible
//! `/v1/embeddings` provider (bring-your-own model).
//!
-//! Model files are fetched once on first use from the HuggingFace Hub and
-//! cached on disk; air-gapped deployments point [`NeuralConfig::local_dir`]
-//! at a directory holding `config.json`, `tokenizer.json`, and the
-//! safetensors weights instead.
+//! Model files are resolved when the embedder is loaded. The server does
+//! that at startup when neural is requested (#1840): a blocked Hub or a
+//! missing local directory refuses the process once, instead of failing
+//! every later ingest. Air-gapped deployments point
+//! [`NeuralConfig::local_dir`] at a directory holding `config.json`,
+//! `tokenizer.json`, and the safetensors weights. A HuggingFace mirror is
+//! `HF_ENDPOINT` (or [`NeuralConfig::hub_endpoint`]).
//!
//! Compiled only under the `neural` cargo feature.
@@ -87,6 +90,12 @@ pub struct NeuralConfig {
/// Air-gapped: load `config.json` / `tokenizer.json` / weights from this
/// directory instead of downloading. Takes precedence over the hub.
pub local_dir: Option<PathBuf>,
+ /// Override the HuggingFace Hub base URL. `None` uses `HF_ENDPOINT` when
+ /// that variable is set, otherwise `https://huggingface.co`. Tests point
+ /// this at a closed local port so a blocked download never touches the
+ /// network. `ApiBuilder::new()` does not read `HF_ENDPOINT` itself
+ /// (hf-hub 0.4 only does that in `from_env`).
+ pub hub_endpoint: Option<String>,
}
impl Default for NeuralConfig {
@@ -95,6 +104,7 @@ impl Default for NeuralConfig {
model_id: DEFAULT_MODEL_ID.to_string(),
cache_dir: None,
local_dir: None,
+ hub_endpoint: None,
}
}
}
@@ -114,13 +124,14 @@ impl NeuralEmbedder {
self.dims
}
- /// Load the model, downloading from the HuggingFace Hub on first use
- /// (unless [`NeuralConfig::local_dir`] is set). **Blocking** — the caller
- /// runs this off the async executor (see [`crate::embedder`]).
+ /// Load the model, downloading from the HuggingFace Hub when it is not
+ /// already cached (unless [`NeuralConfig::local_dir`] is set).
+ /// **Blocking** — the server runs this at startup, off the async
+ /// executor (see [`crate::Embedder::preload_neural`]).
pub fn load(cfg: &NeuralConfig) -> Result<Self> {
let (config_path, tokenizer_path, weights_path) = match &cfg.local_dir {
Some(dir) => resolve_local(dir)?,
- None => resolve_from_hub(&cfg.model_id, cfg.cache_dir.as_deref())?,
+ None => resolve_from_hub(cfg)?,
};
let config_json = std::fs::read_to_string(&config_path)
@@ -443,7 +454,7 @@ fn compute_asset_identity(
) -> std::result::Result<AssetIdentity, AssetIdentityError> {
let (config_path, tokenizer_path, weights_path) = match &cfg.local_dir {
Some(dir) => resolve_local_assets(dir)?,
- None => resolve_from_hub(&cfg.model_id, cfg.cache_dir.as_deref())
+ None => resolve_from_hub(cfg)
// The hub error names the model id and the cache path; neither may
// reach a client, so only the generic label travels.
.map_err(|e| AssetIdentityError::new("the model download", format!("{e:#}")))?,
@@ -537,22 +548,46 @@ fn find_local_weights(dir: &Path) -> std::result::Result<PathBuf, AssetIdentityE
))
}
+/// The single startup error for a neural model that cannot be loaded.
+///
+/// Callers that refuse to start (#1840) use this text so the operator sees
+/// the cause once, plus the two ways out: lexical, or an offline / mirrored
+/// model. Lexical is named as a different command, never as a silent swap —
+/// those vectors are not comparable with neural vectors (#1189).
+pub fn startup_failure_message(model_id: &str, error: &anyhow::Error) -> String {
+ format!(
+ "embedding.mode=neural failed to load model {model_id}: {error:#}. \
+ Refusing to start. Lexical vectors are not comparable with neural vectors, \
+ so this process will not switch to the lexical embedder or keep running and \
+ fail every ingest. To index without a model, restart with: \
+ xerj --embed-mode lexical. To keep neural offline, set \
+ embedding.local_model_dir to a directory containing config.json, \
+ tokenizer.json and model.safetensors. To use a HuggingFace mirror, set \
+ HF_ENDPOINT to the mirror base URL and start again."
+ )
+}
+
/// Download (or read from cache) the model files from the HuggingFace Hub.
///
-/// The first launch with `--embed-mode neural` on a fresh machine pulls the
-/// weights (~90 MB for MiniLM); every launch after that is an instant cache
-/// hit. We surface that clearly so a user staring at a terminal knows the
-/// one-time download is happening rather than a hang. A progress bar is shown
-/// (hf-hub writes it to stderr) for the same reason.
-fn resolve_from_hub(
- model_id: &str,
- cache_dir: Option<&Path>,
-) -> Result<(PathBuf, PathBuf, PathBuf)> {
+/// The server calls this at startup when `--embed-mode neural` is requested,
+/// before it listens. The first start on a fresh machine pulls the weights
+/// (~90 MB for MiniLM); every start after that is a cache hit. A progress bar
+/// is shown (hf-hub writes it to stderr) so a download is not mistaken for a
+/// hang.
+fn resolve_from_hub(cfg: &NeuralConfig) -> Result<(PathBuf, PathBuf, PathBuf)> {
use hf_hub::api::sync::ApiBuilder;
- let mut builder = ApiBuilder::new().with_progress(true);
- if let Some(dir) = cache_dir {
- builder = builder.with_cache_dir(dir.to_path_buf());
+ let model_id = cfg.model_id.as_str();
+ // `ApiBuilder::new()` does not read `HF_ENDPOINT` (hf-hub 0.4 reserved
+ // that for `from_env`, which also relocates the cache via `HF_HOME`).
+ // Keep the historical cache directory and apply only the mirror URL,
+ // unless the caller set `model_cache_dir` / `hub_endpoint`.
+ let mut builder = ApiBuilder::new().with_progress(true).with_retries(0);
+ if let Some(dir) = &cfg.cache_dir {
+ builder = builder.with_cache_dir(dir.clone());
+ }
+ if let Some(endpoint) = hub_endpoint_override(cfg) {
+ builder = builder.with_endpoint(endpoint);
}
let api = builder
.build()
@@ -590,6 +625,22 @@ fn resolve_from_hub(
Ok((config, tokenizer, weights))
}
+/// Explicit [`NeuralConfig::hub_endpoint`], otherwise a non-empty `HF_ENDPOINT`.
+fn hub_endpoint_override(cfg: &NeuralConfig) -> Option<String> {
+ if let Some(endpoint) = cfg
+ .hub_endpoint
+ .as_deref()
+ .map(str::trim)
+ .filter(|endpoint| !endpoint.is_empty())
+ {
+ return Some(endpoint.to_string());
+ }
+ std::env::var("HF_ENDPOINT")
+ .ok()
+ .map(|endpoint| endpoint.trim().to_string())
+ .filter(|endpoint| !endpoint.is_empty())
+}
+
#[cfg(test)]
mod tests {
use super::*;
diff --git a/engine/crates/xerj-common/src/config.rs b/engine/crates/xerj-common/src/config.rs
index 30d36b6b6..81d18e3b6 100644
--- a/engine/crates/xerj-common/src/config.rs
+++ b/engine/crates/xerj-common/src/config.rs
@@ -1556,11 +1556,15 @@ impl LogsConfig {
/// default: lexical, *not* neural semantic understanding.
/// * `"neural"` — a built-in BERT sentence embedder (all-MiniLM-L6-v2 by
/// default) that runs in-process via `candle`. The model weights are
-/// downloaded once on first use (or read from
+/// downloaded and loaded at startup (or read from
/// [`local_model_dir`](Self::local_model_dir) for
-/// air-gapped deployments). The neural backend ships in the standard
-/// binary; a `--no-default-features` slim build omits it and falls back
-/// to lexical.
+/// air-gapped deployments; `HF_ENDPOINT` selects a Hub mirror). If that
+/// load fails, the process refuses to start (#1840) instead of accepting
+/// traffic and failing every ingest. It does not substitute the lexical
+/// embedder: those vectors are not comparable with neural vectors, the
+/// same fail-closed rule as explicit `"proxy"` (#1189). The neural
+/// backend ships in the standard binary; a `--no-default-features` slim
+/// build omits it and falls back to lexical, and identity reports lexical.
/// * `"proxy"` — call an external OpenAI-compatible `/v1/embeddings`
/// endpoint ([`default_endpoint`](Self::default_endpoint)). Lets
/// customers plug in ANY embedding model / provider they already run. An explicit `mode = "proxy"` with a
diff --git a/engine/crates/xerj-engine/src/index.rs b/engine/crates/xerj-engine/src/index.rs
index cb080febb..775defc4c 100644
--- a/engine/crates/xerj-engine/src/index.rs
+++ b/engine/crates/xerj-engine/src/index.rs
@@ -37185,6 +37185,9 @@ fn configured_neural_config(
model_id,
cache_dir: (!cfg.model_cache_dir.is_empty()).then(|| cfg.model_cache_dir.clone().into()),
local_dir: (!cfg.local_model_dir.is_empty()).then(|| cfg.local_model_dir.clone().into()),
+ // The mirror URL is `HF_ENDPOINT`, read inside the hub client. There
+ // is no config field for it; tests inject one on `NeuralConfig` directly.
+ hub_endpoint: None,
}
}
@@ -38599,7 +38602,7 @@ fn make_embedder(cfg: &xerj_common::config::EmbeddingConfig) -> Result<xerj_ai::
|| (mode != "neural" && mode != "lexical" && !cfg.default_endpoint.is_empty());
if mode == "neural" {
- return Ok(make_neural_embedder(cfg));
+ return make_neural_embedder(cfg);
}
if mode == "onnx-experimental" {
@@ -38735,27 +38738,66 @@ fn make_onnx_embedder(_cfg: &xerj_common::config::EmbeddingConfig) -> Result<xer
)))
}
-/// Construct the built-in neural embedder when compiled in; otherwise warn
-/// and fall back to lexical so a `mode=neural` config never bricks startup.
+/// Load the neural model when this binary can, before any ingest.
+///
+/// Explicit `mode=neural` fails closed (#1840), the same shape as explicit
+/// `mode=proxy` (#1189): a download or file error is returned, and the lexical
+/// embedder is not installed in its place. Those vectors are not comparable.
+/// The server calls this at startup so the failure happens once, before it
+/// listens. A slim build without the `neural` feature still returns `Ok` —
+/// `make_neural_embedder` warns and substitutes lexical, and identity reports
+/// lexical, because there is no model to load.
+pub fn ensure_neural_ready(cfg: &xerj_common::config::EmbeddingConfig) -> Result<()> {
+ if !cfg.mode.trim().eq_ignore_ascii_case("neural") {
+ return Ok(());
+ }
+ #[cfg(feature = "neural")]
+ {
+ let neural_cfg = configured_neural_config(cfg);
+ if let Err(error) = xerj_ai::Embedder::preload_neural(&neural_cfg) {
+ return Err(EngineError::Common(xerj_common::XerjError::embedding(
+ xerj_ai::neural::startup_failure_message(&neural_cfg.model_id, &error),
+ )));
+ }
+ Ok(())
+ }
+ #[cfg(not(feature = "neural"))]
+ {
+ Ok(())
+ }
+}
+
+/// Construct the built-in neural embedder when compiled in.
+///
+/// The model is loaded here (and at server startup via [`ensure_neural_ready`]).
+/// A failure is an error, not a lexical embedder. A slim build without the
+/// feature warns and returns lexical so a `mode=neural` config on that binary
+/// still starts; identity reports lexical for that case.
#[cfg(feature = "neural")]
-fn make_neural_embedder(cfg: &xerj_common::config::EmbeddingConfig) -> xerj_ai::Embedder {
+fn make_neural_embedder(cfg: &xerj_common::config::EmbeddingConfig) -> Result<xerj_ai::Embedder> {
// Exactly the resolution the execution identity content-addresses — see
// `configured_neural_config`. Duplicating it here once let the reported
// identity describe a different model than the one that loads.
+ ensure_neural_ready(cfg)?;
let neural_cfg = configured_neural_config(cfg);
- info!(model = %neural_cfg.model_id, "embedding backend: built-in neural BERT (loads on first use)");
- xerj_ai::Embedder::neural(neural_cfg)
+ info!(
+ model = %neural_cfg.model_id,
+ "embedding backend: built-in neural BERT (loaded)"
+ );
+ Ok(xerj_ai::Embedder::neural(neural_cfg))
}
#[cfg(not(feature = "neural"))]
-fn make_neural_embedder(_cfg: &xerj_common::config::EmbeddingConfig) -> xerj_ai::Embedder {
+fn make_neural_embedder(_cfg: &xerj_common::config::EmbeddingConfig) -> Result<xerj_ai::Embedder> {
+ // Identity reports lexical for this build. The warning is the whole
+ // signal: there is no model download that can fail later.
warn!(
"embedding.mode=neural requested but this is a slim build (compiled with \
`--no-default-features`, no neural backend) — falling back to the built-in lexical \
embedder. The standard release binary ships the neural embedder; use it, or rebuild \
xerj-server with default features."
);
- xerj_ai::Embedder::lexical()
+ Ok(xerj_ai::Embedder::lexical())
}
// ── HNSW persistence helpers (module-level, called from sync `Index::open`)
diff --git a/engine/crates/xerj-server/Cargo.toml b/engine/crates/xerj-server/Cargo.toml
index 9c935402f..eb9c2c5c3 100644
--- a/engine/crates/xerj-server/Cargo.toml
+++ b/engine/crates/xerj-server/Cargo.toml
@@ -64,8 +64,9 @@ prost.workspace = true
[features]
# The shipped binary is batteries-included: the built-in neural BERT
# embedder (candle + MiniLM) is ON by default, so end users just add
-# `--embed-mode neural` at runtime and the model downloads itself on first
-# use — no special build, no separate binary. Build with
+# `--embed-mode neural` at runtime and the model downloads and loads at
+# startup — a blocked Hub refuses to start, it does not fail every ingest.
+# No special build, no separate binary. Build with
# `--no-default-features` for a slimmer binary without the neural backend
# (the lexical + proxy backends still work).
default = ["neural", "decide-local"]
diff --git a/engine/crates/xerj-server/src/main.rs b/engine/crates/xerj-server/src/main.rs
index 3a971938a..135c7157f 100644
--- a/engine/crates/xerj-server/src/main.rs
+++ b/engine/crates/xerj-server/src/main.rs
@@ -330,8 +330,13 @@ fn help_text(feedback: bool) -> String {
\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20onnx-experimental\n\
\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20lexical built-in feature-hash (default; offline, not neural)\n\
\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20neural built-in BERT (all-MiniLM-L6-v2) — real semantics,\n\
- \x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20in-process; auto-downloads the model (~90 MB) on\n\
- \x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20first use, then runs from cache. Just add the flag.\n\
+ \x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20in-process. The model (~90 MB) loads at startup:\n\
+ \x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20downloaded from the HuggingFace Hub, or read from\n\
+ \x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20embedding.local_model_dir, then cached. If that load\n\
+ \x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20fails, the process refuses to start. It does not keep\n\
+ \x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20running and fail every ingest, and it does not write\n\
+ \x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20lexical vectors. Restart with: xerj --embed-mode lexical\n\
+ \x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20Or set HF_ENDPOINT to a Hub mirror and start again.\n\
\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20proxy external OpenAI-compatible /v1/embeddings endpoint\n\
\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20auto proxy if embedding.default_endpoint is set, else lexical\n\
\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20onnx-experimental local all-MiniLM-L6-v2-compatible FP32\n\
@@ -2297,10 +2302,141 @@ fn build_runtime(worker_threads: usize) -> Result<tokio::runtime::Runtime> {
.context("build tokio runtime")
}
+/// #1840: a neural startup whose Hub cannot be reached fails once, here,
+/// before any listener exists. This lives in `xerj-server` because that
+/// crate's default features include `neural`, which is what
+/// `cargo test --workspace` actually runs. `xerj-ai`'s own tests do not
+/// enable the feature unless asked.
+#[cfg(all(test, feature = "neural"))]
+mod neural_startup_tests {
+ use xerj_ai::neural::{startup_failure_message, NeuralConfig};
+ use xerj_ai::Embedder;
+
+ /// Closed port on loopback. Connection refused is local and immediate;
+ /// nothing is sent off the machine.
+ const UNREACHABLE_HUB: &str = "http://127.0.0.1:1";
+
+ #[test]
+ fn unreachable_hub_refuses_once_and_does_not_install_lexical() {
+ let cache = tempfile::tempdir().unwrap();
+ let cfg = NeuralConfig {
+ model_id: "sentence-transformers/all-MiniLM-L6-v2".into(),
+ cache_dir: Some(cache.path().to_path_buf()),
+ local_dir: None,
+ hub_endpoint: Some(UNREACHABLE_HUB.into()),
+ };
+ // `NeuralEmbedder` is not `Debug`, so `expect_err` cannot name the Ok side.
+ let error = match Embedder::preload_neural(&cfg) {
+ Ok(_) => panic!("a closed hub must not load a model"),
+ Err(error) => error,
+ };
+ let message = startup_failure_message(&cfg.model_id, &error);
+ assert!(
+ message.contains("127.0.0.1") || message.contains("Connection refused"),
+ "the cause must be the closed local hub, not a real download: {message}"
+ );
+ assert!(
+ message.contains("xerj --embed-mode lexical"),
+ "the error must give the lexical command: {message}"
+ );
+ assert!(
+ message.contains("embedding.local_model_dir"),
+ "the error must name the offline model directory: {message}"
+ );
+ assert!(
+ message.contains("HF_ENDPOINT"),
+ "the error must name the mirror variable: {message}"
+ );
+ assert!(
+ message.contains("Refusing to start"),
+ "the error must say the process will not start: {message}"
+ );
+ assert!(
+ !message
+ .to_ascii_lowercase()
+ .contains("falling back to lexical"),
+ "must not describe a lexical swap: {message}"
+ );
+ // The shared cell stays empty, so a later embed cannot pretend the
+ // model loaded. Preload's return type is the neural model itself:
+ // failure cannot hand back a lexical embedder.
+ assert!(
+ Embedder::preload_neural(&cfg).is_err(),
+ "a second attempt must still fail rather than reuse a lexical stand-in"
+ );
+ }
+
+ #[test]
+ fn missing_local_model_refuses_with_the_same_startup_error() {
+ let dir = tempfile::tempdir().unwrap();
+ let cfg = xerj_common::config::EmbeddingConfig {
+ mode: "neural".into(),
+ local_model_dir: dir.path().to_string_lossy().into_owned(),
+ ..Default::default()
+ };
+ let error = xerj_engine::index::ensure_neural_ready(&cfg)
+ .expect_err("an empty local model directory must refuse startup");
+ let message = error.to_string();
+ assert!(
+ message.contains("embedding.mode=neural"),
+ "must name the setting: {message}"
+ );
+ assert!(
+ message.contains("xerj --embed-mode lexical"),
+ "must give the lexical command: {message}"
+ );
+ assert!(
+ message.contains("model.safetensors") || message.contains("missing"),
+ "must name the missing local asset: {message}"
+ );
+ assert!(
+ !message.contains("127.0.0.1"),
+ "a local directory must not contact the hub: {message}"
+ );
+ }
+}
+
#[cfg(test)]
mod help_text_tests {
use super::{auth_disabled_line, help_text};
+ /// #1840: `--embed-mode neural` must say the model loads at startup and
+ /// that a failed download refuses to start, with the lexical command.
+ #[test]
+ fn neural_help_says_a_failed_download_refuses_to_start() {
+ let help = help_text(true);
+ let neural = help
+ .lines()
+ .position(|line| line.contains("neural built-in BERT"))
+ .expect("neural mode must be documented");
+ let window: String = help
+ .lines()
+ .skip(neural)
+ .take(12)
+ .collect::<Vec<_>>()
+ .join("\n");
+ assert!(
+ window.contains("loads at startup"),
+ "neural help must say the model loads at startup:\n{window}"
+ );
+ assert!(
+ window.contains("refuses to start"),
+ "neural help must say a failed load refuses to start:\n{window}"
+ );
+ assert!(
+ window.contains("xerj --embed-mode lexical"),
+ "neural help must give the lexical command:\n{window}"
+ );
+ assert!(
+ window.contains("embedding.local_model_dir"),
+ "neural help must name the offline model directory:\n{window}"
+ );
+ assert!(
+ window.contains("HF_ENDPOINT"),
+ "neural help must name the Hub mirror variable:\n{window}"
+ );
+ }
+
/// A loopback `--insecure` boot says nothing off this host can connect,
/// then used to say anyone on the network can write. Those two lines
/// cannot both be true.
@@ -3039,6 +3175,35 @@ async fn async_main() -> Result<()> {
);
}
+ // 3d. Explicit mode=neural loads the model before the data directory, the
+ // admin key, or any listener (#1840). A blocked Hub used to boot a
+ // node that then failed every ingest. Explicit proxy already refuses
+ // to start rather than substitute lexical vectors (#1189); neural does
+ // the same, because those vectors are not comparable. A slim build
+ // without the neural feature keeps its documented lexical fallback
+ // (identity reports lexical) — there is no model to load.
+ if cfg.embedding.mode.trim().eq_ignore_ascii_case("neural") {
+ #[cfg(feature = "neural")]
+ {
+ let embedding = cfg.embedding.clone();
+ info!(
+ model = %embedding.neural_model,
+ "embedding.mode=neural: loading the model before opening the data directory"
+ );
+ tokio::task::spawn_blocking(move || {
+ xerj_engine::index::ensure_neural_ready(&embedding)
+ })
+ .await
+ .context("neural embedder load task panicked")??;
+ info!("embedding.mode=neural: model loaded");
+ }
+ #[cfg(not(feature = "neural"))]
+ {
+ // Slim binary. `make_neural_embedder` warns and uses lexical when
+ // an index actually embeds. Do not refuse to start.
+ }
+ }
+
// 4. Data directory
std::fs::create_dir_all(&cfg.server.data_dir)
.with_context(|| format!("create data dir {}", cfg.server.data_dir))?;