AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
feat(ai): wire experimental ONNX through autoindex
bugerc14aa5cb1c3d
diff --git a/docs/EXPERIMENTAL_ONNX.md b/docs/EXPERIMENTAL_ONNX.mdindex cd0203894..67ef128d7 100644--- a/docs/EXPERIMENTAL_ONNX.md+++ b/docs/EXPERIMENTAL_ONNX.md@@ -1,63 +1,155 @@# Experimental ONNX embedding backend-Status: embedding-library prototype, **not a server or CLI mode**.+Status: end-to-end experimental server mode for GNU/Linux glibc builds. It is+off by default. XERJ's default remains the non-neural lexical feature hash;+`--embed-mode neural` remains the built-in Candle backend.-XERJ's supported built-in neural backend remains Candle and-`--embed-mode neural` continues to mean Candle. Default and musl builds do not-compile ONNX Runtime.+## Exact model contract-The opt-in `xerj-ai` feature provides the exact FP32 MiniLM inference path and-a bounded length-aware microbatch API:+This is not a generic ONNX-model interface. The supplied graph must be an FP32+`sentence-transformers/all-MiniLM-L6-v2`-compatible feature-extraction export:++- int64 inputs named `input_ids`, `attention_mask`, and `token_type_ids`;+- rank-3 token output named `last_hidden_state` or `token_embeddings`;+- output width exactly 384;+- a `tokenizer.json` from the same model/export.++XERJ applies attention-mask mean pooling and L2 normalization. It truncates at+512 tokens. A mapping that declares another vector width is rejected.++## Copy-paste workflow++Export a compatible model and inspect its interface:++```bash+python3 -m venv /tmp/xerj-onnx-export+/tmp/xerj-onnx-export/bin/pip install 'optimum[onnxruntime]' onnx+/tmp/xerj-onnx-export/bin/optimum-cli export onnx \+ --model sentence-transformers/all-MiniLM-L6-v2 \+ --task feature-extraction \+ /tmp/xerj-minilm-onnx++/tmp/xerj-onnx-export/bin/python - <<'PY'+import onnx+p = "/tmp/xerj-minilm-onnx/model.onnx"+m = onnx.load(p)+print("inputs:", [(v.name, [d.dim_value for d in v.type.tensor_type.shape.dim])+ for v in m.graph.input])+print("outputs:", [(v.name, [d.dim_value for d in v.type.tensor_type.shape.dim])+ for v in m.graph.output])+PY++sha256sum \+ /tmp/xerj-minilm-onnx/model.onnx \+ /tmp/xerj-minilm-onnx/tokenizer.json+```++Confirm the three required input names, a supported output name, and width 384.+Export tools can change their file layout or output names; XERJ deliberately+fails instead of guessing.++Build the opt-in server:```bashcd engine-cargo run --release -p xerj-ai --features onnx-experimental \- --example onnx_experimental -- /path/model.onnx /path/tokenizer.json+cargo build --release -j 32 -p xerj-server --features onnx-experimental+```++Start it with explicit local assets:++```bash+target/release/xerj \+ --insecure \+ --data-dir /tmp/xerj-onnx-data \+ --embed-mode onnx-experimental \+ --onnx-model /tmp/xerj-minilm-onnx/model.onnx \+ --onnx-tokenizer /tmp/xerj-minilm-onnx/tokenizer.json```-The caller supplies an ONNX model with `input_ids`, `attention_mask`, and-`token_type_ids` inputs and a `[batch, sequence, 384]` `last_hidden_state`-output. The current verified artifact is the official FP32-`sentence-transformers/all-MiniLM-L6-v2` ONNX export.+No model is downloaded at server startup. Incorrect paths, an incompatible+build, and invalid admission limits fail startup with a corrective error. The+first real semantic inference prints one concise activation message containing+the verified model and tokenizer hashes. ONNX Runtime messages below warning+are hidden by default; use `XERJ_ONNX_LOG=info` or `verbose` for diagnosis.++In another terminal, index a corpus:++```bash+target/release/xerj autoindex /path/to/corpus \+ --url http://localhost:9200 \+ --prefix finance-onnx \+ --fresh++target/release/xerj autoindex map \+ --url http://localhost:9200 \+ --prefix finance-onnx+```++ONNX runs only for fields inferred as `semantic_text`, normally a sufficiently+long body field. Short or structured datasets may infer none. Check+`autoindex --dry-run`, the data map's `semantic_field`, and the server's+activation log before claiming an ONNX result.++Query the mapped semantic field:++```bash+curl -s http://localhost:9200/finance-onnx-*/_search \+ -H 'content-type: application/json' \+ -d '{"query":{"semantic":{"field":"body","query":"Which quarter had the largest operating-margin decline?","k":10}},"size":5}'+```++## Restart and vector-space safety++Every ONNX semantic index stores `embedding_identity.json` with SHA-256 model+and tokenizer fingerprints plus dimensions, pooling, and token limit. Restart+with the same assets. XERJ refuses:++- another model or tokenizer at restart;+- switching an ONNX-pinned index to another backend;+- enabling ONNX in place on a populated marker-less semantic index;+- caller-supplied derived vectors whose identity cannot be verified.++The error tells the operator to restore the original assets or re-run+autoindex with `--fresh` and a new prefix. XERJ never silently mixes vector+spaces.++## Admission and errors-Safety properties:+One safe ONNX Runtime session is shared per complete model configuration.+Length-aware microbatches are serialized through it. Before model loading or+tokenization, shared admission enforces:-- safe `ort` API only;-- one session behind a mutex;-- bounded pending inputs, documents per batch, and padded token slots;-- length-aware grouping;-- exact restoration to caller input order;-- clear backpressure and invalid-budget errors.+- `onnx_max_inflight_calls` (default 8);+- `onnx_max_input_bytes_per_call` (default 8 MiB);+- `onnx_max_inflight_input_bytes` (default 32 MiB);+- `onnx_max_pending`, microbatch size, and padded-token limits.-## Measured prototype results+Overload returns retryable HTTP 429. Autoindex does not call that junk and does+not journal the affected source file complete; correct the pressure/config and+rerun the same command to resume.-On the controlled 128-document mixed-length benchmark, exact FP32 ONNX plus-the retained scheduler processed 116.671 documents/s versus 9.045 documents/s-for Candle using the identical precomputed length-aware batch plan: **12.90x-at the `xerj-ai` embedding layer**. The median minimum same-document-Candle/ONNX cosine was `0.9999991655`.+These settings are available under `[embedding]` in the TOML configuration.+The server validates impossible or zero admission limits before serving.-This is not a claim of 12.90x end-to-end XERJ indexing. The benchmark excludes-document extraction, HTTP handling, persistence, and HNSW construction. ONNX-also used 16 intra-op threads; the measured CPU-efficiency improvement was-2.92x, smaller than the wall-time result. A server-level comparison is required-before publishing an indexing-throughput claim.+## Measured performance and limits-Measured stripped full-server binary sizes were:+On the controlled 128-document mixed-length benchmark, optimized Candle+processed 9.045 documents/s and FP32 ONNX with the retained scheduler processed+116.671 documents/s: **12.90x at the embedding layer**. The median minimum+same-document Candle/ONNX cosine was `0.9999991655`; output ordering was+preserved. CPU-efficiency improved 2.92x.-- current Candle build: 36.06 MiB;-- Candle plus ONNX: 54.81 MiB (**+18.75 MiB / +52.0%**);-- ONNX-only experimental build: 52.49 MiB (**+16.43 MiB / +45.6%**).+This is not a 12.90x end-to-end indexing claim. It excludes extraction, HTTP,+lexical indexing, persistence, HNSW construction, and contention. Full+FinanceBench autoindex time must be measured, not projected.-The roughly 90 MiB ONNX model is a separate runtime asset comparable to the-existing Candle safetensors asset. The much larger static ONNX Runtime archive-downloaded while compiling is not added wholesale to the final binary.+Measured stripped server binaries:-The bundled `ort` binaries do not cover XERJ's musl release targets. Default-and musl builds therefore remain Candle-only; production adoption requires a-reproducible musl packaging strategy or an explicitly narrower target matrix.+- Candle: 36.06 MiB;+- Candle plus ONNX: 54.81 MiB (+18.75 MiB / +52.0%);+- ONNX-only experimental build: 52.49 MiB (+16.43 MiB / +45.6%).-This is deliberately not exposed in `xerj --help`. Server integration still-needs model acquisition, backend fingerprinting/reindex compatibility,-platform packaging, lifecycle metrics, cancellation, and end-to-end retrieval-quality gates. Do not describe XERJ as supporting `--embed-mode onnx`.+The approximately 90 MiB model is a separate runtime asset. Bundled `ort`+binaries do not cover XERJ's musl release targets, so standard musl releases+remain Candle-only. ONNX production adoption still needs a supported target+matrix, end-to-end quality gates, and full-corpus throughput/resource results.diff --git a/engine/Cargo.lock b/engine/Cargo.lockindex 7bab24fa1..c50e18b30 100644--- a/engine/Cargo.lock+++ b/engine/Cargo.lock@@ -5140,6 +5140,7 @@ dependencies = ["reqwest","serde","serde_json",+ "sha2","thiserror 2.0.18","tokenizers","tokio",diff --git a/engine/crates/xerj-ai/Cargo.toml b/engine/crates/xerj-ai/Cargo.tomlindex 63db37e5b..e46c00452 100644--- a/engine/crates/xerj-ai/Cargo.toml+++ b/engine/crates/xerj-ai/Cargo.toml@@ -33,13 +33,15 @@ ort = { version = "2.0.0-rc.12", optional = true, default-features = false, feat"std", "ndarray", "tracing", "download-binaries", "tls-rustls","copy-dylibs", "api-24",] }+sha2 = { version = "0.10", optional = true }[features]default = []# Bundles the neural BERT embedder. Adds ~candle + tokenizers to the build.neural = ["dep:candle-core", "dep:candle-nn", "dep:candle-transformers", "dep:tokenizers", "dep:hf-hub"]-# Embedding-layer prototype only; this does not add a server `--embed-mode`.-onnx-experimental = ["dep:ort", "dep:tokenizers"]+# Experimental local ONNX backend; xerj-server exposes it only when the+# feature is propagated and explicitly selected at runtime.+onnx-experimental = ["dep:ort", "dep:tokenizers", "dep:sha2"][dev-dependencies]tokio = { version = "1", features = ["full"] }diff --git a/engine/crates/xerj-ai/src/embedder.rs b/engine/crates/xerj-ai/src/embedder.rsindex 4f18d1e93..8838eada1 100644--- a/engine/crates/xerj-ai/src/embedder.rs+++ b/engine/crates/xerj-ai/src/embedder.rs@@ -1,6 +1,6 @@//! Unified embedding backend.//!-//! XERJ embeds `semantic_text` fields through one of three interchangeable+//! XERJ embeds `semantic_text` fields through one of four interchangeable//! backends, all behind a single [`Embedder`] handle so the engine's ingest//! and query paths never branch on the backend://!@@ -14,17 +14,30 @@//! ([`crate::neural`]), running in-process via `candle`. Compiled only//! under the `neural` cargo feature; the model is loaded lazily on first//! use (download-on-first-run) so startup stays instant.+//! * [`Embedder::Onnx`] — the experimental in-process ONNX Runtime backend.+//! It is compiled only under `onnx-experimental`, requires explicit model+//! and tokenizer paths, and loads the model lazily on first use.//!-//! [`Embedder::is_active`] distinguishes a *real* embedder (proxy or neural)-//! from the lexical fallback — the query path only auto-embeds against the-//! lexical backend for `semantic_text` fields, matching how they were-//! embedded at ingest.+//! [`Embedder::is_active`] distinguishes a neural/proxy/ONNX backend from the+//! lexical fallback. The query path nevertheless auto-embeds `semantic_text`+//! fields with whichever backend indexed them, including lexical, so ingest+//! and query vectors always use the same embedding identity.use anyhow::{anyhow, Result};use crate::embed::EmbeddingProxy;use crate::local::{local_embed, DEFAULT_DIMS};+#[cfg(feature = "onnx-experimental")]+type OnnxCell = tokio::sync::OnceCell<std::sync::Arc<crate::onnx::OnnxEmbedder>>;++#[cfg(feature = "onnx-experimental")]+struct OnnxShared {+ cell: OnnxCell,+ calls: std::sync::Arc<tokio::sync::Semaphore>,+ bytes: std::sync::Arc<tokio::sync::Semaphore>,+}+#[cfg(feature = "neural")]type NeuralCell = tokio::sync::OnceCell<std::sync::Arc<crate::neural::NeuralEmbedder>>;@@ -66,6 +79,9 @@ pub enum Embedder {/// Built-in neural BERT embedder (candle). Lazily loaded on first use.#[cfg(feature = "neural")]Neural(NeuralHandle),+ /// Experimental local ONNX Runtime sentence encoder.+ #[cfg(feature = "onnx-experimental")]+ Onnx(OnnxHandle),}impl Embedder {@@ -85,10 +101,16 @@ impl Embedder {Embedder::Neural(NeuralHandle::new(cfg))}- /// `true` when a *real* embedder (neural or external proxy) is configured;- /// `false` for the lexical fallback. The query path uses this to decide- /// whether to embed arbitrary query text (active) or restrict to- /// `semantic_text` fields embedded the same lexical way at ingest.+ #[cfg(feature = "onnx-experimental")]+ pub fn onnx(cfg: OnnxConfig) -> Self {+ Embedder::Onnx(OnnxHandle::new(cfg))+ }++ /// `true` when a Candle neural, experimental ONNX, or external proxy+ /// embedder is configured; `false` for the lexical fallback. The query+ /// path uses this to decide whether to embed arbitrary query text (active)+ /// or restrict to `semantic_text` fields embedded the same lexical way at+ /// ingest.pub fn is_active(&self) -> bool {!matches!(self, Embedder::Lexical)}@@ -100,6 +122,8 @@ impl Embedder {Embedder::Proxy(_) => "external proxy (OpenAI-compatible /v1/embeddings)",#[cfg(feature = "neural")]Embedder::Neural(_) => "neural BERT (built-in, candle)",+ #[cfg(feature = "onnx-experimental")]+ Embedder::Onnx(_) => "neural BERT (experimental ONNX Runtime)",}}@@ -113,8 +137,305 @@ impl Embedder {.map_err(|e| anyhow!("embedding proxy failed: {e}")),#[cfg(feature = "neural")]Embedder::Neural(handle) => handle.embed_batch(texts).await,+ #[cfg(feature = "onnx-experimental")]+ Embedder::Onnx(handle) => handle.embed_batch(texts).await,+ }+ }+}++#[cfg(feature = "onnx-experimental")]+#[derive(Clone, Debug, Hash, PartialEq, Eq)]+pub struct OnnxConfig {+ pub model_path: std::path::PathBuf,+ pub tokenizer_path: std::path::PathBuf,+ /// Content fingerprints are part of the shared-session cache key and are+ /// rechecked immediately before load, preventing same-path asset swaps.+ pub model_sha256: String,+ pub tokenizer_sha256: String,+ pub intra_threads: usize,+ pub microbatch: crate::onnx::MicrobatchConfig,+ pub max_inflight_calls: usize,+ pub max_input_bytes_per_call: usize,+ pub max_inflight_input_bytes: usize,+}++#[cfg(feature = "onnx-experimental")]+#[derive(Debug, thiserror::Error)]+#[error("{reason}")]+pub struct OnnxAdmissionError {+ pub reason: String,+}++#[cfg(feature = "onnx-experimental")]+pub struct OnnxHandle {+ cfg: OnnxConfig,+ shared: std::sync::Arc<OnnxShared>,+}++#[cfg(feature = "onnx-experimental")]+impl OnnxHandle {+ fn new(cfg: OnnxConfig) -> Self {+ use std::collections::HashMap;+ use std::sync::{Mutex, OnceLock, Weak};+ static CELLS: OnceLock<Mutex<HashMap<OnnxConfig, Weak<OnnxShared>>>> = OnceLock::new();+ let mut cells = CELLS+ .get_or_init(|| Mutex::new(HashMap::new()))+ .lock()+ .unwrap_or_else(|poisoned| poisoned.into_inner());+ if let Some(shared) = cells.get(&cfg).and_then(Weak::upgrade) {+ return Self { cfg, shared };+ }+ cells.retain(|_, shared| shared.strong_count() > 0);+ let shared = std::sync::Arc::new(OnnxShared {+ cell: OnnxCell::new(),+ calls: std::sync::Arc::new(tokio::sync::Semaphore::new(cfg.max_inflight_calls.max(1))),+ bytes: std::sync::Arc::new(tokio::sync::Semaphore::new(+ cfg.max_inflight_input_bytes.max(1),+ )),+ });+ cells.insert(cfg.clone(), std::sync::Arc::downgrade(&shared));+ Self { cfg, shared }+ }++ async fn get(&self) -> Result<std::sync::Arc<crate::onnx::OnnxEmbedder>> {+ self.shared+ .cell+ .get_or_try_init(|| async {+ let cfg = self.cfg.clone();+ let model = tokio::task::spawn_blocking(move || {+ let model_bytes = std::fs::read(&cfg.model_path).map_err(|e| {+ anyhow!("read ONNX model {}: {e}", cfg.model_path.display())+ })?;+ let tokenizer_bytes = std::fs::read(&cfg.tokenizer_path).map_err(|e| {+ anyhow!("read ONNX tokenizer {}: {e}", cfg.tokenizer_path.display())+ })?;+ let actual_model = sha256_bytes(&model_bytes);+ let actual_tokenizer = sha256_bytes(&tokenizer_bytes);+ if actual_model != cfg.model_sha256 || actual_tokenizer != cfg.tokenizer_sha256+ {+ return Err(anyhow!(+ "ONNX assets changed after configuration; refusing to load a \+ different vector space from the same path (model expected {}, \+ actual {}; tokenizer expected {}, actual {}). Restart with the \+ intended assets or reindex under a new prefix",+ cfg.model_sha256,+ actual_model,+ cfg.tokenizer_sha256,+ actual_tokenizer+ ));+ }+ let embedder = crate::onnx::OnnxEmbedder::load_bytes(+ &model_bytes,+ &tokenizer_bytes,+ cfg.intra_threads,+ )?;+ tracing::info!(+ model_sha256 = %cfg.model_sha256,+ tokenizer_sha256 = %cfg.tokenizer_sha256,+ dimensions = crate::onnx::DIMS,+ "experimental ONNX embedding backend active; first semantic inference loaded the verified model"+ );+ Ok::<_, anyhow::Error>(embedder)+ })+ .await+ .map_err(|e| anyhow!("ONNX model load task panicked: {e}"))??;+ Ok::<_, anyhow::Error>(std::sync::Arc::new(model))+ })+ .await+ .cloned()+ }++ async fn embed_batch(&self, texts: Vec<String>) -> Result<Vec<Vec<f32>>> {+ if texts.is_empty() {+ return Ok(Vec::new());+ }+ if texts.len() > self.cfg.microbatch.max_pending {+ return Err(anyhow::Error::new(OnnxAdmissionError {+ reason: format!(+ "ONNX request rejected before tokenization: {} texts exceed max_pending={}; \+ split the request and retry",+ texts.len(),+ self.cfg.microbatch.max_pending+ ),+ }));+ }+ let input_bytes = texts+ .iter()+ .try_fold(0usize, |total, text| total.checked_add(text.len()))+ .ok_or_else(|| {+ anyhow::Error::new(OnnxAdmissionError {+ reason: "ONNX input byte count overflowed; split the request".into(),+ })+ })?;+ let (_call_permit, _byte_permits) = self.try_admit(input_bytes)?;+ let model = self.get().await?;+ let limits = self.cfg.microbatch;+ tokio::task::spawn_blocking(move || model.embed_scheduled_blocking(&texts, limits))+ .await+ .map_err(|e| anyhow!("ONNX embed task panicked: {e}"))?+ }++ fn try_admit(+ &self,+ input_bytes: usize,+ ) -> Result<(+ tokio::sync::OwnedSemaphorePermit,+ tokio::sync::OwnedSemaphorePermit,+ )> {+ if input_bytes > self.cfg.max_input_bytes_per_call {+ return Err(anyhow::Error::new(OnnxAdmissionError {+ reason: format!(+ "ONNX request rejected before tokenization: input is {input_bytes} bytes, \+ per-call limit is {} bytes; split the request and retry",+ self.cfg.max_input_bytes_per_call+ ),+ }));+ }+ let byte_permits = u32::try_from(input_bytes.max(1)).map_err(|_| {+ anyhow::Error::new(OnnxAdmissionError {+ reason: format!(+ "ONNX request rejected before tokenization: {input_bytes} input bytes \+ exceed the semaphore permit range; split the request"+ ),+ })+ })?;+ let call = self+ .shared+ .calls+ .clone()+ .try_acquire_owned()+ .map_err(|_| {+ anyhow::Error::new(OnnxAdmissionError {+ reason: format!(+ "ONNX embedding admission full: {} calls are already admitted \+ (loading, running, or awaiting the serialized session); retry with backoff",+ self.cfg.max_inflight_calls+ ),+ })+ })?;+ let bytes = self+ .shared+ .bytes+ .clone()+ .try_acquire_many_owned(byte_permits)+ .map_err(|_| {+ anyhow::Error::new(OnnxAdmissionError {+ reason: format!(+ "ONNX embedding byte budget full: request needs {input_bytes} bytes, \+ global in-flight limit is {} bytes; retry with backoff",+ self.cfg.max_inflight_input_bytes+ ),+ })+ })?;+ Ok((call, bytes))+ }+}++#[cfg(feature = "onnx-experimental")]+fn sha256_bytes(bytes: &[u8]) -> String {+ use sha2::{Digest, Sha256};+ let mut hasher = Sha256::new();+ hasher.update(bytes);+ format!("{:x}", hasher.finalize())+}++#[cfg(all(test, feature = "onnx-experimental"))]+mod onnx_handle_tests {+ use super::{OnnxConfig, OnnxHandle};+ use crate::onnx::MicrobatchConfig;+ use std::path::PathBuf;+ use std::sync::Arc;++ fn cfg(model_sha256: &str) -> OnnxConfig {+ OnnxConfig {+ model_path: PathBuf::from("/tmp/model.onnx"),+ tokenizer_path: PathBuf::from("/tmp/tokenizer.json"),+ model_sha256: model_sha256.into(),+ tokenizer_sha256: "tokenizer-hash".into(),+ intra_threads: 4,+ microbatch: MicrobatchConfig::default(),+ max_inflight_calls: 2,+ max_input_bytes_per_call: 10,+ max_inflight_input_bytes: 12,}}++ #[test]+ fn same_paths_with_different_content_hashes_never_share_session_cell() {+ let first = OnnxHandle::new(cfg("model-a"));+ let second = OnnxHandle::new(cfg("model-b"));+ assert!(!Arc::ptr_eq(&first.shared, &second.shared));+ }++ #[test]+ fn global_admission_caps_calls_and_releases_permits() {+ let handle = OnnxHandle::new(cfg("admission-calls"));+ let first = handle.try_admit(1).unwrap();+ let second = handle.try_admit(1).unwrap();+ let error = handle.try_admit(1).unwrap_err().to_string();+ assert!(error.contains("admission full"), "{error}");+ drop(first);+ assert!(handle.try_admit(1).is_ok());+ drop(second);+ }++ #[test]+ fn concurrent_handles_share_global_call_cap() {+ let config = cfg("admission-concurrent");+ let first = OnnxHandle::new(config.clone());+ let second = OnnxHandle::new(config.clone());+ let rejected = OnnxHandle::new(config);+ let (ready_tx, ready_rx) = std::sync::mpsc::channel();+ let release = Arc::new(std::sync::Barrier::new(3));+ std::thread::scope(|scope| {+ for handle in [first, second] {+ let ready_tx = ready_tx.clone();+ let release = Arc::clone(&release);+ scope.spawn(move || {+ let _permit = handle.try_admit(1).unwrap();+ ready_tx.send(()).unwrap();+ release.wait();+ });+ }+ ready_rx.recv().unwrap();+ ready_rx.recv().unwrap();+ let error = rejected.try_admit(1).unwrap_err().to_string();+ assert!(error.contains("admission full"), "{error}");+ release.wait();+ });+ assert!(rejected.try_admit(1).is_ok(), "permits must release");+ }++ #[test]+ fn global_byte_budget_and_per_call_limit_are_enforced_before_work() {+ let handle = OnnxHandle::new(cfg("admission-bytes"));+ let held = handle.try_admit(8).unwrap();+ let error = handle.try_admit(5).unwrap_err().to_string();+ assert!(error.contains("byte budget full"), "{error}");+ drop(held);+ assert!(handle.try_admit(5).is_ok());++ let error = handle.try_admit(11).unwrap_err().to_string();+ assert!(error.contains("per-call limit"), "{error}");+ }++ #[tokio::test]+ async fn document_cap_rejects_before_model_load_or_tokenization() {+ let mut config = cfg("admission-docs");+ config.microbatch.max_pending = 1;+ let handle = OnnxHandle::new(config);+ let error = handle+ .embed_batch(vec!["one".into(), "two".into()])+ .await+ .unwrap_err()+ .to_string();+ assert!(error.contains("before tokenization"), "{error}");+ assert!(error.contains("max_pending=1"), "{error}");+ assert!(+ handle.shared.cell.get().is_none(),+ "model must remain unloaded"+ );+ }}/// Lazily-loaded neural backend. The heavy model is loaded once, on the firstdiff --git a/engine/crates/xerj-ai/src/lib.rs b/engine/crates/xerj-ai/src/lib.rsindex 49150bd57..9e3e3c134 100644--- a/engine/crates/xerj-ai/src/lib.rs+++ b/engine/crates/xerj-ai/src/lib.rs@@ -3,12 +3,13 @@//! AI-native features for the xerj search engine.//!//! Provides:-//! - [`embedder`] — Unified backend handle (lexical / proxy / neural) used by the engine+//! - [`embedder`] — Unified backend handle (lexical / proxy / Candle neural /+//! experimental ONNX) used by the engine//! - [`embed`] — Embedding proxy: async HTTP client for OpenAI-compatible embedding APIs//! - [`local`] — Built-in zero-config deterministic text embedder (feature hashing)//! - [`neural`] — Built-in neural BERT sentence embedder via candle (feature `neural`)-//! - `onnx` — Experimental FP32 ONNX backend (feature `onnx-experimental`);-//! not wired into the server+//! - `onnx` — Experimental MiniLM-compatible FP32 ONNX backend+//! (feature `onnx-experimental`; server feature + explicit runtime selection required)//! - [`chunker`] — Text chunking with sentence-aware splitting and overlap//! - [`memory`] — Agent memory: semantic dedup + recency-blended recalldiff --git a/engine/crates/xerj-ai/src/onnx.rs b/engine/crates/xerj-ai/src/onnx.rsindex c721f3dc3..65af3fbeb 100644--- a/engine/crates/xerj-ai/src/onnx.rs+++ b/engine/crates/xerj-ai/src/onnx.rs@@ -1,7 +1,15 @@//! Experimental FP32 ONNX Runtime MiniLM backend.//!-//! This module is an embedding-layer prototype. It is not wired to XERJ's-//! server or CLI. The safe `ort` API requires mutable session access, so one+//! This backend is available end-to-end only in builds with the+//! `onnx-experimental` feature and must be explicitly selected with local+//! model/tokenizer paths. It is deliberately not a generic ONNX interface: the+//! graph must be an FP32 all-MiniLM-L6-v2-compatible encoder with int64 inputs+//! named `input_ids`, `attention_mask`, and `token_type_ids`, and a rank-3,+//! width-384 output named `last_hidden_state` or `token_embeddings`. The+//! tokenizer.json must come from the same model/export. XERJ attention-mask+//! mean-pools and L2-normalizes the token output.+//!+//! The safe `ort` API requires mutable session access, so one//! session is serialized behind a mutex and aggregate throughput comes from//! bounded, length-aware microbatches rather than unsafe concurrent `Run`.@@ -16,7 +24,7 @@ pub const MAX_TOKENS: usize = 512;pub const DIMS: usize = 384;/// Bounds for one offline scheduling window.-#[derive(Clone, Copy, Debug, PartialEq, Eq)]+#[derive(Clone, Copy, Debug, Hash, PartialEq, Eq)]pub struct MicrobatchConfig {/// Maximum inputs accepted in one call. Larger calls receive a clear/// backpressure error instead of allocating an unbounded queue.@@ -58,8 +66,20 @@ pub struct OnnxEmbedder {impl OnnxEmbedder {pub fn load(model_path: &Path, tokenizer_path: &Path, intra_threads: usize) -> Result<Self> {- let mut tokenizer = Tokenizer::from_file(tokenizer_path)- .map_err(|e| anyhow!("load tokenizer {}: {e}", tokenizer_path.display()))?;+ let model = std::fs::read(model_path)+ .with_context(|| format!("read ONNX model {}", model_path.display()))?;+ let tokenizer = std::fs::read(tokenizer_path)+ .with_context(|| format!("read tokenizer {}", tokenizer_path.display()))?;+ Self::load_bytes(&model, &tokenizer, intra_threads)+ }++ pub(crate) fn load_bytes(+ model_bytes: &[u8],+ tokenizer_bytes: &[u8],+ intra_threads: usize,+ ) -> Result<Self> {+ let mut tokenizer = Tokenizer::from_bytes(tokenizer_bytes)+ .map_err(|e| anyhow!("load tokenizer.json bytes: {e}"))?;tokenizer.with_padding(Some(PaddingParams {strategy: PaddingStrategy::BatchLongest,..Default::default()@@ -71,16 +91,29 @@ impl OnnxEmbedder {})).map_err(|e| anyhow!("configure tokenizer truncation: {e}"))?;+ let ort_log_level = match std::env::var("XERJ_ONNX_LOG")+ .unwrap_or_default()+ .to_ascii_lowercase()+ .as_str()+ {+ "verbose" | "trace" => ort::logging::LogLevel::Verbose,+ "info" | "debug" => ort::logging::LogLevel::Info,+ "error" => ort::logging::LogLevel::Error,+ "fatal" => ort::logging::LogLevel::Fatal,+ _ => ort::logging::LogLevel::Warning,+ };let session = Session::builder().map_err(|e| anyhow!("create ONNX Runtime session builder: {e}"))?+ .with_log_level(ort_log_level)+ .map_err(|e| anyhow!("configure ONNX Runtime log level: {e}"))?.with_optimization_level(GraphOptimizationLevel::Level3).map_err(|e| anyhow!("configure ONNX graph optimizations: {e}"))?.with_config_entry("session.intra_op.allow_spinning", "0").map_err(|e| anyhow!("disable ONNX intra-op spinning: {e}"))?.with_intra_threads(intra_threads.max(1)).map_err(|e| anyhow!("configure ONNX intra-op threads: {e}"))?- .commit_from_file(model_path)- .with_context(|| format!("load ONNX model {}", model_path.display()))?;+ .commit_from_memory(model_bytes)+ .context("load ONNX model bytes")?;Ok(Self {session: Mutex::new(session),tokenizer,diff --git a/engine/crates/xerj-autoindex/src/catalog.rs b/engine/crates/xerj-autoindex/src/catalog.rsindex aade95dfa..e64c046d2 100644--- a/engine/crates/xerj-autoindex/src/catalog.rs+++ b/engine/crates/xerj-autoindex/src/catalog.rs@@ -53,7 +53,7 @@ pub fn catalog_mapping() -> Value {pub const GOTCHAS: &[&str] = &["hybrid search: use {\"query\":{\"hybrid\":{\"queries\":[…]}}} ONLY — retriever.rrf is a silent stub and rank.rrf is ignored on this engine",- "semantic_text fields are embedded server-side: the DEFAULT is the built-in LEXICAL feature-hash embedder (384-dim hybrid lexical+vector, NOT neural) — start the server with `--embed-mode neural` (built-in BERT) or `--embed-mode proxy` for true neural semantics",+ "semantic_text fields are embedded server-side: the DEFAULT is the built-in LEXICAL feature-hash embedder (384-dim hybrid lexical+vector, NOT neural) — start the server with `--embed-mode neural` (built-in Candle BERT), `--embed-mode proxy`, or an ONNX-enabled build with `--embed-mode onnx-experimental --onnx-model … --onnx-tokenizer …` for neural semantics; ONNX runs only when this map shows a semantic_field, and its first real inference is confirmed by the server activation log","semantic queries ignore _source filtering and return the ~8KB *_vector field in _source — strip client-side","exact filters use TOP-LEVEL keyword fields (term on .keyword subfields returns 0 hits on this engine)","all dates are normalized to RFC3339 UTC millis; mappings use strict_date_optional_time||epoch_millis",@@ -180,7 +180,7 @@ pub fn build_sample_queries(pd: &PlanDataset, correlations: &[KeyCorr]) -> Vec<Vif let Some(sf) = &pd.semantic_field {out.push(json!({"class": "hybrid_lexical_vector",- "title": format!("Hybrid lexical+vector (RRF) on {sf} — embedder set server-side (lexical by default; neural/proxy if configured)"),+ "title": format!("Hybrid lexical+vector (RRF) on {sf} — embedder set server-side (lexical by default; Candle neural, proxy, or experimental ONNX if configured)"),"request": format!("POST /{}/_search", pd.index),"body": {"query": {"hybrid": {"queries": [{"query": {"match": {(sf.clone()): word.clone()}}, "weight": 1},@@ -296,7 +296,7 @@ pub fn render_map(s.push_str(&format!("### `{}`\n\n", g("index_name")));if let Some(sem) = d.get("semantic_field").filter(|v| v.is_string()) {s.push_str(&format!(- "semantic body field: `{}` (hybrid lexical+vector; embedder set server-side — lexical by default, neural/proxy if configured)\n\n",+ "semantic body field: `{}` (hybrid lexical+vector; embedder set server-side — lexical by default, Candle neural, proxy, or experimental ONNX if configured)\n\n",pretty_val(sem)));}diff --git a/engine/crates/xerj-autoindex/src/cli.rs b/engine/crates/xerj-autoindex/src/cli.rsindex 87263d5be..c046d6fe6 100644--- a/engine/crates/xerj-autoindex/src/cli.rs+++ b/engine/crates/xerj-autoindex/src/cli.rs@@ -73,6 +73,15 @@ pub fn print_help() {--dataset <SLUG> (map) show a single dataset\n\--help, -h this help\n\\n\+ EMBEDDINGS:\n\+ autoindex sends semantic_text to the running server; it does not choose the\n\+ server's embedding backend. The default is lexical (not neural). For the\n\+ experimental ONNX backend, start xerj with `--embed-mode onnx-experimental\n\+ --onnx-model MODEL.onnx --onnx-tokenizer tokenizer.json`, then run autoindex.\n\+ ONNX runs only for fields inferred as semantic_text (normally long body text;\n\+ short/structured datasets may infer none). Use --dry-run or `autoindex map` to\n\+ confirm a semantic field before attributing an indexing result to embeddings.\n\+ \n\EXIT CODES: 0 complete; 3 completed-with-junk (junk recorded, never fatal);\n\2 usage; 1 endpoint unreachable / journal-config mismatch\n");diff --git a/engine/crates/xerj-autoindex/src/esclient.rs b/engine/crates/xerj-autoindex/src/esclient.rsindex 1eb2ff1a6..3b4273345 100644--- a/engine/crates/xerj-autoindex/src/esclient.rs+++ b/engine/crates/xerj-autoindex/src/esclient.rs@@ -14,7 +14,11 @@ pub struct Es {pub struct BulkOutcome {pub item_errors: u64,+ /// Per-item 5xx/429 failures are backend/admission failures, not bad source+ /// records. Callers must not journal the source file complete.+ pub server_errors: u64,pub first_error: Option<String>,+ pub first_server_error: Option<String>,}impl Es {@@ -112,7 +116,9 @@ impl Es {}let v: Value = resp.json().context("parse bulk response")?;let mut item_errors = 0u64;+ let mut server_errors = 0u64;let mut first_error = None;+ let mut first_server_error = None;if v.get("errors").and_then(|e| e.as_bool()).unwrap_or(false) {if let Some(items) = v.get("items").and_then(|i| i.as_array()) {for it in items {@@ -123,6 +129,16 @@ impl Es {if let Some(op) = op {if op.get("error").is_some() {item_errors += 1;+ let item_status =+ op.get("status").and_then(Value::as_u64).unwrap_or(500);+ if item_status == 429 || item_status >= 500 {+ server_errors += 1;+ if first_server_error.is_none() {+ first_server_error = Some(+ op["error"].to_string().chars().take(500).collect(),+ );+ }+ }if first_error.is_none() {first_error = Some(op["error"].to_string().chars().take(300).collect(),@@ -135,7 +151,9 @@ impl Es {}Ok(BulkOutcome {item_errors,+ server_errors,first_error,+ first_server_error,})},)diff --git a/engine/crates/xerj-autoindex/src/infer/mod.rs b/engine/crates/xerj-autoindex/src/infer/mod.rsindex e187dcf86..4f22ca52d 100644--- a/engine/crates/xerj-autoindex/src/infer/mod.rs+++ b/engine/crates/xerj-autoindex/src/infer/mod.rs@@ -515,7 +515,7 @@ pub fn infer_fields(let best = specs.iter().enumerate()- .filter(|(i, s)| {+ .filter(|(_i, s)| {s.es_type == "text"&& s.avg_len >= 80.0&& fields@@ -530,7 +530,7 @@ pub fn infer_fields(specs[i].notes.push(format!("hybrid lexical+vector body — elected because it looks like natural language \(word_ratio {:.2}, {:.1} tokens/value); embedded server-side (lexical by \- default, neural/proxy if configured)",+ default; Candle neural, proxy, or experimental ONNX if configured)",a.map(|x| x.word_ratio()).unwrap_or(0.0),a.map(|x| x.mean_tokens()).unwrap_or(0.0),));diff --git a/engine/crates/xerj-autoindex/src/lib.rs b/engine/crates/xerj-autoindex/src/lib.rsindex fb86a96b6..643c6f6d7 100644--- a/engine/crates/xerj-autoindex/src/lib.rs+++ b/engine/crates/xerj-autoindex/src/lib.rs@@ -491,6 +491,19 @@ fn run_index(cfg: IndexCfg) -> Result<i32> {if buf.len() >= bulk_cut || buf_docs >= 5000 {match es.bulk(std::mem::take(&mut buf)) {Ok(o) => {+ if o.server_errors > 0 {+ send_err = Some(format!(+ "bulk backend failed for {} item(s): {}. \+ Source file was not journaled complete; fix \+ the server/embedding configuration and rerun \+ autoindex to resume",+ o.server_errors,+ o.first_server_error+ .as_deref()+ .unwrap_or("unknown server error")+ ));+ return false;+ }if o.item_errors > 0 {junk_records.fetch_add(o.item_errors, Ordering::Relaxed);@@ -533,8 +546,22 @@ fn run_index(cfg: IndexCfg) -> Result<i32> {if !buf.is_empty() && send_err.is_none() {match es.bulk(std::mem::take(&mut buf)) {Ok(o) => {+ if o.server_errors > 0 {+ send_err = Some(format!(+ "bulk backend failed for {} item(s): {}. Source file \+ was not journaled complete; fix the server/embedding \+ configuration and rerun autoindex to resume",+ o.server_errors,+ o.first_server_error+ .as_deref()+ .unwrap_or("unknown server error")+ ));+ }if o.item_errors > 0 {- junk_records.fetch_add(o.item_errors, Ordering::Relaxed);+ junk_records.fetch_add(+ o.item_errors.saturating_sub(o.server_errors),+ Ordering::Relaxed,+ );}}Err(e) => send_err = Some(format!("{e:#}")),@@ -582,7 +609,12 @@ fn run_index(cfg: IndexCfg) -> Result<i32> {let bulk_errs = bulk_errors.into_inner().unwrap();if !bulk_errs.is_empty() {- eprintln!("bulk errors encountered (first few): {bulk_errs:?}");+ anyhow::bail!(+ "autoindex stopped with bulk/backend failures: {}. Failed source files were not \+ journaled complete; fix the reported server or embedding configuration and rerun \+ the same command to resume safely",+ bulk_errs.join(" | ")+ );}// ── finalize: refresh, verify, correlate, catalog ────────────────────diff --git a/engine/crates/xerj-common/src/config.rs b/engine/crates/xerj-common/src/config.rsindex 833bc8e00..0c058e55c 100644--- a/engine/crates/xerj-common/src/config.rs+++ b/engine/crates/xerj-common/src/config.rs@@ -786,6 +786,26 @@ pub struct EmbeddingConfig {/// safetensors weights from this local directory instead of downloading/// — for air-gapped / offline deployments. Empty (default) = download.pub local_model_dir: String,+ /// Experimental ONNX backend: local FP32 all-MiniLM-L6-v2-compatible+ /// model with int64 BERT inputs and a width-384 token-embedding output.+ /// Required when `mode = "onnx-experimental"`; never auto-downloaded.+ pub onnx_model_path: String,+ /// Experimental ONNX backend: tokenizer.json from the same model/export.+ pub onnx_tokenizer_path: String,+ /// Experimental ONNX Runtime intra-op threads (default: available CPUs).+ pub onnx_intra_threads: usize,+ /// Maximum texts accepted by one ONNX scheduling window.+ pub onnx_max_pending: usize,+ /// Maximum documents in one ONNX inference microbatch.+ pub onnx_max_batch: usize,+ /// Maximum batch × padded-token slots per ONNX inference microbatch.+ pub onnx_padded_token_budget: usize,+ /// Maximum ONNX calls admitted globally per shared model/session.+ pub onnx_max_inflight_calls: usize,+ /// Reject one ONNX call above this many UTF-8 input bytes before tokenizing.+ pub onnx_max_input_bytes_per_call: usize,+ /// Aggregate UTF-8 bytes admitted globally per shared ONNX model/session.+ pub onnx_max_inflight_input_bytes: usize,}impl Default for EmbeddingConfig {@@ -799,6 +819,17 @@ impl Default for EmbeddingConfig {neural_model: "sentence-transformers/all-MiniLM-L6-v2".to_string(),model_cache_dir: String::new(),local_model_dir: String::new(),+ onnx_model_path: String::new(),+ onnx_tokenizer_path: String::new(),+ onnx_intra_threads: std::thread::available_parallelism()+ .map(|n| n.get())+ .unwrap_or(1),+ onnx_max_pending: 4096,+ onnx_max_batch: 64,+ onnx_padded_token_budget: 4096,+ onnx_max_inflight_calls: 8,+ onnx_max_input_bytes_per_call: 8 * 1024 * 1024,+ onnx_max_inflight_input_bytes: 32 * 1024 * 1024,}}}diff --git a/engine/crates/xerj-engine/Cargo.toml b/engine/crates/xerj-engine/Cargo.tomlindex f39e8589c..afc704bab 100644--- a/engine/crates/xerj-engine/Cargo.toml+++ b/engine/crates/xerj-engine/Cargo.toml@@ -51,6 +51,7 @@ default = []# default so the standard build stays lean and cross-compiles without the# candle/tokenizer toolchain; enable via xerj-server's `neural` feature.neural = ["xerj-ai/neural"]+onnx-experimental = ["xerj-ai/onnx-experimental"][dev-dependencies]tempfile = { workspace = true }diff --git a/engine/crates/xerj-engine/src/bulk.rs b/engine/crates/xerj-engine/src/bulk.rsindex 644fcc3a2..5b1e9d229 100644--- a/engine/crates/xerj-engine/src/bulk.rs+++ b/engine/crates/xerj-engine/src/bulk.rs@@ -40,6 +40,18 @@ pub struct BulkItemResult {pub get_source: Option<Value>,}+fn engine_error_http_status(error: &EngineError) -> u16 {+ match error {+ EngineError::Common(xerj_common::XerjError::IndexBlocked { block_type, .. })+ if block_type.contains("read_only_allow_delete") =>+ {+ 429+ }+ EngineError::Common(error) => error.http_status(),+ _ => 500,+ }+}+// ── Bulk processor ────────────────────────────────────────────────────────────/// A parsed bulk action before execution.@@ -820,6 +832,10 @@ pub async fn process_bulk_with_opts(// created-vs-updated / 201-vs-200 semantics turbo can't express.let mut auto_id_batches: HashMap<String, Vec<(usize, String, std::sync::Arc<[u8]>)>> =HashMap::new();+ let mut semantic_index_batches: HashMap<+ String,+ Vec<(usize, Option<String>, std::sync::Arc<[u8]>)>,+ > = HashMap::new();let mut non_index_actions: Vec<ParsedAction> = Vec::new();// Precompute which target indices declare any strict-format date@@ -919,15 +935,20 @@ pub async fn process_bulk_with_opts(// them through the non-index path. Also divert when the index// has date fields with a strict `format`, so the per-item// validation loop can reject malformed values like ES does.- let is_plain_index = action.action_type == "index"+ let has_strict_date =+ index_has_strict_date(&action.target_index, &mut index_needs_date_validation);+ let has_dynamic_copy =+ index_has_dynamic_copy(&action.target_index, &mut index_needs_dynamic_copy);+ let has_embedding = index_has_embedding(&action.target_index, &mut index_needs_embedding);+ let has_plain_metadata = action.action_type == "index"&& action.if_seq_no.is_none()&& action.if_primary_term.is_none()&& action.routing.is_none()&& action.dynamic_templates.is_none()&& action.pipeline.is_none()- && !index_has_strict_date(&action.target_index, &mut index_needs_date_validation)- && !index_has_dynamic_copy(&action.target_index, &mut index_needs_dynamic_copy)- && !index_has_embedding(&action.target_index, &mut index_needs_embedding);+ && !has_strict_date+ && !has_dynamic_copy;+ let is_plain_index = has_plain_metadata && !has_embedding;if is_plain_index {match action.doc_id {None => {@@ -949,6 +970,11 @@ pub async fn process_bulk_with_opts());}}+ } else if has_plain_metadata && has_embedding {+ semantic_index_batches+ .entry(action.target_index)+ .or_default()+ .push((action.item_index, action.doc_id, action.doc_bytes));} else {non_index_actions.push(action);}@@ -958,6 +984,106 @@ pub async fn process_bulk_with_opts(let group_ms = t_group.elapsed().as_millis() as u64;let t_exec = std::time::Instant::now();+ // ── semantic_text index actions: cross-document embedding windows ────+ //+ // Parse and prepare a target-index group together so the embedder sees+ // useful multi-document windows. The normal per-document index method is+ // still used after preparation; it observes the supplied companion vector+ // and therefore preserves WAL, overwrite status, response ordering, and+ // item-level errors.+ for (index_name, batch) in semantic_index_batches {+ let idx = match engine.get_or_create_index(&index_name) {+ Ok(i) => i,+ Err(e) => {+ for (item_idx, id, _) in batch {+ items[item_idx] = Some(BulkItemResult {+ action: "index".into(),+ index: index_name.clone(),+ id: id.unwrap_or_default(),+ status: 500,+ result: None,+ error: Some(e.to_string()),+ get_source: None,+ });+ errors = true;+ }+ continue;+ }+ };++ let mut valid_meta = Vec::with_capacity(batch.len());+ let mut valid_sources = Vec::with_capacity(batch.len());+ for (item_idx, id, bytes) in batch {+ match serde_json::from_slice::<Value>(&bytes) {+ Ok(source) => {+ valid_meta.push((item_idx, id));+ valid_sources.push(source);+ }+ Err(e) => {+ items[item_idx] = Some(BulkItemResult {+ action: "index".into(),+ index: index_name.clone(),+ id: id.unwrap_or_default(),+ status: 400,+ result: None,+ error: Some(format!("failed to parse: invalid document JSON: {e}")),+ get_source: None,+ });+ errors = true;+ }+ }+ }++ let prepared = idx.apply_semantic_embeddings_batch(valid_sources).await;+ for ((item_idx, id), prepared_source) in valid_meta.into_iter().zip(prepared) {+ let source = match prepared_source {+ Ok(source) => source,+ Err(e) => {+ let status = engine_error_http_status(&e);+ items[item_idx] = Some(BulkItemResult {+ action: "index".into(),+ index: index_name.clone(),+ id: id.unwrap_or_default(),+ status,+ result: None,+ error: Some(e.to_string()),+ get_source: None,+ });+ errors = true;+ continue;+ }+ };+ let error_id = id.clone().unwrap_or_default();+ match idx.index_document_prepared(id, source).await {+ Ok(resp) => {+ let status = if resp.result == "updated" { 200 } else { 201 };+ items[item_idx] = Some(BulkItemResult {+ action: "index".into(),+ index: index_name.clone(),+ id: resp.id,+ status,+ result: Some(resp.result),+ error: None,+ get_source: None,+ });+ }+ Err(e) => {+ let status = engine_error_http_status(&e);+ items[item_idx] = Some(BulkItemResult {+ action: "index".into(),+ index: index_name.clone(),+ id: error_id,+ status,+ result: None,+ error: Some(e.to_string()),+ get_source: None,+ });+ errors = true;+ }+ }+ }+ }+// ── Auto-id index actions: ONE turbo batch per target index ──────────//// `index_batch_turbo_raw` appends the whole group with a single WALdiff --git a/engine/crates/xerj-engine/src/index.rs b/engine/crates/xerj-engine/src/index.rsindex 12e6ccb0c..ddb5ada11 100644--- a/engine/crates/xerj-engine/src/index.rs+++ b/engine/crates/xerj-engine/src/index.rs@@ -556,16 +556,20 @@ pub struct Index {/// `Config.merge` at index construction; reads are cheap and merge/// runs hold the snapshot for the duration of one batch.merge_config: xerj_common::config::MergeConfig,+ /// Retained so a semantic field added after index creation can pin the+ /// same embedding identity before its first document is accepted.+ embedding_config: xerj_common::config::EmbeddingConfig,/// Embedding backend for `semantic` / `semantic_text` (v0.7-P2).- /// One of lexical (built-in, default), an external proxy, or the- /// built-in neural BERT embedder — selected by `Config.embedding.mode`.+ /// One of lexical (built-in, default), an external proxy, the built-in+ /// Candle neural BERT embedder, or the experimental ONNX Runtime backend+ /// — selected by `Config.embedding.mode`./// [`xerj_ai::Embedder::is_active`] is false only for the lexical/// fallback; an active backend embeds arbitrary query text, while the/// lexical fallback embeds only `semantic_text` fields (embedded the- /// same way at ingest). Reused across queries (proxy semaphore / neural- /// model are shared behind the `Arc`).- embedder: Arc<xerj_ai::Embedder>,+ /// same way at ingest). Reused across queries (proxy admission state and+ /// neural/ONNX models are shared behind the `Arc`).+ embedder: Arc<RwLock<xerj_ai::Embedder>>,// ── Per-index metrics ─────────────────────────────────────────────────────/// Total search queries executed.@@ -844,6 +848,8 @@ impl Index {schema,dynamic: xerj_common::schema::DynamicMapping::Dynamic,};+ validate_embedding_identity(&index_dir, &managed.schema, &config.embedding, true, false)?;+ let effective_embedder = make_embedder_for_schema(&managed.schema, &config.embedding)?;// Doc count is a sanity cap only — the byte threshold is the primary// driver. Historically this was 10 000, which forced flushes every ~8 MB@@ -940,7 +946,8 @@ impl Index {)),merge_in_progress: Arc::new(std::sync::atomic::AtomicBool::new(false)),merge_config: config.merge.clone(),- embedder: Arc::new(make_embedder(&config.embedding)),+ embedding_config: config.embedding.clone(),+ embedder: Arc::new(RwLock::new(effective_embedder)),dv_cache: Arc::new(dashmap::DashMap::new()),sort_shadow_cache: Arc::new(dashmap::DashMap::new()),sort_shadow_fields: Arc::new(dashmap::DashMap::new()),@@ -993,6 +1000,14 @@ impl Index {// Load schema from disk if it exists.let schema = load_schema(&index_dir).unwrap_or_else(|_| ManagedSchema::dynamic());+ validate_embedding_identity(+ &index_dir,+ &schema.schema,+ &config.embedding,+ false,+ segment_doc_count > 0 || store.version_map.live_count() > 0,+ )?;+ let effective_embedder = make_embedder_for_schema(&schema.schema, &config.embedding)?;// Load persisted settings early so we can build the registry before WAL replay.let settings = load_settings(&index_dir).unwrap_or(Value::Null);@@ -1178,7 +1193,8 @@ impl Index {)),merge_in_progress: Arc::new(std::sync::atomic::AtomicBool::new(false)),merge_config: config.merge.clone(),- embedder: Arc::new(make_embedder(&config.embedding)),+ embedding_config: config.embedding.clone(),+ embedder: Arc::new(RwLock::new(effective_embedder)),dv_cache: Arc::new(dashmap::DashMap::new()),sort_shadow_cache: Arc::new(dashmap::DashMap::new()),sort_shadow_fields: Arc::new(dashmap::DashMap::new()),@@ -1266,6 +1282,27 @@ impl Index {source: Value,if_seq_no: Option<u64>,if_primary_term: Option<u64>,+ ) -> Result<IndexResponse> {+ self.index_document_with_version_inner(id, source, if_seq_no, if_primary_term, false)+ .await+ }++ pub(crate) async fn index_document_prepared(+ &self,+ id: Option<String>,+ source: Value,+ ) -> Result<IndexResponse> {+ self.index_document_with_version_inner(id, source, None, None, true)+ .await+ }++ async fn index_document_with_version_inner(+ &self,+ id: Option<String>,+ source: Value,+ if_seq_no: Option<u64>,+ if_primary_term: Option<u64>,+ semantic_embeddings_prepared: bool,) -> Result<IndexResponse> {// Check write block.if self.is_write_blocked().await {@@ -1321,7 +1358,11 @@ impl Index {// its companion vector field (`<field>_vector`) so it becomes// kNN-searchable. Runs before copy_to / storage so the derived vector// is part of `_source` and gets picked up by HNSW indexing below.- let source = self.apply_semantic_embeddings(source).await?;+ let source = if semantic_embeddings_prepared {+ source+ } else {+ self.apply_semantic_embeddings(source).await?+ };// Apply copy_to: expand the source by copying field values to their target fields.let source = {@@ -4939,18 +4980,19 @@ impl Index {};// Embed the query text with the effective embedder:- // * An ACTIVE backend (neural BERT or external proxy) embeds the- // query text directly — the same backend used at ingest.+ // * An ACTIVE backend (Candle neural BERT, experimental ONNX, or an+ // external proxy) embeds the query text directly — the same+ // backend used at ingest.// * Otherwise the built-in lexical embedder — but ONLY for// `semantic_text` fields, which were embedded that same way at// ingest. A `semantic` query against a plain dense_vector field// with no active backend still returns the original 400 (there// is no comparable stored vector to match against).- let query_vec = if self.embedder.is_active() {+ let embedder = self.embedder.read().await;+ let query_vec = if embedder.is_active() {// `embed_batch` takes Vec<String>; we only have one text but// batching keeps the wire format stable for callers.- let mut vectors = self- .embedder+ let mut vectors = embedder.embed_batch(vec![text.to_string()]).await.map_err(|e| {@@ -4974,8 +5016,10 @@ impl Index {return Err(EngineError::Common(xerj_common::XerjError::invalid_query("semantic query requires either a `semantic_text` field (auto-embedded \with the built-in lexical embedder) or an active embedding backend \- (`embedding.mode = \"neural\"`, or `\"proxy\"` with \- `embedding.default_endpoint` set to an OpenAI-compatible endpoint).",+ (`embedding.mode = \"neural\"`; `\"proxy\"` with \+ `embedding.default_endpoint` set to an OpenAI-compatible endpoint; \+ or `\"onnx-experimental\"` in an ONNX-enabled build with explicit \+ model and tokenizer paths).",)));};@@ -5001,7 +5045,22 @@ impl Index {/// deterministic embedder — the *same* choice made at query time so the/// vectors are comparable. Fields where the target already holds a value/// (caller pre-embedded) or whose value is not a string are left untouched.- async fn apply_semantic_embeddings(&self, mut source: Value) -> Result<Value> {+ async fn apply_semantic_embeddings(&self, source: Value) -> Result<Value> {+ self.apply_semantic_embeddings_batch(vec![source])+ .await+ .pop()+ .expect("one input produces one output")+ }++ /// Prepare embeddings for several documents in shared inference windows.+ ///+ /// The returned vector is position-aligned with `sources`. A failed+ /// backend window is retried per document-field job so one bad item does+ /// not turn otherwise valid bulk items into failures.+ pub(crate) async fn apply_semantic_embeddings_batch(+ &self,+ mut sources: Vec<Value>,+ ) -> Vec<Result<Value>> {// Collect (field, target_field, dims) specs without holding the schema// lock across the (possibly async) embedding calls.let specs: Vec<(String, String, usize)> = {@@ -5026,44 +5085,183 @@ impl Index {.collect()};if specs.is_empty() {- return Ok(source);- }- let Some(obj) = source.as_object_mut() else {- return Ok(source);- };- for (field, target, dims) in specs {- // Respect a caller-supplied vector (e.g. pre-computed offline).- if obj.get(&target).map(|v| !v.is_null()).unwrap_or(false) {- continue;- }- let Some(text) = obj.get(&field).and_then(Value::as_str) else {+ return sources.into_iter().map(Ok).collect();+ }++ struct EmbedJob {+ source_idx: usize,+ field: String,+ target: String,+ dims: usize,+ original_text: String,+ texts: Vec<String>,+ }++ let mut jobs = Vec::<EmbedJob>::new();+ let mut pre_failures: Vec<Option<String>> = (0..sources.len()).map(|_| None).collect();+ let onnx_pinned = self+ .embedding_config+ .mode+ .eq_ignore_ascii_case("onnx-experimental");+ for (source_idx, source) in sources.iter().enumerate() {+ let Some(obj) = source.as_object() else {continue;};- let text = text.to_string();- // Chunk once and embed EACH overlapping chunk. `chunk_vecs` is one- // embedding per passage (>= 1). Short text (<= chunk_size) yields a- // single chunk == the exact pre-chunking behavior.- let chunk_vecs: Vec<Vec<f32>> = if self.embedder.is_active() {- // Active backend (neural or proxy): chunk, then embed each- // passage. The same backend + chunking is used at query time.- let chunks = semantic_chunker().chunk(&text, None);- let chunk_texts: Vec<String> = if chunks.len() <= 1 {- vec![text.clone()]+ for (field, target, dims) in &specs {+ let supplied_vector = obj.get(target).map(|v| !v.is_null()).unwrap_or(false);+ let supplied_chunks = obj+ .get(&format!("{target}_chunks"))+ .map(|v| !v.is_null())+ .unwrap_or(false);+ if onnx_pinned && (supplied_vector || supplied_chunks) {+ pre_failures[source_idx] = Some(format!(+ "field [{field}]: caller-supplied derived vectors are not accepted for \+ an ONNX-pinned semantic index because their model identity cannot be \+ verified; submit text only or use a separate explicitly pre-embedded index"+ ));+ continue;+ }+ if supplied_vector {+ continue;+ }+ let Some(text) = obj.get(field).and_then(Value::as_str) else {+ continue;+ };+ let chunks = semantic_chunker().chunk(text, None);+ let texts = if chunks.len() <= 1 {+ vec![text.to_string()]} else {chunks.iter().map(|c| c.text.clone()).collect()};- let vs = self.embedder.embed_batch(chunk_texts).await.map_err(|e| {- // Item 9: preserve the embedder's 5xx class on ingest too —- // a proxy outage must not masquerade as a 400 bad document.- EngineError::Common(embed_backend_error(- e,- &format!("semantic_text embed failed for field [{field}]"),- ))- })?;- vs.into_iter().filter(|v| !v.is_empty()).collect()- } else {- // Lexical fallback: deterministic per-chunk feature-hash.- local_chunk_vectors(&text, dims)+ jobs.push(EmbedJob {+ source_idx,+ field: field.clone(),+ target: target.clone(),+ dims: *dims,+ original_text: text.to_string(),+ texts,+ });+ }+ }++ let mut outputs: Vec<Option<Vec<Vec<f32>>>> = vec![None; jobs.len()];+ let mut failures = pre_failures;+ let mut admission_rejections = vec![false; sources.len()];++ let embedder = self.embedder.read().await;+ if embedder.is_active() {+ // Bound caller-side windows as well as backend-internal batches.+ // This avoids an arbitrarily large proxy payload and gives ONNX a+ // useful cross-document scheduling window without giant padding.+ const MAX_PASSAGES_PER_WINDOW: usize = 64;+ let mut start = 0;+ while start < jobs.len() {+ let mut end = start;+ let mut passages = 0;+ while end < jobs.len()+ && (end == start || passages + jobs[end].texts.len() <= MAX_PASSAGES_PER_WINDOW)+ {+ passages += jobs[end].texts.len();+ end += 1;+ }+ let texts: Vec<String> = jobs[start..end]+ .iter()+ .flat_map(|job| job.texts.iter().cloned())+ .collect();+ match embedder.embed_batch(texts).await {+ Ok(vectors) if vectors.len() == passages => {+ let mut offset = 0;+ for job_idx in start..end {+ let count = jobs[job_idx].texts.len();+ let job_vectors = &vectors[offset..offset + count];+ if let Some(vector) = job_vectors+ .iter()+ .find(|vector| vector.len() != jobs[job_idx].dims)+ {+ failures[jobs[job_idx].source_idx] = Some(format!(+ "field [{}]: embedding backend returned width {}, mapping requires {}",+ jobs[job_idx].field,+ vector.len(),+ jobs[job_idx].dims+ ));+ } else {+ outputs[job_idx] = Some(job_vectors.to_vec());+ }+ offset += count;+ }+ }+ batch_result => {+ // Preserve per-item bulk behavior on a backend error:+ // retry each document-field job separately and only+ // fail the sources whose own retry fails.+ let batch_error = match batch_result {+ Ok(v) => format!(+ "embedding backend returned {} vectors for {passages} texts",+ v.len()+ ),+ Err(e) => e.to_string(),+ };+ tracing::warn!(+ error = %batch_error,+ jobs = end - start,+ passages,+ "semantic embedding window failed; retrying per item"+ );+ for job_idx in start..end {+ match embedder.embed_batch(jobs[job_idx].texts.clone()).await {+ Ok(v)+ if v.len() == jobs[job_idx].texts.len()+ && v.iter()+ .all(|vector| vector.len() == jobs[job_idx].dims) =>+ {+ outputs[job_idx] = Some(v);+ }+ Ok(v) => {+ let widths = v+ .iter()+ .map(Vec::len)+ .collect::<std::collections::BTreeSet<_>>();+ failures[jobs[job_idx].source_idx] = Some(format!(+ "field [{}]: embedding backend returned {} vectors with widths {:?} for {} texts; mapping requires width {}",+ jobs[job_idx].field,+ v.len(),+ widths,+ jobs[job_idx].texts.len(),+ jobs[job_idx].dims,+ ));+ }+ Err(e) => {+ admission_rejections[jobs[job_idx].source_idx] =+ is_embedding_admission_rejection(&e);+ failures[jobs[job_idx].source_idx] =+ Some(format!("field [{}]: {e}", jobs[job_idx].field));+ }+ }+ }+ }+ }+ start = end;+ }+ } else {+ for (job_idx, job) in jobs.iter().enumerate() {+ // The lexical fallback intentionally keeps its deterministic+ // per-document chunking behavior.+ outputs[job_idx] = Some(local_chunk_vectors(&job.original_text, job.dims));+ }+ }++ for (job_idx, job) in jobs.into_iter().enumerate() {+ if failures[job.source_idx].is_some() {+ continue;+ }+ let chunk_vecs: Vec<Vec<f32>> = outputs[job_idx]+ .take()+ .unwrap_or_default()+ .into_iter()+ .filter(|v| !v.is_empty())+ .collect();+ let Some(obj) = sources[job.source_idx].as_object_mut() else {+ continue;};// Pooled vector for `target`: single chunk is stored as-is (no// re-normalization, matching pre-chunking behavior); multiple@@ -5076,7 +5274,7 @@ impl Index {_ => mean_pool_normalize(&chunk_vecs),};let arr: Vec<Value> = pooled.into_iter().map(|f| Value::from(f as f64)).collect();- obj.insert(target.clone(), Value::Array(arr));+ obj.insert(job.target.clone(), Value::Array(arr));// Per-chunk passage vectors — persisted ONLY when the document// actually spans more than one chunk, under `<target>_chunks` as an// array of vectors. Semantic search prefers these and scores by the@@ -5088,10 +5286,26 @@ impl Index {.iter().map(|cv| Value::Array(cv.iter().map(|f| Value::from(*f as f64)).collect())).collect();- obj.insert(format!("{target}_chunks"), Value::Array(chunks_json));+ obj.insert(format!("{}_chunks", job.target), Value::Array(chunks_json));}}- Ok(source)++ sources+ .into_iter()+ .enumerate()+ .map(|(idx, source)| match failures[idx].take() {+ Some(error) if admission_rejections[idx] => Err(EngineError::Common(+ xerj_common::XerjError::resource_exhausted(format!(+ "semantic_text batch embed rejected before tokenization: {error}"+ )),+ )),+ Some(error) => Err(EngineError::Common(embed_backend_error(+ anyhow::anyhow!(error),+ "semantic_text batch embed failed",+ ))),+ None => Ok(source),+ })+ .collect()}/// Hybrid (multi-query + fusion) executor. Recursively runs each@@ -10158,6 +10372,26 @@ impl Index {/// Add a field to the schema.pub async fn add_field(&self, field: FieldConfig) -> Result<()> {+ let activates_embedder = schema_needs_embedder(std::slice::from_ref(&field))+ && !schema_needs_embedder(&self.schema.read().await.schema.fields);+ let activated = if activates_embedder {+ Some(make_embedder(&self.embedding_config)?)+ } else {+ None+ };+ if schema_has_embeddings(std::slice::from_ref(&field)) {+ if self+ .embedding_config+ .mode+ .eq_ignore_ascii_case("onnx-experimental")+ {+ validate_onnx_dimensions(std::slice::from_ref(&field))?;+ }+ ensure_embedding_identity_for_new_field(&self.data_dir, &self.embedding_config)?;+ }+ if let Some(embedder) = activated {+ *self.embedder.write().await = embedder;+ }let mut schema = self.schema.write().await;schema.add_field(field)?;self.save_schema(&schema).await?;@@ -15990,13 +16224,18 @@ fn build_highlight_fragments(/// `default_endpoint`)./// * `"neural"` — the built-in BERT embedder (requires the `neural`/// cargo feature; falls back to lexical with a warning otherwise).+/// * `"onnx-experimental"` — an explicit FP32 ONNX model and tokenizer+/// loaded through ONNX Runtime (requires the `onnx-experimental` feature)./// * `"lexical"` — the built-in feature-hash embedder./// * `"auto"` (default / unknown) — proxy when `default_endpoint` is set,/// else lexical. Preserves the historical behavior exactly.///-/// Any misconfiguration degrades to the lexical fallback (never a crash);-/// a `semantic` query with no active backend against a plain vector field-/// still returns a helpful 400.+/// Proxy initialization and an unavailable Candle feature degrade to the+/// lexical fallback with a warning. ONNX is deliberately fail-closed: missing+/// assets, a feature-disabled build, or an incompatible model returns an+/// actionable configuration error instead of silently changing vector+/// identity. A `semantic` query with no active backend against a plain vector+/// field still returns a helpful 400./// Map an embedder failure to the correct ES class (item 9). An embedding/// backend outage or misconfiguration is a server-side (5xx) failure, never a/// 400 `invalid_query`: downgrading it hid proxy failures behind a client@@ -16004,23 +16243,389 @@ fn build_highlight_fragments(/// `EmbeddingError` (500, `circuit_breaking_exception`) with `ctx` prepended./// Takes `impl Display` because the `Embedder` surfaces `anyhow::Error` (the/// classification of transient-vs-permanent already happened inside the proxy).-fn embed_backend_error(e: impl std::fmt::Display, ctx: &str) -> xerj_common::XerjError {+fn is_embedding_admission_rejection(e: &anyhow::Error) -> bool {+ #[cfg(feature = "onnx-experimental")]+ if e.downcast_ref::<xerj_ai::embedder::OnnxAdmissionError>()+ .is_some()+ {+ return true;+ }+ false+}++fn embed_backend_error(e: anyhow::Error, ctx: &str) -> xerj_common::XerjError {+ if is_embedding_admission_rejection(&e) {+ return xerj_common::XerjError::resource_exhausted(format!("{ctx}: {e}"));+ }xerj_common::XerjError::embedding(format!("{ctx}: {e}"))}-fn make_embedder(cfg: &xerj_common::config::EmbeddingConfig) -> xerj_ai::Embedder {+const EMBEDDING_IDENTITY_FILE: &str = "embedding_identity.json";++#[derive(Debug, PartialEq, Serialize, Deserialize)]+struct PersistedEmbeddingIdentity {+ version: u32,+ backend: String,+ model_sha256: String,+ tokenizer_sha256: String,+ dimensions: usize,+ pooling: String,+ max_tokens: usize,+}++fn schema_has_embeddings(fields: &[FieldConfig]) -> bool {+ fields+ .iter()+ .any(|field| field.embedding.is_some() || schema_has_embeddings(&field.fields))+}++fn schema_needs_embedder(fields: &[FieldConfig]) -> bool {+ fields.iter().any(|field| {+ field.embedding.is_some()+ || matches!(field.field_type, FieldType::Vector | FieldType::Chunk)+ || schema_needs_embedder(&field.fields)+ })+}++fn validate_onnx_dimensions(fields: &[FieldConfig]) -> Result<()> {+ for field in fields {+ if field.embedding.is_some() {+ let dims = field+ .options+ .dimensions+ .unwrap_or(xerj_ai::local::DEFAULT_DIMS);+ if dims != 384 {+ return Err(EngineError::Common(xerj_common::XerjError::embedding(+ format!(+ "semantic field [{}] declares {dims} dimensions, but the experimental \+ MiniLM ONNX backend returns exactly 384. Change the mapping to 384 or \+ use an embedding backend matching the declared dimensions.",+ field.name+ ),+ )));+ }+ }+ validate_onnx_dimensions(&field.fields)?;+ }+ Ok(())+}++fn sha256_file(path: &Path) -> Result<String> {+ use sha2::{Digest, Sha256};+ use std::collections::HashMap;+ use std::io::Read;+ use std::sync::{Mutex, OnceLock};+ use std::time::UNIX_EPOCH;++ static CACHE: OnceLock<Mutex<HashMap<(PathBuf, u64, u128), String>>> = OnceLock::new();+ let canonical = path.canonicalize().map_err(|e| {+ EngineError::Common(xerj_common::XerjError::embedding(format!(+ "cannot resolve embedding asset {}: {e}",+ path.display()+ )))+ })?;+ let metadata = std::fs::metadata(&canonical)?;+ let modified = metadata+ .modified()+ .ok()+ .and_then(|time| time.duration_since(UNIX_EPOCH).ok())+ .map(|duration| duration.as_nanos())+ .unwrap_or(0);+ let key = (canonical.clone(), metadata.len(), modified);+ let mut cache = CACHE+ .get_or_init(|| Mutex::new(HashMap::new()))+ .lock()+ .unwrap_or_else(|poisoned| poisoned.into_inner());+ if let Some(hash) = cache.get(&key).cloned() {+ return Ok(hash);+ }+ let mut file = std::fs::File::open(&canonical).map_err(|e| {+ EngineError::Common(xerj_common::XerjError::embedding(format!(+ "cannot fingerprint embedding asset {}: {e}",+ path.display()+ )))+ })?;+ let mut hasher = Sha256::new();+ let mut buf = [0_u8; 1024 * 1024];+ loop {+ let n = file.read(&mut buf)?;+ if n == 0 {+ break;+ }+ hasher.update(&buf[..n]);+ }+ let hash = format!("{:x}", hasher.finalize());+ cache.insert(key, hash.clone());+ Ok(hash)+}++fn configured_onnx_identity(+ cfg: &xerj_common::config::EmbeddingConfig,+) -> Result<Option<PersistedEmbeddingIdentity>> {+ if cfg.mode.trim().to_ascii_lowercase() != "onnx-experimental" {+ return Ok(None);+ }+ #[cfg(not(feature = "onnx-experimental"))]+ return Err(EngineError::Common(xerj_common::XerjError::embedding(+ "onnx-experimental was requested but this engine build does not include the \+ onnx-experimental feature; no embedding identity marker was written",+ )));+ #[cfg(feature = "onnx-experimental")]+ Ok(Some(PersistedEmbeddingIdentity {+ version: 1,+ backend: "onnx-experimental".into(),+ model_sha256: sha256_file(Path::new(&cfg.onnx_model_path))?,+ tokenizer_sha256: sha256_file(Path::new(&cfg.onnx_tokenizer_path))?,+ dimensions: 384,+ pooling: "attention-mask-mean+l2-normalize".into(),+ max_tokens: 512,+ }))+}++/// Fail closed rather than mixing vectors produced by different model,+/// tokenizer, pooling, or backend configurations after a restart/resume.+fn validate_embedding_identity(+ index_dir: &Path,+ schema: &Schema,+ cfg: &xerj_common::config::EmbeddingConfig,+ new_index: bool,+ has_documents: bool,+) -> Result<()> {+ if !schema_has_embeddings(&schema.fields) {+ return Ok(());+ }+ let marker_path = index_dir.join(EMBEDDING_IDENTITY_FILE);+ let configured = configured_onnx_identity(cfg)?;+ if configured.is_some() {+ validate_onnx_dimensions(&schema.fields)?;+ }+ if marker_path.exists() {+ let persisted: PersistedEmbeddingIdentity =+ serde_json::from_slice(&std::fs::read(&marker_path)?).map_err(|e| {+ EngineError::Common(xerj_common::XerjError::embedding(format!(+ "cannot read {}: {e}; restore the marker or reindex this index",+ marker_path.display()+ )))+ })?;+ return match configured {+ Some(current) if current == persisted => Ok(()),+ Some(current) => Err(EngineError::Common(xerj_common::XerjError::embedding(+ format!(+ "embedding identity mismatch for index {}: persisted backend={} \+ model={} tokenizer={}, configured backend={} model={} tokenizer={}. \+ Refusing to mix incompatible vector spaces. Restore the original assets \+ or re-run autoindex with --fresh and a new index prefix.",+ index_dir.display(),+ persisted.backend,+ persisted.model_sha256,+ persisted.tokenizer_sha256,+ current.backend,+ current.model_sha256,+ current.tokenizer_sha256,+ ),+ ))),+ None => Err(EngineError::Common(xerj_common::XerjError::embedding(+ format!(+ "index {} contains ONNX vectors but the server is configured for \+ embedding.mode={:?}. Restart with onnx-experimental and the original \+ assets, or reindex under a new prefix.",+ index_dir.display(),+ cfg.mode+ ),+ ))),+ };+ }+ let Some(identity) = configured else {+ return Ok(());+ };+ if !new_index && has_documents {+ return Err(EngineError::Common(xerj_common::XerjError::embedding(+ format!(+ "existing semantic index {} has no embedding identity marker. It may contain \+ vectors from another backend, so ONNX cannot be enabled safely in place. \+ Re-run autoindex with --fresh and a new --prefix, or rebuild the index from \+ source.",+ index_dir.display()+ ),+ )));+ }+ write_file_atomic(&marker_path, &serde_json::to_vec_pretty(&identity)?)?;+ Ok(())+}++fn ensure_embedding_identity_for_new_field(+ index_dir: &Path,+ cfg: &xerj_common::config::EmbeddingConfig,+) -> Result<()> {+ let Some(configured) = configured_onnx_identity(cfg)? else {+ return Ok(());+ };+ let marker_path = index_dir.join(EMBEDDING_IDENTITY_FILE);+ if marker_path.exists() {+ let persisted: PersistedEmbeddingIdentity =+ serde_json::from_slice(&std::fs::read(&marker_path)?).map_err(|e| {+ EngineError::Common(xerj_common::XerjError::embedding(format!(+ "cannot read {}: {e}; restore the marker or reindex this index",+ marker_path.display()+ )))+ })?;+ if persisted != configured {+ return Err(EngineError::Common(xerj_common::XerjError::embedding(+ format!(+ "embedding identity mismatch for index {}; refusing to add a semantic \+ field in a different vector space. Restore the original ONNX assets or \+ create a new index.",+ index_dir.display()+ ),+ )));+ }+ return Ok(());+ }+ // This field does not exist yet, so existing documents cannot contain its+ // derived vectors. Pin the identity before publishing the mapping; all+ // later writes and restarts are then checked against it.+ write_file_atomic(&marker_path, &serde_json::to_vec_pretty(&configured)?)?;+ Ok(())+}++#[cfg(test)]+mod embedding_identity_tests {+ use super::*;++ fn semantic_schema() -> Schema {+ let mut schema = Schema::empty();+ schema+ .add_field(FieldConfig::new("body", FieldType::Text).with_embedding(+ xerj_common::types::EmbeddingConfig {+ endpoint: None,+ model: None,+ target_field: None,+ },+ ))+ .unwrap();+ schema+ }++ fn onnx_config(dir: &Path, suffix: &str) -> xerj_common::config::EmbeddingConfig {+ let model = dir.join(format!("model-{suffix}.onnx"));+ let tokenizer = dir.join(format!("tokenizer-{suffix}.json"));+ std::fs::write(&model, format!("model-{suffix}")).unwrap();+ std::fs::write(&tokenizer, format!("tokenizer-{suffix}")).unwrap();+ xerj_common::config::EmbeddingConfig {+ mode: "onnx-experimental".into(),+ onnx_model_path: model.to_string_lossy().into_owned(),+ onnx_tokenizer_path: tokenizer.to_string_lossy().into_owned(),+ ..Default::default()+ }+ }++ #[test]+ fn onnx_identity_is_persisted_and_matching_restart_is_allowed() {+ let dir = tempfile::tempdir().unwrap();+ let cfg = onnx_config(dir.path(), "a");+ validate_embedding_identity(dir.path(), &semantic_schema(), &cfg, true, false).unwrap();+ assert!(dir.path().join(EMBEDDING_IDENTITY_FILE).is_file());+ validate_embedding_identity(dir.path(), &semantic_schema(), &cfg, false, true).unwrap();+ }++ #[test]+ fn changed_onnx_assets_are_rejected() {+ let dir = tempfile::tempdir().unwrap();+ let first = onnx_config(dir.path(), "a");+ validate_embedding_identity(dir.path(), &semantic_schema(), &first, true, false).unwrap();+ let changed = onnx_config(dir.path(), "b");+ let error =+ validate_embedding_identity(dir.path(), &semantic_schema(), &changed, false, true)+ .unwrap_err()+ .to_string();+ assert!(error.contains("identity mismatch"), "{error}");+ assert!(error.contains("new index prefix"), "{error}");+ }++ #[test]+ fn markerless_existing_index_only_fails_closed_for_onnx() {+ let dir = tempfile::tempdir().unwrap();+ let lexical = xerj_common::config::EmbeddingConfig::default();+ validate_embedding_identity(dir.path(), &semantic_schema(), &lexical, false, true).unwrap();+ let onnx = onnx_config(dir.path(), "a");+ let error = validate_embedding_identity(dir.path(), &semantic_schema(), &onnx, false, true)+ .unwrap_err()+ .to_string();+ assert!(error.contains("no embedding identity marker"), "{error}");+ }++ #[test]+ fn later_semantic_field_pins_identity_before_first_write() {+ let dir = tempfile::tempdir().unwrap();+ let onnx = onnx_config(dir.path(), "a");+ ensure_embedding_identity_for_new_field(dir.path(), &onnx).unwrap();+ assert!(dir.path().join(EMBEDDING_IDENTITY_FILE).is_file());+ ensure_embedding_identity_for_new_field(dir.path(), &onnx).unwrap();+ }++ #[test]+ fn onnx_rejects_non_384_semantic_mapping() {+ let dir = tempfile::tempdir().unwrap();+ let onnx = onnx_config(dir.path(), "a");+ let mut schema = semantic_schema();+ schema.fields[0].options.dimensions = Some(768);+ let error = validate_embedding_identity(dir.path(), &schema, &onnx, true, false)+ .unwrap_err()+ .to_string();+ assert!(error.contains("declares 768 dimensions"), "{error}");+ assert!(error.contains("exactly 384"), "{error}");+ }++ #[cfg(feature = "onnx-experimental")]+ #[test]+ fn onnx_admission_rejection_maps_to_retryable_429() {+ let error = anyhow::Error::new(xerj_ai::embedder::OnnxAdmissionError {+ reason: "global admission full; retry".into(),+ });+ let mapped = embed_backend_error(error, "semantic query embed failed");+ assert_eq!(mapped.http_status(), 429);+ assert!(mapped.to_string().contains("retry"));+ }++ #[cfg(feature = "onnx-experimental")]+ #[test]+ fn non_vector_schema_does_not_touch_onnx_assets() {+ let config = xerj_common::config::EmbeddingConfig {+ mode: "onnx-experimental".into(),+ onnx_model_path: "/definitely/missing/model.onnx".into(),+ onnx_tokenizer_path: "/definitely/missing/tokenizer.json".into(),+ ..Default::default()+ };+ let embedder = make_embedder_for_schema(&Schema::empty(), &config).unwrap();+ assert!(embedder.describe().contains("lexical"));++ let mut schema = semantic_schema();+ schema.fields[0].options.dimensions = Some(384);+ let error = make_embedder_for_schema(&schema, &config)+ .err()+ .expect("semantic schema must activate and validate ONNX")+ .to_string();+ assert!(error.contains("readable FP32 model"), "{error}");+ }+}++fn make_embedder(cfg: &xerj_common::config::EmbeddingConfig) -> Result<xerj_ai::Embedder> {let mode = cfg.mode.trim().to_ascii_lowercase();let want_proxy = mode == "proxy"|| (mode != "neural" && mode != "lexical" && !cfg.default_endpoint.is_empty());if mode == "neural" {- return make_neural_embedder(cfg);+ return Ok(make_neural_embedder(cfg));+ }++ if mode == "onnx-experimental" {+ return make_onnx_embedder(cfg);}if want_proxy {if cfg.default_endpoint.is_empty() {warn!("embedding.mode=proxy but embedding.default_endpoint is empty — falling back to lexical");- return xerj_ai::Embedder::lexical();+ return Ok(xerj_ai::Embedder::lexical());}let proxy_cfg = xerj_ai::embed::EmbeddingProxyConfig {endpoint: cfg.default_endpoint.clone(),@@ -16030,7 +16635,7 @@ fn make_embedder(cfg: &xerj_common::config::EmbeddingConfig) -> xerj_ai::Embeddemax_concurrent: 4,max_retries: 3,};- return match xerj_ai::embed::EmbeddingProxy::new(proxy_cfg) {+ return Ok(match xerj_ai::embed::EmbeddingProxy::new(proxy_cfg) {Ok(p) => {info!(endpoint = %cfg.default_endpoint, "embedding backend: external proxy");xerj_ai::Embedder::proxy(p)@@ -16039,10 +16644,100 @@ fn make_embedder(cfg: &xerj_common::config::EmbeddingConfig) -> xerj_ai::Embeddewarn!(error = %e, "embedding proxy init failed — falling back to lexical");xerj_ai::Embedder::lexical()}- };+ });}- xerj_ai::Embedder::lexical()+ Ok(xerj_ai::Embedder::lexical())+}++fn make_embedder_for_schema(+ schema: &Schema,+ cfg: &xerj_common::config::EmbeddingConfig,+) -> Result<xerj_ai::Embedder> {+ if schema_needs_embedder(&schema.fields) {+ make_embedder(cfg)+ } else {+ Ok(xerj_ai::Embedder::lexical())+ }+}++#[cfg(feature = "onnx-experimental")]+fn make_onnx_embedder(cfg: &xerj_common::config::EmbeddingConfig) -> Result<xerj_ai::Embedder> {+ use std::path::PathBuf;+ let model_path = PathBuf::from(cfg.onnx_model_path.trim());+ let tokenizer_path = PathBuf::from(cfg.onnx_tokenizer_path.trim());+ if cfg.onnx_model_path.trim().is_empty() || !model_path.is_file() {+ return Err(EngineError::Common(xerj_common::XerjError::embedding(+ format!(+ "embedding.mode=onnx-experimental requires a readable FP32 model file; \+ set --onnx-model <PATH> or embedding.onnx_model_path (got {:?})",+ cfg.onnx_model_path+ ),+ )));+ }+ if cfg.onnx_tokenizer_path.trim().is_empty() || !tokenizer_path.is_file() {+ return Err(EngineError::Common(xerj_common::XerjError::embedding(+ format!(+ "embedding.mode=onnx-experimental requires the matching tokenizer.json; \+ set --onnx-tokenizer <PATH> or embedding.onnx_tokenizer_path (got {:?})",+ cfg.onnx_tokenizer_path+ ),+ )));+ }+ if cfg.onnx_max_pending == 0+ || cfg.onnx_max_batch == 0+ || cfg.onnx_padded_token_budget == 0+ || cfg.onnx_max_inflight_calls == 0+ || cfg.onnx_max_input_bytes_per_call == 0+ || cfg.onnx_max_inflight_input_bytes == 0+ || cfg.onnx_max_input_bytes_per_call > cfg.onnx_max_inflight_input_bytes+ || cfg.onnx_max_inflight_input_bytes > u32::MAX as usize+ {+ return Err(EngineError::Common(xerj_common::XerjError::embedding(+ format!(+ "invalid ONNX limits: pending={}, batch={}, padded_tokens={}, inflight_calls={}, \+ bytes_per_call={}, inflight_bytes={}; all must be non-zero, bytes_per_call \+ must not exceed inflight_bytes, and inflight_bytes must be <= {}",+ cfg.onnx_max_pending,+ cfg.onnx_max_batch,+ cfg.onnx_padded_token_budget,+ cfg.onnx_max_inflight_calls,+ cfg.onnx_max_input_bytes_per_call,+ cfg.onnx_max_inflight_input_bytes,+ u32::MAX+ ),+ )));+ }+ let onnx_cfg = xerj_ai::embedder::OnnxConfig {+ model_sha256: sha256_file(&model_path)?,+ tokenizer_sha256: sha256_file(&tokenizer_path)?,+ model_path,+ tokenizer_path,+ intra_threads: cfg.onnx_intra_threads.max(1),+ microbatch: xerj_ai::onnx::MicrobatchConfig {+ max_pending: cfg.onnx_max_pending,+ max_batch: cfg.onnx_max_batch,+ padded_token_budget: cfg.onnx_padded_token_budget,+ },+ max_inflight_calls: cfg.onnx_max_inflight_calls,+ max_input_bytes_per_call: cfg.onnx_max_input_bytes_per_call,+ max_inflight_input_bytes: cfg.onnx_max_inflight_input_bytes,+ };+ info!(+ model = %onnx_cfg.model_path.display(),+ tokenizer = %onnx_cfg.tokenizer_path.display(),+ "embedding backend: experimental ONNX Runtime (loads on first use)"+ );+ Ok(xerj_ai::Embedder::onnx(onnx_cfg))+}++#[cfg(not(feature = "onnx-experimental"))]+fn make_onnx_embedder(_cfg: &xerj_common::config::EmbeddingConfig) -> Result<xerj_ai::Embedder> {+ Err(EngineError::Common(xerj_common::XerjError::embedding(+ "embedding.mode=onnx-experimental requested, but this binary was built without it; \+ rebuild xerj-server with --features onnx-experimental. XERJ will not silently \+ substitute lexical vectors because that would mislabel search quality.",+ )))}/// Construct the built-in neural embedder when compiled in; otherwise warndiff --git a/engine/crates/xerj-engine/tests/integration.rs b/engine/crates/xerj-engine/tests/integration.rsindex ed8042035..da84261f2 100644--- a/engine/crates/xerj-engine/tests/integration.rs+++ b/engine/crates/xerj-engine/tests/integration.rs@@ -4905,6 +4905,107 @@ async fn test_semantic_text_match_survives_flush() {);}+#[tokio::test]+async fn test_semantic_text_bulk_preserves_item_order_status_and_vectors() {+ let dir = TempDir::new().unwrap();+ let engine = make_engine(&dir);++ let mut schema = Schema::empty();+ let mut fc = FieldConfig::new("content", FieldType::Text);+ fc.options.dimensions = Some(16);+ fc.options.similarity = Some("cosine".to_string());+ fc.embedding = Some(xerj_common::types::EmbeddingConfig {+ endpoint: None,+ model: None,+ target_field: Some("content_vector".to_string()),+ });+ schema.fields.push(fc);+ engine.create_index("sem-bulk", schema).unwrap();++ let body = concat!(+ "{\"index\":{\"_index\":\"sem-bulk\",\"_id\":\"a\"}}\n",+ "{\"content\":\"alpha financial report\"}\n",+ "{\"index\":{\"_index\":\"sem-bulk\",\"_id\":\"b\"}}\n",+ "{\"content\":\"beta quarterly earnings\"}\n",+ "{\"index\":{\"_index\":\"sem-bulk\",\"_id\":\"a\"}}\n",+ "{\"content\":\"alpha annual report updated\"}\n",+ );+ let result = xerj_engine::bulk::process_bulk(&engine, None, body).await;+ assert!(!result.errors, "{:?}", result.items);+ assert_eq!(result.items.len(), 3);+ assert_eq!(result.items[0].id, "a");+ assert_eq!(result.items[0].status, 201);+ assert_eq!(result.items[1].id, "b");+ assert_eq!(result.items[1].status, 201);+ assert_eq!(result.items[2].id, "a");+ assert_eq!(result.items[2].status, 200);++ let idx = engine.get_index("sem-bulk").unwrap();+ let a = idx.get_document("a").await.unwrap().unwrap();+ let b = idx.get_document("b").await.unwrap().unwrap();+ assert_eq!(a["content"], "alpha annual report updated");+ assert_eq!(a["content_vector"].as_array().unwrap().len(), 16);+ assert_eq!(b["content_vector"].as_array().unwrap().len(), 16);+}++#[cfg(feature = "onnx-experimental")]+#[tokio::test]+async fn test_semantic_bulk_preserves_onnx_admission_429() {+ let dir = TempDir::new().unwrap();+ let model = dir.path().join("model.onnx");+ let tokenizer = dir.path().join("tokenizer.json");+ std::fs::write(&model, b"not loaded: admission rejects first").unwrap();+ std::fs::write(&tokenizer, b"not loaded: admission rejects first").unwrap();++ let mut config = Config::default();+ config.server.data_dir = dir.path().join("data").to_string_lossy().into_owned();+ config.embedding.mode = "onnx-experimental".into();+ config.embedding.onnx_model_path = model.to_string_lossy().into_owned();+ config.embedding.onnx_tokenizer_path = tokenizer.to_string_lossy().into_owned();+ config.embedding.onnx_max_input_bytes_per_call = 1;+ config.embedding.onnx_max_inflight_input_bytes = 1;+ let engine = Engine::new(config).unwrap();++ let mut schema = Schema::empty();+ let mut field = FieldConfig::new("content", FieldType::Text);+ field.options.dimensions = Some(384);+ field.embedding = Some(xerj_common::types::EmbeddingConfig {+ endpoint: None,+ model: None,+ target_field: Some("content_vector".into()),+ });+ schema.fields.push(field);+ engine.create_index("sem-onnx-overload", schema).unwrap();++ let body = concat!(+ "{\"index\":{\"_index\":\"sem-onnx-overload\",\"_id\":\"a\"}}\n",+ "{\"content\":\"too large\"}\n",+ );+ let result = xerj_engine::bulk::process_bulk(&engine, None, body).await;+ assert!(result.errors);+ assert_eq!(result.items.len(), 1);+ assert_eq!(result.items[0].status, 429, "{:?}", result.items[0]);+ assert!(+ result.items[0]+ .error+ .as_deref()+ .unwrap_or_default()+ .contains("rejected before tokenization"),+ "{:?}",+ result.items[0]+ );+ assert!(+ engine+ .get_index("sem-onnx-overload")+ .unwrap()+ .get_document("a")+ .await+ .unwrap()+ .is_none(),+ "rejected bulk item must not be persisted"+ );+}+/// Blocker 5: snapshot RESTORE ignored the request `indices` filter and/// rewrote EVERY index in the snapshot with snapshot-time state, silently/// destroying all writes made since. The filter must select exactly thediff --git a/engine/crates/xerj-server/Cargo.toml b/engine/crates/xerj-server/Cargo.tomlindex b7e04a9c5..189e3aeb4 100644--- a/engine/crates/xerj-server/Cargo.toml+++ b/engine/crates/xerj-server/Cargo.toml@@ -65,6 +65,7 @@ prost.workspace = true# (the lexical + proxy backends still work).default = ["neural"]neural = ["xerj-engine/neural"]+onnx-experimental = ["xerj-engine/onnx-experimental"][target.'cfg(not(target_env = "msvc"))'.dependencies]tikv-jemallocator = "0.6"diff --git a/engine/crates/xerj-server/src/main.rs b/engine/crates/xerj-server/src/main.rsindex 6f3c34cd7..d68995fd8 100644--- a/engine/crates/xerj-server/src/main.rs+++ b/engine/crates/xerj-server/src/main.rs@@ -78,6 +78,8 @@ struct CliArgs {data_dir: Option<String>,insecure: bool,embed_mode: Option<String>,+ onnx_model: Option<String>,+ onnx_tokenizer: Option<String>,}fn parse_args() -> CliArgs {@@ -86,6 +88,8 @@ fn parse_args() -> CliArgs {let mut data_dir = None;let mut insecure = false;let mut embed_mode = None;+ let mut onnx_model = None;+ let mut onnx_tokenizer = None;while let Some(arg) = args.next() {match arg.as_str() {@@ -101,6 +105,8 @@ fn parse_args() -> CliArgs {"--embed-mode" => {embed_mode = args.next();}+ "--onnx-model" => onnx_model = args.next(),+ "--onnx-tokenizer" => onnx_tokenizer = args.next(),"--help" | "-h" => {print_help();std::process::exit(0);@@ -121,6 +127,8 @@ fn parse_args() -> CliArgs {data_dir,insecure,embed_mode,+ onnx_model,+ onnx_tokenizer,}}@@ -135,13 +143,20 @@ fn print_help() {--config, -c <PATH> Path to TOML config file\n\--data-dir, -d <PATH> Override data directory\n\--insecure, -k Disable TLS\n\- --embed-mode <MODE> Embedding backend: lexical | neural | proxy | auto\n\+ --embed-mode <MODE> Embedding backend: lexical | neural | proxy | auto |\n\+ onnx-experimental\n\lexical built-in feature-hash (default; offline, not neural)\n\neural built-in BERT (all-MiniLM-L6-v2) — real semantics,\n\in-process; auto-downloads the model (~90 MB) on\n\first use, then runs from cache. Just add the flag.\n\proxy external OpenAI-compatible /v1/embeddings endpoint\n\auto proxy if embedding.default_endpoint is set, else lexical\n\+ onnx-experimental local all-MiniLM-L6-v2-compatible FP32\n\+ ONNX prototype; width 384, named BERT inputs and\n\+ token output; requires matching tokenizer.json;\n\+ no auto-download\n\+ --onnx-model <PATH> compatible FP32 MiniLM ONNX model\n\+ --onnx-tokenizer <PATH> tokenizer.json from the same model/export\n\--help, -h Show this help\n\--version, -V Print version and exit\n\\n\@@ -153,7 +168,9 @@ fn print_help() {ENVIRONMENT:\n\XERJ_LOG Log level filter (default: info)\n\XERJ_CONFIG Config file path\n\- XERJ_EMBED_MODE Embedding backend (lexical|neural|proxy|auto)\n",+ XERJ_EMBED_MODE Embedding backend (lexical|neural|proxy|auto|onnx-experimental)\n\+ XERJ_ONNX_MODEL FP32 ONNX model path\n\+ XERJ_ONNX_TOKENIZER matching tokenizer.json path\n",env!("CARGO_PKG_VERSION"));}@@ -246,7 +263,8 @@ fn load_config(args: &CliArgs) -> Result<Config> {}// Embedding backend override: `--embed-mode` flag or `XERJ_EMBED_MODE`- // env (flag wins). Accepts `lexical` | `neural` | `proxy` | `auto`.+ // env (flag wins). Also accepts `onnx-experimental` in a feature-enabled+ // build with explicit model and tokenizer paths.if let Some(mode) = args.embed_mode.clone()@@ -254,12 +272,69 @@ fn load_config(args: &CliArgs) -> Result<Config> {{let mode = mode.trim().to_ascii_lowercase();match mode.as_str() {- "lexical" | "neural" | "proxy" | "auto" => {+ "lexical" | "neural" | "proxy" | "auto" | "onnx-experimental" => {info!("embedding.mode = {mode} (from CLI/env)");cfg.embedding.mode = mode;}other => {- warn!("ignoring unknown --embed-mode '{other}' (use lexical|neural|proxy|auto)");+ anyhow::bail!("unknown --embed-mode '{other}'; use lexical|neural|proxy|auto|onnx-experimental");+ }+ }+ }+ if let Some(path) = args+ .onnx_model+ .clone()+ .or_else(|| std::env::var("XERJ_ONNX_MODEL").ok())+ {+ cfg.embedding.onnx_model_path = path;+ }+ if let Some(path) = args+ .onnx_tokenizer+ .clone()+ .or_else(|| std::env::var("XERJ_ONNX_TOKENIZER").ok())+ {+ cfg.embedding.onnx_tokenizer_path = path;+ }+ if cfg.embedding.mode == "onnx-experimental" {+ #[cfg(not(feature = "onnx-experimental"))]+ anyhow::bail!(+ "onnx-experimental was requested, but this binary was built without it; \+ rebuild with `cargo build --release -p xerj-server --features onnx-experimental`"+ );+ #[cfg(feature = "onnx-experimental")]+ {+ let model = Path::new(&cfg.embedding.onnx_model_path);+ let tokenizer = Path::new(&cfg.embedding.onnx_tokenizer_path);+ if !model.is_file() {+ anyhow::bail!(+ "onnx-experimental needs a readable FP32 model file at {:?}; \+ pass --onnx-model <PATH> (no model is downloaded automatically)",+ cfg.embedding.onnx_model_path+ );+ }+ if !tokenizer.is_file() {+ anyhow::bail!(+ "onnx-experimental needs the matching tokenizer.json at {:?}; \+ pass --onnx-tokenizer <PATH>",+ cfg.embedding.onnx_tokenizer_path+ );+ }+ if cfg.embedding.onnx_max_inflight_calls == 0+ || cfg.embedding.onnx_max_input_bytes_per_call == 0+ || cfg.embedding.onnx_max_inflight_input_bytes == 0+ || cfg.embedding.onnx_max_input_bytes_per_call+ > cfg.embedding.onnx_max_inflight_input_bytes+ || cfg.embedding.onnx_max_inflight_input_bytes > u32::MAX as usize+ {+ anyhow::bail!(+ "invalid ONNX admission limits: inflight_calls={}, bytes_per_call={}, \+ inflight_bytes={}; use non-zero values, keep bytes_per_call <= \+ inflight_bytes, and inflight_bytes <= {}",+ cfg.embedding.onnx_max_inflight_calls,+ cfg.embedding.onnx_max_input_bytes_per_call,+ cfg.embedding.onnx_max_inflight_input_bytes,+ u32::MAX+ );}}}@@ -461,9 +536,18 @@ async fn build_tls_config(cfg: &Config) -> Result<Option<RustlsConfig>> {// ─────────────────────────────────────────────────────────────────────────────fn init_tracing(logging: &xerj_common::config::LoggingConfig) {- let filter = EnvFilter::try_from_env("XERJ_LOG")+ let mut filter = EnvFilter::try_from_env("XERJ_LOG").or_else(|_| EnvFilter::try_from_env("RUST_LOG")).unwrap_or_else(|_| EnvFilter::new("info"));+ let onnx_log = std::env::var("XERJ_ONNX_LOG")+ .unwrap_or_default()+ .to_ascii_lowercase();+ if !matches!(onnx_log.as_str(), "info" | "debug" | "verbose" | "trace") {+ // `ort`'s tracing bridge emits allocator and graph-optimizer internals+ // at INFO independently of the ONNX Runtime session severity. Keep+ // normal XERJ logs concise; XERJ_ONNX_LOG explicitly opts back in.+ filter = filter.add_directive("ort=warn".parse().expect("valid ort log directive"));+ }let builder = tracing_subscriber::fmt().with_env_filter(filter)@@ -727,6 +811,8 @@ async fn run_cli_index(cmd: IndexCmdArgs) -> Result<()> {data_dir: cmd.data_dir.clone(),insecure: true,embed_mode: None,+ onnx_model: None,+ onnx_tokenizer: None,};let mut cfg = load_config(&fake_cli)?;// Tracing after config so the [logging] format applies (RC4-W4 item 6).