Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

perf(binary-protocol): _bulk shares one embedding pass across the batch (#903) (#912)

xerj-org0f2ebccb389a
diff --git a/CHANGELOG.md b/CHANGELOG.md
index d59292b92..638dd0ae7 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -115,9 +115,48 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
documents ~15 docs/s is the cost of CPU transformer inference, and moving it
needs quantization, a GPU provider or a smaller model. What was genuinely
broken was mixed-length work, which is what pointing `autoindex` at a real
- folder produces. Binary-protocol `_bulk` still embeds one document per
- inference call (`xerj-api/src/binary_protocol.rs`); the HTTP path this
- release measures does not.
+ folder produces. Binary-protocol `_bulk` embedded one document per inference
+ call when this was written (`xerj-api/src/binary_protocol.rs`); that half is
+ the entry below.
+
+- **Binary-protocol `_bulk` shares one embedding pass across the batch instead
+ of one inference call per document**
+ ([#903](https://github.com/xerj-org/xerj/issues/903), split out of #366).
+ `handle_bulk` looped `index_document` per document, and `index_document`
+ embeds its own `semantic_text` fields — so a 64-document bulk cost 64
+ inference calls of one passage each. It now collects the batch and hands it
+ to `Index::index_documents_batched`, which routes it through the same
+ `apply_semantic_embeddings_batch` entry the HTTP `_bulk` path uses: passages
+ are windowed at `embedding.onnx_scheduling_window` (default 64) across
+ documents, publication stays per document, and one document's failure still
+ fails only that document. There is no second batching rule.
+
+ Measured end to end over a real TCP connection through the handler with the
+ harness added here (`cargo run --release -p xerj-api --features neural
+ --example binary_bulk_throughput -- <documents> <per-bulk>`), on this repo's
+ 32-core x86_64 box with `--embed-mode neural` (built-in Candle backend,
+ `sentence-transformers/all-MiniLM-L6-v2`). Corpus: 1 000 lines of
+ `apache/lucene`'s `lucene/CHANGES.txt`, 40–400 bytes each so every document
+ is exactly one passage. Medians of interleaved runs; the box was shared with
+ other builds, so the spread is wide and the medians are quoted, not the best
+ runs:
+
+ | documents per bulk | before | after |
+ |---|---|---|
+ | 16 (n=3) | 60.7 docs/s | 82.6 docs/s |
+ | 64 (n=5) | 64.0 docs/s | 102.7 docs/s |
+ | 256 (n=3) | 84.3 docs/s | 124.9 docs/s |
+
+ **Bounds, stated plainly.** The win is a short-document win. A document long
+ enough to fill a scheduling window on its own already got its own inference
+ call: `semantic_embedding_window_end` admits a whole document even past the
+ window (`index.rs`), so before and after are the same call for it. On the
+ default lexical feature-hashing embedder there is no inference to batch and
+ no change was measured — 872 → 865 docs/s (medians of n=7 interleaved,
+ ranges fully overlapping). And the module this fixes is a library entry
+ point: `serve_binary_protocol` is not bound by any listener in `xerj-server`
+ (`xerj-engine/src/index_guard.rs` says so, and nothing in the tree calls
+ it), so no shipped-server ingest path changes speed with this release.
### Fixed
diff --git a/engine/crates/xerj-api/Cargo.toml b/engine/crates/xerj-api/Cargo.toml
index 8ea6bcdf0..91b7ece86 100644
--- a/engine/crates/xerj-api/Cargo.toml
+++ b/engine/crates/xerj-api/Cargo.toml
@@ -6,6 +6,10 @@ license.workspace = true
authors.workspace = true
[features]
+# Built-in Candle neural embedder, for the `binary_bulk_throughput`
+# example (#903) and for anyone linking this crate directly. The shipped
+# binary turns it on through `xerj-server`'s own `neural` feature.
+neural = ["xerj-engine/neural"]
onnx-experimental = ["xerj-engine/onnx-experimental"]
[dependencies]
diff --git a/engine/crates/xerj-api/examples/binary_bulk_throughput.rs b/engine/crates/xerj-api/examples/binary_bulk_throughput.rs
new file mode 100644
index 000000000..0557c8f5e
--- /dev/null
+++ b/engine/crates/xerj-api/examples/binary_bulk_throughput.rs
@@ -0,0 +1,221 @@
+//! End-to-end throughput harness for the binary-protocol `_bulk` handler
+//! (issue #903), driven over a real TCP connection through
+//! [`xerj_api::binary_protocol::serve_binary_protocol`].
+//!
+//! `xerj-ai/examples/neural_throughput.rs` measures the encoder in isolation.
+//! This one measures what a client actually gets: documents per second from
+//! the moment the BULK frame is written to the moment its `RESP_OK` comes
+//! back, with the whole ingest path (embedding, WAL, memtable, HNSW) in the
+//! middle. It exists so the #903 before/after can be quoted from a run rather
+//! than reasoned about from the call shape.
+//!
+//! It is only interesting with an ACTIVE embedding backend — the default
+//! lexical feature-hashing embedder does no batched inference at all, so its
+//! numbers say nothing about batching. Run it as:
+//!
+//! ```sh
+//! cargo run --release -p xerj-api --features neural \
+//! --example binary_bulk_throughput -- <documents> <documents-per-bulk>
+//! ```
+//!
+//! Environment:
+//!
+//! * `XERJ_BULK_CORPUS` — a text file, one document per line. Lines shorter
+//! than 40 or longer than 400 bytes are skipped so every document is exactly
+//! one passage (the semantic chunker cuts at 512 characters), which keeps
+//! documents/s and passages/s the same number. Without it the harness
+//! synthesizes short lines and says so.
+//! * `XERJ_NEURAL_LOCAL_DIR` — load MiniLM weights from a local directory
+//! instead of the HuggingFace cache.
+//! * `XERJ_BULK_MODE` — `neural` (default), `lexical`, or `proxy`.
+
+use std::io::{Read, Write};
+use std::net::TcpStream;
+use std::sync::Arc;
+use std::time::Instant;
+
+use serde_json::{json, Value};
+use xerj_api::binary_protocol::{op, serve_binary_protocol};
+use xerj_api::state::AppState;
+use xerj_common::types::{FieldConfig, FieldType, Schema};
+
+/// all-MiniLM-L6-v2's hidden size, which is also the lexical fallback's width.
+const DIMS: usize = 384;
+
+fn corpus(count: usize) -> (Vec<String>, &'static str) {
+ match std::env::var("XERJ_BULK_CORPUS") {
+ Ok(path) => {
+ let text = std::fs::read_to_string(&path).expect("read XERJ_BULK_CORPUS");
+ let docs: Vec<String> = text
+ .lines()
+ .map(str::trim)
+ .filter(|line| (40..=400).contains(&line.len()))
+ .map(str::to_owned)
+ .take(count)
+ .collect();
+ assert!(
+ docs.len() == count,
+ "corpus {path} yielded {} usable lines, needed {count}",
+ docs.len()
+ );
+ (docs, "real")
+ }
+ Err(_) => (
+ (0..count)
+ .map(|i| {
+ format!(
+ "synthetic record {i}: a short line of prose standing in for a log \
+ message or a chat turn"
+ )
+ })
+ .collect(),
+ "synthetic",
+ ),
+ }
+}
+
+fn frame(stream: &mut TcpStream, op_code: u8, payload: &Value) -> Value {
+ let body = serde_json::to_vec(payload).expect("serialize");
+ let mut header = [0u8; 5];
+ header[..4].copy_from_slice(&(body.len() as u32).to_le_bytes());
+ header[4] = op_code;
+ stream.write_all(&header).expect("write header");
+ stream.write_all(&body).expect("write payload");
+ stream.flush().expect("flush");
+
+ let mut resp_header = [0u8; 5];
+ stream.read_exact(&mut resp_header).expect("read header");
+ let length = u32::from_le_bytes([
+ resp_header[0],
+ resp_header[1],
+ resp_header[2],
+ resp_header[3],
+ ]) as usize;
+ let mut resp = vec![0u8; length];
+ stream.read_exact(&mut resp).expect("read payload");
+ assert_eq!(
+ resp_header[4],
+ op::RESP_OK,
+ "{}",
+ String::from_utf8_lossy(&resp)
+ );
+ serde_json::from_slice(&resp).expect("response json")
+}
+
+#[tokio::main]
+async fn main() {
+ let mut args = std::env::args().skip(1);
+ let documents: usize = args
+ .next()
+ .map(|a| a.parse().expect("documents"))
+ .unwrap_or(1000);
+ let per_bulk: usize = args
+ .next()
+ .map(|a| a.parse().expect("documents-per-bulk"))
+ .unwrap_or(64);
+ let mode = std::env::var("XERJ_BULK_MODE").unwrap_or_else(|_| "neural".to_string());
+
+ let dir = tempfile::tempdir().expect("tempdir");
+ let mut config = xerj_common::config::Config::default();
+ config.server.data_dir = dir.path().to_string_lossy().into_owned();
+ config.storage.wal_sync = xerj_common::config::WalSync::Async;
+ config.embedding.mode = mode.clone();
+ if let Ok(local) = std::env::var("XERJ_NEURAL_LOCAL_DIR") {
+ config.embedding.local_model_dir = local;
+ }
+
+ let metrics = xerj_common::metrics::Metrics::new().expect("metrics");
+ let engine = xerj_engine::Engine::new(config.clone()).expect("engine");
+
+ let mut schema = Schema::empty();
+ let mut body = FieldConfig::new("body", FieldType::Text);
+ body.options.dimensions = Some(DIMS);
+ body.options.similarity = Some("cosine".into());
+ body.embedding = Some(xerj_common::types::EmbeddingConfig {
+ endpoint: None,
+ model: None,
+ target_field: Some("body_vector".into()),
+ });
+ schema.add_field(body).expect("semantic field");
+ let mut companion = FieldConfig::new("body_vector", FieldType::Vector);
+ companion.options.dimensions = Some(DIMS);
+ companion.options.similarity = Some("cosine".into());
+ schema.add_field(companion).expect("companion field");
+ engine.create_index("bulkbench", schema).expect("create");
+
+ let state = Arc::new(AppState::new(config, engine, metrics));
+
+ // Take an ephemeral port from the kernel, release it, and hand the address
+ // to the server — `serve_binary_protocol` binds for itself.
+ let probe = std::net::TcpListener::bind("127.0.0.1:0").expect("probe bind");
+ let addr = probe.local_addr().expect("probe addr");
+ drop(probe);
+ tokio::spawn(serve_binary_protocol(addr, Arc::clone(&state)));
+
+ let mut stream = None;
+ for _ in 0..200 {
+ match TcpStream::connect(addr) {
+ Ok(s) => {
+ stream = Some(s);
+ break;
+ }
+ Err(_) => std::thread::sleep(std::time::Duration::from_millis(25)),
+ }
+ }
+ let mut stream = stream.expect("connect to binary protocol");
+ stream.set_nodelay(true).expect("nodelay");
+
+ // Warm-up: the first bulk pays model load and tokenizer init, which is a
+ // one-off cost and not what this measures. Its documents are separate ids
+ // so the timed run is all inserts, never updates.
+ let (warm, _) = corpus(per_bulk);
+ let warm_docs: Vec<Value> = warm
+ .iter()
+ .enumerate()
+ .map(|(i, text)| json!({ "_id": format!("warm-{i}"), "body": text }))
+ .collect();
+ let warm_resp = frame(
+ &mut stream,
+ op::BULK,
+ &json!({ "index": "bulkbench", "docs": warm_docs }),
+ );
+ assert_eq!(warm_resp["errors"], json!(false), "warm-up: {warm_resp}");
+
+ let (texts, provenance) = corpus(documents);
+ let bytes: usize = texts.iter().map(String::len).sum();
+ let longest = texts.iter().map(String::len).max().unwrap_or(0);
+ let batches: Vec<Vec<Value>> = texts
+ .chunks(per_bulk)
+ .enumerate()
+ .map(|(b, chunk)| {
+ chunk
+ .iter()
+ .enumerate()
+ .map(|(i, text)| json!({ "_id": format!("d{b}-{i}"), "body": text }))
+ .collect()
+ })
+ .collect();
+
+ let started = Instant::now();
+ let mut indexed = 0i64;
+ for batch in &batches {
+ let resp = frame(
+ &mut stream,
+ op::BULK,
+ &json!({ "index": "bulkbench", "docs": batch }),
+ );
+ assert_eq!(resp["errors"], json!(false), "bulk: {resp}");
+ indexed += resp["indexed"].as_i64().unwrap_or(0);
+ }
+ let elapsed = started.elapsed();
+
+ assert_eq!(indexed, documents as i64, "every document must be indexed");
+ println!(
+ "mode={mode} corpus={provenance} documents={documents} per_bulk={per_bulk} \
+ bulks={} mean_doc_bytes={} longest_doc_bytes={longest} elapsed_s={:.3} docs_per_s={:.2}",
+ batches.len(),
+ bytes / documents,
+ elapsed.as_secs_f64(),
+ documents as f64 / elapsed.as_secs_f64(),
+ );
+}
diff --git a/engine/crates/xerj-api/src/binary_protocol.rs b/engine/crates/xerj-api/src/binary_protocol.rs
index e71405987..fe4a48c52 100644
--- a/engine/crates/xerj-api/src/binary_protocol.rs
+++ b/engine/crates/xerj-api/src/binary_protocol.rs
@@ -382,13 +382,26 @@ async fn handle_bulk(
let mut indexed = 0i32;
let mut errors = false;
- for doc in req.docs {
- let id = doc
- .get("_id")
- .and_then(Value::as_str)
- .map(str::to_owned)
- .unwrap_or_else(|| Uuid::new_v4().to_string());
- if idx.index_document(Some(id), doc).await.is_ok() {
+ // One embedding pass for the whole batch, not one per document. The
+ // engine's `index_documents_batched` routes the batch through the same
+ // `apply_semantic_embeddings_batch` entry the HTTP `_bulk` path uses, so
+ // both bulk entry points share one batching rule; publication is still
+ // per document and per-document failures stay isolated (#903).
+ let docs: Vec<(Option<String>, Value)> = req
+ .docs
+ .into_iter()
+ .map(|doc| {
+ let id = doc
+ .get("_id")
+ .and_then(Value::as_str)
+ .map(str::to_owned)
+ .unwrap_or_else(|| Uuid::new_v4().to_string());
+ (Some(id), doc)
+ })
+ .collect();
+
+ for result in idx.index_documents_batched(docs).await {
+ if result.is_ok() {
indexed += 1;
state.metrics.record_doc_indexed(&req.index);
} else {
@@ -457,3 +470,242 @@ pub async fn serve_binary_protocol(
});
}
}
+
+#[cfg(test)]
+mod tests {
+ //! #903: the binary `_bulk` handler must not embed one document per
+ //! inference call.
+ //!
+ //! The backend here is an OpenAI-compatible proxy served on an ephemeral
+ //! local port, because `EmbeddingProxy::embed_batch` issues exactly one
+ //! HTTP POST per call — so the request count *is* the inference-call
+ //! count, with no model download and no timing noise. The default lexical
+ //! embedder is deliberately not used: it is not an active backend, it runs
+ //! no batched inference, and a test against it would prove nothing here.
+
+ use std::sync::atomic::{AtomicUsize, Ordering};
+
+ use serde_json::json;
+ use xerj_common::types::{FieldConfig, FieldType, Schema};
+
+ use super::*;
+
+ const DIMS: usize = 8;
+
+ /// What the embedding backend was actually asked to do.
+ struct BackendCalls {
+ calls: Arc<AtomicUsize>,
+ widest_call: Arc<AtomicUsize>,
+ texts: Arc<AtomicUsize>,
+ }
+
+ /// A mock OpenAI-compatible `/v1/embeddings` endpoint on an ephemeral port.
+ async fn mock_embedding_backend() -> (String, BackendCalls) {
+ let calls = Arc::new(AtomicUsize::new(0));
+ let widest_call = Arc::new(AtomicUsize::new(0));
+ let texts = Arc::new(AtomicUsize::new(0));
+ let (c, w, t) = (
+ Arc::clone(&calls),
+ Arc::clone(&widest_call),
+ Arc::clone(&texts),
+ );
+ let app = axum::Router::new().route(
+ "/v1/embeddings",
+ axum::routing::post(move |axum::Json(body): axum::Json<Value>| {
+ let (c, w, t) = (Arc::clone(&c), Arc::clone(&w), Arc::clone(&t));
+ async move {
+ let n = body
+ .get("input")
+ .and_then(Value::as_array)
+ .map(Vec::len)
+ .unwrap_or(0);
+ c.fetch_add(1, Ordering::SeqCst);
+ t.fetch_add(n, Ordering::SeqCst);
+ w.fetch_max(n, Ordering::SeqCst);
+ let data: Vec<Value> = (0..n)
+ .map(|i| {
+ let embedding: Vec<f64> = (0..DIMS)
+ .map(|d| ((i + d) % 7) as f64 / 7.0 + 0.1)
+ .collect();
+ json!({ "index": i, "embedding": embedding })
+ })
+ .collect();
+ axum::Json(json!({ "data": data }))
+ }
+ }),
+ );
+ let listener = TcpListener::bind("127.0.0.1:0")
+ .await
+ .expect("bind mock backend");
+ let addr = listener.local_addr().expect("mock backend addr");
+ tokio::spawn(async move {
+ let _ = axum::serve(listener, app).await;
+ });
+ (
+ format!("http://{addr}/v1/embeddings"),
+ BackendCalls {
+ calls,
+ widest_call,
+ texts,
+ },
+ )
+ }
+
+ /// An index with one `semantic_text`-shaped field, served by the binary
+ /// protocol on an ephemeral port. The temp dir must outlive the engine.
+ async fn node(endpoint: String) -> (std::net::SocketAddr, tempfile::TempDir) {
+ let dir = tempfile::tempdir().expect("tempdir");
+ let mut config = xerj_common::config::Config::default();
+ config.server.data_dir = dir.path().to_string_lossy().into_owned();
+ config.storage.wal_sync = xerj_common::config::WalSync::Async;
+ config.embedding.mode = "proxy".into();
+ config.embedding.default_endpoint = endpoint;
+ config.embedding.default_model = "mock-embed".into();
+
+ let metrics = xerj_common::metrics::Metrics::new().expect("metrics");
+ let engine = xerj_engine::Engine::new(config.clone()).expect("engine");
+
+ let mut schema = Schema::empty();
+ let mut body = FieldConfig::new("body", FieldType::Text);
+ body.options.dimensions = Some(DIMS);
+ body.options.similarity = Some("cosine".into());
+ body.embedding = Some(xerj_common::types::EmbeddingConfig {
+ endpoint: None,
+ model: None,
+ target_field: Some("body_vector".into()),
+ });
+ schema.add_field(body).expect("semantic field");
+ let mut companion = FieldConfig::new("body_vector", FieldType::Vector);
+ companion.options.dimensions = Some(DIMS);
+ companion.options.similarity = Some("cosine".into());
+ schema.add_field(companion).expect("companion field");
+ engine.create_index("notes", schema).expect("create index");
+
+ let state = Arc::new(crate::state::AppState::new(config, engine, metrics));
+ let listener = TcpListener::bind("127.0.0.1:0")
+ .await
+ .expect("bind binary protocol");
+ let addr = listener.local_addr().expect("binary protocol addr");
+ tokio::spawn(async move {
+ while let Ok((stream, _)) = listener.accept().await {
+ tokio::spawn(handle_connection(stream, Arc::clone(&state)));
+ }
+ });
+ (addr, dir)
+ }
+
+ /// One framed request/response round trip on a fresh connection.
+ async fn round_trip(addr: std::net::SocketAddr, op_code: u8, payload: Value) -> Value {
+ let mut stream = TcpStream::connect(addr).await.expect("connect");
+ let body = serde_json::to_vec(&payload).expect("serialize");
+ let mut header = [0u8; 5];
+ header[..4].copy_from_slice(&(body.len() as u32).to_le_bytes());
+ header[4] = op_code;
+ stream.write_all(&header).await.expect("write header");
+ stream.write_all(&body).await.expect("write payload");
+ stream.flush().await.expect("flush");
+
+ let mut resp_header = [0u8; 5];
+ stream
+ .read_exact(&mut resp_header)
+ .await
+ .expect("read header");
+ let length = u32::from_le_bytes([
+ resp_header[0],
+ resp_header[1],
+ resp_header[2],
+ resp_header[3],
+ ]) as usize;
+ let mut resp = vec![0u8; length];
+ stream.read_exact(&mut resp).await.expect("read payload");
+ assert_eq!(
+ resp_header[4],
+ op::RESP_OK,
+ "response: {}",
+ String::from_utf8_lossy(&resp)
+ );
+ serde_json::from_slice(&resp).expect("response json")
+ }
+
+ /// #903: a bulk of N single-passage documents must reach the embedding
+ /// backend as ONE inference call carrying all N passages, not N calls of
+ /// one passage each. Before the fix `handle_bulk` looped `index_document`
+ /// per document and this assertion saw 12 calls of width 1.
+ #[tokio::test]
+ async fn binary_bulk_embeds_the_whole_batch_in_one_inference_call() {
+ let (endpoint, backend) = mock_embedding_backend().await;
+ let (addr, _dir) = node(endpoint).await;
+
+ const DOCS: usize = 12;
+ let docs: Vec<Value> = (0..DOCS)
+ .map(|i| json!({ "_id": format!("d{i}"), "body": format!("passage number {i}") }))
+ .collect();
+ let response = round_trip(addr, op::BULK, json!({ "index": "notes", "docs": docs })).await;
+
+ assert_eq!(
+ response["errors"],
+ json!(false),
+ "bulk response: {response}"
+ );
+ assert_eq!(
+ response["indexed"],
+ json!(DOCS as i64),
+ "bulk response: {response}"
+ );
+ assert_eq!(
+ backend.texts.load(Ordering::SeqCst),
+ DOCS,
+ "every document's passage must still be embedded exactly once"
+ );
+ assert_eq!(
+ backend.widest_call.load(Ordering::SeqCst),
+ DOCS,
+ "the whole batch must share one inference window"
+ );
+ assert_eq!(
+ backend.calls.load(Ordering::SeqCst),
+ 1,
+ "{DOCS} single-passage documents must cost 1 inference call, not one per document"
+ );
+ }
+
+ /// Batching must not widen a single document's failure into the batch's:
+ /// publication stays per document, and the response still counts only what
+ /// was actually indexed.
+ #[tokio::test]
+ async fn a_rejected_document_does_not_fail_its_neighbours() {
+ let (endpoint, backend) = mock_embedding_backend().await;
+ let (addr, _dir) = node(endpoint).await;
+
+ // `body_vector` is the engine-owned target of the semantic field, so
+ // hand-supplied passage metadata for it is rejected per document.
+ let metadata_field = format!(
+ "{}body_vector",
+ xerj_query::executor::PASSAGE_METADATA_PREFIX
+ );
+ let docs = vec![
+ json!({ "_id": "ok-1", "body": "first passage" }),
+ json!({
+ "_id": "bad",
+ "body": "second passage",
+ metadata_field.clone(): { "field": "body", "chunks": [[0, 3]] }
+ }),
+ json!({ "_id": "ok-2", "body": "third passage" }),
+ ];
+ let response = round_trip(addr, op::BULK, json!({ "index": "notes", "docs": docs })).await;
+
+ assert_eq!(response["total"], json!(3), "bulk response: {response}");
+ assert_eq!(response["indexed"], json!(2), "bulk response: {response}");
+ assert_eq!(response["errors"], json!(true), "bulk response: {response}");
+ assert_eq!(
+ backend.calls.load(Ordering::SeqCst),
+ 1,
+ "the surviving documents still share one inference call"
+ );
+
+ let found = round_trip(addr, op::GET, json!({ "index": "notes", "id": "ok-2" })).await;
+ assert_eq!(found["found"], json!(true), "get response: {found}");
+ let missing = round_trip(addr, op::GET, json!({ "index": "notes", "id": "bad" })).await;
+ assert_eq!(missing["found"], json!(false), "get response: {missing}");
+ }
+}
diff --git a/engine/crates/xerj-engine/src/index.rs b/engine/crates/xerj-engine/src/index.rs
index 10ad285e4..805f3603e 100644
--- a/engine/crates/xerj-engine/src/index.rs
+++ b/engine/crates/xerj-engine/src/index.rs
@@ -8587,6 +8587,39 @@ impl Index {
.await
}
+ /// Index several documents, sharing ONE embedding pass across the batch.
+ ///
+ /// Publication is per document and identical to [`Index::index_document`];
+ /// the only difference is that `semantic_text` auto-embedding runs once for
+ /// the whole batch through `apply_semantic_embeddings_batch` — the same
+ /// entry the HTTP `_bulk` path uses — instead of once per document. With an
+ /// active backend that is one inference call per `onnx_scheduling_window`
+ /// of passages rather than one call per document (a single document whose
+ /// own passages exceed the window is still admitted whole, so it keeps its
+ /// own call — see `semantic_embedding_window_end`).
+ ///
+ /// The returned vector is position-aligned with `docs`: a document whose
+ /// embedding or publication failed carries its own error and does not
+ /// affect its neighbours.
+ pub async fn index_documents_batched(
+ &self,
+ docs: Vec<(Option<String>, Value)>,
+ ) -> Vec<Result<IndexResponse>> {
+ if docs.is_empty() {
+ return Vec::new();
+ }
+ let (ids, sources): (Vec<Option<String>>, Vec<Value>) = docs.into_iter().unzip();
+ let prepared = self.apply_semantic_embeddings_batch(sources).await;
+ let mut responses = Vec::with_capacity(ids.len());
+ for (id, prepared_source) in ids.into_iter().zip(prepared) {
+ responses.push(match prepared_source {
+ Ok(source) => self.index_document_prepared(id, source).await,
+ Err(e) => Err(e),
+ });
+ }
+ responses
+ }
+
async fn index_document_with_version_inner(
&self,
id: Option<String>,