AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
docs(vectors): say that a mapping's quantization value is not validated
xerj-org7bb6eb77f575
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 80b35923e..760fd5877 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -100,6 +100,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0`Config::default()`. An example that deliberately differs (the blocks whosepoint is switching TLS or cluster mode on) has to carry `# not a default` onthe line, so the reader is told as well as the test.+ - **Known gap, found while correcting the `scalar4` prose and disclosed+ rather than quietly fixed:** the guard reads `config.html` against+ `Config::default()`, so it covers the *config* half only. Nothing checks a+ *mapping*, and a mapping is not validated either —+ `es_compat.rs` matches `"scalar8" | "int8" | "none"` on a `dense_vector`+ field's `quantization` and falls through `_ => {}`, so+ `"quantization": "scalar4"` (or `"binary"`, or a typo like `"sq8"`) is+ accepted with a 200, echoed back verbatim by `GET /_mapping`, and ignored+ — the field is stored at full-precision f32 while the mapping says it is+ quantized. Measured through the ES-compat router at this commit. It is+ pre-existing and outside this diff; it is another instance of the+ accepted-and-ignored class in+ [#204](https://github.com/xerj-org/xerj/issues/204), is filed with the+ repro and a proposed 400 as+ [#275](https://github.com/xerj-org/xerj/issues/275), and+ `landing/docs/vectors.html` now says so on the page instead of leaving+ "not a mode you can select" to imply the value is rejected.- **Four other claims from [#207](https://github.com/xerj-org/xerj/issues/207)now match the source.**diff --git a/landing/docs/vectors.html b/landing/docs/vectors.htmlindex bb0c4fb00..702ba5de2 100644--- a/landing/docs/vectors.html+++ b/landing/docs/vectors.html@@ -157,6 +157,7 @@<li><strong>binary</strong> — 1-bit per dimension. Not implemented: the server <strong>refuses to start</strong> on <code class="inline">default_quantization = "binary"</code> rather than silently storing full precision while claiming a 32× saving.</li></ul><p>A 4-bit quantizer exists in the vector crate but no config or mapping value reaches it, so <code class="inline">scalar4</code> is not a mode you can select in this build.</p>+ <p><strong>Known gap — a mapping's <code class="inline">quantization</code> value is not validated</strong> (<a href="https://github.com/xerj-org/xerj/issues/275">#275</a>). Startup refuses an unimplemented <code class="inline">[vector] default_quantization</code>, but the per-field mapping key does not: <code class="inline">"quantization": "scalar4"</code> — or <code class="inline">"binary"</code>, or a typo such as <code class="inline">"sq8"</code> — is accepted with a <code class="inline">200</code>, echoed back verbatim by <code class="inline">GET /_mapping</code>, and then ignored, so the field is stored at full-precision f32 while the mapping you read back says otherwise. Measured on rc.14. Until it is a <code class="inline">400</code>, <code class="inline">"scalar8"</code> (or <code class="inline">"int8"</code>, or <code class="inline">index_options.type: int8_hnsw</code>/<code class="inline">int8_flat</code>) is the only value that changes anything, and reading the mapping back is not a way to confirm it took.</p><h2 id="knn-query">KNN query</h2><pre class="code">{