AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
feat(search): rerank stage with a pluggable judge provider (#946)
xerj-orgaaeae25cd79b
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex b3a7f0e71..8c84db4ba 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -161,6 +161,48 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0never silently degrades to one leg. The MCP `xerj_memory_recall` toolexposes the same two parameters. Raised by @Vinz2168 from a shared-memoryagent integration where neither single mode was enough.+- **`rerank` stage on `_search`: hand the top hits to an external relevance+ judge and reorder by the calibrated probability that comes back.**+ `"rerank": {}` sends the question and the text of the top `window` hits+ (default 30, max 300) to TypeSafe AI's Jev; the 0–1 probability replaces+ `_score`, so `rerank.min_score` is an absolute cut-off, which a BM25 score+ cannot be. **It is the only search-time feature that sends document text+ off the node** (`[embedding] default_endpoint` and the WAL tap also send text+ off the node when an operator configures them; docs/RERANK.md lists every+ outbound connection a node can open, and a test fails when the engine source+ gains one that list does not name): inert until+ an operator sets `[rerank] api_key` (or+ `TYPESAFE_API_KEY`; config wins over env), opt-in per request, forbidden+ outright by `[rerank] enabled = false`, and only fields the response returns+ are sent — `rerank.fields` is an exhaustive allow-list. Failure policy is+ *degrade on deadline, surface on contract*: a slow provider is a 200 with the+ engine's order and `_rerank.applied: false`; no key is 503, disabled 403, a+ rejected key or malformed body 502, all with no hits; a provider that answers+ for nothing it was sent is a 200 with `applied: false`. When applied, every+ `_score` is a probability or `null` (a hit with no verdict, counted in+ `_rerank.unjudged`, sorts last) — never an engine score beside probabilities.+ `hits.total` and `aggs`+ stay the engine's; paging happens inside the window; `sort`, `search_after`,+ `collapse`, scroll and `size: 0` are 400s; and `_msearch`, search templates,+ `_async_search`, `_rank_eval` (per request, under `failures`), the native+ `/v1` search API and gRPC refuse the block instead of dropping it. Every+ caller-chosen cost knob has a server-side ceiling, the strings included:+ `instructions` is capped at 2,000 characters because the provider's wire+ format repeats it once per judged document, the question at 4,000 and+ `model` at 128; a provider response over 2 MiB is a 502, not an allocation.+ `GET /_xerj/rerank` reports whether a provider is configured+ and never the key; `/v1/metrics` gains `xerj_rerank_requests_total{outcome}`,+ `xerj_rerank_documents_judged_total` and+ `xerj_rerank_provider_tokens_total{kind}`; the MCP `xerj_search` and+ `xerj_hybrid_search` tools take an optional `rerank` argument. **Ranking+ quality with the real model is not verified** — no provider key was+ available, and every HTTP test runs against an in-process test double.+ New crate `xerj-rerank`; reference in `docs/RERANK.md`; two benchmarks,+ `benchmarks/beir-hybrid` and `benchmarks/decisions-as-retrieval`, both+ measured with `--embed-mode neural`, not the default lexical embedder. The+ one-question-per-document request shape follows `hev/jev-rerank`+ (Apache-2.0); the failure policy and the retry back-off constants follow+ Meilisearch's personalization module (MIT; adapted, cited in code).### Fixeddiff --git a/README.md b/README.mdindex eb3a320e6..ec43c6deb 100644--- a/README.md+++ b/README.md@@ -410,6 +410,12 @@ limits each one does not lift:snapshot API, and what restore replaces.- [Security model](./docs/SECURITY_MODEL.md) for authentication, the reserved`.xerj-memory-*` namespace, API keys and what is not enforced.+- [Reranking](./docs/RERANK.md) for the optional `rerank` search stage: an external judge+ reorders the top hits by a calibrated probability. It is the only search-time feature+ that sends document text off the node (proxy embeddings and the WAL tap also send text+ when an operator configures them; the page lists every outbound connection a node can+ open), it needs an operator-configured provider key, and+ its ranking quality with the real model is not verified.- [XERJ vs Lucene 10.3.1](./docs/XERJ_VS_LUCENE.md) for a source-pinned, six-axis comparisonof their storage and search designs.diff --git a/ROADMAP.md b/ROADMAP.mdindex 71c626a57..ad7786b49 100644--- a/ROADMAP.md+++ b/ROADMAP.md@@ -86,7 +86,7 @@ The 1.0 bar: **every public claim verified against the release binary, and every**Stage 1 — in flight now** (each lands as its own PR; none is merged at the time of writing):-- **Judged search — a rerank stage.** Retrieve with BM25 or `hybrid`, then have a second stage re-judge the top *N* before the page is returned. *Today on `main`:* the ES `rescore` block (query rescorer) and the `hybrid` query type with `rrf|linear`; there is no rerank stage. *In flight:* branch `feat/rerank-stage` — a `rerank` block on `_search` that hands the top hits to an external relevance judge, opt-in per request and inert until an operator configures a key. It is the one feature that would send document text off the machine, which is exactly why a *local* judge is on this list. *The bar for a local model is a measured one.* On the BEIR test splits with the opt-in neural embedder (`--embed-mode neural`, all-MiniLM-L6-v2, CPU) the shipped `hybrid` RRF query is already the best arm we have — nDCG@10 **0.699–0.704 on SciFact and 0.345 on NFCorpus** over three runs, against 0.657 / 0.302 for BM25 alone, 0.676 / 0.329 for vectors alone, and 0.686 / 0.332 for "BM25 top-30 re-ordered by the same bi-encoder", which is *worse* than hybrid on both. So re-ordering with the embedder we already ship is not worth building, and **a local cross-encoder ships only if it beats 0.699 / 0.345 on the same harness, by more than the run-to-run spread** ([`benchmarks/neural-path-triage/`](./benchmarks/neural-path-triage/)). Those figures are with the neural embedder; the default embedder is lexical feature hashing and was not part of this run. The hybrid figure is the only arm that does not reproduce — 0.6993, 0.7023 and 0.7044 on SciFact across three runs of unchanged indices — because tied RRF scores are ordered by a per-process hash seed ([#940](https://github.com/xerj-org/xerj/issues/940)); a gain inside that spread is not a gain.+- **Judged search — a rerank stage.** Retrieve with BM25 or `hybrid`, then have a second stage re-judge the top *N* before the page is returned. *Today on `main`:* the ES `rescore` block (query rescorer) and the `hybrid` query type with `rrf|linear`; there is no rerank stage. *In flight:* branch `feat/rerank-stage` — a `rerank` block on `_search` that hands the top hits to an external relevance judge, opt-in per request and inert until an operator configures a key. It is the only *search-time* feature that would send document text off the machine — proxy embeddings (`[embedding] default_endpoint`) send it at write time and the WAL tap replays writes to an external `_bulk` endpoint, both operator-configured and off by default ([every outbound connection](./docs/RERANK.md#every-way-data-leaves-a-xerj-node)) — which is exactly why a *local* judge is on this list. *The bar for a local model is a measured one.* On the BEIR test splits with the opt-in neural embedder (`--embed-mode neural`, all-MiniLM-L6-v2, CPU) the shipped `hybrid` RRF query is already the best arm we have — nDCG@10 **0.699–0.704 on SciFact and 0.345 on NFCorpus** over three runs, against 0.657 / 0.302 for BM25 alone, 0.676 / 0.329 for vectors alone, and 0.686 / 0.332 for "BM25 top-30 re-ordered by the same bi-encoder", which is *worse* than hybrid on both. So re-ordering with the embedder we already ship is not worth building, and **a local cross-encoder ships only if it beats 0.699 / 0.345 on the same harness, by more than the run-to-run spread** ([`benchmarks/neural-path-triage/`](./benchmarks/neural-path-triage/)). Those figures are with the neural embedder; the default embedder is lexical feature hashing and was not part of this run. The hybrid figure is the only arm that does not reproduce — 0.6993, 0.7023 and 0.7044 on SciFact across three runs of unchanged indices — because tied RRF scores are ordered by a per-process hash seed ([#940](https://github.com/xerj-org/xerj/issues/940)); a gain inside that spread is not a gain.- **Share links and a guest reading room.** Hand one person a link to one corpus or one document, read-only, without creating them an account. *Today on `main`:* nothing — there is no share or guest code in the tree, and reading a corpus needs an account session or an API key. *In flight:* branches `feat/share-links` (scoped, expiring links; a share can never name a system index) and `feat/console-corpus-reader` (the corpus browser and the reader the link opens). Both branches carry commits their authors marked unverified; they are not claims yet.- **Mail ingest.** *Today on `main`, unreleased:* `xerj autoindex` extracts `.eml` / MIME messages — headers, body, and attachments indexed as their own records ([#921](https://github.com/xerj-org/xerj/pull/921)) — and, since [#949](https://github.com/xerj-org/xerj/pull/949), streams `mbox` mailboxes and extracted Google Takeout exports (archives are never opened) and writes `replies_to` / `attachment_of` edges from the mail headers. Measured on a synthetic mailbox only; the node's ingest memory is the limit ([#948](https://github.com/xerj-org/xerj/issues/948)).- **`autoindex` resilience.** One dataset the server refuses currently aborts the whole run ([#929](https://github.com/xerj-org/xerj/issues/929)), the `--no-graph` progress line is indistinguishable from a hang ([#931](https://github.com/xerj-org/xerj/issues/931)), and `xc-index.sh --fresh` fails on a previously indexed corpus ([#930](https://github.com/xerj-org/xerj/issues/930)). *In flight:* PR [#934](https://github.com/xerj-org/xerj/pull/934) (`fix/autoindex-resilience`). It is on this list because every later item assumes a folder can be indexed unattended.diff --git a/benchmarks/beir-hybrid/README.md b/benchmarks/beir-hybrid/README.mdnew file mode 100644index 000000000..756362f44--- /dev/null+++ b/benchmarks/beir-hybrid/README.md@@ -0,0 +1,73 @@+# BEIR: what XERJ's shipped retrieval scores, with a free local model++**Run:** 2026-09-18, `xerj v1.0.0-rc.74`, `--embed-mode neural` (built-in Candle BERT,+`sentence-transformers/all-MiniLM-L6-v2`, 384-dim, ~90 MB, CPU). One node, one shard,+default settings. Metric: nDCG@10 on each dataset's BEIR `test` split, computed by+`eval.py` from live queries — nothing is assumed or carried over.++| Arm | SciFact (300 q, 5,183 docs) | NFCorpus (323 q, 3,633 docs) |+|---|---:|---:|+| BM25 (`multi_match` on title, text) | 0.6572 | 0.3016 |+| MiniLM vectors only (`semantic`) | 0.6764 | 0.3291 |+| BM25 top-30 → reorder by MiniLM | 0.6855 | 0.3323 |+| **Hybrid RRF (`hybrid`, server-side)** | **0.6993** | **0.3448** |++Raw output: [`results/`](./results).++## What this says++1. **Hybrid is the best arm XERJ has, on both datasets, and it already ships.** No new+ code is needed to get it.+2. **Reranking a BM25 shortlist with the same bi-encoder is worse than fusing with it.**+ That is why `xerj-rerank` has no "local MiniLM" provider: it would be a regression+ with a good name. A local second stage is only worth building with a cross-encoder,+ and only if it beats 0.699 / 0.345 here.++## Against hosted rerankers — read the caveat++Numbers published in [`hev/jev-rerank`](https://github.com/hev/jev-rerank)'s README for+the same datasets and metric:++| | SciFact | NFCorpus |+|---|---:|---:|+| Jev (TypeSafe AI) | 0.768 | 0.358 |+| Voyage rerank-3 | 0.755 | 0.357 |+| Cohere rerank-v3.5 | 0.745 | 0.340 |+| **XERJ hybrid, local MiniLM, no API** | **0.699** | **0.345** |++**We did not run those systems.** They rerank that project's own first-stage shortlist,+which is not ours, so this is the same dataset and metric but not a controlled+comparison. What can be said: on NFCorpus XERJ's free local hybrid is inside the band of+the hosted rerankers; on SciFact it is 4.6–6.9 points behind them. A controlled+comparison needs a `TYPESAFE_API_KEY` and the `rerank` stage, which now exists.++## Defects this run surfaced++- **Neural indexing was ~3–7 documents/second on CPU in this run.** SciFact (5,183+ short abstracts) took about 25 minutes. That is consistent with the ~3 docs/s figure+ recorded in `xerj-autoindex/src/infer/mod.rs`, and it is the binding constraint on any+ "index your mail on a laptop" story.+- **Hybrid and filtered-semantic queries cost ~220–370 ms p50** on a 5k-document index,+ against ~18 ms for BM25. Most of that is the per-query BERT forward pass on CPU; the+ machine was also compiling during the run, so treat the absolute values as upper+ bounds. The first SciFact vector-arm p50 (0.4 ms) is a **cache artefact** — an earlier+ aborted run had already embedded those queries — and must not be quoted.+- **BM25 returned zero hits for 25 of 323 NFCorpus queries** under default-OR+ `multi_match`. **Explained, and not a defect:** exactly 25 of the test queries+ share no token with any document's title or text — they are single words such+ as `deafness`, `eggnog`, `Fosamax` and `Zoloft` that occur nowhere in the+ corpus. [`lexical_gap.py`](./lexical_gap.py) counts them without a running+ node ([output](./results/nfcorpus-lexical-gap.txt)); the same check reports 0+ for SciFact, where BM25 also had `empty=0`. This is the dataset's lexical gap,+ and it is the clearest reason the vector arm matters on NFCorpus: no reorder+ of a BM25 shortlist can repair an empty list.++## Reproduce++```sh+# a throwaway node on private ports, neural embedder+xerj -c xerj.toml -d ./data --insecure --embed-mode neural &+curl -LO https://public.ukp.informatik.tu-darmstadt.de/thakur/BEIR/datasets/scifact.zip && unzip scifact.zip+XERJ_URL=http://localhost:9410 python3 load.py scifact/corpus.jsonl scifact+XERJ_URL=http://localhost:9410 python3 eval.py 30 scifact+```diff --git a/benchmarks/beir-hybrid/eval.py b/benchmarks/beir-hybrid/eval.pynew file mode 100644index 000000000..f0c8c95ca--- /dev/null+++ b/benchmarks/beir-hybrid/eval.py@@ -0,0 +1,47 @@+"""nDCG@10 on BEIR SciFact (test split) for retrieval arms on a live XERJ node.+Every number printed comes from a query against the node; nothing is assumed."""+import json, math, sys, time, urllib.request, collections+import os+U=os.environ.get("XERJ_URL","http://localhost:9410"); IDX=sys.argv[2] if len(sys.argv)>2 else "scifact"; WINDOW=int(sys.argv[1]) if len(sys.argv)>1 else 30+def post(p,b):+ r=urllib.request.Request(U+p,data=json.dumps(b).encode(),method="POST",headers={"content-type":"application/json"})+ try: return json.loads(urllib.request.urlopen(r,timeout=120).read())+ except urllib.error.HTTPError as e: return {"_err":e.code,"body":e.read().decode()[:300]}+queries={}+for l in open(IDX+"/queries.jsonl"):+ d=json.loads(l); queries[d["_id"]]=d["text"]+qrels=collections.defaultdict(dict)+for i,l in enumerate(open(IDX+"/qrels/test.tsv")):+ if i==0: continue+ q,d,s=l.split("\t"); qrels[q][d]=int(s)+def ndcg10(ranked,rel):+ dcg=sum(rel.get(d,0)/math.log2(i+2) for i,d in enumerate(ranked[:10]))+ ideal=sorted(rel.values(),reverse=True)[:10]+ idcg=sum(g/math.log2(i+2) for i,g in enumerate(ideal))+ return dcg/idcg if idcg else 0.0+def ids(res): return [h["_id"] for h in res.get("hits",{}).get("hits",[])]+BM=lambda q:{"multi_match":{"query":q,"fields":["title","text"]}}+SEM=lambda q:{"semantic":{"field":"body","query":q}}+def bm25(q,n=100): return ids(post(f"/{IDX}/_search",{"size":n,"_source":False,"query":BM(q)}))+def sem(q,n=100): return ids(post(f"/{IDX}/_search",{"size":n,"_source":False,"query":SEM(q)}))+def hybrid(q,n=100):+ return ids(post(f"/{IDX}/_search",{"size":n,"_source":False,"query":{"hybrid":{"queries":[{"query":BM(q)},{"query":SEM(q)}],"fusion":"rrf"}}}))+def rerank_window(q,w):+ """BM25 top-w, reordered by the embedding score; the tail keeps BM25 order."""+ base=bm25(q,100); win=base[:w]+ r=post(f"/{IDX}/_search",{"size":w,"_source":False,"query":{"bool":{"must":[SEM(q)],"filter":[{"ids":{"values":win}}]}}})+ order=ids(r)+ if "_err" in r: raise SystemExit(f"filtered semantic failed: {r}")+ seen=set(order); return order+[d for d in win if d not in seen]+base[w:]+arms={"bm25":bm25,"minilm (vector only)":sem,"hybrid rrf (server)":hybrid,f"bm25 top-{WINDOW} -> minilm rerank":lambda q:rerank_window(q,WINDOW)}+probe=post(f"/{IDX}/_search",{"size":1,"query":SEM("test")})+if "_err" in probe: raise SystemExit(f"semantic arm unavailable: {probe}")+print(f"queries={len(qrels)} docs={post(f'/{IDX}/_count',{}).get('count')}")+for name,fn in arms.items():+ t=time.time(); scores=[]; empty=0; lat=[]+ for qid,rel in qrels.items():+ t0=time.time(); ranked=fn(queries[qid]); lat.append(time.time()-t0)+ if not ranked: empty+=1+ scores.append(ndcg10(ranked,rel))+ lat.sort()+ print(f"{name:34s} nDCG@10={sum(scores)/len(scores):.4f} empty={empty:3d} p50={lat[len(lat)//2]*1000:6.1f}ms p95={lat[int(len(lat)*.95)]*1000:6.1f}ms")diff --git a/benchmarks/beir-hybrid/lexical_gap.py b/benchmarks/beir-hybrid/lexical_gap.pynew file mode 100644index 000000000..e3e0d4f65--- /dev/null+++ b/benchmarks/beir-hybrid/lexical_gap.py@@ -0,0 +1,40 @@+#!/usr/bin/env python3+"""Why did BM25 return nothing for some queries? Count the lexical gap.++A BM25 arm that returns zero hits is either an engine defect or a query whose+words occur nowhere in the corpus. This separates the two WITHOUT a running+node: it tokenises the way a no-stemming standard analyzer does (lowercase,+split on non-alphanumerics) and counts the test queries that share no token+with any document's title or text.++ python3 lexical_gap.py nfcorpus++If the count equals the `empty=` figure eval.py printed for the BM25 arm, the+empty result lists are the dataset's lexical gap, not a bug.+"""+import csv, json, re, sys++def tok(s):+ return set(re.findall(r"[a-z0-9]+", s.lower()))++def main(root):+ vocab, docs = set(), 0+ for line in open(f"{root}/corpus.jsonl"):+ d = json.loads(line)+ docs += 1+ vocab |= tok(d.get("title", "")) | tok(d.get("text", ""))+ rows = list(csv.reader(open(f"{root}/qrels/test.tsv"), delimiter="\t"))[1:]+ test_ids = {r[0] for r in rows}+ queries = {}+ for line in open(f"{root}/queries.jsonl"):+ q = json.loads(line)+ if q["_id"] in test_ids:+ queries[q["_id"]] = q["text"]+ gap = sorted((i, q) for i, q in queries.items() if not (tok(q) & vocab))+ print(f"docs={docs} vocab={len(vocab)} test_queries={len(queries)}")+ print(f"test queries sharing NO token with any document: {len(gap)}")+ for i, q in gap:+ print(f" {i}\t{q}")++if __name__ == "__main__":+ main(sys.argv[1] if len(sys.argv) > 1 else "nfcorpus")diff --git a/benchmarks/beir-hybrid/load.py b/benchmarks/beir-hybrid/load.pynew file mode 100644index 000000000..e0c42051f--- /dev/null+++ b/benchmarks/beir-hybrid/load.py@@ -0,0 +1,24 @@+import json, urllib.request, sys+import os+U=os.environ.get("XERJ_URL","http://localhost:9410")+def req(m,p,b=None,ct="application/json"):+ d = b if isinstance(b,(bytes,type(None))) else json.dumps(b).encode()+ r=urllib.request.Request(U+p,data=d,method=m,headers={"content-type":ct})+ try: return json.loads(urllib.request.urlopen(r,timeout=600).read())+ except urllib.error.HTTPError as e: return {"_err":e.code,"body":e.read().decode()[:400]}+DS=sys.argv[2]+print(req("DELETE","/"+DS))+print(req("PUT","/"+DS,{"mappings":{"properties":{+ "title":{"type":"text"},"text":{"type":"text"},+ "body":{"type":"semantic_text"}}}}))+docs=[json.loads(l) for l in open(sys.argv[1])]+B=200+for i in range(0,len(docs),B):+ lines=[]+ for d in docs[i:i+B]:+ lines.append(json.dumps({"index":{"_index":DS,"_id":d["_id"]}}))+ lines.append(json.dumps({"title":d["title"],"text":d["text"],"body":(d["title"]+". "+d["text"])}))+ r=req("POST","/_bulk",("\n".join(lines)+"\n").encode(),"application/x-ndjson")+ if r.get("errors") or "_err" in r: print("bulk problem",str(r)[:300]); break+ if i%1000==0: print("indexed",i,flush=True)+print(req("POST","/"+DS+"/_refresh")); print(req("GET","/"+DS+"/_count"))diff --git a/benchmarks/beir-hybrid/results/nfcorpus-lexical-gap.txt b/benchmarks/beir-hybrid/results/nfcorpus-lexical-gap.txtnew file mode 100644index 000000000..8aa174215--- /dev/null+++ b/benchmarks/beir-hybrid/results/nfcorpus-lexical-gap.txt@@ -0,0 +1,27 @@+docs=3633 vocab=26359 test_queries=323+test queries sharing NO token with any document: 25+ PLAIN-1008 deafness+ PLAIN-1098 eggnog+ PLAIN-1214 Fosamax+ PLAIN-1309 halibut+ PLAIN-1496 leeks+ PLAIN-1611 mesquite+ PLAIN-1621 Mevacor+ PLAIN-1679 myelopathy+ PLAIN-1752 okra+ PLAIN-1827 Peoria+ PLAIN-1867 pineapples+ PLAIN-2134 Splenda+ PLAIN-2209 taro+ PLAIN-2271 Tufts+ PLAIN-2281 turnips+ PLAIN-2375 whiting+ PLAIN-2396 Yale+ PLAIN-2408 Zoloft+ PLAIN-510 airport scanners+ PLAIN-520 Alli+ PLAIN-551 amnesia+ PLAIN-583 antinutrients+ PLAIN-645 bagels+ PLAIN-817 canker sores+ PLAIN-997 Czechoslovakiadiff --git a/benchmarks/beir-hybrid/results/nfcorpus.txt b/benchmarks/beir-hybrid/results/nfcorpus.txtnew file mode 100644index 000000000..f0aaa3e60--- /dev/null+++ b/benchmarks/beir-hybrid/results/nfcorpus.txt@@ -0,0 +1,5 @@+queries=323 docs=3633+bm25 nDCG@10=0.3016 empty= 25 p50= 17.0ms p95= 46.8ms+minilm (vector only) nDCG@10=0.3291 empty= 0 p50= 230.6ms p95= 268.9ms+hybrid rrf (server) nDCG@10=0.3448 empty= 0 p50= 280.1ms p95= 317.6ms+bm25 top-30 -> minilm rerank nDCG@10=0.3323 empty= 25 p50= 223.8ms p95= 261.2msdiff --git a/benchmarks/beir-hybrid/results/scifact.txt b/benchmarks/beir-hybrid/results/scifact.txtnew file mode 100644index 000000000..5f459291c--- /dev/null+++ b/benchmarks/beir-hybrid/results/scifact.txt@@ -0,0 +1,5 @@+queries=300 docs=5183+bm25 nDCG@10=0.6572 empty= 0 p50= 18.6ms p95= 23.2ms+minilm (vector only) nDCG@10=0.6764 empty= 0 p50= 0.4ms p95= 262.2ms+hybrid rrf (server) nDCG@10=0.6993 empty= 0 p50= 371.7ms p95= 387.6ms+bm25 top-30 -> minilm rerank nDCG@10=0.6855 empty= 0 p50= 297.8ms p95= 343.2msdiff --git a/benchmarks/decisions-as-retrieval/README.md b/benchmarks/decisions-as-retrieval/README.mdnew file mode 100644index 000000000..720a16ec3--- /dev/null+++ b/benchmarks/decisions-as-retrieval/README.md@@ -0,0 +1,57 @@+# Typed decisions without a model call: classification as retrieval++TypeSafe's Jev answers typed questions — pick one of N (`choice`), yes/no with a+probability (`noul`). Where a team already has **labelled history**, XERJ can answer the+same shape of question with no model call at all: index the labelled examples, classify+new text by a rank-weighted vote over its *k* nearest neighbours, and report the winning+label's vote share as the probability.++**Run:** 2026-09-18, `xerj v1.0.0-rc.74`, `--embed-mode neural` (all-MiniLM-L6-v2, CPU),+k = 10, 1,000 held-out test items per dataset (seeded shuffle). Raw output in+[`results/`](./results).++### Banking77 — 77-way intent routing (the `choice` shape). 10,003 labelled examples indexed.++| Arm | Accuracy | ECE | Decided at confidence ≥ 0.8 | …and right | ms/item |+|---|---:|---:|---:|---:|---:|+| BM25 — no model of any kind | 0.819 | 0.089 | 50.6% | 0.996 | 2.2 |+| MiniLM neighbours | 0.933 | **0.012** | **86.9%** | 0.979 | 8.6 |+| Hybrid RRF | **0.937** | 0.052 | 79.2% | 0.989 | 12.4 |++### SMS spam — yes/no detection (the `noul` shape). 4,000 labelled examples indexed.++| Arm | Accuracy | spam P / R / F1 | ECE | ms/item |+|---|---:|---|---:|---:|+| BM25 — no model of any kind | 0.983 | 0.964 / 0.925 / 0.944 | 0.017 | 0.9 |+| MiniLM neighbours | 0.982 | 0.910 / 0.973 / 0.940 | 0.009 | 74.5 |+| Hybrid RRF | **0.989** | 0.966 / 0.959 / **0.962** | 0.015 | 67.2 |++## What this is, and what it is not++- **It is not zero-shot, and Jev is.** These numbers need labelled examples; Jev needs+ none. Where there is no history — a new policy, a new category — this does not apply+ and a judge model does. We did not run Jev on these datasets and claim nothing about+ how it would score.+- **The probability is usable.** An expected calibration error of 0.01–0.02 means a+ reported 0.9 is right about nine times in ten, so the confidence can gate real+ decisions.+- **That gate is the point.** On Banking77, 87% of traffic clears confidence 0.8 and is+ routed at 97.9% accuracy with no tokens spent. Only the remaining 13% needs a judge+ model or a person. This is the same economics as semantic detections: a free local+ stage takes the bulk, and the paid stage sees only what is genuinely uncertain.+- **Spam detection at 98.3% needs no model at all** — plain BM25 over 4,000 examples,+ under a millisecond per message.+- Latency was measured on a machine that was also compiling; treat it as an upper bound.+ The gap between datasets is query length: short banking utterances embed in a few+ milliseconds, longer SMS bodies in tens.++## Reproduce++```sh+xerj -c xerj.toml -d ./data --insecure --embed-mode neural &+B=https://raw.githubusercontent.com/PolyAI-LDN/task-specific-datasets/master/banking_data+curl -so b77_train.csv $B/train.csv; curl -so b77_test.csv $B/test.csv+curl -so sms.tsv https://raw.githubusercontent.com/justmarkham/pycon-2016-tutorial/master/data/sms.tsv+XERJ_URL=http://localhost:9410 python3 load.py+XERJ_URL=http://localhost:9410 N=1000 K=10 python3 eval.py+```diff --git a/benchmarks/decisions-as-retrieval/eval.py b/benchmarks/decisions-as-retrieval/eval.pynew file mode 100644index 000000000..878297b79--- /dev/null+++ b/benchmarks/decisions-as-retrieval/eval.py@@ -0,0 +1,47 @@+"""Jev's choice/noul use cases as retrieval: classify by voting over the k nearest+labelled examples; the winning label's vote share is reported as its probability."""+import csv, json, os, sys, time, urllib.request, collections+U=os.environ.get("XERJ_URL","http://localhost:9410"); K=int(os.environ.get("K","10"))+def post(p,b):+ r=urllib.request.Request(U+p,data=json.dumps(b).encode(),method="POST",headers={"content-type":"application/json"})+ try: return json.loads(urllib.request.urlopen(r,timeout=120).read())+ except urllib.error.HTTPError as e: return {"_err":e.code,"body":e.read().decode()[:200]}+ARMS={"bm25 (no model)":lambda t:{"match":{"text":t}},+ "minilm":lambda t:{"semantic":{"field":"body","query":t}},+ "hybrid rrf":lambda t:{"hybrid":{"queries":[{"query":{"match":{"text":t}}},{"query":{"semantic":{"field":"body","query":t}}}],"fusion":"rrf"}}}+def classify(idx,qb):+ r=post(f"/{idx}/_search",{"size":K,"_source":["label"],"query":qb})+ hs=r.get("hits",{}).get("hits",[])+ if not hs: return None,0.0+ # rank-weighted vote so the ordering matters, not just membership+ v=collections.Counter()+ for i,h in enumerate(hs): v[h["_source"]["label"]]+=1.0/(i+1)+ lab,w=v.most_common(1)[0]; return lab,w/sum(v.values())+def ece(conf_correct,bins=10):+ n=len(conf_correct); tot=0.0+ for b in range(bins):+ xs=[(c,k) for c,k in conf_correct if b/bins<c<=(b+1)/bins or (b==0 and c==0)]+ if xs: tot+=len(xs)/n*abs(sum(k for _,k in xs)/len(xs)-sum(c for c,_ in xs)/len(xs))+ return tot+def run(name,idx,test,positive=None):+ print(f"\n== {name}: {len(test)} test items, k={K}")+ for arm,qf in ARMS.items():+ cc=[]; tp=fp=fn=0; none=0; t0=time.time()+ for text,gold in test:+ pred,conf=classify(idx,qf(text))+ if pred is None: none+=1+ cc.append((conf,1 if pred==gold else 0))+ if positive:+ tp+=pred==positive and gold==positive; fp+=pred==positive and gold!=positive; fn+=pred!=positive and gold==positive+ acc=sum(k for _,k in cc)/len(cc); ms=(time.time()-t0)/len(test)*1000+ hi=[k for c,k in cc if c>=0.8]+ line=f" {arm:16s} acc={acc:.4f} ECE={ece(cc):.3f} conf>=0.8: {len(hi)/len(cc):5.1%} of items at {sum(hi)/max(len(hi),1):.4f} acc no-hit={none} {ms:6.1f}ms/item"+ if positive:+ p=tp/max(tp+fp,1); r=tp/max(tp+fn,1); line+=f" | {positive}: P={p:.3f} R={r:.3f} F1={2*p*r/max(p+r,1e-9):.3f}"+ print(line,flush=True)+b=[(r["text"],r["category"]) for r in csv.DictReader(open("b77_test.csv"))]+N=int(os.environ.get("N","1000"))+import random; random.Random(11).shuffle(b)+run("Banking77 — 77-way intent routing (Jev `choice`)","b77",b[:N])+s=[tuple(x) for x in json.load(open("sms_test.json"))]+run("SMS spam — yes/no detection (Jev `noul`)","sms",s[:N],positive="spam")diff --git a/benchmarks/decisions-as-retrieval/load.py b/benchmarks/decisions-as-retrieval/load.pynew file mode 100644index 000000000..8019a1993--- /dev/null+++ b/benchmarks/decisions-as-retrieval/load.py@@ -0,0 +1,22 @@+import csv, json, sys, urllib.request, random, os+U=os.environ.get("XERJ_URL","http://localhost:9410")+def req(m,p,b=None,ct="application/json"):+ d=b if isinstance(b,(bytes,type(None))) else json.dumps(b).encode()+ r=urllib.request.Request(U+p,data=d,method=m,headers={"content-type":ct})+ try: return json.loads(urllib.request.urlopen(r,timeout=900).read())+ except urllib.error.HTTPError as e: return {"_err":e.code,"body":e.read().decode()[:300]}+def load(idx,rows):+ req("DELETE","/"+idx)+ print(req("PUT","/"+idx,{"mappings":{"properties":{"label":{"type":"keyword"},"text":{"type":"text"},"body":{"type":"semantic_text"}}}}))+ for i in range(0,len(rows),250):+ lines=[]+ for j,(t,l) in enumerate(rows[i:i+250]):+ lines.append(json.dumps({"index":{"_index":idx,"_id":str(i+j)}})); lines.append(json.dumps({"text":t,"body":t,"label":l}))+ r=req("POST","/_bulk",("\n".join(lines)+"\n").encode(),"application/x-ndjson")+ if r.get("errors") or "_err" in r: print("bulk problem",str(r)[:300]); sys.exit(1)+ if i%1000==0: print(idx,"indexed",i,flush=True)+ req("POST",f"/{idx}/_refresh"); print(idx,req("GET",f"/{idx}/_count").get("count"))+b=[(r["text"],r["category"]) for r in csv.DictReader(open("b77_train.csv"))]+load("b77",b)+s=[l.rstrip("\n").split("\t",1) for l in open("sms.tsv")]; s=[(t,l) for l,t in s]+random.Random(7).shuffle(s); json.dump(s[4000:],open("sms_test.json","w")); load("sms",s[:4000])diff --git a/benchmarks/decisions-as-retrieval/results/run-2026-09-18.txt b/benchmarks/decisions-as-retrieval/results/run-2026-09-18.txtnew file mode 100644index 000000000..cb459ed58--- /dev/null+++ b/benchmarks/decisions-as-retrieval/results/run-2026-09-18.txt@@ -0,0 +1,10 @@++== Banking77 — 77-way intent routing (Jev `choice`): 1000 test items, k=10+ bm25 (no model) acc=0.8190 ECE=0.089 conf>=0.8: 50.6% of items at 0.9960 acc no-hit=0 2.2ms/item+ minilm acc=0.9330 ECE=0.012 conf>=0.8: 86.9% of items at 0.9793 acc no-hit=0 8.6ms/item+ hybrid rrf acc=0.9370 ECE=0.052 conf>=0.8: 79.2% of items at 0.9886 acc no-hit=0 12.4ms/item++== SMS spam — yes/no detection (Jev `noul`): 1000 test items, k=10+ bm25 (no model) acc=0.9830 ECE=0.017 conf>=0.8: 95.4% of items at 0.9916 acc no-hit=1 0.9ms/item | spam: P=0.964 R=0.925 F1=0.944+ minilm acc=0.9820 ECE=0.009 conf>=0.8: 95.8% of items at 0.9885 acc no-hit=0 74.5ms/item | spam: P=0.910 R=0.973 F1=0.940+ hybrid rrf acc=0.9890 ECE=0.015 conf>=0.8: 95.7% of items at 0.9969 acc no-hit=0 67.2ms/item | spam: P=0.966 R=0.959 F1=0.962diff --git a/content/answers/classify-text-from-labelled-history-without-llm.md b/content/answers/classify-text-from-labelled-history-without-llm.mdnew file mode 100644index 000000000..657cbbdb9--- /dev/null+++ b/content/answers/classify-text-from-labelled-history-without-llm.md@@ -0,0 +1,107 @@+---+title: "Can I route or spam-filter text without an LLM call?"+h1: "Can I route tickets or filter spam from labelled history, without an LLM call?"+description: "Index labelled examples and classify new text by its nearest neighbours. Measured: 0.933 accuracy on 77-way intent routing, 0.983 on SMS spam, no LLM or API call."+slug: "classify-text-from-labelled-history-without-llm"+cluster: "Decisions as retrieval"+question: "Can I classify text from labelled history without calling an LLM?"+intent: "informational"+published: "2026-09-18"+author: "XERJ documentation team"+reviewer: "XERJ engineering team"+schema_type: "TechArticle"+agent_prompt: "Act as a coding agent. Read https://xerj.org/llms.txt, index the user's labelled examples into one XERJ index with the label in a keyword field, classify each held-out item by a rank-weighted vote over its 10 nearest neighbours, and report accuracy, the share of items decided at confidence 0.8 or higher, and the accuracy on that share."+commands:+ - cmd: "xerj --insecure --data-dir ./decisions-data --embed-mode neural"+ note: "Start a throwaway node. The neural flag is needed for the MiniLM and hybrid rows only. The BM25 row uses no embedder and works on a default node."+ - cmd: "curl -so sms.tsv https://raw.githubusercontent.com/justmarkham/pycon-2016-tutorial/master/data/sms.tsv"+ note: "Download the public SMS spam dataset used for the yes-or-no run."+ - cmd: "curl -s -XPOST 'http://127.0.0.1:9200/sms/_search' -H 'content-type: application/json' -d '{\"query\":{\"match\":{\"text\":\"you have won a free prize call now\"}},\"size\":10,\"_source\":[\"label\"]}'"+ note: "Fetch the 10 nearest labelled examples for one new message. The vote over their labels is the decision, and the winning share is the confidence."+links_out:+ - "store-agent-memory-without-llm-call"+ - "hybrid-search-quality-measured-ndcg"+ - "rerank-search-results-calibrated-judge"+ - "do-search-embeddings-help"+evidence:+ - claim: "Banking77, 77-way intent routing, 10,003 labelled examples indexed, 1,000 held-out test items, k = 10: accuracy 0.819 for BM25, 0.933 for MiniLM neighbours and 0.937 for hybrid RRF; expected calibration error 0.089, 0.012 and 0.052; 50.6%, 86.9% and 79.2% of items decided at confidence 0.8 or higher, at accuracy 0.996, 0.979 and 0.989; 2.2, 8.6 and 12.4 ms per item."+ source: "benchmarks/decisions-as-retrieval/results/run-2026-09-18.txt"+ - claim: "SMS spam, yes-or-no detection, 4,000 labelled examples indexed, 1,000 held-out test items, k = 10: accuracy 0.983 for BM25, 0.982 for MiniLM neighbours and 0.989 for hybrid RRF; spam F1 0.944, 0.940 and 0.962; expected calibration error 0.017, 0.009 and 0.015; 0.9, 74.5 and 67.2 ms per item."+ source: "benchmarks/decisions-as-retrieval/results/run-2026-09-18.txt"+ - claim: "The run was made on 2026-09-18 with xerj v1.0.0-rc.74 and --embed-mode neural using all-MiniLM-L6-v2 on CPU, with a seeded shuffle for the held-out split, on a machine that was also compiling."+ source: "benchmarks/decisions-as-retrieval/README.md"+faq:+ - q: "Can I classify text without calling an LLM?"+ a: "Yes, when you have labelled history. Index the labelled examples, fetch the 10 nearest for each new item, and take a rank-weighted vote over their labels. No model call is made per decision."+ - q: "How accurate is nearest-neighbour classification on a real dataset?"+ a: "On Banking77, a 77-way intent dataset, XERJ measured 0.819 with BM25 and 0.933 with MiniLM neighbours. On SMS spam it measured 0.983 with BM25 alone. The MiniLM figures need `--embed-mode neural`."+ - q: "Can I trust the confidence value?"+ a: "In this run, yes. Expected calibration error was 0.012 on Banking77 and 0.009 on SMS spam for the MiniLM arm: averaged over confidence bins, the reported confidence and the observed accuracy differed by about one point."+ - q: "Does this work with no labelled examples?"+ a: "No. This is not zero-shot. A new category or a new policy has no history to vote from, and that case needs a judge model or a person."+ - q: "Do I need a neural embedder for this?"+ a: "Not for spam. BM25 alone reached 0.983 on SMS spam with no embedder at all. For 77-way intent routing the neural arm added about 11 points of accuracy over BM25."+ - q: "Did XERJ compare this against an LLM judge?"+ a: "No. XERJ did not run any judge model on these datasets and claims nothing about how one would score."+---++**TL;DR** — If you already have labelled examples, you can route and filter text with no LLM or API call. Index the examples, fetch the 10 nearest for each new item, and vote. Measured: 0.933 accuracy on 77-way intent routing and 0.983 on SMS spam. The 0.933 figure needs `--embed-mode neural`, which runs one local MiniLM embedding per item; the spam figure is plain BM25, with no model of any kind.++## The method++A judge model answers typed questions: pick one of N, or yes-or-no with a probability. Where a team already has labelled history, a search index can answer the same shape of question.++1. Index every labelled example, with the label in a keyword field.+2. For a new item, fetch its 10 nearest neighbours.+3. Take a rank-weighted vote over the neighbours' labels.+4. Report the winning label, and its share of the vote as the confidence.++No model runs per decision on the BM25 arm. On the neural arms one local embedding runs per item, on the node, with no API call.++## Intent routing: Banking77++Banking77 is a public dataset of banking support messages with 77 intent labels. The run indexed 10,003 labelled examples and classified 1,000 held-out items.++| Arm | Accuracy | Calibration error | Decided at confidence 0.8 or higher | Accuracy on that share | ms per item |+| --- | --- | --- | --- | --- | --- |+| BM25, no model of any kind | 0.819 | 0.089 | 50.6% | 0.996 | 2.2 |+| MiniLM neighbours | 0.933 | 0.012 | 86.9% | 0.979 | 8.6 |+| Hybrid RRF | 0.937 | 0.052 | 79.2% | 0.989 | 12.4 |++## Spam detection: SMS spam++The SMS spam dataset is a public yes-or-no task. The run indexed 4,000 labelled messages and classified 1,000 held-out ones.++| Arm | Accuracy | Spam F1 | Calibration error | ms per item |+| --- | --- | --- | --- | --- |+| BM25, no model of any kind | 0.983 | 0.944 | 0.017 | 0.9 |+| MiniLM neighbours | 0.982 | 0.940 | 0.009 | 74.5 |+| Hybrid RRF | 0.989 | 0.962 | 0.015 | 67.2 |++Plain BM25 over 4,000 examples reached 0.983 in under a millisecond per message. The neural arm did not improve accuracy here, and it cost far more time.++## Which embedder these numbers need++The BM25 rows use no embedder, so they hold on a default XERJ node. The MiniLM and hybrid rows were measured with `--embed-mode neural` and the all-MiniLM-L6-v2 model on CPU.++XERJ's default embedder is lexical feature hashing. It has no model in it. Do not read the MiniLM or hybrid rows as what a default node scores.++## The confidence gate is the point++An expected calibration error of 0.012 means the reported confidence is usable: averaged over confidence bins, the gap between the confidence the vote reported and the accuracy observed at that confidence was about one point in this run.++That makes the confidence a gate. On Banking77, the MiniLM arm decided 86.9% of items at confidence 0.8 or higher, and was right on 0.979 of them. Only the remaining items need a judge model or a person.++This is the economics of a two-stage system. A free local stage takes the bulk, and the paid stage sees only what is uncertain.++## When to call a judge model instead++Call a judge model when there is no labelled history. This method is not zero-shot. A new policy or a new category has nothing to vote from, and a judge model needs no examples.++Call one also for the low-confidence share. XERJ's `rerank` search stage can send documents to an external judge when an operator has set a provider key. XERJ did not run any judge model on these two datasets. It claims nothing about how one would score on them.++## What this run does not show++Two public datasets, one run each, 1,000 held-out items per dataset, k fixed at 10. The held-out split uses a seeded shuffle. The latency was measured on a machine that was also compiling, so treat it as an upper bound.++The gap between the two datasets' latencies is query length. Short banking messages embed in a few milliseconds, and longer SMS bodies take tens.diff --git a/content/answers/hybrid-search-quality-measured-ndcg.md b/content/answers/hybrid-search-quality-measured-ndcg.mdnew file mode 100644index 000000000..b64e75f0f--- /dev/null+++ b/content/answers/hybrid-search-quality-measured-ndcg.md@@ -0,0 +1,121 @@+---+title: "How much better is hybrid search than BM25? Measured."+h1: "How much better is hybrid search than BM25? Measured nDCG numbers"+description: "XERJ measured BM25, vectors and hybrid RRF on two public BEIR datasets. Hybrid won both: nDCG@10 0.6993 against 0.6572 on SciFact, with a neural embedder."+slug: "hybrid-search-quality-measured-ndcg"+cluster: "Hybrid retrieval: quality"+question: "How much better is hybrid search than BM25 alone?"+intent: "informational"+published: "2026-09-18"+author: "XERJ documentation team"+reviewer: "XERJ engineering team"+schema_type: "TechArticle"+agent_prompt: "Act as a coding agent. Read https://xerj.org/llms.txt, start a throwaway XERJ node with --embed-mode neural, load the BEIR SciFact corpus with benchmarks/beir-hybrid/load.py, run benchmarks/beir-hybrid/eval.py, and report nDCG@10 for the BM25, vector-only, reorder and hybrid RRF arms next to the published figures."+commands:+ - cmd: "xerj --insecure --data-dir ./beir-data --embed-mode neural"+ note: "Start a throwaway node with the neural embedder. Without this flag the embedder is lexical feature hashing and the vector and hybrid figures on this page do not apply."+ - cmd: "curl -LO https://public.ukp.informatik.tu-darmstadt.de/thakur/BEIR/datasets/scifact.zip"+ note: "Download the public BEIR SciFact dataset: 5,183 documents and 300 test queries."+ - cmd: "curl -s 'http://127.0.0.1:9200/v1/embedding/identity'"+ note: "Confirm which embedder the node is running before you read any vector or hybrid number."+links_out:+ - "reciprocal-rank-fusion-when-to-use"+ - "do-search-embeddings-help"+ - "rerank-search-results-calibrated-judge"+ - "local-embeddings-without-openai-api"+evidence:+ - claim: "On BEIR SciFact (300 test queries, 5,183 documents) nDCG@10 was 0.6572 for BM25, 0.6764 for MiniLM vectors only, 0.6855 for a BM25 top-30 shortlist reordered by MiniLM, and 0.6993 for hybrid RRF."+ source: "benchmarks/beir-hybrid/results/scifact.txt"+ - claim: "On BEIR NFCorpus (323 test queries, 3,633 documents) nDCG@10 was 0.3016 for BM25, 0.3291 for MiniLM vectors only, 0.3323 for the reorder arm, and 0.3448 for hybrid RRF. BM25 returned zero hits for 25 of the 323 queries."+ source: "benchmarks/beir-hybrid/results/nfcorpus.txt"+ - claim: "Exactly 25 of the 323 NFCorpus test queries share no token with any document title or text, which accounts for every empty BM25 result list; the same check reports 0 for SciFact."+ source: "benchmarks/beir-hybrid/results/nfcorpus-lexical-gap.txt"+ - claim: "The run was made on 2026-09-18 with xerj v1.0.0-rc.74 and --embed-mode neural, using the built-in all-MiniLM-L6-v2 model at 384 dimensions on CPU, one node and one shard with default settings. Neural indexing ran at about 3 to 7 documents per second, SciFact took about 25 minutes, and hybrid queries cost about 220 to 370 ms at p50 against about 18 ms for BM25."+ source: "benchmarks/beir-hybrid/README.md"+ - claim: "The nDCG@10 figures for Jev (0.768 SciFact, 0.358 NFCorpus), Voyage rerank-3 (0.755, 0.357) and Cohere rerank-v3.5 (0.745, 0.340) are published in the hev/jev-rerank README. XERJ did not run them."+ source: "https://github.com/hev/jev-rerank"+faq:+ - q: "How much better is hybrid search than BM25?"+ a: "On two public BEIR datasets XERJ's hybrid RRF scored nDCG@10 0.6993 against 0.6572 for BM25 on SciFact, and 0.3448 against 0.3016 on NFCorpus. Both runs used `--embed-mode neural`."+ - q: "Do these numbers apply to a default XERJ node?"+ a: "Only the BM25 row does. The vector and hybrid rows need `--embed-mode neural`. The default embedder is lexical feature hashing, which has no model in it."+ - q: "Is reranking a BM25 shortlist with the same embedding model a good idea?"+ a: "Not in this run. Reordering the BM25 top 30 by MiniLM scored 0.6855 on SciFact, below 0.6993 for fusing with the same model. Fusion beat reordering on both datasets."+ - q: "How does XERJ hybrid compare to hosted rerankers?"+ a: "XERJ did not run them. Figures published elsewhere put Jev at 0.768 on SciFact and XERJ hybrid at 0.6993, but they use a different first stage, so it is not a controlled comparison."+ - q: "What does hybrid search cost in latency?"+ a: "In this run hybrid queries cost about 220 to 370 ms at p50 against about 18 ms for BM25, on a 5,000-document index on CPU. The machine was also compiling, so read those as upper bounds."+ - q: "How fast is neural indexing?"+ a: "About 3 to 7 documents per second on CPU in this run. SciFact's 5,183 short abstracts took about 25 minutes to index."+---++**TL;DR** — On two public BEIR datasets, XERJ's hybrid RRF beat BM25 alone and beat vectors alone: nDCG@10 0.6993 against 0.6572 on SciFact, and 0.3448 against 0.3016 on NFCorpus. These runs used `--embed-mode neural` with all-MiniLM-L6-v2. The default embedder is lexical feature hashing, and these vector figures do not apply to it.++## The numbers++The metric is nDCG@10 on each dataset's BEIR `test` split. The evaluation script computes it from live queries against a running node. Nothing is carried over from another run.++| Arm | SciFact (300 queries, 5,183 docs) | NFCorpus (323 queries, 3,633 docs) |+| --- | --- | --- |+| BM25 (`multi_match` on title and text) | 0.6572 | 0.3016 |+| MiniLM vectors only (`semantic`) | 0.6764 | 0.3291 |+| BM25 top 30, reordered by MiniLM | 0.6855 | 0.3323 |+| Hybrid RRF (`hybrid`, server-side) | 0.6993 | 0.3448 |++The run was made on 2026-09-18 with xerj v1.0.0-rc.74, on one node and one shard with default settings. The raw output is committed next to the scripts in `benchmarks/beir-hybrid`.++## Which embedder these numbers need++Read this before you quote a row. XERJ embeds with lexical feature hashing by default. That embedder has no model in it and cannot connect synonyms.++The vector, reorder and hybrid rows were measured with `--embed-mode neural`. That flag loads the built-in all-MiniLM-L6-v2 model, 384 dimensions, on CPU. A default node does not score these figures on those three rows. The BM25 row uses no embedder at all, so it holds on any node.++## Fusion beat reordering++The third row is the one that changes a design decision. It takes the BM25 top 30 and reorders them with the same MiniLM model. It scored 0.6855 on SciFact and 0.3323 on NFCorpus.++Hybrid RRF uses the same model and scored 0.6993 and 0.3448. Fusing with the model beat reranking with it, on both datasets.++The reason is recall. A reorder can only move documents that BM25 already returned. Fusion also admits documents that only the vector list found. On NFCorpus BM25 returned zero hits for 25 of the 323 queries, and no reorder can fix an empty list.++Those 25 empty lists are not an engine defect. Each of those queries is a word or phrase that occurs nowhere in the corpus, such as `deafness`, `eggnog` or `Zoloft`. A script in the benchmark folder counts them without a running node, and it reports 0 for SciFact, where BM25 had no empty lists.++This is why XERJ ships no local MiniLM rerank provider. It would score below the hybrid search that already exists.++## What it costs++Neural mode is not free. These figures come from the same run.++| Cost | Measured |+| --- | --- |+| Neural indexing speed | about 3 to 7 documents per second on CPU |+| Time to index SciFact | about 25 minutes |+| BM25 query, p50 | about 18 ms |+| Hybrid query, p50 | about 220 to 370 ms |++Most of the hybrid latency is one BERT forward pass per query on CPU. The machine was also compiling during the run, so treat the absolute latencies as upper bounds. One vector-arm latency in the SciFact raw log is a cache artefact from an earlier aborted run. It is not quoted here and must not be.++## How this relates to hosted rerankers++The `hev/jev-rerank` README publishes nDCG@10 for three hosted rerankers on the same datasets.++| System | SciFact | NFCorpus | Run by |+| --- | --- | --- | --- |+| Jev (TypeSafe AI) | 0.768 | 0.358 | that README, not XERJ |+| Voyage rerank-3 | 0.755 | 0.357 | that README, not XERJ |+| Cohere rerank-v3.5 | 0.745 | 0.340 | that README, not XERJ |+| XERJ hybrid RRF, local MiniLM | 0.6993 | 0.3448 | XERJ, `--embed-mode neural` |++XERJ did not run those three systems. They rerank that project's own first-stage shortlist, which is not XERJ's. The dataset and the metric match, and nothing else is controlled.++What the table supports is narrow. On NFCorpus the local hybrid figure sits inside the range of the published reranker figures. On SciFact it is lower than all three. A controlled comparison needs a provider key and XERJ's `rerank` stage, and XERJ has not run one.++## When to choose a hosted reranker instead++Choose a hosted reranker when ranking quality on hard queries matters most. You pay for it in latency, in cost, and in text that leaves the machine. The published SciFact figures for all three hosted systems are higher than XERJ's local hybrid figure. A reranker also returns a calibrated probability, which a fused score is not.++Choose local hybrid when text must stay on the machine, or when a paid call per search is not acceptable. XERJ supports both. `hybrid` runs on the node with the built-in lexical or neural embedder. With proxy embeddings, the query text goes to that embeddings API. The `rerank` stage calls a judge when an operator has set a key.++## What this run does not show++Two datasets, one model and one run each. Both datasets are scientific and medical text, so the gap may differ on code, logs or chat. The node was a single shard on a shared host. These figures show the ordering of the four arms on this data, not what your corpus will score.diff --git a/content/answers/rerank-search-results-calibrated-judge.md b/content/answers/rerank-search-results-calibrated-judge.mdnew file mode 100644index 000000000..df2ae0e52--- /dev/null+++ b/content/answers/rerank-search-results-calibrated-judge.md@@ -0,0 +1,150 @@+---+title: "How do I rerank search results with a relevance judge?"+h1: "How do I rerank search results with a calibrated relevance judge?"+description: "Add a rerank block to a XERJ search: the top 30 hits go to an external judge, a 0 to 1 probability replaces the score, and min_score becomes a real cut-off."+slug: "rerank-search-results-calibrated-judge"+cluster: "Hybrid retrieval: reranking"+question: "How do I rerank search results with a calibrated relevance judge?"+intent: "informational"+published: "2026-09-18"+author: "XERJ documentation team"+reviewer: "XERJ engineering team"+schema_type: "TechArticle"+agent_prompt: "Act as a coding agent. Read https://xerj.org/llms.txt, ask the operator whether a rerank provider key is set on the XERJ node and whether document text may leave the machine, then run one search without the rerank block and one with it, and report the _rerank block of the second response, including whether applied is true."+commands:+ - cmd: "curl -s 'http://127.0.0.1:9200/_xerj/rerank'"+ note: "Ask the node whether a rerank provider is configured. The answer names where the key came from and never prints the key. A node started with --insecure needs no credentials. On any other node the route is superuser-only, so send the admin key from the admin.key file in the data directory."+ - cmd: "curl -s -XPOST 'http://127.0.0.1:9200/kb/_search' -H 'content-type: application/json' -d '{\"query\":{\"match\":{\"body\":\"vitamin d bone density\"}},\"size\":5,\"rerank\":{\"min_score\":0.5}}'"+ note: "Rerank the top 30 hits and keep the ones the judge scores at 0.5 or higher. This request sends the text of those hits to the provider."+ - cmd: "curl -s -XPOST 'http://127.0.0.1:9200/kb/_search' -H 'content-type: application/json' -d '{\"query\":{\"match\":{\"body\":\"vitamin d bone density\"}},\"_source\":[\"title\"],\"rerank\":{}}'"+ note: "Send titles only. The judge sees exactly what the response returns, so source filtering is also the control on what leaves the machine."+links_out:+ - "reciprocal-rank-fusion-when-to-use"+ - "hybrid-search-quality-measured-ndcg"+ - "how-xerj-combines-search"+ - "/docs/config"+evidence:+ - claim: "Reranking is the only search-time feature that sends document text off the node. Proxy embeddings and the WAL tap also send text off the node when configured; the neural model download and cluster Raft messages carry no document or query text. A test checks that list against every outbound network client in the engine source."+ source: "engine/crates/xerj-rerank/tests/egress_inventory.rs"+ - claim: "The rerank stage defaults to a window of 30 hits, allows at most 300, sends at most 30 documents per provider call, runs 8 calls in flight by default and 16 at most, and defaults to a 10000 ms stage budget capped at 60000 ms."+ source: "engine/crates/xerj-rerank/src/lib.rs"+ - claim: "The failure policy, every refused combination, the 400, 403, 502 and 503 statuses, paging inside the window, and the rule that only returned text is sent are each pinned by an HTTP test against an in-process stub of the provider wire format."+ source: "engine/crates/xerj-api/tests/rerank_stage_http.rs"+ - claim: "The nDCG@10 figures for Jev (0.768 SciFact, 0.358 NFCorpus), Voyage rerank-3 (0.755, 0.357) and Cohere rerank-v3.5 (0.745, 0.340) are published in the hev/jev-rerank README. XERJ did not run them, and they rerank that project's own first-stage shortlist."+ source: "https://github.com/hev/jev-rerank"+ - claim: "XERJ hybrid RRF scored nDCG@10 0.6993 on SciFact and 0.3448 on NFCorpus with --embed-mode neural and all-MiniLM-L6-v2, not with the default lexical embedder."+ source: "benchmarks/beir-hybrid/README.md"+faq:+ - q: "How do I rerank search results in XERJ?"+ a: "Add a `rerank` object to the body of `POST /{index}/_search`. An empty object uses every default: the top 30 hits are judged and reordered by relevance probability."+ - q: "Does reranking send my documents to a third party?"+ a: "Yes. It is the only search-time feature that sends document text off the node. Proxy embeddings and the WAL tap also send text off the node when an operator configures them, and all three are off by default. The question and the text of up to `window` hits go to the provider. Only fields the response returns are sent."+ - q: "What happens when the rerank provider is slow?"+ a: "The search still answers with HTTP 200. You get the engine's order and scores, and the `_rerank` block says `applied: false` with the reason. A slow provider degrades; it does not fail the search."+ - q: "What happens when the provider key is wrong or missing?"+ a: "The search fails loudly. A missing key is HTTP 503, a provider 401 is HTTP 502, both with type `rerank_exception` and no hits. XERJ does not hand back lexical order as if it had been reranked."+ - q: "Is the ranking quality of the rerank model verified?"+ a: "No. XERJ had no provider key, so ranking quality with the real model is not verified by this project. The tests use a test double. Only the mechanism is verified."+ - q: "Can I combine rerank with sort or aggregations?"+ a: "Aggregations work and still describe the full match set. `sort`, `search_after`, `collapse`, `scroll` and `size: 0` are refused with an HTTP 400 that names the conflict."+ - q: "Can an AI agent ask for reranking through MCP?"+ a: "Yes. The `xerj_search` and `xerj_hybrid_search` tools take an optional `rerank` argument. It only works when the operator has set a provider key on the node."+---++**TL;DR** — Add `"rerank": {}` to a XERJ `_search` body. The top 30 hits go to an external judge, which returns a 0 to 1 relevance probability per document. That probability replaces `_score`, so `min_score` becomes a real cut-off. It sends document text off the machine. XERJ has not verified ranking quality with the real model.++## What the rerank stage does++The engine retrieves and ranks as usual. The stage then sends the question and the top `window` hits to a relevance judge. The judge answers one yes-or-no question per document and returns a probability that the document is relevant.++XERJ reorders the hits by that probability. The probability replaces `_score`, and `hits.max_score` becomes the top hit's probability.++```json+{+ "query": { "match": { "body": "vitamin d bone density" } },+ "size": 5,+ "rerank": { "min_score": 0.5 }+}+```++The provider is TypeSafe AI's Jev model, reached through its System One API. XERJ asks one question per document rather than one multiple-choice question across all of them, because a real corpus has more than one relevant document. That request shape is the approach used by the `hev/jev-rerank` project, which is Apache-2.0.++## Why a probability is worth more than a better order++A BM25 score orders results but has no absolute meaning. A score of 7.2 on one query is not comparable to 7.2 on another query, so you cannot use it as a threshold.++A calibrated probability means the same thing on every query. `rerank.min_score: 0.5` drops every document the judge scored below 0.5, whatever the query was. The top-level `min_score` stays the engine's threshold on engine scores, and the two compose.++## This sends your data off the machine++Reranking is the only search-time feature that sends document text to a third party. Two other features send text off the node, and both are off by default. Proxy embeddings (`[embedding] default_endpoint`) send document text at indexing time and query text at search time to an external embeddings API. The WAL tap replays every write on tapped indices to an external `_bulk` endpoint. An operator has to turn either one on.++The node has two more outbound connections, and neither carries document or query text. Neural mode downloads its model from the HuggingFace Hub once. Cluster mode, which a default single-node deployment never starts, sends Raft messages, such as index names and mappings, to your own peers. The reference, `docs/RERANK.md`, lists every outbound connection a node can open.++A rerank request POSTs the question and the text of up to `window` hits to the provider.++Three controls exist. Reranking does nothing until an operator sets a provider key. A search only triggers it by carrying a `rerank` block. An operator can forbid it with `enabled = false` under `[rerank]`, which refuses every rerank request with HTTP 403.++The judge sees exactly what the response returns, and nothing else. Source filtering is therefore also the control on what leaves the machine: `"_source": ["title"]` sends titles only. `rerank.fields` is exhaustive, so `["body"]` sends the body and not even the title. Document ids, index names, numbers and vectors are never sent.++## Set the provider key++Put the key in the config file or in the environment. A value in the config file wins over the environment. A stray variable therefore cannot silently change where document text is sent.++```toml+[rerank]+enabled = true+api_key = "ts-..."+endpoint = "https://api.typesafe.ai/v1/systemone"+```++The environment fallbacks are `TYPESAFE_API_KEY` and `TYPESAFE_ENDPOINT`. There is no per-request key, because a key in a search body would land in logs. `GET /_xerj/rerank` reports whether a key is set and where it came from. It never returns the key.++## Read the _rerank block++Every reranked response carries a `_rerank` block ahead of `hits`. Read `applied` before you trust the order.++| Field | Meaning |+| --- | --- |+| `applied` | `true` means the order is the judge's. `false` means the engine's order stands, and `reason` says why. |+| `score_kind` | `probability` or `engine`. |+| `judged` | Documents the provider returned a verdict for. |+| `pruned_below_min_score` | Judged hits removed by `rerank.min_score`. |+| `fields_without_text` | Named fields that no hit in the window returned text for. |+| `usage` | Tokens the provider reported for this search. |++`hits.total` and aggregations stay the engine's. They describe the full match set, not the window. When four documents match and two are judged irrelevant, `total` is 4 and `pruned_below_min_score` is 2.++## What happens when the provider fails++The policy is split: degrade on deadline, surface on contract.++| What happened | Result |+| --- | --- |+| The provider missed `timeout_ms` | HTTP 200, engine order, `applied: false`. |+| The `rerank` block is malformed or the combination is refused | HTTP 400, nothing sent. |+| The operator disabled reranking | HTTP 403, nothing sent. |+| No provider key on the node | HTTP 503, nothing sent. |+| The provider answered 401 or a malformed body | HTTP 502, no hits. |++A slow third party is not a reason to deny you results you already have. A wrong key does not fix itself, and hiding it would mislead you about which ranking you hold. This split follows the approach Meilisearch uses in its personalization module (MIT). The approach and the retry back-off constants were adapted from it, and the code cites the lines.++## Cost facts++Every document in the window is a paid judgement. The window defaults to 30 and the server caps it at 300. A provider call carries at most 30 documents, so `window: 35` is two calls. Scores from different calls stay comparable, because each one is an absolute probability rather than a rank inside its batch.++XERJ sends 8 calls at once by default and 16 at most. The `usage` field is the provider's own token count for the search. XERJ does not price it.++Every request is judged from scratch. There is no verdict cache, so three page requests over one 30-document window are three provider calls and 90 paid judgements. To keep the cost down, fetch the window once and cut the pages out of it in your own code. Page two continues page one only if the provider returns the same probabilities on a repeat call, which XERJ has not verified for the real model.++The strings have ceilings too. `instructions` is limited to 2,000 characters, because the provider's wire format repeats it once per judged document. The question is limited to 4,000 characters and `model` to 128. A longer value is an HTTP 400 that names the field and the limit.++## What is not verified++XERJ had no provider key. Ranking quality with the real Jev model is not verified by this project. Every test of the stage runs against a test double that scores by a table the test supplies.++The `hev/jev-rerank` README publishes nDCG@10 of 0.768 on SciFact and 0.358 on NFCorpus for Jev. XERJ did not run those systems, and they rerank that project's own first-stage shortlist, not XERJ's. XERJ's own hybrid search scored 0.6993 and 0.3448 on the same datasets. Those two figures were measured with `--embed-mode neural` and the all-MiniLM-L6-v2 model. The default embedder is lexical feature hashing, which has no model in it. This is not a controlled comparison.++## When reranking is not worth it++Do not rerank an exact identifier lookup, a filter, or an aggregation-only request. The first stage already answers those, and the stage adds a paid network call per search. Rerank when the right document is probably in the top 30 but not at the top, or when you need an absolute relevance cut-off.diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex 4276f1d2f..6f2a8e99f 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -60,6 +60,7 @@ pipeline plugins).| `xerj-common` | Shared types: `Config`, `Schema`, `FieldType`, `XerjError`. || `xerj-cluster` | Embedded Raft consensus for cluster metadata (no external dependencies). || `xerj-console-api` | Bundled console backend (dashboards, auth, prefs) mounted at `/_xerj-console/api/v1/*`. |+| `xerj-rerank` | Optional second-stage reranking for `_search`: calls an external relevance judge after hits are rendered. The only search-time component that makes an outbound call with document text (proxy embeddings and the WAL tap also send text off the node when configured; RERANK.md lists every outbound connection); never linked by `xerj-engine`. See [RERANK.md](./RERANK.md). || `xerj-wasm` | Pluggable transform pipeline with an optional WASM backend. || `tests/es-compat-yaml` | `es-yaml-runner`: executes ES REST-spec YAML suites against a live server. |diff --git a/docs/RERANK.md b/docs/RERANK.mdnew file mode 100644index 000000000..ba429caa6--- /dev/null+++ b/docs/RERANK.md@@ -0,0 +1,534 @@+# Reranking search results (`rerank`)++> **This feature sends your data off the machine.** A search that carries a+> `rerank` block POSTs the question and the text of up to `window` hits+> (default 30, maximum 300) to a third-party API. It is the only+> **search-time** feature that sends document text off the node. Two other+> features send text off the node, both operator configuration and off by+> default: `[embedding] default_endpoint` (`--embed-mode proxy`) sends document+> text at write time and query text at search time to an external embeddings+> API, and the WAL tap (`PUT /_xerj/wal_tap`) replays every write on tapped+> indices to an external `_bulk` endpoint. The node's other outbound+> connections carry no document or query text: the one-time HuggingFace model+> download for `--embed-mode neural`, and Raft messages to your own peers in+> cluster mode. Reranking is off until an operator supplies a provider key, a+> request only triggers it by asking for it, and an operator can forbid it+> outright with `[rerank] enabled = false`. Read+> [Every way data leaves a XERJ node](#every-way-data-leaves-a-xerj-node) before+> you turn it on.++`rerank` is an optional second stage on `POST /{index}/_search`. The engine+retrieves and ranks as usual; the stage then hands the top hits to an external+relevance judge, which returns a probability per document that it answers the+question. Hits are reordered by that probability, and the probability replaces+`_score`.++It exists for one reason above ordering. A BM25 or fused score orders results+but has no absolute meaning: a `_score` of 7.2 is not comparable across queries+and cannot be used as a cut-off. A calibrated probability can, so+`rerank.min_score` is a threshold that means the same thing on every query.++> **Not verified: ranking quality with the real model.** No provider key was+> available to this project, so XERJ has **not** measured how well the Jev model+> ranks. Every test of this stage runs against a test double that scores by a+> table the test supplies. What is verified is the mechanism — the wire format,+> the reorder, paging, pruning, the failure policy, and what is and is not sent.+> The quality figures published by the provider's ecosystem are quoted in+> [Quality numbers](#quality-numbers-and-what-we-did-not-measure) with that+> caveat attached.++The stage is in `engine/crates/xerj-api/src/rerank_stage.rs`; the provider+client, the request parser and the failure policy are the leaf crate+`engine/crates/xerj-rerank`. The end-to-end tests are+`engine/crates/xerj-api/tests/rerank_stage_http.rs`.++## Set it up++A node needs a provider key. There are two places to put it, and **a value in+the config file wins over the environment** — the same rule as+`cluster.auth_secret`, so a stray variable in a service's environment cannot+silently re-point where document text is sent.++```toml+# xerj.toml+[rerank]+enabled = true # false = refuse every rerank request (403)+api_key = "ts-..." # or the TYPESAFE_API_KEY environment variable+endpoint = "https://api.typesafe.ai/v1/systemone" # or TYPESAFE_ENDPOINT; this is the default+```++| Setting | Environment fallback | Default | Meaning |+|---|---|---|---|+| `rerank.enabled` | none | `true` | `false` refuses every `rerank` request with HTTP 403, whatever the environment holds. Nothing is sent. |+| `rerank.api_key` | `TYPESAFE_API_KEY` | empty | The provider key. Empty in both places means reranking is not configured: requests get HTTP 503. |+| `rerank.endpoint` | `TYPESAFE_ENDPOINT` | `https://api.typesafe.ai/v1/systemone` | Where documents are POSTed. Must be an absolute `http(s)://` URL with no `user:password@` in it. The node refuses to start otherwise, **whichever place the value came from** — the config file at config load, the environment variable at boot (`TYPESAFE_ENDPOINT: rerank.endpoint must not carry credentials…`). |++The settings are resolved once, at startup. Changing the environment of a+running node does nothing; restart it.++There is deliberately **no per-request key**. A key in a search body would land+in slow-query logs, audit trails and client-side request dumps.++### Check whether it is configured++```sh+curl -s -H "Authorization: ApiKey $ADMIN_KEY" http://localhost:9200/_xerj/rerank+```++```json+{+ "enabled": true,+ "configured": true,+ "providers": ["jev"],+ "api_key": { "set": true, "source": "config" },+ "endpoint": { "url": "https://api.typesafe.ai/v1/systemone", "source": "default" },+ "defaults": { "provider": "jev", "model": "jev-latest", "window": 30, "batch": 30,+ "max_concurrency": 8, "max_doc_chars": 1200, "timeout_ms": 10000 },+ "limits": { "max_docs_per_call": 30, "max_window": 300, "max_concurrency": 16,+ "max_doc_chars": 8000, "max_timeout_ms": 60000,+ "max_instructions_chars": 2000, "max_query_chars": 4000,+ "max_model_chars": 128, "max_fields": 64, "max_field_name_chars": 256 },+ "data_egress": "A search that carries a `rerank` block sends the text of up to `window` hits, and the query, to the endpoint above. It is the only search-time feature that sends document text off the node. Two other features send text off the node, both operator configuration and off by default: `[embedding] default_endpoint` (`--embed-mode proxy`) sends document text at write time and query text at search time to an external embeddings API, and the WAL tap (`PUT /_xerj/wal_tap`) replays every write on tapped indices to an external `_bulk` endpoint. The node's other outbound connections carry no document or query text: the one-time HuggingFace model download for `--embed-mode neural`, Raft messages (index names, mappings, shard assignments) to the configured peers in cluster mode, and object storage — the `S3Backend` client, which nothing on the segment path constructs today, and `xerj autoindex s3://`, which names buckets and keys to the endpoint you configure and reads objects IN."+}+```++The endpoint reports **that** a key is set and **where it came from**+(`"config"` or `"env"`). It never returns the key, and the URL is printed with+any userinfo, query string and fragment removed. `configured` is `false` when a+key exists but `enabled = false`: a key on a node that forbids reranking is not+a working configuration.++`/_xerj/*` is the node's operator namespace and is superuser-only. A caller+without superuser rights finds out from the search itself: HTTP 503 means no+key, HTTP 403 means the operator disabled it.++## Request++```json+POST /kb/_search+{+ "query": { "match": { "body": "vitamin d supplementation bone density" } },+ "size": 5,+ "rerank": { "min_score": 0.5 }+}+```++`"rerank": {}` is valid and uses every default.++| Field | Type | Default | Limit | Meaning |+|---|---|---|---|---|+| `provider` | string | `"jev"` | `jev`, `typesafe` (alias), `none`, `disabled` | `none` / `disabled` skips the call and reports `applied: false`. Anything else is a 400. |+| `model` | string | `"jev-latest"` | ≤ 128 characters | Sent to the provider verbatim. |+| `window` | integer | 30 | 1–300 | How many of the engine's top hits are judged. **Every document in the window is a paid judgement.** |+| `min_score` | number | none | 0–1 | Drop hits whose probability is below this. It is a probability; 7 is a 400. |+| `query` | string | inferred | non-empty, ≤ 4,000 characters | The question documents are judged against. The limit also applies to a question read from the search query. See [The question](#the-question). |+| `fields` | string[] | every returned string field | 1–64 names, each ≤ 256 characters | Which returned fields are sent. **Exhaustive**: `["body"]` sends the body and nothing else, not even the title. See [What leaves the machine](#what-leaves-the-machine). |+| `instructions` | string | a generic relevance question | ≤ 2,000 characters | Overrides what "relevant" means. A support corpus and a code corpus do not mean the same thing by it. The provider's wire format carries the instructions inside **every** per-document question, so this string is sent once per judged document — which is why it has a ceiling. |+| `timeout_ms` | integer | 10000 | 1–60000 | Wall-clock budget for the **whole stage**, not per call. |+| `max_doc_chars` | integer | 1200 | 1–8000 | Title and text are each cut to this many characters, on a character boundary. |+| `batch` | integer | 30 | clamped to 30 | Documents per provider call. Values above 30 are lowered to 30, not refused: the ceiling is the provider's, not a caller mistake. |+| `max_concurrency` | integer | 8 | 1–16 | Provider calls in flight at once. |++Unknown keys are **refused by name**, not ignored. A caller who misspells+`min_score` as `treshold` gets a 400 naming `treshold`, instead of unpruned+results that look pruned.++The ceilings are the server's because the caller picks the window and the+operator pays for it. That includes the strings. `instructions` is repeated+once per judged document, so before it had a ceiling a single `size: 1` search+carrying a 1 MB `instructions` string at `window: 40` put 40 MB on the wire to+the provider (measured in review, against a test double). With the ceilings,+the most one search can send is window × (title + text, each cut at+`max_doc_chars`, + `instructions`) plus the question once per call. That is+arithmetic, not a measurement: at the default window and `max_doc_chars` with+instructions at their limit, 30 × (1,200 + 1,200 + 2,000) + 4,000 = 136,000+characters; at every maximum at once, 300 × (8,000 + 8,000 + 2,000) ++10 × 4,000 = 5.44 million characters, before JSON framing. An over-long value+is a 400 that names the field and the limit; the value itself is never echoed+back. The ceilings are also readable from `GET /_xerj/rerank` under `limits`.++### The question++The judge needs one natural-language question. XERJ reads it from the search+query only when the query is a shape that carries exactly one:++| Query shape | Question read from |+|---|---|+| `match`, `match_phrase` | the field's query string |+| `multi_match`, `semantic`, `simple_query_string` | `query` |+| anything else — `bool`, `hybrid`, `term`, `match_all`, a top-level `knn` with no text query, no `query` at all | **refused (400)**: pass `rerank.query` |++This is deliberately shallow. A `bool` tree has no single question in it, and+picking one would rank against a guess without anyone noticing. `rerank.query`+always wins when it is present, and the `_rerank` block reports the question+that was actually used.++## Response++The stage adds one top-level block, `_rerank`, placed **before** `hits` on the+wire so that a reader which truncates long output from the bottom still sees+which ranking it is holding.++Applied:++```json+{+ "_rerank": {+ "applied": true,+ "provider": "jev",+ "model": "jev-latest",+ "score_kind": "probability",+ "window": 4,+ "judged": 4,+ "unjudged": 0,+ "skipped_no_text": 0,+ "pruned_below_min_score": 2,+ "dropped_unjudged": 0,+ "partial_failures": 0,+ "usage": { "input_tokens": 100, "output_tokens": 10 },+ "query": "vitamin d supplementation bone density",+ "took_ms": 212+ },+ "took": 215,+ "hits": { "total": { "value": 4, "relation": "eq" }, "max_score": 0.9, "hits": [ ... ] }+}+```++Degraded (the provider missed the deadline — see [Failure+policy](#failure-policy-degrade-on-deadline-surface-on-contract)):++```json+{+ "_rerank": {+ "applied": false,+ "reason": "rerank deadline exceeded after 301ms",+ "provider": "jev",+ "score_kind": "engine",+ "query": "vitamin d supplementation bone density",+ "took_ms": 301+ },+ "hits": { ... the engine's order, the engine's scores ... }+}+```++| `_rerank` field | Meaning |+|---|---|+| `applied` | `true`: the order is the judge's. `false`: the order and scores are the engine's; `reason` says why. **Always read this.** |+| `score_kind` | `"probability"` when `_score` is a 0–1 relevance probability, `"engine"` when it is still BM25 / fusion. |+| `window` | Hits considered: `min(rerank.window, hits the engine returned)`. |+| `judged` | Documents the provider returned a verdict for. One verdict per document: an answer keyed to a document that was never sent, or sent in another call, is not counted. This is the figure `xerj_rerank_documents_judged_total` meters. |+| `unjudged` | Hits in the response that have no verdict — the provider did not answer for them, or they were skipped as blank. Their `_score` is `null` and they sort after every judged hit. Without `min_score`, `judged + unjudged` equals `window`. |+| `skipped_no_text` | Hits in the window with no string content in the response. They are not sent: a blank document costs a judgement and a verdict on nothing means nothing. |+| `pruned_below_min_score` | Judged hits removed by `rerank.min_score`. |+| `dropped_unjudged` | With `min_score` set, hits that were never judged cannot be shown to clear the bar, so they are dropped and counted here. |+| `fields_without_text` | Present only when there is something to report: the names in `rerank.fields` that **no hit in the window** returned any text for — a typo, a field these hits lack, or a field the projection removed. Those documents were judged without it. |+| `partial_failures` | Provider calls that did not deliver a verdict while others did: calls that failed, and calls the deadline stopped from ever being sent. The scored hits are still correctly ordered — each probability is absolute — but the window was not fully covered; `unjudged` says by how much. |+| `usage` | Tokens the provider reported, summed over every call that answered. Zero when the provider omits it. |+| `query` | The question that was judged, inferred or given. |+| `took_ms` | Time spent in the stage. The response's own `took` includes it. |++What changes on a hit: `_score` becomes the probability, and `hits.max_score`+becomes the top hit's probability. Hits that were not judged get `_score:+null` and sort **after** every judged hit, in the engine's order — an unjudged+document is not evidence of irrelevance, but it cannot be ranked against+documents that were scored. Equal probabilities keep the engine's order.++**`_score` is one kind per response.** When `applied` is `true`, every+`_score` is a probability or `null`; the engine's BM25 value is never left on+some hits beside probabilities on others. A client that sorts by `_score` or+scales by `hits.max_score` cannot tell a `1.63` BM25 from a `0.9` probability,+and `max_score` would be smaller than a later hit's score. `null` is what+Elasticsearch itself puts in `_score` when a hit has no comparable score, so+every client already handles it. `_score` and `hits.max_score` carry the+provider's probability at double precision: `0.9` from the provider is `0.9` on+the wire, so a threshold applied client-side agrees with `rerank.min_score`.++If the provider answered but not for a single document that was sent (every+answer keyed to something else), that is not a reranking: the response is the+degraded shape, `applied: false`, with the reason, and the operator's meters+count it as degraded, not applied.++## How it interacts with the rest of `_search`++The stage owns exactly one thing: the order, and with `min_score` the+membership, of `hits.hits`. Everything else in the response stays the engine's+and describes the **full match set**, not the window. Each row below is pinned+by a test in `rerank_stage_http.rs`.++| Combined with | Behaviour |+|---|---|+| `aggs` | **Unchanged.** Aggregations are computed over every matching document, and are identical with and without `rerank` — including when `min_score` prunes the page to nothing. |+| `hits.total`, `track_total_hits` | **The engine's total.** `true`, an integer cap and `false` each report exactly what they report without `rerank`. Four documents matched and two were judged irrelevant: `total` is 4, and `pruned_below_min_score` is 2. Both facts are true and the response says both. |+| `size`, `from` | Paging happens **inside the reranked window**. XERJ widens the engine's page to `window`, judges all of it, then cuts `from`/`size` out of the judge's order, so page two continues page one **as long as the provider gives the same probabilities on a repeat call** — there is no verdict cache, so every page request judges the whole window again (and pays for it again; see [Cost and concurrency facts](#cost-and-concurrency-facts)). Whether the real model is deterministic has not been verified by this project. To page without that dependency, ask for the window once (`size` = `window`) and page client-side. `from + size` must be ≤ `window`, or the request is a 400: a page cut across the window boundary would mix two rankings. |+| `highlight`, `fields`, `inner_hits`, `matched_queries` | **Travel with their hit.** The stage moves rendered hits, so nothing is recomputed and nothing is swapped between hits. |+| `"fields": ["_passage"]` | The matching passage is **judgeable**, and leads the text the judge reads. Name it in `rerank.fields` to send only the passage. |+| the large-response hint (`_xerj.hints`) | Its ready-to-send corrected request **keeps the `rerank` block**. A suggestion without it would be a different search, returning the engine's order. Its narrower projection also narrows what the judge is sent. |+| `_source` filtering | Decides what the judge sees — see the next section. `_source: false` with no `fields`, `docvalue_fields`, `stored_fields` or `script_fields` clause is a 400 (nothing can be judged); beside one that returns the text, it works, and the judge reads what came back — with or without `rerank.fields`: `"_source": false, "fields": ["body"], "rerank": {}` judges the `body` values returned under `fields`. |+| `explain` | The engine's explanation is kept, unaltered, under a new top node whose `value` equals the new `_score` and whose description says the score is a rerank probability. An `_explanation.value` that disagreed with `_score` would mislead. |+| `profile` | Unchanged; it profiles the engine. The stage reports its own time in `_rerank.took_ms`. |+| top-level `min_score` | The **engine's** threshold, applied to **engine** scores before the window is cut. It shapes the match set and `hits.total` exactly as it does without `rerank`, and it is never compared against a probability. `rerank.min_score` is the probability threshold. The two compose: engine bar first, probability bar second. |+| `knn` (top level) | Works. A vector is not a question, so a `knn` with no text `query` needs `rerank.query`. Beside a text query the question is read from the text query, and vector-only hits are inside the window the judge sees. Vectors are never sent. |+| `rescore` | Runs first, inside the engine. `rerank` has the last word. |+| `pit` (point in time) | Works: a PIT fixes the snapshot, not the order. Paging it with `search_after` is refused like any other `search_after`. |+| `index.max_result_window` smaller than `rerank.window` | **400**, in the caller's terms: `` `rerank.window` (30) exceeds `index.max_result_window` (20) on `kbsmall` ``. The stage asks the engine for `window` hits, so every participating index — including each one a wildcard expands to — must be able to return that many. Lower the window or raise the setting. |+| `"rerank": null` | The same as no `rerank` key, on every surface: `_search` runs without the stage, and an `_msearch` item or template carrying it is not refused. |+| multi-index and wildcard searches (`/a,b/_search`, `/kb*/_search`) | Work. The window is cut from the merged result, so hits from different indices are ranked against each other; `_index` stays with its `_id`. |+| `sort` (body or `?sort=`) | **400.** An explicit sort already fixes the order. |+| `search_after`, `collapse`, `scroll` | **400.** Each of them depends on the engine's order. |+| `size: 0` | **400.** The provider would be paid to judge documents the response does not return. |++### Endpoints that refuse `rerank`++Only `POST /{index}/_search` (and its `GET` and no-index forms) runs the stage.+These endpoints answer a body that carries `rerank` with a 400 that says so,+rather than dropping the key and returning the engine's order under a 200:++| Endpoint | Refusal |+|---|---|+| `_msearch` | **Per item.** The item that carries `rerank` gets a `status: 400` entry; the rest of the batch still runs. |+| `_search/template`, `_msearch/template` | 400 when the *rendered* template contains `rerank` (per item for `_msearch/template`). |+| `_async_search` | 400. A stored search would also be a stored third-party call that nobody is waiting on. |+| `_search_scroll`, `?scroll=`, and the `_search/scroll` continuation | 400. A scroll streams the engine's order. |+| `_rank_eval` | **Per request.** A `requests[]` entry whose `request` carries `rerank` is reported under `failures` by its id and contributes nothing to `metric_score`; the other requests are still evaluated. `_rank_eval` runs the engine's ranking only, so scoring it while ignoring the block would publish "reranking changed nothing" about an order the judge never saw. To measure a reranked order, run the `_search` requests yourself and score the returned ids. |++`_count`, `_validate/query` and `_explain` take a query, not a search body, and+return no ordering; they ignore a `rerank` key as they ignore every other+search-body key. The native `/v1` search API and the gRPC Search RPC refuse the+block the same way the endpoints above do, and `"rerank": null` is an absent+block on all of them.++## What leaves the machine++**The judge sees exactly what the response returns, and nothing else.**+Candidate text is read from each hit as the response will carry it: the+projected `_source`, then the hit's `fields`. That makes the rule auditable —+whatever you were about to receive is what was sent.++Sent to the provider, per search:++- the question (`rerank.query`, or the one read from the query);+- `rerank.instructions`, if given, and the model name;+- for each hit in the window, cut to `max_doc_chars` characters each:+ - with no `rerank.fields`: the `title` field if the hit returns one; then the+ matching passage, if the request asked for `"fields": ["_passage"]`; then+ every other top-level string field (and array of strings) in the returned+ `_source`; then every string value the hit returns under `fields` — which+ is where `fields`, `docvalue_fields`, `stored_fields` and `script_fields`+ put theirs — for a name `_source` did not already return, in name order.+ A value present in both is sent once. The passage goes **first** because+ text is cut at+ `max_doc_chars`: on a long document an appended passage would be the part+ that gets cut, and it is the part that says why the hit matched;+ - with `rerank.fields`: **those fields and nothing else.** The list is+ exhaustive. `["body"]` does not send the title; `["title", "body"]` does.+ A name is looked up in the returned `_source` first, then in the hit's+ `fields`. `"_passage"` is accepted as a name and sends the passage text;+- the provider key, as a bearer token in the `Authorization` header and nowhere+ else. If the provider echoes it back in an error body, XERJ redacts it before+ the body reaches the search caller or the log: whoever runs a search is not+ necessarily the operator who owns the key.++Never sent: document `_id`s and index names (documents are keyed `d0`, `d1`, … by+position), numbers, booleans, vectors, nested objects, and any field the+response does not return.++### Every way data leaves a XERJ node++Reranking is the only thing a **search request** can do that sends document+text off the node. It is not the only outbound connection a node can make. This+is the complete list, taken from the engine source — every outbound HTTP or TCP+client under `engine/crates` — and kept complete by a test+(`engine/crates/xerj-rerank/tests/egress_inventory.rs`) that fails when a new+client appears in a source file this list does not account for:++| Path | What leaves the node | When | Default |+|---|---|---|---|+| **Rerank provider** — `[rerank]`, `TYPESAFE_API_KEY` / `TYPESAFE_ENDPOINT` | The question, `rerank.instructions`, the model name, and the returned text of up to `window` hits (above) | A search that carries a `rerank` block | Off: no key is configured |+| **Proxy embeddings** — `[embedding] default_endpoint`, `--embed-mode proxy` | Document text of the fields it embeds, and the text of queries it embeds | Document text at write time (index, `_bulk`, update, reindex); query text at search time, for every query it embeds (`semantic`, hybrid, semantic memory recall) | Off: `default_endpoint` is empty |+| **WAL tap** — `[wal_tap]`, `PUT /_xerj/wal_tap` | Every write on the tapped indices (indexed documents and deletes), as `_bulk` to `{target_url}/_bulk`; never system indices | Continuously, every `poll_interval_ms` | Off: `enabled = false` |+| **Neural model download** — `--embed-mode neural` | HTTPS requests to the HuggingFace Hub naming the model (`config.json`, `tokenizer.json`, `model.safetensors`); no document or query text | The first time the neural embedder loads without `embedding.local_model_dir`; later starts read the local cache | Off: the default embedder is lexical |+| **Cluster transport** — `[cluster] enabled = true` | Raft messages to the configured `peers`: index names, mappings, shard assignments, node addresses, cluster settings; no document text and no queries | While cluster mode runs | Off: single node |+| **Object storage backend** — `[storage]`, `S3Backend` | S3 requests to the configured endpoint: bucket and key names, ranged reads, and the credentials from the environment; no document text and no query text | Never today — nothing on the segment path constructs it and `storage.backend = "s3"` refuses to start (#965 would wire it) | Off, and refused |+| **`xerj autoindex s3://`** (client, not the node) — `--endpoint-url`, `AWS_*` | S3 requests to the configured endpoint: bucket and key names and the credentials; the objects travel INTO the machine, and their text then goes to the node URL you gave the client | While that command runs | Off: only when you pass an `s3://` root |++A node started with the defaults opens none of these connections, and makes no+telemetry, update-check or licence call. Two things outside the node complete+the picture:++- **The Console in a browser** asks Google Fonts for its typefaces. That is+ the browser's request, not the node's, and it carries no index data; the+ [air-gapped recipe](./recipes/air-gapped-deployment.md) names the three pages+ that link the fonts, and a blocked request falls back to system fonts.+- **The command-line clients in the same binary** (`xerj autoindex`,+ `xerj mcp`, `xerj search` and the others) send what they read to the node URL+ you give them, `http://localhost:9200` unless you say otherwise.+ `xerj feedback --open-pr` runs your `gh` to open a GitHub pull request with+ the report it drafted; without that flag it does nothing over the network.++So `_source` filtering is also an egress control:++```json+{ "query": { "match": { "body": "refund" } },+ "_source": ["title"],+ "rerank": {} }+```++sends titles only. The `body` text is not in the response, so it is not in the+request to the provider either.++To send the **matching passage and nothing else** — usually the best trade on+long documents, for both the judge and the amount of text that leaves:++```json+{ "query": { "match": { "body": "refund" } },+ "_source": ["title"],+ "fields": ["_passage"],+ "rerank": { "fields": ["_passage"] } }+```++Whether judging the passage ranks better than judging the head of the document+has not been measured; what is certain is that less text is sent.++What was returned is checked on the **rendered hits**, not predicted from the+request, so a document is never judged blind:++- `rerank.fields` names a field that `_source` filtering removes and `fields`+ does not request: **400** before the search runs, naming the field;+- no hit in the window returns any text to judge: **400** `"nothing to judge"`,+ and nothing is sent;+- some named fields return text and others return none anywhere in the window:+ the search proceeds, and `_rerank.fields_without_text` lists the ones that+ contributed nothing.++Known limitation: today a `fields` entry is dropped when an explicit `_source`+includes list omits that field ([#932](https://github.com/xerj-org/xerj/issues/932)),+so `"_source": ["title"], "fields": ["body"], "rerank": {"fields": ["body"]}`+is refused as nothing to judge. `"_source": false, "fields": ["body"]` works.++To rule reranking out entirely on a node, set `[rerank] enabled = false`.++## Failure policy: degrade on deadline, surface on contract++A blanket "fall back to the engine's order on any error" would be the wrong+default. A caller who asked for calibrated relevance and silently got lexical+order has been misled. So the policy is split:++| What happened | Result |+|---|---|+| The provider did not answer inside `timeout_ms` | **Degrade.** HTTP 200, the engine's order and scores, `_rerank.applied: false` with the reason. A slow third party is not a reason to deny the caller results it already has. |+| `rerank.provider` is `none` / `disabled` | Degrade, same shape. |+| Malformed `rerank` block, unknown provider, a refused combination | **400** `illegal_argument_exception`. Nothing is sent. |+| The operator set `enabled = false` | **403** `rerank_exception`. Nothing is sent. |+| No key configured | **503** `rerank_exception`, naming `[rerank] api_key` and `TYPESAFE_API_KEY`. Nothing is sent. A search **without** `rerank` on the same node is unaffected. |+| Provider answered 401 / 403 / another 4xx | **502** `rerank_exception` with the provider's status. Not retried. |+| Provider answered 200 with a body that is not the documented shape | **502** `rerank_exception`. |+| Provider answered 200 with a body over 2 MiB | **502** `rerank_exception` ("larger than 2097152 bytes"). A real answer for 30 documents is a few kilobytes; the node stops reading at the ceiling instead of letting the endpoint choose how much memory a search allocates. |+| Provider unreachable | **502** `rerank_exception`. |+| Provider answered 429, 529 or 5xx | Retried, up to 3 attempts per call, with a short exponential back-off and an extra pause after a 429 — but never past the deadline. If the deadline runs out first, degrade; if the retries run out first, 502. |++Contract and configuration faults do not fix themselves, and hiding them costs+the operator more than it saves. A surfaced fault returns **no hits**: handing+back the engine's order next to an error would invite a caller to ignore the+error.++If some provider calls succeed and others fail, or the deadline stops later+calls from being sent, the stage keeps what it has: the judged hits are ordered+by probability, the rest follow with `_score: null` in the engine's order, and+`partial_failures` says how many calls did not deliver.++The split follows the one Meilisearch uses for its reranking call in its+personalization module (return the original hits when the deadline is exceeded,+return the error otherwise). The approach, and the retry back-off constants,+were adapted from that module (MIT); the code comment in `xerj-rerank` cites+the lines.++## Cost and concurrency facts++- **30 documents per provider call.** The `hev/jev-rerank` README reports a+ ~32k-token request budget, "~30–50 typical passages per call", and uses 30;+ XERJ uses the same figure and has not measured the provider's limit itself. A+ larger window is split: `window: 35` is two calls (30 + 5), and `window: 300`,+ the maximum, is ten.+- Scores from different calls are comparable, because each answer is an absolute+ probability rather than a rank within its batch. That is what makes splitting+ safe.+- **8 calls in flight by default, 16 at most**, sent in waves. The provider's+ rate limits are undocumented; the same README reports sustained 429s at about+ 24 requests in flight (their observation, not ours), and both ceilings sit+ under that. A search that provokes a 429 is slower than one that paced itself.+- One question is asked **per document** rather than one multiple-choice+ question across them. Real corpora have more than one relevant document, and a+ multiple-choice question forces a single winner. This one-`noul`-per-document+ approach is the one used by [`hev/jev-rerank`](https://github.com/hev/jev-rerank)+ (Apache-2.0).+- The node keeps one HTTP connection pool for the provider, so consecutive+ searches do not pay a TLS handshake each.+- **Every request is judged from scratch.** There is no verdict cache: three+ page requests (`from` 0, 10, 20) over one 30-document window are three+ provider calls and 90 paid judgements, not 30. Fetch the window once and page+ client-side when cost or page-to-page consistency matters.+- `_rerank.usage` is the meter reading for the search. XERJ does not price it.+ It is advisory: a provider that reports a token count that is not a+ non-negative number meters as zero for that count, and never costs the+ caller a ranking whose verdicts were all valid.++## From an agent (MCP)++`xerj_search` and `xerj_hybrid_search` take an optional `rerank` argument with+the same fields. `true` or `{}` means defaults.++- With a **plain-string** `query`, the MCP server passes your string as+ `rerank.query`, because the plain-string path expands into a `bool` query that+ the node will not read a question from.+- `xerj_hybrid_search` **requires** `rerank.query`: a hybrid search has several+ sub-queries and no single question.+- The MCP server cannot supply a provider key. Without one on the node the tool+ call returns the node's 503; repeat the search without `rerank`.++## Quality numbers, and what we did not measure++XERJ has not run the Jev model. The figures below for Jev, Voyage and Cohere are+**published in the [`hev/jev-rerank`](https://github.com/hev/jev-rerank) README,+were not run by this project, and rerank that project's own first-stage+shortlist, which is not XERJ's.** Same datasets, same metric, not a controlled+comparison.++nDCG@10 on the BEIR `test` split:++| | SciFact | NFCorpus | Run by |+|---|---:|---:|---|+| Jev (TypeSafe AI), 30 Nouls in one request — the shape this stage sends | 0.768 | 0.358 | hev/jev-rerank README — not by us |+| Voyage rerank-3 | 0.755 | 0.357 | hev/jev-rerank README — not by us |+| Cohere rerank-v3.5 | 0.745 | 0.340 | hev/jev-rerank README — not by us |+| XERJ BM25 | 0.6572 | 0.3016 | us |+| XERJ MiniLM vectors only | 0.6764 | 0.3291 | us, `--embed-mode neural` |+| XERJ BM25 top-30, reordered by MiniLM | 0.6855 | 0.3323 | us, `--embed-mode neural` |+| XERJ hybrid RRF | 0.6993 | 0.3448 | us, `--embed-mode neural` |++**Our rows were measured with `--embed-mode neural` and the+`all-MiniLM-L6-v2` model, not with the default embedder.** XERJ's default+embedder is lexical feature hashing, which has no model in it; do not read the+vector or hybrid rows as what a default node scores. The BM25 row uses no+embedder at all. Method, raw output and the reproduce commands are in+[`benchmarks/beir-hybrid`](../benchmarks/beir-hybrid).++Two things those runs do say:++- Reordering a BM25 shortlist with the same bi-encoder scored **below** fusing+ with it, on both datasets. That is why there is no "local MiniLM" rerank+ provider: it would be a regression with a good name.+- Where a team already has labelled history, some yes/no and pick-one decisions+ need no judge model at all —+ [`benchmarks/decisions-as-retrieval`](../benchmarks/decisions-as-retrieval)+ measures that, also under `--embed-mode neural`.++## Credits++- The one-`noul`-per-document request shape:+ [`hev/jev-rerank`](https://github.com/hev/jev-rerank), Apache-2.0.+- The degrade-on-deadline, surface-on-contract failure policy, and the retry+ back-off shape (exponential in the attempt, a flat extra pause after a 429):+ Meilisearch's personalization module (MIT, not part of its Enterprise+ Edition). Approach and constants adapted; cited in `xerj-rerank/src/lib.rs`.+- The provider API is TypeSafe AI's System One+ (`POST /v1/systemone`, <https://docs.typesafe.ai/api>).diff --git a/docs/ZERO_TOKEN_DIRECTION.md b/docs/ZERO_TOKEN_DIRECTION.mdindex 9ac778aa8..35bfff986 100644--- a/docs/ZERO_TOKEN_DIRECTION.md+++ b/docs/ZERO_TOKEN_DIRECTION.md@@ -42,9 +42,14 @@ stage that answers "which of these actually answer it", over the top *N* only.type with `rrf` or `linear` fusion. There is no rerank stage on `main`. Thebranch `feat/rerank-stage` adds a `rerank` block on `_search` that hands thetop hits to an external relevance judge; it is opt-in per request and inert-until an operator configures a key, and it is the one feature that would send-document text off the machine. That is exactly why a *local* judge is on this-list.+until an operator configures a key. It is the only *search-time*+feature that sends document text off the node, not the only feature that does:+proxy embeddings (`[embedding] default_endpoint`) and the WAL tap send text off+the node too, and all three are off by default. `docs/RERANK.md` lists every+outbound connection a node can open, and `xerj-rerank/tests/egress_inventory.rs`+holds that list against the engine's outbound clients. That a query-time stage+sends the text of whatever a query matched is exactly why a *local* judge is on+this list.**The bar a local model has to clear.** Before building one we measured whatalready ships. BEIR test splits, `--embed-mode neural` (all-MiniLM-L6-v2, CPU),diff --git a/docs/recipes/air-gapped-deployment.md b/docs/recipes/air-gapped-deployment.mdindex 6b77b2907..3f8dd5610 100644--- a/docs/recipes/air-gapped-deployment.md+++ b/docs/recipes/air-gapped-deployment.md@@ -34,6 +34,9 @@ those two files before entering the enclave.| External proxy | Disabled (`default_endpoint = ""`) | Only the configured endpoint is contacted || ONNX | Experimental and not in the standard release | Build the feature and provide local assets explicitly || WAL tap | Disabled | It remains inert unless enabled with a target URL; see the durable overlay note below |+| Rerank provider | Inert: no key is configured | The only search-time feature that POSTs document text to a third party. The external proxy and WAL tap rows above also send text off the node when enabled; the neural model download and the cluster transport carry no document text; all of them are off by default. It stays inert with no `rerank.api_key` and no `TYPESAFE_API_KEY` in the service environment, and a search must carry a `rerank` block to trigger it. Set `[rerank] enabled = false` to refuse it outright, whatever the environment holds; `GET /_xerj/rerank` confirms `"enabled": false`. See [RERANK.md](../RERANK.md#every-way-data-leaves-a-xerj-node) for every outbound connection a node can open |+| Object storage backend | Refused: `storage.backend = "s3"` does not start | The `S3Backend` client exists (#966) but nothing on the segment path constructs it, so an air-gapped node opens no bucket connection. If #965 wires it, the endpoint you configure is the only host contacted |+| `xerj autoindex s3://` | Off: only when you pass an `s3://` root | A client, not the node. It contacts the endpoint you name (`--endpoint-url`) and nothing else; a folder root contacts nothing || Cluster | Disabled | Single-node startup does not initialize the Raft transport || REST / ES-compatible listeners | `127.0.0.1` | The default is loopback-only || `autoindex` and MCP clients | Localhost defaults | They connect to `localhost`; they do not create a public listener |diff --git a/engine/Cargo.lock b/engine/Cargo.lockindex 5bf668d45..1b9e4f8c7 100644--- a/engine/Cargo.lock+++ b/engine/Cargo.lock@@ -7244,6 +7244,7 @@ dependencies = ["xerj-common","xerj-engine","xerj-query",+ "xerj-rerank","xerj-wasm","xxhash-rust",]@@ -7520,6 +7521,7 @@ dependencies = ["serde_json","tokio","xerj-common",+ "xerj-rerank",][[package]]@@ -7537,6 +7539,18 @@ dependencies = ["xerj-common",]+[[package]]+name = "xerj-rerank"+version = "1.0.0-rc.74"+dependencies = [+ "reqwest",+ "serde",+ "serde_json",+ "thiserror 2.0.18",+ "tokio",+ "tracing",+]+[[package]]name = "xerj-server"version = "1.0.0-rc.74"diff --git a/engine/Cargo.toml b/engine/Cargo.tomlindex 851e23ba1..e4685393a 100644--- a/engine/Cargo.toml+++ b/engine/Cargo.toml@@ -8,6 +8,7 @@ members = ["crates/xerj-vector","crates/xerj-logs","crates/xerj-query",+ "crates/xerj-rerank","crates/xerj-api","crates/xerj-ai","crates/xerj-compress",diff --git a/engine/README.md b/engine/README.mdindex 92c1d509e..a12d8269f 100644--- a/engine/README.md+++ b/engine/README.md@@ -296,6 +296,7 @@ HTTP Request| `xerj-console-api` | Console backend at `/_xerj-console/api/v1` — auth, prefs, dashboards, saved views, data sources || `xerj-cluster` | Embedded Raft consensus for cluster metadata, written from scratch || `xerj-mcp` | MCP stdio server exposing XERJ's REST surface to agent hosts as tools |+| `xerj-rerank` | Second-stage reranking client for `_search`: strict `rerank` block parser, provider call with batching and a deadline, degrade-on-deadline / surface-on-contract failure policy. A leaf the API layer calls; the engine does not link it || `xerj-wasm` | Pluggable ingest-time document transform pipeline |Every crate under `crates/` appears in this table — `crates/xerj-engine/tests/docs_capability_lists.rs`diff --git a/engine/crates/xerj-api/Cargo.toml b/engine/crates/xerj-api/Cargo.tomlindex 91b7ece86..dd7a4a660 100644--- a/engine/crates/xerj-api/Cargo.toml+++ b/engine/crates/xerj-api/Cargo.toml@@ -16,6 +16,7 @@ onnx-experimental = ["xerj-engine/onnx-experimental"]xerj-common = { path = "../xerj-common" }xerj-engine = { path = "../xerj-engine" }xerj-query = { path = "../xerj-query" }+xerj-rerank = { path = "../xerj-rerank" }xerj-wasm = { path = "../xerj-wasm" }# Note: this crate intentionally does NOT depend on xerj-console-api.# Xerj Console is a peer product mounted at the server level (`xerj-server`)diff --git a/engine/crates/xerj-api/src/es_compat.rs b/engine/crates/xerj-api/src/es_compat.rsindex ceca53ad8..712da8dc3 100644--- a/engine/crates/xerj-api/src/es_compat.rs+++ b/engine/crates/xerj-api/src/es_compat.rs@@ -3447,6 +3447,12 @@ pub struct EsSearchBody {/// (live: a 1 ms-timeout wildcard ran 3.7 s with `timed_out: false`).#[serde(default)]pub timeout: Option<String>,+ /// Second-stage relevance reranking over the top `window` hits — see+ /// [`crate::rerank_stage`]. Declared here because serde drops unknown+ /// top-level keys silently, and a `rerank` block that vanished without a+ /// word would return lexical order to a caller who asked for something else.+ #[serde(default)]+ pub rerank: Option<Value>,}impl Default for EsSearchBody {@@ -3482,6 +3488,7 @@ impl Default for EsSearchBody {slice: None,pit: None,timeout: None,+ rerank: None,}}}@@ -3533,6 +3540,10 @@ const ES_SEARCH_TOP_LEVEL_KEYS: &[&str] = &["profile","stats","ext",+ // The one key here that is NOT Elasticsearch's: XERJ's second-stage+ // reranking (`crate::rerank_stage`). Listed so the unknown-key gate lets it+ // through to a stage that validates its contents strictly.+ "rerank",];/// A minimal position-tracking JSON scanner used solely to locate the first@@ -7837,7 +7848,7 @@ fn search_response_hint(index: &str, body: &EsSearchBody, hits: &[EsHit]) -> Optreturn None;}- Some(json!({+ let mut hint = json!({"hints": [{"reason": format!("this response carried {source_bytes} bytes of `_source` across \@@ -7859,7 +7870,16 @@ fn search_response_hint(index: &str, body: &EsSearchBody, hits: &[EsHit]) -> Opt},},}],- }))+ });+ // The suggestion is "ready to send", so it must still be the caller's+ // search. A request that asked for `rerank` and followed a hint without it+ // would come back in the engine's order, believing it had been reranked.+ // The suggested projection also narrows what the judge is sent — to the+ // title, the path and the matching passage — which is the same trade.+ if let Some(rerank) = &body.rerank {+ hint["hints"][0]["try"]["body"]["rerank"] = rerank.clone();+ }+ Some(hint)}/// A short list of existing index names to suggest when `_search` named an@@ -9334,6 +9354,34 @@ async fn search_impl(}));}+ // Validate `rerank` and widen the page to its window *before* the search+ // runs; the stage itself runs on the rendered hits further down. Placed+ // after every URL parameter has been merged into `body` — `?sort=`,+ // `?_source=false`, `?_source_excludes=`, `?size=`, `?from=` are all things+ // the stage refuses or pages by, and a check that ran before the merge let+ // the URL spelling of each one straight through.+ let rerank_plan = match crate::rerank_stage::RerankPlan::prepare(&mut body, is_scroll_request) {+ Ok(plan) => plan,+ Err(reason) => {+ crate::rerank_stage::record_refused(&state.metrics);+ return (+ StatusCode::BAD_REQUEST,+ Json(json!({+ "error": {+ "root_cause": [{+ "type": "illegal_argument_exception",+ "reason": reason,+ }],+ "type": "illegal_argument_exception",+ "reason": reason,+ },+ "status": 400,+ })),+ )+ .into_response();+ }+ };+// `?typed_keys` — prefix agg names with their type in the response.let typed_keys = params.typed_keys.is_some();@@ -10872,31 +10920,47 @@ async fn search_impl(}}+ // `rerank` widened the engine's page to `rerank.window`, so the engine's+ // own `from + size > max_result_window` check would fire on a page size+ // the caller never sent (they asked for `size: 5`, the stage asked the+ // engine for 30). Say it in the caller's terms, per index, before the+ // engine runs — a wildcard search tripped on one small index the same way.+ if let Some(plan) = &rerank_plan {+ for ix in &participating_indices {+ let max_w = index_max_result_window(&state, ix);+ if plan.window() > max_w {+ let reason = format!(+ "`rerank.window` ({}) exceeds `index.max_result_window` ({max_w}) on \+ `{ix}`: the stage judges that many of the engine's top hits, so the \+ engine must be able to return them. Lower `rerank.window` or raise \+ the index setting",+ plan.window()+ );+ crate::rerank_stage::record_refused(&state.metrics);+ return (+ StatusCode::BAD_REQUEST,+ Json(json!({+ "error": {+ "root_cause": [{+ "type": "illegal_argument_exception",+ "reason": reason,+ }],+ "type": "illegal_argument_exception",+ "reason": reason,+ },+ "status": 400,+ })),+ )+ .into_response();+ }+ }+ }+// Enforce per-index `index.max_result_window` for `ids` clauses.// ES rejects queries whose `ids.values` list is longer than the// index's configured max_result_window (default 10000).for ix in &participating_indices {- let max_w = state- .engine- .index_settings- .get(ix)- .map(|v| v.clone())- .and_then(|s| {- let as_int = |v: &Value| {- v.as_u64()- .or_else(|| v.as_str().and_then(|s| s.parse().ok()))- };- s.pointer("/index/max_result_window")- .and_then(as_int)- .or_else(|| {- s.get("index")- .and_then(|i| i.get("index.max_result_window"))- .and_then(as_int)- })- .or_else(|| s.get("index.max_result_window").and_then(as_int))- })- .map(|v| v as usize)- .unwrap_or(10_000);+ let max_w = index_max_result_window(&state, ix);fn max_ids_in_json(q: &Value) -> usize {match q {Value::Object(obj) => {@@ -14176,6 +14240,67 @@ async fn search_impl(shards_block.failed = shards_failed_count;}+ // ── rerank stage ────────────────────────────────────────────────────────+ // Runs on the rendered hits, so everything a hit carries — `highlight`,+ // `fields`, `inner_hits`, `matched_queries`, `_ignored` — moves with it.+ // Runs BEFORE the response hints and the `_savings` re-sum below, because+ // both describe the page this response actually emits and the stage is+ // what cuts the widened window back down to that page. `hits.total` and+ // `aggregations` are deliberately untouched: they describe the full match+ // set, not the window.+ let mut hits = hits;+ let mut max_score = max_score;+ let mut took_ms = took_ms;+ let mut savings_claim = savings_claim;+ let rerank_info = match &rerank_plan {+ None => None,+ Some(plan) => {+ let applied = plan.apply(&state.rerank, &mut hits, &mut max_score).await;+ crate::rerank_stage::record_outcome(&state.metrics, &applied);+ match applied {+ Ok(info) => {+ // Hand the caller's own page back to everything downstream+ // that reads it (`terminated_early`, the payload hint's+ // suggested request) — the widening was the stage's business.+ let (from, size) = plan.requested_page();+ body.from = from;+ body.size = size;+ // `took` is what the caller waited, and the provider call is+ // most of it. The engine-only figure was already recorded in+ // the latency histograms above, which is where it belongs.+ took_ms = started.elapsed().as_millis() as u64;+ // `_savings` is summed over the hits a response emits; the+ // earlier sum was over the whole window.+ if let Some((record, _)) = savings_claim {+ let emitted: u64 = hits+ .iter()+ .filter_map(|h| savings_by_hit.get(&(h.index.clone(), h.id.clone())))+ .sum();+ savings_claim = Some((record, emitted));+ }+ Some(info)+ }+ Err(e) => {+ let status = crate::rerank_stage::status_for(&e);+ let kind = crate::rerank_stage::error_type_for(&e);+ let reason = format!("rerank failed: {e}");+ return (+ StatusCode::from_u16(status).unwrap_or(StatusCode::BAD_GATEWAY),+ Json(json!({+ "error": {+ "root_cause": [{ "type": kind, "reason": reason }],+ "type": kind,+ "reason": reason,+ },+ "status": status,+ })),+ )+ .into_response();+ }+ }+ }+ };+// Computed before `hits` is moved into `response_body` below.// Two independent diagnostics, merged into one `hints` array rather than// one winning: a query can both name a nonexistent field AND return an@@ -14331,6 +14456,21 @@ async fn search_impl(// everything else including `hits`. `serde_json` runs with// `preserve_order` workspace-wide (see xerj-common/Cargo.toml), so the// rebuilt map's insertion order is exactly the wire order.+ // `_rerank` says which ranking the caller is looking at — applied, or the+ // engine's own after a degrade. It goes ahead of `hits` for the same reason+ // the hint below does: a reader that truncates long output from the bottom+ // must still see it.+ if let Some(info) = rerank_info {+ if let Some(obj) = response_body.as_object_mut() {+ let mut rebuilt = serde_json::Map::with_capacity(obj.len() + 1);+ rebuilt.insert("_rerank".to_string(), info);+ for (k, v) in obj.iter() {+ rebuilt.insert(k.clone(), v.clone());+ }+ *obj = rebuilt;+ }+ }+if let Some(hint) = response_hint {if let Some(obj) = response_body.as_object_mut() {let mut rebuilt = serde_json::Map::with_capacity(obj.len() + 1);@@ -20814,6 +20954,34 @@ pub async fn delete_index_template(// DELETE /_search/scroll — clear scroll// ─────────────────────────────────────────────────────────────────────────────+/// An index's `index.max_result_window`, in any of the three spellings a+/// settings document can carry it (nested `index.max_result_window`, a flat+/// dotted key under `index`, or a top-level dotted key), defaulting to ES's+/// 10,000. Shared by the `ids` clause check and the `rerank.window` check.+fn index_max_result_window(state: &AppState, ix: &str) -> usize {+ state+ .engine+ .index_settings+ .get(ix)+ .map(|v| v.clone())+ .and_then(|s| {+ let as_int = |v: &Value| {+ v.as_u64()+ .or_else(|| v.as_str().and_then(|s| s.parse().ok()))+ };+ s.pointer("/index/max_result_window")+ .and_then(as_int)+ .or_else(|| {+ s.get("index")+ .and_then(|i| i.get("index.max_result_window"))+ .and_then(as_int)+ })+ .or_else(|| s.get("index.max_result_window").and_then(as_int))+ })+ .map(|v| v as usize)+ .unwrap_or(10_000)+}+#[derive(Debug, Deserialize, Default)]pub struct ScrollQueryParams {pub scroll: Option<String>,@@ -20828,6 +20996,16 @@ pub async fn search_with_scroll(let started = Instant::now();let body = body.into_or_default();+ // A scroll streams the engine's order; `rerank` cannot apply to it, and+ // this handler used to rebuild the body with `rerank: None` — dropping it.+ if body.rerank.is_some() {+ return (+ StatusCode::BAD_REQUEST,+ Json(crate::rerank_stage::unsupported_on("_search_scroll")),+ )+ .into_response();+ }+// RC4 blocker 11: same open-scroll cap as the `?scroll=` path on// `_search` — checked up front so the full-corpus snapshot below is// never even computed for a request that cannot get a context.@@ -20887,6 +21065,7 @@ pub async fn search_with_scroll(slice: body.slice.clone(),pit: body.pit.clone(),timeout: body.timeout.clone(),+ rerank: None,};// page_size: what the caller requested (or default 10)let page_size = body.size;@@ -21181,6 +21360,13 @@ pub struct ScrollBody {pub scroll: Option<String>,#[serde(default)]pub scroll_id: Option<String>,+ /// Declared so a continuation body carrying `rerank` is refused rather+ /// than ignored: serde drops unknown keys, and a scroll continuation+ /// streams the engine's order, so a caller who added `rerank` here was+ /// getting lexical order under a 200 — the same accepted-and-ignored+ /// class the `?scroll=` path already refuses.+ #[serde(default)]+ pub rerank: Option<Value>,}#[derive(Debug, Deserialize, Default)]@@ -21200,6 +21386,13 @@ pub async fn next_scroll() -> impl IntoResponse {// scroll_id may come from body OR query paramlet body = body.into_or_default();+ if body.rerank.is_some() {+ return (+ StatusCode::BAD_REQUEST,+ Json(crate::rerank_stage::unsupported_on("_search/scroll")),+ )+ .into_response();+ }let scroll_id = match body.scroll_id.clone().or_else(|| params.scroll_id.clone()) {Some(id) if !id.is_empty() => id,_ => {@@ -23717,6 +23910,15 @@ async fn msearch_impl(continue;}+ // `rerank` is a `_search` stage and this path has its own, smaller+ // renderer. `parse_request` ignores keys it does not know, so without+ // this the item came back 200 in the engine's order — refused per item+ // instead, and the rest of the batch still runs.+ if crate::rerank_stage::carries_rerank(&search_body_val) {+ responses.push(crate::rerank_stage::unsupported_on("_msearch"));+ continue;+ }+// Determine index name: header wins, then the path index (for// /{index}/_msearch), then "*" (all indices).let index_name = header@@ -31734,6 +31936,16 @@ pub async fn search_template(return ApiError::new(xerj_common::XerjError::invalid_query(msg)).into_response();}+ // A template that renders a `rerank` block: this minimal path does not run+ // the stage, and `parse_request` would drop the key without a word.+ if crate::rerank_stage::carries_rerank(&search_body_val) {+ return (+ StatusCode::BAD_REQUEST,+ Json(crate::rerank_stage::unsupported_on("_search/template")),+ )+ .into_response();+ }+// Parse and execute as a normal search.let search_req = match xerj_query::parse_request(&search_body_val).map_err(|e| xerj_common::XerjError::invalid_query(e.to_string()))@@ -31919,6 +32131,13 @@ async fn msearch_template_impl(continue;};+ // Same refusal as `_msearch` and `_search/template`: the rendered body+ // carries a `rerank` block this path would silently drop.+ if crate::rerank_stage::carries_rerank(&search_body_val) {+ responses.push(crate::rerank_stage::unsupported_on("_msearch/template"));+ continue;+ }+// AND in any alias filter before parsing (#451 class C), same as// `_msearch`: a filtered alias's document boundary must apply to the// rendered template search too.@@ -36548,6 +36767,16 @@ pub async fn async_search_submit(body: OptionalJson<EsSearchBody>,) -> impl IntoResponse {let mut body = body.into_or_default();+ // An async search stores its response and hands it back later; a stored+ // rerank would also be a stored third-party call nobody is waiting on.+ // Refused rather than dropped.+ if body.rerank.is_some() {+ return (+ StatusCode::BAD_REQUEST,+ Json(crate::rerank_stage::unsupported_on("_async_search")),+ )+ .into_response();+ }// ES 8.x default: absent track_total_hits → totals tracked up to 10,000// only ({"value":10000,"relation":"gte"} beyond), same as `_search`.if body.track_total_hits.is_none() {@@ -37209,6 +37438,25 @@ pub async fn rank_eval(};for req_spec in &body.requests {+ // `_rank_eval` runs the engine's ranking and nothing else: the request+ // below is rebuilt from `query` and `size`, so a `rerank` block would+ // vanish and the metric would be computed over the engine's order —+ // "reranking changed nothing", under a 200, from the one endpoint whose+ // job is measuring ranking quality. Reported per request in `failures`+ // (the rest of the batch still runs), like every other request this+ // handler cannot run, and nothing is sent to the provider.+ if crate::rerank_stage::carries_rerank(&req_spec.request) {+ crate::rerank_stage::record_refused(&state.metrics);+ // The same `illegal_argument_exception` body every other surface+ // that does not run the stage answers with, not the+ // `search_phase_execution_exception` a failed search gets: the+ // request was never run.+ failures.insert(+ req_spec.id.clone(),+ crate::rerank_stage::unsupported_on("_rank_eval")["error"].clone(),+ );+ continue;+ }let query_val = req_spec.request.get("query")diff --git a/engine/crates/xerj-api/src/lib.rs b/engine/crates/xerj-api/src/lib.rsindex 8cfe4a8d1..03349cbdc 100644--- a/engine/crates/xerj-api/src/lib.rs+++ b/engine/crates/xerj-api/src/lib.rs@@ -74,6 +74,7 @@ pub mod graph_api;pub mod ism_api;pub mod memory_api;pub mod native;+pub mod rerank_stage;pub mod responses;pub mod router;pub mod state;diff --git a/engine/crates/xerj-api/src/native.rs b/engine/crates/xerj-api/src/native.rsindex 1289e6719..d927a1f68 100644--- a/engine/crates/xerj-api/src/native.rs+++ b/engine/crates/xerj-api/src/native.rs@@ -61,6 +61,11 @@ pub struct NativeSearchRequest {pub sort: Option<Value>,#[serde(default)]pub fields: Option<Vec<String>>,+ /// Declared only so it can be REFUSED. This struct ignores unknown keys, so+ /// without the field a `rerank` block vanished and the caller got the+ /// engine's order back under a 200, believing it had been reranked.+ #[serde(default)]+ pub rerank: Option<Value>,}fn default_size() -> usize {@@ -492,6 +497,19 @@ pub async fn search(// recorded once at completion via `record_query`.let _search_guard = state.metrics.active_search_guard();+ if req.rerank.is_some() {+ crate::rerank_stage::record_refused(&state.metrics);+ let error = xerj_common::XerjError::invalid_query(crate::rerank_stage::unsupported_reason(+ "the native search API",+ ));+ return native_error(+ error,+ Some(&request_id),+ started.elapsed().as_millis() as u64,+ )+ .into_response();+ }+if !native_fields_supported(req.fields.as_deref()) {let error = xerj_common::XerjError::invalid_query("native search only supports the `_passage` pseudo-field; remove `fields` and read \diff --git a/engine/crates/xerj-api/src/rerank_stage.rs b/engine/crates/xerj-api/src/rerank_stage.rsnew file mode 100644index 000000000..8aa72ea2f--- /dev/null+++ b/engine/crates/xerj-api/src/rerank_stage.rs@@ -0,0 +1,1185 @@+//! The `rerank` stage of `_search`.+//!+//! Runs after the engine has produced and rendered its hits, and before the+//! response is assembled. It never touches the engine: it widens the requested+//! page to the rerank window, hands the window to a [`xerj_rerank::Provider`],+//! and reorders the rendered hits by what comes back.+//!+//! # What the stage owns, and what it leaves alone+//!+//! It owns exactly one thing: the order (and, with `rerank.min_score`, the+//! membership) of `hits.hits`. Everything else in the response stays the+//! engine's — `hits.total`, `aggregations`, `suggest`, `profile` all describe+//! the full match set, not the window. That is the split Meilisearch's+//! personalisation rerank makes too+//! (`crates/meilisearch/src/routes/indexes/search.rs:788-801`: only+//! `search_result.hits` is replaced, facets and the estimated total are left as+//! computed; approach adapted, no code copied).+//!+//! # The judge sees what the caller sees+//!+//! Candidate text is read from the hit as the response will carry it — the+//! projected `_source`, then the hit's `fields`. Nothing the response does not+//! return is sent to the provider. That is a privacy property as much as a+//! design one: reranking is the only search-time feature that sends document+//! text off the node (proxy embeddings send query text at search time and+//! document text at write time; the WAL tap sends writes; the full list is+//! `docs/RERANK.md`, "Every way data leaves a XERJ node", kept complete by+//! `xerj-rerank/tests/egress_inventory.rs`), and "exactly what you were about+//! to receive, no more" is a rule a caller can audit. The cost is that+//! `_source` filtering which removes the text also removes it from the judge,+//! so that combination is refused by name instead of being judged blind.+//!+//! Two rules follow from treating this as an egress control rather than a+//! convenience. `rerank.fields` is EXHAUSTIVE: naming `["body"]` sends the body+//! and nothing else, not even the title. And what was actually returned is+//! checked on the rendered hits, not predicted from the request — a named field+//! that contributed no text is reported as `_rerank.fields_without_text`, and a+//! window with no text at all is a 400, never a batch of blank documents.+//!+//! # Strictness+//!+//! The stage refuses what it cannot do coherently. An explicit `sort` means the+//! caller has already said how results are ordered; paging past the window+//! means pages would be cut from two different orderings. Both are a 400 rather+//! than quietly honoured in a way that misleads. Surfaces that do not run the+//! stage at all (`_msearch`, search templates, `_async_search`, scroll and its+//! `_search/scroll` continuation, `_rank_eval`, the native `/v1` search API and+//! the gRPC Search RPC) refuse a body carrying `rerank` for the same reason: dropping+//! the key silently returns lexical order to a caller who asked for something+//! else. `"rerank": null` is the same as no `rerank` key on every surface.+//!+//! # One kind of `_score` per response+//!+//! When the stage applies, `_score` is a probability on every hit that has+//! one and `null` on every hit that does not — a hit the provider returned no+//! verdict for, or one that was skipped as blank. The engine's BM25 value is+//! never left in `_score` next to probabilities: a client that sorts by+//! `_score` or scales by `hits.max_score` (Kibana's score bars, the+//! elasticsearch-py helpers) cannot tell a `1.63` BM25 from a `0.9`+//! probability, and `max_score` would be smaller than a later hit's score.+//! `null` is what Elasticsearch itself puts in `_score` when a hit has no+//! comparable score (a field sort without `track_scores`), so every client+//! already handles it. The count is reported as `_rerank.unjudged`.++use axum::extract::State;+use axum::Json;+use serde_json::{json, Value};+use std::time::{Duration, Instant};+use xerj_rerank::{+ Candidate, Deadline, Provider, ProviderSettings, RerankConfig, RerankError, RerankOutcome,+};++use crate::es_compat::EsSearchBody;+use crate::responses::EsHit;+use crate::state::AppState;++/// A validated rerank request, produced before the search runs.+pub struct RerankPlan {+ cfg: RerankConfig,+ query: String,+ requested_size: usize,+ from: usize,+}++/// Read a plain question out of the simple query shapes that carry one.+///+/// Deliberately shallow. A `bool` tree, a `hybrid` with two different arms or a+/// `query_string` with operators has no single "question", and picking one+/// would rank against a guess. Those callers pass `rerank.query`.+fn infer_query(q: &Value) -> Option<String> {+ let obj = q.as_object()?;+ if obj.len() != 1 {+ return None;+ }+ let (kind, params) = obj.iter().next()?;+ match kind.as_str() {+ "multi_match" | "semantic" | "simple_query_string" => {+ params.get("query")?.as_str().map(str::to_string)+ }+ "match" | "match_phrase" => {+ let field = params.as_object()?.values().next()?;+ match field {+ Value::String(s) => Some(s.clone()),+ Value::Object(o) => o.get("query")?.as_str().map(str::to_string),+ _ => None,+ }+ }+ _ => None,+ }+}++/// `*`-glob match, the only wildcard `_source` filtering supports.+fn glob(pattern: &str, name: &str) -> bool {+ let mut parts = pattern.split('*');+ // `split` always yields at least one piece: the literal head.+ let head = parts.next().unwrap_or_default();+ let Some(mut rest) = name.strip_prefix(head) else {+ return false;+ };+ let mut pieces: Vec<&str> = parts.collect();+ let Some(tail) = pieces.pop() else {+ // No `*` at all: the head had to be the whole name.+ return rest.is_empty();+ };+ for piece in pieces {+ match rest.find(piece) {+ Some(at) => rest = &rest[at + piece.len()..],+ None => return false,+ }+ }+ rest.ends_with(tail)+}++/// Whether a `_source` filter pattern keeps `field` — the field itself, an+/// ancestor object of it, or (for includes) a descendant of it.+fn pattern_covers(pattern: &str, field: &str) -> bool {+ glob(pattern, field)+ || field+ .match_indices('.')+ .any(|(i, _)| glob(pattern, &field[..i]))+}++/// Whether the request's `_source` clause leaves `field` in the response.+fn source_keeps(source: Option<&Value>, field: &str) -> bool {+ let strings = |v: &Value| -> Vec<String> {+ match v {+ Value::String(s) => vec![s.clone()],+ Value::Array(a) => a+ .iter()+ .filter_map(|x| x.as_str().map(str::to_string))+ .collect(),+ _ => Vec::new(),+ }+ };+ let (includes, excludes) = match source {+ None | Some(Value::Bool(true)) | Some(Value::Null) => return true,+ Some(Value::Bool(false)) => return false,+ Some(v @ (Value::String(_) | Value::Array(_))) => (strings(v), Vec::new()),+ Some(Value::Object(o)) => (+ o.get("includes")+ .or_else(|| o.get("include"))+ .map(strings)+ .unwrap_or_default(),+ o.get("excludes")+ .or_else(|| o.get("exclude"))+ .map(strings)+ .unwrap_or_default(),+ ),+ Some(_) => return true,+ };+ if excludes.iter().any(|p| pattern_covers(p, field)) {+ return false;+ }+ includes.is_empty()+ || includes+ .iter()+ .any(|p| pattern_covers(p, field) || p.starts_with(&format!("{field}.")))+}++/// Whether the request's `fields` clause asks for `field`.+fn fields_requests(fields: Option<&Value>, field: &str) -> bool {+ let Some(Value::Array(arr)) = fields else {+ return false;+ };+ arr.iter().any(|entry| {+ let name = match entry {+ Value::String(s) => s.as_str(),+ Value::Object(o) => o.get("field").and_then(Value::as_str).unwrap_or(""),+ _ => "",+ };+ !name.is_empty() && glob(name, field)+ })+}++/// Prose collected for the judge, cut at `max_doc_chars` WHILE it is+/// collected.+///+/// The provider crate cuts title and text to `max_doc_chars` when it builds the+/// request, so nothing past that point is ever sent. The first draft collected+/// every returned string field whole, cloned the result into `sendable`, into+/// per-call batches and into each task, and only then cut it: on a window of+/// 300 documents of ~1 MB each that was roughly 0.45 GB of transient copies of+/// text that could never leave the node (measured in review). Stopping at the+/// budget here bounds every later copy by `window × 2 × max_doc_chars`+/// characters, and what is sent is character-for-character what it was.+struct Prose {+ buf: String,+ /// Characters still allowed.+ room: usize,+}++impl Prose {+ fn new(max_chars: usize) -> Self {+ Self {+ buf: String::new(),+ room: max_chars,+ }+ }++ /// Append one piece of prose, newline-separated from the last.+ fn push_str(&mut self, s: &str) {+ if self.room == 0 || s.trim().is_empty() {+ return;+ }+ if !self.buf.is_empty() {+ self.buf.push('\n');+ self.room -= 1;+ if self.room == 0 {+ return;+ }+ }+ let cut = xerj_rerank::clip(s, self.room);+ // `cut` is at most `room` characters; the count is only walked when+ // the piece fit whole.+ self.room = if cut.len() < s.len() {+ 0+ } else {+ self.room - cut.chars().count()+ };+ self.buf.push_str(cut);+ }++ /// Append every string under `v` — a string, or an array of strings.+ /// Numbers, booleans, vectors and nested objects are not prose and are not+ /// sent.+ fn push(&mut self, v: &Value) {+ match v {+ Value::String(s) => self.push_str(s),+ Value::Array(items) => items+ .iter()+ .filter(|i| i.is_string())+ .for_each(|i| self.push(i)),+ _ => {}+ }+ }++ fn is_empty(&self) -> bool {+ self.buf.is_empty()+ }+}++/// `a.b.c` looked up first as a literal key, then as a path.+fn lookup<'a>(obj: &'a serde_json::Map<String, Value>, field: &str) -> Option<&'a Value> {+ if let Some(v) = obj.get(field) {+ return Some(v);+ }+ let (head, rest) = field.split_once('.')?;+ lookup(obj.get(head)?.as_object()?, rest)+}++impl RerankPlan {+ /// Validate the request and widen the page to the rerank window.+ ///+ /// Must run AFTER the URL parameters (`?sort=`, `?_source=`, `?size=`,+ /// `?from=`…) have been merged into `body`, or a `?sort=` slips past the+ /// refusal below. Returns `Ok(None)` when the body has no `rerank` block.+ /// `Err` carries the reason for a 400.+ pub fn prepare(body: &mut EsSearchBody, scrolling: bool) -> Result<Option<Self>, String> {+ let Some(raw) = body.rerank.as_ref() else {+ return Ok(None);+ };+ let cfg = RerankConfig::from_json(raw).map_err(|e| e.to_string())?;++ if scrolling {+ return Err(+ "`rerank` cannot be combined with `scroll`: a scroll streams the \+ engine's order and a reranked window has a different one"+ .into(),+ );+ }+ if body.sort.is_some() {+ return Err(+ "`rerank` cannot be combined with `sort`: an explicit sort already \+ fixes the order, and reranking would discard it"+ .into(),+ );+ }+ if body.search_after.is_some() {+ return Err("`rerank` cannot be combined with `search_after`".into());+ }+ if body.collapse.is_some() {+ return Err("`rerank` cannot be combined with `collapse`".into());+ }+ // `_source: false` on its own returns no text, so there is certainly+ // nothing to judge and the search need not run. Beside a clause that+ // puts values on the hit — `fields`, `docvalue_fields`,+ // `stored_fields`, `script_fields` — it is a coherent request ("do not+ // return the source, return `body` through `fields`, judge on that")+ // and is let through to the post-render check in `apply`, which looks+ // at what actually came back. The first draft gated on `fields` alone+ // and refused `_source: false` + `docvalue_fields` with a message that+ // named `fields` as the only way, contradicting the comment below.+ let non_empty = |v: &Option<Value>| match v {+ Some(Value::Array(a)) => !a.is_empty(),+ Some(Value::Object(o)) => !o.is_empty(),+ Some(Value::String(s)) => !s.trim().is_empty(),+ _ => false,+ };+ let asks_for_fields = non_empty(&body.fields)+ || non_empty(&body.docvalue_fields)+ || non_empty(&body.stored_fields)+ || non_empty(&body.script_fields);+ if matches!(body.source, Some(Value::Bool(false))) && !asks_for_fields {+ return Err(+ "`rerank` needs document text: `_source: false` with no `fields` (or \+ `docvalue_fields`, `stored_fields`, `script_fields`) clause leaves nothing \+ to judge. Return the text through `fields`, or use `rerank.fields` to \+ limit what is sent instead"+ .into(),+ );+ }+ if body.size == 0 {+ return Err(+ "`rerank` cannot be combined with `size: 0`: the provider would be paid to \+ judge documents the response does not return. Drop `rerank` for an \+ aggregation-only request, or ask for at least one hit"+ .into(),+ );+ }+ // `checked_add`, not `+`: `from` is the caller's, and `u64::MAX + 1`+ // wrapped to 0 in a release build (overflow checks are off there),+ // passed this check, paid for a whole window of judgements and+ // returned an empty page under a 200 — while the same request without+ // `rerank` is the engine's 400. `from` is zeroed below, so the engine's+ // own `from + size` guard never sees the caller's value; this is the+ // only place it can be refused.+ let page_end = body.from.checked_add(body.size);+ if page_end.is_none_or(|end| end > cfg.window) {+ return Err(format!(+ "`from` + `size` ({}) exceeds `rerank.window` ({}): results past the window \+ keep the engine's order, so a page cut across the boundary would mix two \+ rankings. Raise `rerank.window` or page within it",+ page_end.map_or_else(+ || format!("{} + {}", body.from, body.size),+ |end| end.to_string()+ ),+ cfg.window+ ));+ }+ // The judge reads the hit as the response carries it, so a field the+ // caller told us to judge but also told us not to return cannot be+ // judged. Say which one instead of sending blank documents.+ //+ // This is a prediction, and it is only made where it cannot be wrong:+ // `docvalue_fields`, `stored_fields` and `script_fields` also put values+ // on a hit, so when any of them is present the question is left to the+ // check on the rendered hits in `apply`.+ let other_channels = body.docvalue_fields.is_some()+ || body.stored_fields.is_some()+ || body.script_fields.is_some();+ if let (Some(fields), false) = (&cfg.fields, other_channels) {+ for f in fields {+ if !source_keeps(body.source.as_ref(), f)+ && !fields_requests(body.fields.as_ref(), f)+ {+ return Err(format!(+ "`rerank.fields` names `{f}`, but the `_source` filter removes it from \+ the response and `fields` does not ask for it. The judge only sees \+ what the response returns, so nothing would be left to judge: include \+ `{f}` in `_source`, or request it through `fields`"+ ));+ }+ }+ }++ let query = match cfg.query.clone() {+ Some(q) => q,+ None => body.query.as_ref().and_then(infer_query).ok_or_else(|| {+ "`rerank.query` is required: the search query is not a shape a single \+ question can be read from (supported for inference: match, match_phrase, \+ multi_match, semantic, simple_query_string). A `bool`, a `hybrid`, a \+ top-level `knn` with no text query, or no `query` at all needs the question \+ spelled out"+ .to_string()+ })?,+ };++ // `rerank.query` was checked against its ceiling by the parser; a+ // question read out of the search query is the same string on the same+ // wire and gets the same ceiling.+ if cfg.query.is_none() {+ xerj_rerank::check_inferred_question(&query).map_err(|e| e.to_string())?;+ }++ let requested_size = body.size;+ let from = body.from;+ body.size = body.size.max(cfg.window);+ // The stage pages the reranked window itself; the engine must return+ // the window from the top.+ body.from = 0;++ Ok(Some(Self {+ cfg,+ query,+ requested_size,+ from,+ }))+ }++ /// The page the caller asked for, before the stage widened it. The search+ /// handler restores these on `body` for everything downstream that reads+ /// them (response hints, `terminated_early`), once the engine has run.+ pub fn requested_page(&self) -> (usize, usize) {+ (self.from, self.requested_size)+ }++ /// How many of the engine's top hits the stage asked for. The search+ /// handler checks it against each participating index's+ /// `index.max_result_window` before the engine runs, so the caller hears+ /// "`rerank.window` (30) exceeds `index.max_result_window` (20)" rather+ /// than the engine's `from + size > max_result_window` about a page size+ /// they never sent.+ pub fn window(&self) -> usize {+ self.cfg.window+ }++ /// Collect the prose a hit returns under one field name: `_source` first,+ /// then the hit's `fields` — which is how fetched-not-stored values get+ /// judged.+ fn field_text(hit: &EsHit, name: &str, out: &mut Prose) {+ let src = hit.source.as_ref().and_then(Value::as_object);+ if let Some(v) = src+ .and_then(|o| lookup(o, name))+ .or_else(|| hit.fields.as_ref().and_then(|f| f.get(name)))+ {+ if name == xerj_query::executor::PASSAGE_RESPONSE_FIELD {+ // `_passage` is `[{field, ordinal, start_offset, end_offset,+ // text, page?}]`, not a string: the prose is its `text`. The+ // one object shape that is read, because it is the engine's own+ // and its `text` is by construction a slice of a returned field.+ for passage in v.as_array().into_iter().flatten() {+ if let Some(text) = passage.get("text") {+ out.push(text);+ }+ }+ } else {+ out.push(v);+ }+ }+ }++ fn candidate(&self, ordinal: usize, hit: &EsHit) -> Candidate {+ // `rerank.fields` is an egress control, so it is exhaustive: a caller+ // who wrote `["body"]` has said what may leave the machine, and the+ // title is not on that list. Unrestricted, the title is promoted to its+ // own slot because a judge reads it differently from body text.+ let title_allowed = match &self.cfg.fields {+ None => true,+ Some(fields) => fields.iter().any(|f| f == "title"),+ };+ // Title and text are each cut at `max_doc_chars` as they are+ // collected — see [`Prose`].+ let mut title = Prose::new(self.cfg.max_doc_chars);+ if title_allowed {+ Self::field_text(hit, "title", &mut title);+ }++ let mut text = Prose::new(self.cfg.max_doc_chars);+ match &self.cfg.fields {+ Some(fields) => {+ for f in fields.iter().filter(|f| f.as_str() != "title") {+ Self::field_text(hit, f, &mut text);+ }+ }+ None => {+ // The matching passage goes FIRST when the response carries+ // one. Text is cut at `max_doc_chars`, so on a long document an+ // appended passage would be the part that gets cut — and it is+ // the part that says why this hit matched.+ let passage = xerj_query::executor::PASSAGE_RESPONSE_FIELD;+ Self::field_text(hit, passage, &mut text);+ let source = hit.source.as_ref().and_then(Value::as_object);+ if let Some(obj) = source {+ obj.iter()+ .filter(|(k, _)| k.as_str() != "title")+ .for_each(|(_, v)| text.push(v));+ }+ // Then what the hit returns under `fields` — where `fields`,+ // `docvalue_fields`, `stored_fields` and `script_fields` put+ // their values. "The judge sees what the response returns"+ // includes them: the docs and `prepare`'s own refusal both say+ // "return the text through `fields`", and until this loop+ // existed `_source: false` + `fields: ["body"]` + `rerank: {}`+ // followed that advice into a second 400 ("nothing to judge").+ // A name `_source` already returned at the top level is+ // skipped, so a value present in both is sent once. `fields`+ // is a `HashMap`; the names are sorted so the same response+ // always sends the same text (the cut at `max_doc_chars` would+ // otherwise fall on a different field from call to call).+ if let Some(returned) = &hit.fields {+ let mut names: Vec<&String> = returned+ .keys()+ .filter(|k| {+ k.as_str() != "title"+ && k.as_str() != passage+ && !source.is_some_and(|o| o.contains_key(k.as_str()))+ })+ .collect();+ names.sort();+ for name in names {+ text.push(&returned[name]);+ }+ }+ }+ }+ Candidate {+ ordinal,+ title: (!title.is_empty()).then_some(title.buf),+ text: text.buf,+ }+ }++ /// Fields named in `rerank.fields` that no hit in the window returned any+ /// prose for.+ ///+ /// Checked on the RENDERED hits rather than predicted from the request,+ /// because prediction was wrong once already: `prepare` reasons that a+ /// `fields` clause returns a value `_source` filtering removed, and the+ /// engine does not always do that. A named field that silently contributes+ /// nothing means the caller believes documents were judged on text the+ /// judge never saw — so it is reported, whatever the cause (a projection, a+ /// typo in the name, or a field these hits simply lack).+ fn fields_without_text(&self, window: &[EsHit]) -> Vec<String> {+ let Some(fields) = &self.cfg.fields else {+ return Vec::new();+ };+ fields+ .iter()+ .filter(|f| {+ window.iter().all(|h| {+ // One character is enough to know the field had prose.+ let mut probe = Prose::new(1);+ Self::field_text(h, f, &mut probe);+ probe.is_empty()+ })+ })+ .cloned()+ .collect()+ }++ /// Rerank `hits` in place and return the `_rerank` block for the response.+ ///+ /// `Err` means a fault the caller must see (see `xerj_rerank::Policy`);+ /// degradable causes come back as `Ok` with `"applied": false`.+ pub async fn apply(+ &self,+ settings: &ProviderSettings,+ hits: &mut Vec<EsHit>,+ max_score: &mut Option<f64>,+ ) -> Result<Value, RerankError> {+ let started = Instant::now();+ let window = self.cfg.window.min(hits.len());+ let all: Vec<Candidate> = hits[..window]+ .iter()+ .enumerate()+ .map(|(i, h)| self.candidate(i, h))+ .collect();+ // A hit with no prose at all is not sent: a blank document costs a+ // judgement and the verdict on nothing means nothing.+ let sendable: Vec<Candidate> = all+ .iter()+ .filter(|c| c.title.is_some() || !c.text.trim().is_empty())+ .cloned()+ .collect();+ let skipped_no_text = all.len() - sendable.len();+ let fields_without_text = self.fields_without_text(&hits[..window]);++ // Misconfiguration surfaces whether or not this search had hits.+ let provider = Provider::from_settings(&self.cfg, settings)?;++ if window > 0 && sendable.is_empty() {+ return Err(RerankError::Config(format!(+ "nothing to judge: none of the top {window} hits carries a string field in \+ the response{}. The judge only sees what the response returns — name the \+ text field(s) in `rerank.fields` and make sure `_source` (or `fields`) \+ returns them",+ match &self.cfg.fields {+ Some(f) => format!(" under `rerank.fields` {f:?}"),+ None => String::new(),+ }+ )));+ }++ let deadline = Deadline::new(self.cfg.timeout.max(Duration::from_millis(1)));+ let outcome = provider+ .rerank(&self.query, &sendable, &self.cfg, &deadline)+ .await?;++ let mut info = match outcome {+ RerankOutcome::Degraded { reason } => {+ tracing::warn!(%reason, "rerank degraded; engine order kept");+ json!({+ "applied": false,+ "reason": reason,+ "provider": self.cfg.provider,+ "score_kind": "engine",+ })+ }+ RerankOutcome::Reordered {+ scores,+ partial_failures,+ usage,+ } => {+ // A verdict counts only for a document that was actually sent.+ // The provider is a third party: an answer keyed to a hit that+ // was skipped as blank, or to a key nobody sent, would otherwise+ // score a document the judge never saw and inflate `judged` —+ // which is the operator's billing meter. The provider crate+ // already keeps one verdict per document per batch; this is+ // the same rule applied to whatever `Provider` arm answered.+ let sent: std::collections::HashSet<usize> =+ sendable.iter().map(|c| c.ordinal).collect();+ let mut seen = std::collections::HashSet::with_capacity(scores.len());+ let scores: Vec<xerj_rerank::Scored> = scores+ .into_iter()+ .filter(|s| sent.contains(&s.ordinal) && seen.insert(s.ordinal))+ .collect();+ // Documents were sent and not one came back with a verdict:+ // the provider answered, but not about anything it was asked.+ // That is the engine's order, and the caller must be told so —+ // `applied: true` over untouched BM25 scores was the silent+ // fake this stage exists to avoid, and it counted as "applied"+ // on the operator's meter.+ if !sendable.is_empty() && scores.is_empty() {+ let reason = "provider returned no verdict for any document that was sent";+ tracing::warn!(reason, "rerank degraded; engine order kept");+ return self.finish(+ started,+ hits,+ json!({+ "applied": false,+ "reason": reason,+ "provider": self.cfg.provider,+ "score_kind": "engine",+ }),+ );+ }+ let ordered = xerj_rerank::apply_scores(&all, &scores, self.cfg.min_score);+ let judged = scores.len();+ let below_min = match self.cfg.min_score {+ Some(min) => scores.iter().filter(|s| s.score < min).count(),+ None => 0,+ };+ let mut slots: Vec<Option<EsHit>> = hits.drain(..).map(Some).collect();+ let mut out = Vec::with_capacity(slots.len());+ let mut unjudged = 0usize;+ for s in &ordered {+ if let Some(mut h) = slots.get_mut(s.ordinal).and_then(Option::take) {+ // Unjudged hits carry a negative sort key internally.+ if s.score >= 0.0 {+ let p = s.score;+ // `explain` described the engine's score. Keep it,+ // under a node that says where `_score` now comes+ // from — an `_explanation.value` that disagrees+ // with `_score` would be a lie by omission.+ if let Some(engine) = h.explanation.take() {+ h.explanation = Some(json!({+ "value": p,+ "description": format!(+ "rerank: relevance probability from provider `{}` \+ (model `{}`); replaces the engine score explained below",+ self.cfg.provider, self.cfg.model+ ),+ "details": [engine],+ }));+ }+ h.score = Some(p);+ } else {+ // No verdict. `_score` is `null`, never the engine's+ // BM25 value beside probabilities (see the module+ // doc). The explanation, if any, still describes+ // the engine score, and says so.+ unjudged += 1;+ if let Some(engine) = h.explanation.take() {+ let engine_value = engine.get("value").cloned();+ h.explanation = Some(json!({+ "value": engine_value,+ "description": format!(+ "rerank: no verdict from provider `{}` for this hit; \+ `_score` is null and the engine score is explained below",+ self.cfg.provider+ ),+ "details": [engine],+ }));+ }+ h.score = None;+ }+ out.push(h);+ }+ }+ // Past the window nothing was judged. With a threshold those+ // hits cannot be shown to clear it, so they are dropped; with+ // none they follow in the engine's order, also with no score.+ if self.cfg.min_score.is_none() {+ for mut h in slots.into_iter().skip(window).flatten() {+ h.score = None;+ unjudged += 1;+ out.push(h);+ }+ }+ let dropped_unjudged = match self.cfg.min_score {+ Some(_) => window.saturating_sub(judged),+ None => 0,+ };+ *hits = out;+ // Judged hits sort first and their probabilities descend, so+ // the first hit's score is the maximum of every `_score` on the+ // page — the invariant `max_score` is for.+ *max_score = hits.first().and_then(|h| h.score);+ json!({+ "applied": true,+ "provider": self.cfg.provider,+ "model": self.cfg.model,+ "score_kind": "probability",+ "window": window,+ "judged": judged,+ "unjudged": unjudged,+ "skipped_no_text": skipped_no_text,+ "pruned_below_min_score": below_min,+ "dropped_unjudged": dropped_unjudged,+ "partial_failures": partial_failures,+ "usage": {+ "input_tokens": usage.input_tokens,+ "output_tokens": usage.output_tokens,+ },+ })+ }+ };+ if let Some(obj) = info.as_object_mut() {+ // Present only when there is something to say: a field the caller+ // asked to be judged on that contributed no text at all.+ if !fields_without_text.is_empty() && obj.get("applied") == Some(&json!(true)) {+ obj.insert("fields_without_text".into(), json!(fields_without_text));+ }+ }+ self.finish(started, hits, info)+ }++ /// Stamp the fields every `_rerank` block carries and cut the widened+ /// window back to the page the caller asked for.+ fn finish(+ &self,+ started: Instant,+ hits: &mut Vec<EsHit>,+ mut info: Value,+ ) -> Result<Value, RerankError> {+ if let Some(obj) = info.as_object_mut() {+ // The question that was judged — inferred or given. A caller who+ // let it be inferred should be able to see what was inferred.+ obj.insert("query".into(), json!(self.query));+ obj.insert(+ "took_ms".into(),+ json!(started.elapsed().as_millis() as u64),+ );+ }+ // The page was widened to the window; cut it back to what was asked.+ let page: Vec<EsHit> = std::mem::take(hits)+ .into_iter()+ .skip(self.from)+ .take(self.requested_size)+ .collect();+ *hits = page;+ Ok(info)+ }+}++/// Whether a raw search body carries a `rerank` block that means something.+///+/// `"rerank": null` is treated as absent everywhere: `EsSearchBody.rerank` is+/// an `Option<Value>` and serde reads `null` as `None`, so `_search` already+/// ignored it, while the surfaces that inspect the raw JSON (`_msearch`, the+/// templates, `_async_search`) saw a present key and refused. One rule.+pub fn carries_rerank(body: &Value) -> bool {+ body.get("rerank").is_some_and(|r| !r.is_null())+}++/// Count a request the stage refused before the search ran (`prepare` said no).+pub fn record_refused(metrics: &xerj_common::metrics::Metrics) {+ metrics+ .rerank_requests+ .with_label_values(&["refused"])+ .inc();+}++/// Count what [`RerankPlan::apply`] did.+///+/// `refused` means nothing reached the provider (400 / 403 / 503); `failed`+/// means it was called and the contract broke (502). The split matters to the+/// operator reading the counter: only `applied`, `degraded` and `failed` can+/// have cost money.+pub fn record_outcome(+ metrics: &xerj_common::metrics::Metrics,+ result: &Result<Value, RerankError>,+) {+ let outcome = match result {+ Ok(info) if info["applied"] == json!(true) => {+ metrics+ .rerank_documents_judged+ .inc_by(info["judged"].as_u64().unwrap_or(0));+ for (kind, key) in [("input", "input_tokens"), ("output", "output_tokens")] {+ metrics+ .rerank_provider_tokens+ .with_label_values(&[kind])+ .inc_by(info["usage"][key].as_u64().unwrap_or(0));+ }+ "applied"+ }+ Ok(_) => "degraded",+ Err(e) if status_for(e) == 502 => "failed",+ Err(_) => "refused",+ };+ metrics.rerank_requests.with_label_values(&[outcome]).inc();+}++/// HTTP status for a rerank fault that must reach the caller.+pub fn status_for(e: &RerankError) -> u16 {+ match e {+ RerankError::Config(_) | RerankError::UnknownProvider(_) => 400,+ // The operator has forbidden it. Not the caller's syntax (400) and not+ // something a retry or a key fixes (503).+ RerankError::DisabledByOperator => 403,+ // The server is not configured to reach the provider: not the+ // caller's mistake, and not a gateway fault either.+ RerankError::MissingKey(_) => 503,+ RerankError::Transport(_)+ | RerankError::Status { .. }+ | RerankError::Malformed(_)+ | RerankError::Deadline { .. } => 502,+ }+}++/// ES-shaped error `type` for a rerank fault: a request the caller can fix is+/// an `illegal_argument_exception` like every other 400; the rest are XERJ's.+pub fn error_type_for(e: &RerankError) -> &'static str {+ match e {+ RerankError::Config(_) | RerankError::UnknownProvider(_) => "illegal_argument_exception",+ _ => "rerank_exception",+ }+}++/// Why an endpoint that does not run the rerank stage refuses the block. One+/// sentence, shared by the ES-shaped 400 below, the native REST API and gRPC,+/// so the three surfaces cannot tell a caller three different things.+pub fn unsupported_reason(endpoint: &str) -> String {+ format!(+ "`rerank` is not supported on {endpoint}: only `POST /{{index}}/_search` runs the rerank \+ stage. Send this search there, or remove the `rerank` block — it is refused here \+ rather than silently ignored"+ )+}++/// The 400 for an endpoint that does not run the rerank stage.+///+/// `xerj_query::parse_request` ignores keys it does not know, so without this a+/// `rerank` block on `_msearch`, a search template, `_async_search` or a scroll+/// vanished without a word and the caller got the engine's order back under a+/// 200 — the accepted-and-ignored class this project treats as a defect.+pub fn unsupported_on(endpoint: &str) -> Value {+ let reason = unsupported_reason(endpoint);+ json!({+ "error": {+ "root_cause": [{ "type": "illegal_argument_exception", "reason": reason }],+ "type": "illegal_argument_exception",+ "reason": reason,+ },+ "status": 400,+ })+}++/// `GET /_xerj/rerank` — whether a rerank provider is configured.+///+/// Reports THAT a key is set and where it came from; never the key, and the+/// endpoint only with userinfo, query and fragment stripped. Superuser-only,+/// like the rest of the `/_xerj/*` operator namespace (`authz.rs`).+pub async fn rerank_status(State(state): State<AppState>) -> Json<Value> {+ Json(status_document(&state.rerank))+}++/// The body of [`rerank_status`], split out so it is testable without a router.+pub fn status_document(settings: &ProviderSettings) -> Value {+ let defaults = RerankConfig::default();+ json!({+ "enabled": settings.enabled,+ "configured": settings.enabled && settings.has_key(),+ "providers": ["jev"],+ "api_key": {+ "set": settings.has_key(),+ "source": settings.key_source().map(|s| s.as_str()),+ },+ "endpoint": {+ "url": settings.endpoint_for_display(),+ "source": settings.endpoint_source().as_str(),+ },+ "defaults": {+ "provider": defaults.provider,+ "model": defaults.model,+ "window": defaults.window,+ "batch": defaults.batch,+ "max_concurrency": defaults.max_concurrency,+ "max_doc_chars": defaults.max_doc_chars,+ "timeout_ms": defaults.timeout.as_millis() as u64,+ },+ "limits": {+ "max_docs_per_call": xerj_rerank::JEV_MAX_DOCS_PER_CALL,+ "max_window": xerj_rerank::MAX_WINDOW,+ "max_concurrency": xerj_rerank::MAX_CONCURRENCY,+ "max_doc_chars": xerj_rerank::MAX_DOC_CHARS,+ "max_timeout_ms": xerj_rerank::MAX_TIMEOUT_MS,+ "max_instructions_chars": xerj_rerank::MAX_INSTRUCTIONS_CHARS,+ "max_query_chars": xerj_rerank::MAX_QUERY_CHARS,+ "max_model_chars": xerj_rerank::MAX_MODEL_CHARS,+ "max_fields": xerj_rerank::MAX_FIELDS,+ "max_field_name_chars": xerj_rerank::MAX_FIELD_NAME_CHARS,+ },+ "data_egress": xerj_rerank::DATA_EGRESS,+ })+}++#[cfg(test)]+mod tests {+ use super::*;++ #[test]+ fn infers_a_question_only_from_simple_shapes() {+ assert_eq!(+ infer_query(&json!({"match": {"body": "vitamin d"}})).as_deref(),+ Some("vitamin d")+ );+ assert_eq!(+ infer_query(&json!({"match": {"body": {"query": "vitamin d"}}})).as_deref(),+ Some("vitamin d")+ );+ assert_eq!(+ infer_query(&json!({"multi_match": {"query": "q", "fields": ["a"]}})).as_deref(),+ Some("q")+ );+ assert_eq!(+ infer_query(&json!({"semantic": {"field": "body", "query": "q"}})).as_deref(),+ Some("q")+ );+ // No single question in these — must not guess.+ assert!(infer_query(&json!({"bool": {"must": [{"match": {"a": "x"}}]}})).is_none());+ assert!(infer_query(&json!({"match_all": {}})).is_none());+ assert!(infer_query(&json!({"term": {"a": "x"}})).is_none());+ }++ #[test]+ fn fault_statuses() {+ assert_eq!(status_for(&RerankError::Config("x".into())), 400);+ assert_eq!(status_for(&RerankError::UnknownProvider("x".into())), 400);+ assert_eq!(status_for(&RerankError::MissingKey("K")), 503);+ assert_eq!(status_for(&RerankError::DisabledByOperator), 403);+ assert_eq!(+ status_for(&RerankError::Status {+ status: 401,+ body: String::new()+ }),+ 502+ );+ assert_eq!(+ error_type_for(&RerankError::Config("x".into())),+ "illegal_argument_exception"+ );+ assert_eq!(+ error_type_for(&RerankError::MissingKey("K")),+ "rerank_exception"+ );+ }++ #[test]+ fn source_filter_knows_what_survives() {+ let inc = json!(["title", "meta.*"]);+ assert!(source_keeps(Some(&inc), "title"));+ assert!(source_keeps(Some(&inc), "meta.summary"));+ assert!(!source_keeps(Some(&inc), "body"));++ let exc = json!({"excludes": ["body", "secret.*"]});+ assert!(!source_keeps(Some(&exc), "body"));+ assert!(!source_keeps(Some(&exc), "secret.note"));+ assert!(source_keeps(Some(&exc), "title"));++ // Excluding the parent object removes the child.+ let exc_parent = json!({"excludes": ["meta"]});+ assert!(!source_keeps(Some(&exc_parent), "meta.summary"));+ // Including a child keeps (part of) the parent.+ let inc_child = json!({"includes": ["meta.summary"]});+ assert!(source_keeps(Some(&inc_child), "meta"));++ assert!(source_keeps(None, "body"));+ assert!(source_keeps(Some(&json!(true)), "body"));+ assert!(!source_keeps(Some(&json!(false)), "body"));+ assert!(source_keeps(Some(&json!("bo*")), "body"));+ }++ #[test]+ fn glob_matches_like_source_filtering_does() {+ assert!(glob("body", "body"));+ assert!(!glob("body", "body2"));+ assert!(glob("*", "anything"));+ assert!(glob("meta.*", "meta.summary"));+ assert!(!glob("meta.*", "metadata"));+ assert!(glob("*_text", "body_text"));+ assert!(glob("a*b*c", "a-x-b-y-c"));+ assert!(!glob("a*b*c", "a-x-c"));+ assert!(glob("été*", "été-2026"), "multi-byte names must not panic");+ }++ #[test]+ fn text_is_strings_and_string_arrays_only() {+ let mut out = Prose::new(1_000);+ out.push(&json!("alpha"));+ out.push(&json!(["beta", 7, "gamma"]));+ out.push(&json!(42));+ out.push(&json!({"nested": "no"}));+ out.push(&json!([0.1, 0.2]));+ out.push(&json!(" "));+ assert_eq!(out.buf, "alpha\nbeta\ngamma");+ }++ /// Cutting while collecting must send exactly what collecting everything+ /// and cutting afterwards sent — for every budget, across piece+ /// boundaries, on the separator itself, and inside multi-byte text — while+ /// never holding more than the budget.+ #[test]+ fn prose_cut_while_collected_equals_collected_then_cut() {+ let pieces = [+ "héllo wörld",+ " ",+ "второй кусок",+ "",+ "日本語のテキスト",+ "tail",+ ];+ let whole = pieces+ .iter()+ .filter(|p| !p.trim().is_empty())+ .copied()+ .collect::<Vec<_>>()+ .join("\n");+ for budget in 0..=whole.chars().count() + 2 {+ let mut prose = Prose::new(budget);+ for p in pieces {+ prose.push_str(p);+ }+ assert_eq!(+ prose.buf,+ xerj_rerank::clip(&whole, budget),+ "budget {budget}"+ );+ assert!(prose.buf.chars().count() <= budget, "budget {budget}");+ }+ }++ fn hit(source: Value, fields: Option<Value>) -> EsHit {+ let mut value = json!({+ "_index": "i", "_id": "1", "_score": 1.0, "_source": source,+ "matched_queries": null+ });+ if let Some(f) = fields {+ value["fields"] = f;+ }+ serde_json::from_value(value).expect("an EsHit")+ }++ fn plan(rerank: Value) -> RerankPlan {+ RerankPlan {+ cfg: RerankConfig::from_json(&rerank).expect("a valid block"),+ query: "q".into(),+ requested_size: 10,+ from: 0,+ }+ }++ /// A megabyte-sized field costs the stage `max_doc_chars`, not a megabyte:+ /// the candidate is what gets cloned into batches and tasks.+ #[test]+ fn a_candidate_never_holds_more_than_max_doc_chars_of_each_part() {+ let big = "x".repeat(1_000_000);+ let h = hit(json!({"title": big, "body": big, "more": [big, big]}), None);+ let c = plan(json!({"max_doc_chars": 50})).candidate(0, &h);+ assert_eq!(c.title.as_deref().map(|t| t.chars().count()), Some(50));+ assert_eq!(c.text.chars().count(), 50);+ let c = plan(json!({"max_doc_chars": 50, "fields": ["more", "body"]})).candidate(0, &h);+ assert_eq!(c.title, None, "`rerank.fields` is exhaustive");+ assert_eq!(c.text.chars().count(), 50);+ }++ /// Default mode reads `_source`, then the hit's `fields` for names+ /// `_source` did not already return — once, and in a stable order.+ #[test]+ fn default_mode_reads_returned_fields_once_and_in_name_order() {+ let h = hit(+ json!({"title": "T", "body": "from source"}),+ Some(json!({+ "body": ["from source"],+ "zeta": ["last"],+ "alpha": ["first"],+ "n": [7],+ "title": ["T"]+ })),+ );+ let c = plan(json!({})).candidate(0, &h);+ assert_eq!(c.title.as_deref(), Some("T"));+ assert_eq!(c.text, "from source\nfirst\nlast");++ // `_source: false`: everything comes from `fields`, title included.+ let mut h = hit(+ json!({}),+ Some(json!({"body": ["only here"], "title": ["T"]})),+ );+ h.source = None;+ let c = plan(json!({})).candidate(0, &h);+ assert_eq!(c.title.as_deref(), Some("T"));+ assert_eq!(c.text, "only here");+ }++ #[test]+ fn dotted_fields_resolve_as_a_literal_key_then_as_a_path() {+ let doc = json!({"a.b": "literal", "a": {"b": "path", "c": {"d": "deep"}}});+ let obj = doc.as_object().unwrap();+ assert_eq!(lookup(obj, "a.b"), Some(&json!("literal")));+ assert_eq!(lookup(obj, "a.c.d"), Some(&json!("deep")));+ assert_eq!(lookup(obj, "a.x"), None);+ }++ /// `xerj-common` validates the environment endpoint at boot and cannot+ /// link `xerj-rerank`, so it carries its own copy of the variable name.+ #[test]+ fn the_env_endpoint_variable_name_is_the_same_in_both_crates() {+ assert_eq!(+ xerj_common::config::RerankProviderConfig::ENV_ENDPOINT,+ xerj_rerank::JevProvider::ENV_ENDPOINT+ );+ }++ #[test]+ fn status_document_never_carries_the_key() {+ let s = ProviderSettings::with_key_and_endpoint(+ "sk-do-not-print",+ "https://u:[email protected]/v1/systemone?k=v",+ );+ let doc = status_document(&s);+ let text = doc.to_string();+ assert!(!text.contains("sk-do-not-print"), "{text}");+ assert!(!text.contains("u:p@"), "{text}");+ assert!(!text.contains("k=v"), "{text}");+ assert_eq!(doc["configured"], true);+ assert_eq!(doc["api_key"]["set"], true);+ assert_eq!(doc["api_key"]["source"], "config");+ assert_eq!(doc["endpoint"]["url"], "https://judge.example/v1/systemone");++ let unset = status_document(&ProviderSettings::resolve(true, "", "", None, None));+ assert_eq!(unset["configured"], false);+ assert_eq!(unset["api_key"]["set"], false);+ assert!(unset["api_key"]["source"].is_null());+ assert_eq!(unset["endpoint"]["source"], "default");++ let off = status_document(&ProviderSettings::resolve(false, "k", "", None, None));+ assert_eq!(off["enabled"], false);+ assert_eq!(+ off["configured"], false,+ "a key on a node that forbids reranking is not a working configuration"+ );+ }+}diff --git a/engine/crates/xerj-api/src/router.rs b/engine/crates/xerj-api/src/router.rsindex a04db22ab..f7397741d 100644--- a/engine/crates/xerj-api/src/router.rs+++ b/engine/crates/xerj-api/src/router.rs@@ -855,6 +855,10 @@ pub fn build_es_compat_router(state: AppState) -> Router {.delete(wal_tap_api::delete_wal_tap),).route("/_xerj/wal_tap/_stats", get(wal_tap_api::wal_tap_stats))+ // ── Rerank provider status ─────────────────────────────────────────────────+ // Whether a rerank provider is configured — never the key. Superuser+ // only: it inherits the `/_xerj/*` rule in `authz.rs`.+ .route("/_xerj/rerank", get(crate::rerank_stage::rerank_status))// ── Legacy index templates (v1) ────────────────────────────────────────────.route("/_template/:name",diff --git a/engine/crates/xerj-api/src/state.rs b/engine/crates/xerj-api/src/state.rsindex f7e31f822..174b89215 100644--- a/engine/crates/xerj-api/src/state.rs+++ b/engine/crates/xerj-api/src/state.rs@@ -533,6 +533,14 @@ pub struct AppState {/// Live background scoring tasks, keyed by datafeed id. Aborted on `_stop`/// (and replaced on a re-`_start`). In-memory only.pub ml_datafeed_tasks: Arc<DashMap<String, tokio::task::JoinHandle<()>>>,+ /// Resolved rerank-provider settings (key, endpoint, kill switch, shared+ /// HTTP client) — the injection seam for [`crate::rerank_stage`].+ ///+ /// Resolved once here from `[rerank]` in the config file with the+ /// `TYPESAFE_*` environment variables as the fallback, and never read from+ /// the environment again. A test points a node at its own stub by replacing+ /// this field, so parallel tests never race on process-wide env state.+ pub rerank: Arc<xerj_rerank::ProviderSettings>,}impl AppState {@@ -565,7 +573,13 @@ impl AppState {} else {(Arc::new(DashMap::new()), Arc::new(DashMap::new()))};+ let rerank = Arc::new(xerj_rerank::ProviderSettings::from_config_and_env(+ config.rerank.enabled,+ &config.rerank.api_key,+ &config.rerank.endpoint,+ ));Self {+ rerank,config: Arc::new(config),engine: Arc::new(engine),metrics: Arc::new(metrics),diff --git a/engine/crates/xerj-api/tests/rerank_stage_http.rs b/engine/crates/xerj-api/tests/rerank_stage_http.rsnew file mode 100644index 000000000..3862333c7--- /dev/null+++ b/engine/crates/xerj-api/tests/rerank_stage_http.rs@@ -0,0 +1,3026 @@+//! End-to-end checks for the `rerank` stage of `_search`, over HTTP, against an+//! in-process stub that speaks TypeSafe's documented System One wire format:+//!+//! ```text+//! POST /v1/systemone+//! request { state, model, questions: { id: { type:"noul", instructions, criteria } } }+//! response { model, answers: { id: { type:"noul", noul: 0..1 } }, usage }+//! ```+//!+//! THE STUB IS A TEST DOUBLE, NOT A JUDGE. It scores by a table the test hands+//! it (or by word overlap), so ordering is checkable. Nothing here says+//! anything about ranking quality with the real model — that has not been+//! verified by this project; no provider key was available.+//!+//! # Why there is no `set_var` in this file+//!+//! The provider used to read `TYPESAFE_API_KEY` / `TYPESAFE_ENDPOINT` from the+//! process environment on every request. Cargo runs the tests of one binary as+//! parallel threads of one process, so a test that sets those variables races+//! every other test that reads them. Each test here builds its own stub on an+//! ephemeral port and its own node, and points the node at the stub through+//! `AppState::rerank` — the injection seam — so nothing process-wide is touched.++use std::collections::HashMap;+use std::sync::{Arc, Mutex};+use std::time::{Duration, Instant};++use axum::body::Body;+use axum::http::{HeaderMap, Request, StatusCode};+use axum::response::IntoResponse;+use axum::routing::post;+use serde_json::{json, Value};+use tower::ServiceExt;+use xerj_rerank::ProviderSettings;++// ─────────────────────────────────────────────────────────────────────────────+// The stub provider+// ─────────────────────────────────────────────────────────────────────────────++#[derive(Clone)]+enum Verdicts {+ /// Fraction of the query's words found in the document (what the scratch+ /// `stub.py` did).+ Overlap,+ /// Probability by document title; titles not listed score 0.0.+ ByTitle(HashMap<String, f64>),+}++#[derive(Clone)]+enum Fault {+ None,+ /// Answer every call with this status and an error body.+ Status(u16),+ /// Answer the first `n` calls with `status`, then behave.+ StatusThenOk {+ n: usize,+ status: u16,+ },+ /// 200 with a body that is not JSON.+ Garbage,+ /// Sleep this long before answering correctly.+ Delay(Duration),+ /// 401 with a body that echoes the bearer token back, as some APIs do.+ EchoKey,+ /// Answer correctly, and ALSO return a 0.99 verdict for `d0`..`d9` whether+ /// or not those documents were sent — a provider that misbehaves.+ AnswerUnsent,+ /// Answer every other document in each call and stay silent on the rest.+ Partial,+ /// Answer ONLY for keys nobody sent (`d999`, `x`): a provider that parses+ /// but says nothing about the documents it was asked about.+ WrongKeys,+ /// Answer correctly, and ALSO echo a 0.5 verdict for `d0`..`d59` on every+ /// call — a provider that leaks other batches' keys back.+ EchoOtherBatches,+ /// Answer every document correctly, under a `usage` block whose numbers+ /// are not the unsigned integers the documentation promises.+ OddUsage,+ /// 200 with a valid JSON body of several megabytes: a provider (or+ /// something answering in its place) that does not know when to stop.+ HugeBody,+}++#[derive(Clone)]+struct Captured {+ authorization: Option<String>,+ body: Value,+}++#[derive(Clone)]+struct Stub {+ verdicts: Arc<Mutex<Verdicts>>,+ fault: Arc<Mutex<Fault>>,+ seen: Arc<Mutex<Vec<Captured>>>,+ endpoint: String,+}++impl Stub {+ async fn start() -> Self {+ let listener = tokio::net::TcpListener::bind("127.0.0.1:0")+ .await+ .expect("bind an ephemeral port");+ let addr = listener.local_addr().expect("local addr");+ let stub = Stub {+ verdicts: Arc::new(Mutex::new(Verdicts::Overlap)),+ fault: Arc::new(Mutex::new(Fault::None)),+ seen: Arc::new(Mutex::new(Vec::new())),+ endpoint: format!("http://{addr}/v1/systemone"),+ };+ let app = axum::Router::new()+ .route("/v1/systemone", post(systemone))+ .with_state(stub.clone());+ tokio::spawn(async move {+ let _ = axum::serve(listener, app).await;+ });+ stub+ }++ fn by_title(&self, table: &[(&str, f64)]) {+ *self.verdicts.lock().unwrap() =+ Verdicts::ByTitle(table.iter().map(|(t, p)| (t.to_string(), *p)).collect());+ }+ fn fault(&self, f: Fault) {+ *self.fault.lock().unwrap() = f;+ }+ fn calls(&self) -> Vec<Captured> {+ self.seen.lock().unwrap().clone()+ }+ /// Everything that left the node, as one string — for "this text was never+ /// sent" assertions.+ fn everything_sent(&self) -> String {+ self.calls()+ .iter()+ .map(|c| c.body.to_string())+ .collect::<Vec<_>>()+ .join("\n")+ }+}++fn words(s: &str) -> std::collections::HashSet<String> {+ s.to_lowercase()+ .split(|c: char| !c.is_alphanumeric())+ .filter(|w| !w.is_empty())+ .map(str::to_string)+ .collect()+}++async fn systemone(+ axum::extract::State(stub): axum::extract::State<Stub>,+ headers: HeaderMap,+ body: String,+) -> axum::response::Response {+ let parsed: Value = serde_json::from_str(&body).unwrap_or(Value::Null);+ let nth = {+ let mut seen = stub.seen.lock().unwrap();+ seen.push(Captured {+ authorization: headers+ .get("authorization")+ .and_then(|v| v.to_str().ok())+ .map(str::to_string),+ body: parsed.clone(),+ });+ seen.len()+ };+ let fault = stub.fault.lock().unwrap().clone();+ match fault {+ Fault::Status(code) => {+ return (+ StatusCode::from_u16(code).unwrap(),+ axum::Json(json!({"error": "stub fault"})),+ )+ .into_response();+ }+ Fault::StatusThenOk { n, status } if nth <= n => {+ return (+ StatusCode::from_u16(status).unwrap(),+ axum::Json(json!({"error": "stub fault"})),+ )+ .into_response();+ }+ Fault::EchoKey => {+ let auth = headers+ .get("authorization")+ .and_then(|v| v.to_str().ok())+ .unwrap_or_default();+ return (+ StatusCode::UNAUTHORIZED,+ axum::Json(json!({"error": format!("invalid api key: {auth}")})),+ )+ .into_response();+ }+ Fault::Garbage => return (StatusCode::OK, "not json").into_response(),+ Fault::HugeBody => {+ let padding = "x".repeat(3 * 1024 * 1024);+ return axum::Json(json!({"model": "m", "answers": {}, "padding": padding}))+ .into_response();+ }+ Fault::Delay(d) => tokio::time::sleep(d).await,+ _ => {}+ }++ let query = words(parsed["state"]["query"].as_str().unwrap_or_default());+ let verdicts = stub.verdicts.lock().unwrap().clone();+ let mut answers = serde_json::Map::new();+ if matches!(fault, Fault::WrongKeys) {+ answers.insert("d999".into(), json!({"type": "noul", "noul": 0.9}));+ answers.insert("x".into(), json!({"type": "noul", "noul": 0.9}));+ return axum::Json(json!({+ "model": parsed["model"],+ "answers": answers,+ "usage": {"input_tokens": 100, "output_tokens": 10},+ }))+ .into_response();+ }+ if let Some(docs) = parsed["state"]["documents"].as_object() {+ for (i, (key, doc)) in docs.iter().enumerate() {+ if matches!(fault, Fault::Partial) && i % 2 == 1 {+ continue;+ }+ let title = doc["title"].as_str().unwrap_or_default();+ let text = doc["text"].as_str().unwrap_or_default();+ let p = match &verdicts {+ Verdicts::Overlap => {+ let have = words(&format!("{title} {text}"));+ query.intersection(&have).count() as f64 / query.len().max(1) as f64+ }+ Verdicts::ByTitle(table) => table.get(title).copied().unwrap_or(0.0),+ };+ answers.insert(key.clone(), json!({"type": "noul", "noul": p}));+ }+ }+ if matches!(fault, Fault::AnswerUnsent) {+ for i in 0..10 {+ answers+ .entry(format!("d{i}"))+ .or_insert_with(|| json!({"type": "noul", "noul": 0.99}));+ }+ }+ if matches!(fault, Fault::EchoOtherBatches) {+ for i in 0..60 {+ answers+ .entry(format!("d{i}"))+ .or_insert_with(|| json!({"type": "noul", "noul": 0.5}));+ }+ }+ let usage = if matches!(fault, Fault::OddUsage) {+ json!({"input_tokens": 9_223_372_036_854_775_808u64, "output_tokens": -5})+ } else {+ json!({"input_tokens": 100, "output_tokens": 10})+ };+ axum::Json(json!({+ "model": parsed["model"],+ "answers": answers,+ "usage": usage,+ }))+ .into_response()+}++// ─────────────────────────────────────────────────────────────────────────────+// The node+// ─────────────────────────────────────────────────────────────────────────────++struct Node {+ app: axum::Router,+ /// The XERJ-native REST API (`/v1/...`), over the SAME state as `app`.+ native: axum::Router,+ _dir: tempfile::TempDir,+}++async fn node_with(settings: ProviderSettings) -> Node {+ let dir = tempfile::tempdir().expect("tempdir");+ let mut config = xerj_common::config::Config::default();+ config.server.data_dir = dir.path().to_string_lossy().into_owned();+ config.storage.wal_sync = xerj_common::config::WalSync::Async;+ let metrics = xerj_common::metrics::Metrics::new().expect("metrics");+ let engine = xerj_engine::Engine::new(config.clone()).expect("engine");+ let mut state = xerj_api::state::AppState::new(config, engine, metrics);+ // The seam. Whatever TYPESAFE_* the developer's shell happens to hold was+ // read into `state.rerank` by `AppState::new`; it is replaced wholesale+ // here, so no test can reach the real provider or depend on the ambient+ // environment.+ state.rerank = Arc::new(settings);+ Node {+ native: xerj_api::router::build_native_router(state.clone()),+ app: xerj_api::router::build_es_compat_router(state),+ _dir: dir,+ }+}++async fn node(stub: &Stub) -> Node {+ node_with(ProviderSettings::with_key_and_endpoint(+ "test-key",+ &stub.endpoint,+ ))+ .await+}++impl Node {+ async fn raw(&self, method: &str, path: &str, body: String) -> (StatusCode, String) {+ let response = self+ .app+ .clone()+ .oneshot(+ Request::builder()+ .method(method)+ .uri(path)+ .header("content-type", "application/json")+ .body(Body::from(body))+ .expect("request"),+ )+ .await+ .expect("response");+ let status = response.status();+ let bytes = axum::body::to_bytes(response.into_body(), usize::MAX)+ .await+ .expect("body");+ (status, String::from_utf8_lossy(&bytes).into_owned())+ }++ async fn call(&self, method: &str, path: &str, body: Value) -> (StatusCode, Value) {+ let (status, text) = self.raw(method, path, body.to_string()).await;+ (status, serde_json::from_str(&text).unwrap_or(Value::Null))+ }++ async fn search(&self, path: &str, body: Value) -> (StatusCode, Value) {+ self.call("POST", path, body).await+ }++ /// The four documents the scratch script used.+ async fn seed_kb(&self) {+ self.seed(+ "kb",+ &[+ (+ "1",+ "Bone health basics",+ "vitamin supplements are popular. vitamin vitamin vitamin vitamin.",+ "health",+ 1,+ ),+ (+ "2",+ "Trial results",+ "vitamin d supplementation improved bone density in the treatment group",+ "trial",+ 2,+ ),+ (+ "3",+ "Cooking",+ "vitamin rich vegetables for dinner",+ "food",+ 3,+ ),+ (+ "4",+ "Density of materials",+ "bone china has high density",+ "materials",+ 4,+ ),+ ],+ )+ .await;+ }++ async fn seed(&self, index: &str, docs: &[(&str, &str, &str, &str, i64)]) {+ let (st, b) = self+ .call(+ "PUT",+ &format!("/{index}"),+ json!({"mappings": {"properties": {+ "title": {"type": "text"},+ "body": {"type": "text"},+ "cat": {"type": "keyword"},+ "n": {"type": "integer"}+ }}}),+ )+ .await;+ assert!(st.is_success(), "create {index}: {st} {b}");+ for (id, title, body, cat, n) in docs {+ let (st, b) = self+ .call(+ "PUT",+ &format!("/{index}/_doc/{id}"),+ json!({"title": title, "body": body, "cat": cat, "n": n}),+ )+ .await;+ assert!(st.is_success(), "index {index}/{id}: {st} {b}");+ }+ let (st, b) = self+ .call("POST", &format!("/{index}/_refresh"), json!({}))+ .await;+ assert!(st.is_success(), "refresh {index}: {st} {b}");+ }+}++const Q: &str = "vitamin d supplementation bone density";++fn match_q() -> Value {+ json!({"match": {"body": Q}})+}++fn ids(r: &Value) -> Vec<String> {+ r["hits"]["hits"]+ .as_array()+ .map(|a| {+ a.iter()+ .map(|h| h["_id"].as_str().unwrap_or_default().to_string())+ .collect()+ })+ .unwrap_or_default()+}++fn reason(r: &Value) -> String {+ r["error"]["reason"]+ .as_str()+ .unwrap_or_default()+ .to_string()+}++/// A judge that inverts whatever the engine said about the four `kb` docs.+fn inverted(stub: &Stub) {+ stub.by_title(&[+ ("Cooking", 0.9),+ ("Bone health basics", 0.7),+ ("Density of materials", 0.3),+ ("Trial results", 0.1),+ ]);+}+const INVERTED_IDS: [&str; 4] = ["3", "1", "4", "2"];++// ─────────────────────────────────────────────────────────────────────────────+// 1. The seventeen checks the scratch script made, plus the wire check+// ─────────────────────────────────────────────────────────────────────────────++#[tokio::test]+async fn no_rerank_block_leaves_the_response_untouched_and_calls_nobody() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;++ let (st, r) = node+ .search("/kb/_search", json!({"query": match_q(), "size": 4}))+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert!(r.get("_rerank").is_none(), "{r}");+ assert_eq!(ids(&r).len(), 4);+ assert!(+ stub.calls().is_empty(),+ "a search without `rerank` must not contact the provider"+ );+}++#[tokio::test]+async fn applied_order_follows_the_judge_not_the_engine() {+ let stub = Stub::start().await;+ inverted(&stub);+ let node = node(&stub).await;+ node.seed_kb().await;++ let (_, base) = node+ .search("/kb/_search", json!({"query": match_q(), "size": 4}))+ .await;+ assert_ne!(+ ids(&base),+ INVERTED_IDS,+ "the fixture is only a test if the judge disagrees with the engine"+ );++ let (st, text) = node+ .raw(+ "POST",+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "rerank": {"provider": "jev"}}).to_string(),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{text}");+ let r: Value = serde_json::from_str(&text).unwrap();+ assert_eq!(ids(&r), INVERTED_IDS, "{r}");++ let rr = &r["_rerank"];+ assert_eq!(rr["applied"], true, "{rr}");+ assert_eq!(rr["provider"], "jev");+ assert_eq!(rr["model"], "jev-latest");+ assert_eq!(rr["score_kind"], "probability");+ assert_eq!(rr["window"], 4);+ assert_eq!(rr["judged"], 4);+ assert_eq!(rr["skipped_no_text"], 0);+ assert_eq!(rr["partial_failures"], 0);+ assert_eq!(rr["query"], Q, "the inferred question is reported back");+ assert_eq!(rr["usage"]["input_tokens"], 100);+ assert_eq!(rr["usage"]["output_tokens"], 10);+ assert!(rr["took_ms"].is_u64(), "{rr}");++ // `_rerank` precedes `hits` on the wire: a reader that truncates long+ // output from the bottom must still see which ranking it is holding.+ let at = |needle: &str| text.find(needle).unwrap_or(usize::MAX);+ assert!(at("\"_rerank\"") < at("\"hits\""), "{text}");+}++#[tokio::test]+async fn scores_are_probabilities_and_max_score_tracks_the_top_hit() {+ let stub = Stub::start().await;+ inverted(&stub);+ let node = node(&stub).await;+ node.seed_kb().await;++ let (_, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "rerank": {}}),+ )+ .await;+ let scores: Vec<f64> = r["hits"]["hits"]+ .as_array()+ .unwrap()+ .iter()+ .map(|h| h["_score"].as_f64().unwrap())+ .collect();+ // To the digit: the provider said 0.9 and the wire says 0.9, not the+ // 0.8999999761581421 an f32 in the middle produced. A caller who re-applies+ // a threshold client-side must agree with `rerank.min_score`.+ assert_eq!(scores, vec![0.9, 0.7, 0.3, 0.1]);+ assert_eq!(r["hits"]["max_score"].as_f64(), Some(scores[0]));+}++#[tokio::test]+async fn size_and_from_page_inside_the_reranked_window() {+ let stub = Stub::start().await;+ inverted(&stub);+ let node = node(&stub).await;+ node.seed_kb().await;++ let (_, first) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 2, "rerank": {}}),+ )+ .await;+ let (_, second) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "from": 2, "size": 2, "rerank": {}}),+ )+ .await;+ assert_eq!(ids(&first), &INVERTED_IDS[..2], "{first}");+ assert_eq!(+ ids(&second),+ &INVERTED_IDS[2..],+ "page two continues the JUDGE's order, not the engine's: {second}"+ );+ // The whole window was judged both times, even though two hits came back.+ assert_eq!(first["_rerank"]["judged"], 4);+ assert_eq!(second["_rerank"]["judged"], 4);+ // `?size=` / `?from=` are the same request spelled in the URL.+ let (_, url_page) = node+ .search(+ "/kb/_search?from=2&size=2",+ json!({"query": match_q(), "rerank": {}}),+ )+ .await;+ assert_eq!(ids(&url_page), &INVERTED_IDS[2..], "{url_page}");+}++#[tokio::test]+async fn min_score_prunes_by_probability_and_the_total_stays_the_engines() {+ let stub = Stub::start().await;+ inverted(&stub);+ let node = node(&stub).await;+ node.seed_kb().await;++ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "rerank": {"min_score": 0.5}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(ids(&r), ["3", "1"], "{r}");+ assert_eq!(r["_rerank"]["pruned_below_min_score"], 2);+ assert_eq!(r["_rerank"]["dropped_unjudged"], 0);+ // Four documents matched. Two were judged irrelevant. Both facts are true+ // and the response says both: the total is the engine's.+ assert_eq!(r["hits"]["total"]["value"], 4, "{r}");+ assert_eq!(r["hits"]["total"]["relation"], "eq");+}++#[tokio::test]+async fn every_refusal_is_a_400_that_names_the_problem_and_sends_nothing() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;++ let q = match_q();+ let cases: Vec<(&str, &str, Value, &str)> = vec![+ (+ "sort",+ "/kb/_search",+ json!({"query": q, "sort": [{"n": "asc"}], "rerank": {}}),+ "sort",+ ),+ (+ "?sort= in the URL",+ "/kb/_search?sort=n:asc",+ json!({"query": q, "rerank": {}}),+ "sort",+ ),+ (+ "misspelt field",+ "/kb/_search",+ json!({"query": q, "rerank": {"treshold": 0.5}}),+ "treshold",+ ),+ (+ "min_score outside 0..1",+ "/kb/_search",+ json!({"query": q, "rerank": {"min_score": 7}}),+ "probability",+ ),+ (+ "bool without rerank.query",+ "/kb/_search",+ json!({"query": {"bool": {"must": [{"match": {"body": "vitamin"}}]}}, "rerank": {}}),+ "rerank.query",+ ),+ (+ "no query at all",+ "/kb/_search",+ json!({"rerank": {}}),+ "rerank.query",+ ),+ (+ "paging past the window",+ "/kb/_search",+ json!({"query": q, "from": 25, "size": 10, "rerank": {}}),+ "window",+ ),+ (+ "?from= past the window",+ "/kb/_search?from=25&size=10",+ json!({"query": q, "rerank": {}}),+ "window",+ ),+ (+ // `from + size` used to be an unchecked add: u64::MAX + 1 wrapped+ // to 0, passed the window check, paid for a full window of+ // judgements and returned an empty page under a 200 — while the+ // same request without `rerank` is a 400.+ "from + size overflowing",+ "/kb/_search",+ json!({"query": q, "from": u64::MAX, "size": 1, "rerank": {}}),+ "window",+ ),+ (+ "?from= overflowing in the URL",+ "/kb/_search?from=18446744073709551615&size=1",+ json!({"query": q, "rerank": {}}),+ "window",+ ),+ (+ "_source:false",+ "/kb/_search",+ json!({"query": q, "_source": false, "rerank": {}}),+ "_source",+ ),+ (+ "unknown provider",+ "/kb/_search",+ json!({"query": q, "rerank": {"provider": "voyage"}}),+ "voyage",+ ),+ (+ "search_after",+ "/kb/_search",+ json!({"query": q, "search_after": [1], "rerank": {}}),+ "s",+ ),+ (+ "collapse",+ "/kb/_search",+ json!({"query": q, "collapse": {"field": "cat"}, "rerank": {}}),+ "collapse",+ ),+ (+ "scroll",+ "/kb/_search?scroll=1m",+ json!({"query": q, "rerank": {}}),+ "scroll",+ ),+ (+ "size 0",+ "/kb/_search",+ json!({"query": q, "size": 0, "rerank": {}}),+ "size: 0",+ ),+ (+ "window over the ceiling",+ "/kb/_search",+ json!({"query": q, "rerank": {"window": 301}}),+ "rerank.window",+ ),+ (+ "window zero",+ "/kb/_search",+ json!({"query": q, "rerank": {"window": 0}}),+ "rerank.window",+ ),+ (+ "concurrency over the ceiling",+ "/kb/_search",+ json!({"query": q, "rerank": {"max_concurrency": 17}}),+ "max_concurrency",+ ),+ (+ "timeout over the ceiling",+ "/kb/_search",+ json!({"query": q, "rerank": {"timeout_ms": 60001}}),+ "timeout_ms",+ ),+ (+ "rerank is not an object",+ "/kb/_search",+ json!({"query": q, "rerank": true}),+ "object",+ ),+ (+ "empty rerank.query",+ "/kb/_search",+ json!({"query": q, "rerank": {"query": " "}}),+ "rerank.query",+ ),+ (+ "empty rerank.fields",+ "/kb/_search",+ json!({"query": q, "rerank": {"fields": []}}),+ "rerank.fields",+ ),+ (+ "judged field excluded from _source",+ "/kb/_search",+ json!({"query": q, "_source": {"excludes": ["body"]}, "rerank": {"fields": ["body"]}}),+ "body",+ ),+ (+ "judged field not in _source includes",+ "/kb/_search",+ json!({"query": q, "_source": ["title"], "rerank": {"fields": ["body"]}}),+ "body",+ ),+ (+ "judged field excluded in the URL",+ "/kb/_search?_source_excludes=body",+ json!({"query": q, "rerank": {"fields": ["body"]}}),+ "body",+ ),+ ];+ for (name, path, body, needle) in cases {+ let (st, r) = node.search(path, body).await;+ assert_eq!(st, StatusCode::BAD_REQUEST, "{name}: {r}");+ assert_eq!(r["status"], 400, "{name}: {r}");+ assert!(+ r["error"]["type"].is_string(),+ "{name}: an ES-shaped error envelope, not a bare string: {r}"+ );+ assert!(+ reason(&r).contains(needle),+ "{name}: the reason must name `{needle}`, got: {}",+ reason(&r)+ );+ }+ assert!(+ stub.calls().is_empty(),+ "a refused request must not have sent a single document to the provider"+ );+}++#[tokio::test]+async fn a_bool_query_works_once_the_question_is_spelled_out() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;++ let (st, r) = node+ .search(+ "/kb/_search",+ json!({+ "query": {"bool": {"must": [{"match": {"body": "vitamin"}}]}},+ "size": 4,+ "rerank": {"query": "bone density trial"}+ }),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["applied"], true);+ assert_eq!(r["_rerank"]["query"], "bone density trial");+ assert_eq!(+ stub.calls()[0].body["state"]["query"],+ "bone density trial",+ "the judge is asked the caller's question, not a guess at the bool tree"+ );+}++#[tokio::test]+async fn provider_401_surfaces_as_502_rerank_exception_not_as_lexical_order() {+ let stub = Stub::start().await;+ stub.fault(Fault::Status(401));+ let node = node(&stub).await;+ node.seed_kb().await;++ let (st, r) = node+ .search("/kb/_search", json!({"query": match_q(), "rerank": {}}))+ .await;+ assert_eq!(st, StatusCode::BAD_GATEWAY, "{r}");+ assert_eq!(r["status"], 502);+ assert_eq!(r["error"]["type"], "rerank_exception");+ assert!(reason(&r).contains("401"), "{r}");+ assert!(+ r.get("hits").is_none(),+ "a contract fault must not hand back the engine's order as if nothing happened: {r}"+ );+ assert_eq!(stub.calls().len(), 1, "a 401 is not retried");+}++#[tokio::test]+async fn malformed_provider_body_surfaces_as_502() {+ let stub = Stub::start().await;+ stub.fault(Fault::Garbage);+ let node = node(&stub).await;+ node.seed_kb().await;++ let (st, r) = node+ .search("/kb/_search", json!({"query": match_q(), "rerank": {}}))+ .await;+ assert_eq!(st, StatusCode::BAD_GATEWAY, "{r}");+ assert_eq!(r["error"]["type"], "rerank_exception");+ assert!(reason(&r).contains("malformed"), "{r}");+}++#[tokio::test]+async fn an_unreachable_provider_surfaces_as_502() {+ // Port 1 refuses immediately; no stub involved.+ let node = node_with(ProviderSettings::with_key_and_endpoint(+ "test-key",+ "http://127.0.0.1:1/v1/systemone",+ ))+ .await;+ node.seed_kb().await;+ let (st, r) = node+ .search("/kb/_search", json!({"query": match_q(), "rerank": {}}))+ .await;+ assert_eq!(st, StatusCode::BAD_GATEWAY, "{r}");+ assert_eq!(r["error"]["type"], "rerank_exception");+}++#[tokio::test]+async fn a_slow_provider_degrades_and_the_engine_order_stands() {+ let stub = Stub::start().await;+ inverted(&stub);+ stub.fault(Fault::Delay(Duration::from_secs(3)));+ let node = node(&stub).await;+ node.seed_kb().await;++ let (_, base) = node+ .search("/kb/_search", json!({"query": match_q(), "size": 4}))+ .await;++ let started = Instant::now();+ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "rerank": {"timeout_ms": 300}}),+ )+ .await;+ let waited = started.elapsed();++ assert_eq!(st, StatusCode::OK, "degrade on deadline is a 200: {r}");+ assert_eq!(r["_rerank"]["applied"], false, "{r}");+ assert_eq!(r["_rerank"]["score_kind"], "engine");+ assert!(+ r["_rerank"]["reason"]+ .as_str()+ .unwrap_or_default()+ .contains("deadline"),+ "{r}"+ );+ assert_eq!(ids(&r), ids(&base), "the engine's order, untouched");+ assert_eq!(+ r["hits"]["hits"][0]["_score"], base["hits"]["hits"][0]["_score"],+ "and the engine's scores — not a probability nobody computed"+ );+ assert!(+ waited < Duration::from_millis(2500),+ "the caller waited {waited:?} on a 300 ms budget — the deadline did not bound the stage"+ );+}++#[tokio::test]+async fn a_rate_limit_that_clears_is_retried_inside_the_budget() {+ let stub = Stub::start().await;+ inverted(&stub);+ stub.fault(Fault::StatusThenOk { n: 1, status: 429 });+ let node = node(&stub).await;+ node.seed_kb().await;++ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["applied"], true, "{r}");+ assert_eq!(ids(&r), INVERTED_IDS);+ assert_eq!(stub.calls().len(), 2, "one 429, one retry");+}++#[tokio::test]+async fn a_server_with_no_key_answers_503_and_sends_nothing() {+ let stub = Stub::start().await;+ // Resolved with no config key and NO environment — deterministic whatever+ // the developer's shell exports.+ let node = node_with(ProviderSettings::resolve(+ true,+ "",+ &stub.endpoint,+ None,+ None,+ ))+ .await;+ node.seed_kb().await;++ let (st, r) = node+ .search("/kb/_search", json!({"query": match_q(), "rerank": {}}))+ .await;+ assert_eq!(st, StatusCode::SERVICE_UNAVAILABLE, "{r}");+ assert_eq!(r["status"], 503);+ assert_eq!(r["error"]["type"], "rerank_exception");+ assert!(reason(&r).contains("TYPESAFE_API_KEY"), "{r}");+ assert!(+ reason(&r).contains("[rerank]"),+ "names the config section too: {r}"+ );+ assert!(stub.calls().is_empty());++ // Without a `rerank` block the same node searches normally.+ let (st, r) = node+ .search("/kb/_search", json!({"query": match_q()}))+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+}++#[tokio::test]+async fn an_operator_kill_switch_is_a_403_and_sends_nothing() {+ let stub = Stub::start().await;+ let node = node_with(ProviderSettings::resolve(+ false,+ "test-key",+ &stub.endpoint,+ None,+ None,+ ))+ .await;+ node.seed_kb().await;++ let (st, r) = node+ .search("/kb/_search", json!({"query": match_q(), "rerank": {}}))+ .await;+ assert_eq!(st, StatusCode::FORBIDDEN, "{r}");+ assert_eq!(r["error"]["type"], "rerank_exception");+ assert!(reason(&r).contains("disabled"), "{r}");+ assert!(+ stub.calls().is_empty(),+ "enabled = false means nothing leaves the host"+ );+}++#[tokio::test]+async fn wire_shape_bearer_auth_model_and_one_noul_per_document_keyed_identically() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;++ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "rerank": {"model": "jev-2026-09"}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");++ let calls = stub.calls();+ assert_eq!(calls.len(), 1, "four documents fit one call");+ let call = &calls[0];+ assert_eq!(call.authorization.as_deref(), Some("Bearer test-key"));+ assert_eq!(call.body["model"], "jev-2026-09");+ assert_eq!(call.body["state"]["query"], Q);++ let documents = call.body["state"]["documents"].as_object().unwrap();+ let questions = call.body["questions"].as_object().unwrap();+ assert_eq!(documents.len(), 4);+ let mut dk: Vec<&String> = documents.keys().collect();+ let mut qk: Vec<&String> = questions.keys().collect();+ dk.sort();+ qk.sort();+ assert_eq!(dk, qk, "one question per document, keyed identically");+ for (key, question) in questions {+ assert_eq!(question["type"], "noul", "{key}");+ assert!(question["instructions"].is_string(), "{key}");+ assert!(question["criteria"]["true"].is_string(), "{key}");+ assert!(question["criteria"]["false"].is_string(), "{key}");+ }+ for (key, doc) in documents {+ assert!(doc["text"].is_string(), "{key}");+ assert!(doc["title"].is_string(), "{key}");+ }+ // The key is a bearer token and nothing else: never in the body.+ assert!(!call.body.to_string().contains("test-key"));+}++#[tokio::test]+async fn custom_instructions_reach_every_question() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;+ let (st, _) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "rerank": {"instructions": "Is this a clinical result?"}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK);+ for (_, q) in stub.calls()[0].body["questions"].as_object().unwrap() {+ assert_eq!(q["instructions"], "Is this a clinical result?");+ }+}++#[tokio::test]+async fn a_window_larger_than_thirty_is_split_and_ordinals_survive() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ let docs: Vec<(String, String)> = (0..35)+ .map(|i| (format!("{i}"), format!("doc number {i}")))+ .collect();+ let seeded: Vec<(&str, &str, &str, &str, i64)> = docs+ .iter()+ .map(|(id, title)| (id.as_str(), title.as_str(), "common term here", "c", 1))+ .collect();+ node.seed("many", &seeded).await;+ // Only one document is relevant, and it sits wherever the engine put it.+ stub.by_title(&[("doc number 33", 0.95)]);++ let (st, r) = node+ .search(+ "/many/_search",+ json!({"query": {"match": {"body": "common"}}, "size": 35, "rerank": {"window": 35}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["judged"], 35, "{r}");+ assert_eq!(+ ids(&r)[0],+ "33",+ "the one relevant document wins across batches"+ );++ let calls = stub.calls();+ assert_eq!(calls.len(), 2, "35 documents = 30 + 5");+ let mut keys: Vec<String> = Vec::new();+ for c in &calls {+ let docs = c.body["state"]["documents"].as_object().unwrap();+ assert!(docs.len() <= 30, "the provider ceiling is 30 per call");+ keys.extend(docs.keys().cloned());+ }+ keys.sort();+ keys.dedup();+ assert_eq!(keys.len(), 35, "every document is judged exactly once");+ // Usage is summed across the calls that answered.+ assert_eq!(r["_rerank"]["usage"]["input_tokens"], 200);+}++// ─────────────────────────────────────────────────────────────────────────────+// 2. Interactions with the rest of `_search`+// ─────────────────────────────────────────────────────────────────────────────++/// Twelve matching documents in three categories, a window of three.+async fn seed_twelve(node: &Node) {+ let docs: Vec<(String, String, &str)> = (0..12)+ .map(|i| {+ (+ format!("{i}"),+ format!("note {i}"),+ ["red", "green", "blue"][i % 3],+ )+ })+ .collect();+ let seeded: Vec<(&str, &str, &str, &str, i64)> = docs+ .iter()+ .map(|(id, title, cat)| (id.as_str(), title.as_str(), "shared keyword", *cat, 1))+ .collect();+ node.seed("twelve", &seeded).await;+}++#[tokio::test]+async fn aggregations_describe_the_full_match_set_not_the_window() {+ let stub = Stub::start().await;+ stub.by_title(&[("note 7", 0.9)]);+ let node = node(&stub).await;+ seed_twelve(&node).await;++ let body = |rerank: bool| {+ let mut b = json!({+ "query": {"match": {"body": "shared"}},+ "size": 2,+ "aggs": {"by_cat": {"terms": {"field": "cat"}}, "n_sum": {"sum": {"field": "n"}}}+ });+ if rerank {+ b["rerank"] = json!({"window": 3, "min_score": 0.5});+ }+ b+ };+ let (_, plain) = node.search("/twelve/_search", body(false)).await;+ let (st, r) = node.search("/twelve/_search", body(true)).await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["applied"], true, "{r}");+ assert_eq!(r["_rerank"]["window"], 3);++ // The window was 3, the page 2, and pruning left at most one hit — and+ // none of that is allowed to leak into the facets or the total.+ assert_eq!(+ r["aggregations"], plain["aggregations"],+ "aggregations must be byte-identical with and without `rerank`"+ );+ let bucket_sum: u64 = r["aggregations"]["by_cat"]["buckets"]+ .as_array()+ .unwrap()+ .iter()+ .map(|b| b["doc_count"].as_u64().unwrap())+ .sum();+ assert_eq!(bucket_sum, 12, "{r}");+ assert_eq!(r["aggregations"]["n_sum"]["value"].as_f64(), Some(12.0));+ assert_eq!(r["hits"]["total"]["value"], 12, "{r}");+ assert!(ids(&r).len() <= 2);+}++#[tokio::test]+async fn track_total_hits_keeps_its_meaning() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ seed_twelve(&node).await;+ let q = json!({"match": {"body": "shared"}});++ let (_, exact) = node+ .search(+ "/twelve/_search",+ json!({"query": q, "size": 2, "track_total_hits": true, "rerank": {"window": 5}}),+ )+ .await;+ assert_eq!(+ exact["hits"]["total"],+ json!({"value": 12, "relation": "eq"}),+ "{exact}"+ );++ let (_, capped) = node+ .search(+ "/twelve/_search",+ json!({"query": q, "size": 2, "track_total_hits": 5, "rerank": {"window": 5}}),+ )+ .await;+ let (_, capped_plain) = node+ .search(+ "/twelve/_search",+ json!({"query": q, "size": 2, "track_total_hits": 5}),+ )+ .await;+ assert_eq!(+ capped["hits"]["total"], capped_plain["hits"]["total"],+ "an integer cap reports what it reports without `rerank`"+ );++ let (st, off) = node+ .search(+ "/twelve/_search",+ json!({"query": q, "size": 2, "track_total_hits": false, "rerank": {"window": 5}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{off}");+ assert!(off["hits"].get("total").is_none(), "{off}");+ assert_eq!(off["_rerank"]["applied"], true);+ assert_eq!(ids(&off).len(), 2);+}++#[tokio::test]+async fn highlight_and_fields_travel_with_their_hit_through_the_reorder() {+ let stub = Stub::start().await;+ inverted(&stub);+ let node = node(&stub).await;+ node.seed_kb().await;++ let body = |rerank: bool| {+ let mut b = json!({+ "query": match_q(),+ "size": 4,+ "highlight": {"fields": {"body": {}}},+ "fields": ["cat", "n"]+ });+ if rerank {+ b["rerank"] = json!({});+ }+ b+ };+ let (_, plain) = node.search("/kb/_search", body(false)).await;+ let (_, r) = node.search("/kb/_search", body(true)).await;+ assert_eq!(ids(&r), INVERTED_IDS, "{r}");++ let by_id = |resp: &Value| -> HashMap<String, (Value, Value)> {+ resp["hits"]["hits"]+ .as_array()+ .unwrap()+ .iter()+ .map(|h| {+ (+ h["_id"].as_str().unwrap().to_string(),+ (h["highlight"].clone(), h["fields"].clone()),+ )+ })+ .collect()+ };+ let (before, after) = (by_id(&plain), by_id(&r));+ for (id, (hl, fields)) in &after {+ assert!(!hl.is_null(), "hit {id} lost its highlight: {r}");+ assert!(!fields.is_null(), "hit {id} lost its fields: {r}");+ assert_eq!(+ &before[id].0, hl,+ "hit {id} carries another hit's highlight"+ );+ assert_eq!(+ &before[id].1, fields,+ "hit {id} carries another hit's fields"+ );+ }+}++#[tokio::test]+async fn the_judge_sees_exactly_what_the_response_returns() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;++ // `_source: ["title"]` — the body is not in the response, so it is not in+ // the request to the provider either. This is the privacy contract.+ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "_source": ["title"], "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["applied"], true);+ let sent = stub.everything_sent();+ assert!(+ sent.contains("Trial results"),+ "titles were returned, so titles were judged"+ );+ assert!(+ !sent.contains("treatment group") && !sent.contains("bone china"),+ "body text the caller excluded from the response must not leave the machine: {sent}"+ );+ for h in r["hits"]["hits"].as_array().unwrap() {+ assert!(h["_source"].get("body").is_none(), "{h}");+ }++ // A projection that leaves no prose at all: refused, nothing sent.+ let before = stub.calls().len();+ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "_source": ["n"], "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::BAD_REQUEST, "{r}");+ assert_eq!(r["error"]["type"], "illegal_argument_exception");+ assert!(reason(&r).contains("nothing to judge"), "{r}");+ assert_eq!(stub.calls().len(), before, "blank documents are never sent");++ // `fields` is part of the response too: text returned that way is judgeable+ // even with `_source` switched off entirely.+ let (st, r) = node+ .search(+ "/kb/_search",+ json!({+ "query": match_q(), "size": 4,+ "_source": false, "fields": ["body"],+ "rerank": {"fields": ["body"]}+ }),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["applied"], true, "{r}");+ let last = stub.calls().last().unwrap().body.clone();+ assert!(last.to_string().contains("treatment group"), "{last}");+ assert!(+ !last.to_string().contains("Trial results"),+ "titles were neither returned nor named, so they were not sent: {last}"+ );+}++/// The invariant behind the privacy contract's other half: a caller who names+/// the field to judge on is NEVER handed an `applied: true` that was judged on+/// blank text.+///+/// Written against the invariant rather than one outcome on purpose. Today the+/// engine drops a `fields` entry that an explicit `_source` includes list+/// filtered away, so this request returns no `body` and must be refused. If the+/// engine starts returning it, as Elasticsearch does, the same request becomes+/// a legitimate rerank on the body. Both are correct; judging blind is not.+#[tokio::test]+async fn a_named_field_the_response_does_not_return_is_never_judged_blind() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;++ let (st, r) = node+ .search(+ "/kb/_search",+ json!({+ "query": match_q(), "size": 4,+ "_source": ["title"], "fields": ["body"],+ "rerank": {"fields": ["body"]}+ }),+ )+ .await;+ if st == StatusCode::BAD_REQUEST {+ assert_eq!(r["error"]["type"], "illegal_argument_exception", "{r}");+ assert!(reason(&r).contains("nothing to judge"), "{r}");+ assert!(reason(&r).contains("body"), "names the field: {r}");+ assert!(stub.calls().is_empty(), "blank documents are never sent");+ } else {+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["applied"], true, "{r}");+ for c in stub.calls() {+ for (key, doc) in c.body["state"]["documents"].as_object().unwrap() {+ assert!(+ !doc["text"].as_str().unwrap_or_default().trim().is_empty(),+ "document {key} was judged on blank text: {doc}"+ );+ }+ }+ }+}++/// A named field that contributes nothing is reported, not swallowed: a typo in+/// `rerank.fields` must not pass for a judgement on that field.+#[tokio::test]+async fn a_named_field_with_no_text_anywhere_in_the_window_is_reported() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;++ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "rerank": {"fields": ["body", "bdoy"]}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["applied"], true, "{r}");+ assert_eq!(r["_rerank"]["fields_without_text"], json!(["bdoy"]), "{r}");++ // Nothing to report means the key is absent, not an empty array.+ let (_, clean) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "rerank": {"fields": ["body"]}}),+ )+ .await;+ assert!(+ clean["_rerank"].get("fields_without_text").is_none(),+ "{clean}"+ );++ // Every named field empty: that is nothing to judge, and nothing is sent.+ let before = stub.calls().len();+ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "rerank": {"fields": ["bdoy"]}}),+ )+ .await;+ assert_eq!(st, StatusCode::BAD_REQUEST, "{r}");+ assert!(reason(&r).contains("bdoy"), "{r}");+ assert_eq!(stub.calls().len(), before);+}++#[tokio::test]+async fn rerank_fields_is_exhaustive_about_what_is_sent() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;+ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "rerank": {"fields": ["body"]}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");++ // Asserted on the structure, not by substring: the title "Density of+ // materials" ends in the `cat` value "materials", and a substring check+ // reported a leak that was not there.+ let bodies = [+ "vitamin supplements are popular. vitamin vitamin vitamin vitamin.",+ "vitamin d supplementation improved bone density in the treatment group",+ "vitamin rich vegetables for dinner",+ "bone china has high density",+ ];+ let calls = stub.calls();+ let documents = calls[0].body["state"]["documents"].as_object().unwrap();+ assert_eq!(documents.len(), 4);+ for (key, doc) in documents {+ let keys: Vec<&String> = doc.as_object().unwrap().keys().collect();+ assert_eq!(+ keys,+ ["text"],+ "{key}: `rerank.fields` named only `body`, so not even the title may be sent"+ );+ let text = doc["text"].as_str().unwrap();+ assert!(+ bodies.contains(&text),+ "{key}: the text is exactly the body — no `cat`, no `n`, nothing appended: {text:?}"+ );+ }++ // Naming the title brings it back, in its own slot.+ let (_, _) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "rerank": {"fields": ["title", "body"]}}),+ )+ .await;+ let last = stub.calls().last().unwrap().body.clone();+ for (key, doc) in last["state"]["documents"].as_object().unwrap() {+ assert!(doc["title"].is_string(), "{key}: {doc}");+ assert!(+ bodies.contains(&doc["text"].as_str().unwrap()),+ "{key}: {doc}"+ );+ }+}++#[tokio::test]+async fn hits_with_no_prose_are_skipped_not_sent_blank() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;+ let (st, b) = node.call("PUT", "/kb/_doc/9", json!({"n": 9})).await;+ assert!(st.is_success(), "{b}");+ node.call("POST", "/kb/_refresh", json!({})).await;++ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": {"match_all": {}}, "size": 5, "rerank": {"query": "bone density"}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["window"], 5);+ assert_eq!(r["_rerank"]["judged"], 4);+ assert_eq!(r["_rerank"]["skipped_no_text"], 1);+ assert_eq!(+ stub.calls()[0].body["state"]["documents"]+ .as_object()+ .unwrap()+ .len(),+ 4+ );+ // Unjudged is not a verdict of irrelevant: it sorts after the judged hits.+ // Its `_score` is null — never the engine's BM25 value beside+ // probabilities — and the block counts it.+ assert_eq!(ids(&r).last().map(String::as_str), Some("9"), "{r}");+ assert!(r["hits"]["hits"][4]["_score"].is_null(), "{r}");+ assert_eq!(r["_rerank"]["unjudged"], 1, "{r}");++ // With a threshold it cannot be shown to clear the bar, so it goes — and+ // the response says that is why.+ let (_, r) = node+ .search(+ "/kb/_search",+ json!({"query": {"match_all": {}}, "size": 5,+ "rerank": {"query": "bone density", "min_score": 0.0}}),+ )+ .await;+ assert!(!ids(&r).contains(&"9".to_string()), "{r}");+ assert_eq!(r["_rerank"]["dropped_unjudged"], 1, "{r}");+}++#[tokio::test]+async fn explain_is_wrapped_so_it_never_contradicts_the_score() {+ let stub = Stub::start().await;+ inverted(&stub);+ let node = node(&stub).await;+ node.seed_kb().await;++ let (_, plain) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "explain": true}),+ )+ .await;+ let engine: HashMap<String, Value> = plain["hits"]["hits"]+ .as_array()+ .unwrap()+ .iter()+ .map(|h| {+ (+ h["_id"].as_str().unwrap().to_string(),+ h["_explanation"].clone(),+ )+ })+ .collect();++ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "explain": true, "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ for h in r["hits"]["hits"].as_array().unwrap() {+ let id = h["_id"].as_str().unwrap();+ let ex = &h["_explanation"];+ assert_eq!(+ ex["value"], h["_score"],+ "hit {id}: `_explanation.value` must agree with `_score`"+ );+ assert!(+ ex["description"]+ .as_str()+ .unwrap_or_default()+ .contains("rerank"),+ "{ex}"+ );+ assert_eq!(+ ex["details"][0], engine[id],+ "hit {id}: the engine's own explanation is kept underneath, unaltered"+ );+ }+}++#[tokio::test]+async fn profile_survives_and_took_covers_the_provider_call() {+ let stub = Stub::start().await;+ stub.fault(Fault::Delay(Duration::from_millis(200)));+ let node = node(&stub).await;+ node.seed_kb().await;++ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "profile": true, "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert!(+ r["profile"]["shards"].is_array(),+ "profile is the engine's and still there: {r}"+ );+ assert_eq!(r["_rerank"]["applied"], true);+ let stage = r["_rerank"]["took_ms"].as_u64().unwrap();+ let took = r["took"].as_u64().unwrap();+ assert!(stage >= 200, "the stage reports its own time: {stage} ms");+ assert!(+ took >= stage,+ "`took` ({took} ms) is what the caller waited, so it includes the stage ({stage} ms)"+ );+}++#[tokio::test]+async fn top_level_min_score_cuts_engine_scores_and_rerank_min_score_cuts_probabilities() {+ let stub = Stub::start().await;+ inverted(&stub);+ let node = node(&stub).await;+ node.seed_kb().await;++ let (_, plain) = node+ .search("/kb/_search", json!({"query": match_q(), "size": 4}))+ .await;+ let hits = plain["hits"]["hits"].as_array().unwrap();+ // A bar between the second and third engine scores.+ let bar = (hits[1]["_score"].as_f64().unwrap() + hits[2]["_score"].as_f64().unwrap()) / 2.0;+ let survivors: Vec<String> = ids(&plain)[..2].to_vec();++ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "min_score": bar, "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ // Top-level `min_score` is the engine's, applied to ENGINE scores before+ // the window is cut — so it shapes the match set and the total, exactly as+ // it does without `rerank`…+ assert_eq!(r["hits"]["total"]["value"], 2, "{r}");+ assert_eq!(+ r["_rerank"]["judged"], 2,+ "only survivors are sent to the judge"+ );+ let mut got = ids(&r);+ got.sort();+ let mut want = survivors.clone();+ want.sort();+ assert_eq!(got, want);+ // …and it is NOT compared against the probabilities that replace `_score`:+ // both survivors score below `bar` as probabilities and are still here.+ for h in r["hits"]["hits"].as_array().unwrap() {+ assert!(h["_score"].as_f64().unwrap() <= 1.0);+ }++ // The two thresholds compose: engine bar first, probability bar second.+ let (_, both) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "min_score": bar,+ "rerank": {"min_score": 0.2}}),+ )+ .await;+ for h in both["hits"]["hits"].as_array().unwrap() {+ assert!(h["_score"].as_f64().unwrap() >= 0.2, "{both}");+ }+ assert_eq!(both["hits"]["total"]["value"], 2);+}++#[tokio::test]+async fn a_knn_section_needs_the_question_spelled_out_unless_a_text_query_carries_it() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ let (st, b) = node+ .call(+ "PUT",+ "/vec",+ json!({"mappings": {"properties": {+ "title": {"type": "text"}, "body": {"type": "text"},+ "v": {"type": "dense_vector", "dims": 3}+ }}}),+ )+ .await;+ assert!(st.is_success(), "{b}");+ for (id, title, body, v) in [+ ("1", "alpha one", "alpha beta", [0.1, 0.2, 0.3]),+ ("2", "alpha two", "alpha gamma", [0.2, 0.1, 0.4]),+ ("3", "only by vector", "zzz qqq", [0.1, 0.2, 0.3]),+ ] {+ let (st, b) = node+ .call(+ "PUT",+ &format!("/vec/_doc/{id}"),+ json!({"title": title, "body": body, "v": v}),+ )+ .await;+ assert!(st.is_success(), "{b}");+ }+ node.call("POST", "/vec/_refresh", json!({})).await;+ let knn = json!({"field": "v", "query_vector": [0.1, 0.2, 0.3], "k": 3, "num_candidates": 10});++ // A vector is not a question.+ let (st, r) = node+ .search("/vec/_search", json!({"knn": knn, "rerank": {}}))+ .await;+ assert_eq!(st, StatusCode::BAD_REQUEST, "{r}");+ assert!(reason(&r).contains("rerank.query"), "{r}");+ assert!(+ reason(&r).contains("knn"),+ "the reason names the knn case: {r}"+ );++ // Spelled out, a pure-kNN search reranks.+ stub.by_title(&[+ ("only by vector", 0.8),+ ("alpha two", 0.6),+ ("alpha one", 0.1),+ ]);+ let (st, r) = node+ .search(+ "/vec/_search",+ json!({"knn": knn, "size": 3, "_source": ["title", "body"],+ "rerank": {"query": "which one is reachable only by vector?"}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["applied"], true, "{r}");+ assert_eq!(ids(&r), ["3", "2", "1"], "{r}");++ // knn beside a text query: the question is read from the text query, and+ // the vector-only document is inside the window the judge sees.+ let (st, r) = node+ .search(+ "/vec/_search",+ json!({"query": {"match": {"body": "alpha"}}, "knn": knn, "size": 3,+ "_source": ["title", "body"], "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["query"], "alpha");+ assert_eq!(r["_rerank"]["judged"], 3, "{r}");+ assert_eq!(ids(&r)[0], "3", "{r}");+ // No vector was sent to a text judge.+ assert!(+ !stub.everything_sent().contains("0.2"),+ "{}",+ stub.everything_sent()+ );+}++#[tokio::test]+async fn msearch_items_carrying_rerank_are_refused_per_item() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;++ let ndjson = format!(+ "{}\n{}\n{}\n{}\n",+ json!({"index": "kb"}),+ json!({"query": match_q(), "rerank": {}}),+ json!({"index": "kb"}),+ json!({"query": match_q(), "size": 2}),+ );+ let (st, text) = node.raw("POST", "/_msearch", ndjson).await;+ assert_eq!(st, StatusCode::OK, "{text}");+ let r: Value = serde_json::from_str(&text).unwrap();+ let items = r["responses"].as_array().unwrap();+ assert_eq!(items.len(), 2, "{r}");+ assert_eq!(items[0]["status"], 400, "{r}");+ assert!(reason(&items[0]).contains("_msearch"), "{r}");+ assert!(+ reason(&items[0]).contains("_search"),+ "says where to send it: {r}"+ );+ assert_eq!(+ items[1]["hits"]["hits"].as_array().map(Vec::len),+ Some(2),+ "the rest of the batch still runs: {r}"+ );+ assert!(stub.calls().is_empty());+}++#[tokio::test]+async fn templates_async_search_and_scroll_refuse_rerank_instead_of_dropping_it() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;++ let (st, r) = node+ .search(+ "/kb/_search/template",+ json!({"source": {"query": {"match": {"body": "{{q}}"}}, "rerank": {}},+ "params": {"q": "vitamin"}}),+ )+ .await;+ assert_eq!(st, StatusCode::BAD_REQUEST, "_search/template: {r}");+ assert!(reason(&r).contains("_search/template"), "{r}");++ let ndjson = format!(+ "{}\n{}\n",+ json!({"index": "kb"}),+ json!({"source": {"query": {"match": {"body": "{{q}}"}}, "rerank": {}}, "params": {"q": "vitamin"}}),+ );+ let (st, text) = node.raw("POST", "/_msearch/template", ndjson).await;+ assert_eq!(st, StatusCode::OK, "{text}");+ let r: Value = serde_json::from_str(&text).unwrap();+ assert_eq!(r["responses"][0]["status"], 400, "{r}");+ assert!(+ reason(&r["responses"][0]).contains("_msearch/template"),+ "{r}"+ );++ let (st, r) = node+ .search(+ "/kb/_async_search",+ json!({"query": match_q(), "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::BAD_REQUEST, "_async_search: {r}");+ assert!(reason(&r).contains("_async_search"), "{r}");++ let (st, r) = node+ .search(+ "/kb/_search_scroll?scroll=1m",+ json!({"query": match_q(), "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::BAD_REQUEST, "_search_scroll: {r}");++ // A template WITHOUT the block is unaffected.+ let (st, r) = node+ .search(+ "/kb/_search/template",+ json!({"source": {"query": {"match": {"body": "{{q}}"}}}, "params": {"q": "vitamin"}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert!(stub.calls().is_empty());+}++#[tokio::test]+async fn a_multi_index_search_is_reranked_across_its_indices() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;+ node.seed(+ "kb2",+ &[+ (+ "a",+ "Second index winner",+ "vitamin d and bone density, again",+ "trial",+ 1,+ ),+ ("b", "Second index loser", "vitamin gummies", "food", 2),+ ],+ )+ .await;+ stub.by_title(&[+ ("Second index winner", 0.99),+ ("Cooking", 0.80),+ ("Trial results", 0.60),+ ("Second index loser", 0.05),+ ]);++ let (st, r) = node+ .search(+ "/kb,kb2/_search",+ json!({"query": match_q(), "size": 6, "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["judged"], 6, "{r}");+ assert_eq!(r["hits"]["total"]["value"], 6);+ let got: Vec<(String, String)> = r["hits"]["hits"]+ .as_array()+ .unwrap()+ .iter()+ .map(|h| {+ (+ h["_index"].as_str().unwrap().to_string(),+ h["_id"].as_str().unwrap().to_string(),+ )+ })+ .collect();+ assert_eq!(got[0], ("kb2".to_string(), "a".to_string()), "{r}");+ assert_eq!(got[1], ("kb".to_string(), "3".to_string()), "{r}");+ assert_eq!(got[2], ("kb".to_string(), "2".to_string()), "{r}");+ // `_index` still belongs to its `_id` after the reorder.+ for (index, id) in &got {+ let want = if id == "a" || id == "b" { "kb2" } else { "kb" };+ assert_eq!(index, want, "hit {id} claims index {index}");+ }++ // Same through a wildcard.+ let (st, r) = node+ .search(+ "/kb*/_search",+ json!({"query": match_q(), "size": 6, "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["hits"]["hits"][0]["_id"], "a");+}++#[tokio::test]+async fn rescore_runs_first_and_rerank_has_the_last_word() {+ let stub = Stub::start().await;+ inverted(&stub);+ let node = node(&stub).await;+ node.seed_kb().await;+ let (st, r) = node+ .search(+ "/kb/_search",+ json!({+ "query": match_q(), "size": 4,+ "rescore": {"window_size": 4, "query": {"rescore_query": {"match": {"body": "china"}}}},+ "rerank": {}+ }),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(ids(&r), INVERTED_IDS, "{r}");+}++// ─────────────────────────────────────────────────────────────────────────────+// 3. Visibility without exposure+// ─────────────────────────────────────────────────────────────────────────────++#[tokio::test]+async fn the_status_endpoint_says_whether_it_is_configured_and_never_the_key() {+ let stub = Stub::start().await;+ let node = node_with(ProviderSettings::with_key_and_endpoint(+ "sk-this-must-never-be-printed",+ &stub.endpoint,+ ))+ .await;+ let (st, text) = node.raw("GET", "/_xerj/rerank", String::new()).await;+ assert_eq!(st, StatusCode::OK, "{text}");+ assert!(!text.contains("sk-this-must-never-be-printed"), "{text}");+ let r: Value = serde_json::from_str(&text).unwrap();+ assert_eq!(r["enabled"], true);+ assert_eq!(r["configured"], true);+ assert_eq!(r["api_key"], json!({"set": true, "source": "config"}));+ assert_eq!(r["endpoint"]["url"], stub.endpoint);+ assert_eq!(r["limits"]["max_docs_per_call"], 30);+ // The string ceilings are limits an operator (and an agent) can read,+ // like the numeric ones — and they are the crate's constants, not copies.+ assert_eq!(+ r["limits"]["max_instructions_chars"],+ xerj_rerank::MAX_INSTRUCTIONS_CHARS+ );+ assert_eq!(r["limits"]["max_query_chars"], xerj_rerank::MAX_QUERY_CHARS);+ assert_eq!(r["limits"]["max_model_chars"], xerj_rerank::MAX_MODEL_CHARS);+ assert_eq!(r["limits"]["max_fields"], xerj_rerank::MAX_FIELDS);+ assert_eq!(r["defaults"]["max_concurrency"], 8);+ assert!(r["data_egress"]+ .as_str()+ .unwrap()+ .contains("sends the text"));++ let bare = node_with(ProviderSettings::resolve(true, "", "", None, None)).await;+ let (_, r) = bare.call("GET", "/_xerj/rerank", Value::Null).await;+ assert_eq!(r["configured"], false, "{r}");+ assert_eq!(r["api_key"]["set"], false);+ assert_eq!(r["endpoint"]["source"], "default");+}++// ─────────────────────────────────────────────────────────────────────────────+// 4. The surfaces that do not run the stage, and the operator's meter+// ─────────────────────────────────────────────────────────────────────────────++#[tokio::test]+async fn the_native_search_api_refuses_rerank_instead_of_dropping_it() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;++ let send = |body: Value| {+ let app = node.native.clone();+ async move {+ let response = app+ .oneshot(+ Request::builder()+ .method("POST")+ .uri("/v1/indices/kb/search")+ .header("content-type", "application/json")+ .body(Body::from(body.to_string()))+ .unwrap(),+ )+ .await+ .unwrap();+ let status = response.status();+ let bytes = axum::body::to_bytes(response.into_body(), usize::MAX)+ .await+ .unwrap();+ (status, String::from_utf8_lossy(&bytes).into_owned())+ }+ };++ // Its request struct ignores unknown keys, so this used to be a 200 in the+ // engine's order for a caller who believed it had been reranked.+ let (st, text) = send(json!({"q": "vitamin", "rerank": {}})).await;+ assert_eq!(st, StatusCode::BAD_REQUEST, "{text}");+ assert!(text.contains("rerank"), "{text}");+ assert!(text.contains("_search"), "says where to send it: {text}");+ assert!(stub.calls().is_empty());++ // Without the block the native search is untouched.+ let (st, text) = send(json!({"q": "vitamin"})).await;+ assert_eq!(st, StatusCode::OK, "{text}");+}++/// The operator pays per judged document, on a volume the caller chooses.+#[tokio::test]+async fn the_operator_can_meter_rerank_from_prometheus() {+ let stub = Stub::start().await;+ inverted(&stub);+ let node = node(&stub).await;+ node.seed_kb().await;++ // applied: 4 judged, the stub reports 100 in / 10 out.+ let (st, _) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK);+ // refused before the search ran: nothing sent, nothing billed.+ let (st, _) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "rerank": {"treshold": 1}}),+ )+ .await;+ assert_eq!(st, StatusCode::BAD_REQUEST);+ // failed: the provider was called and answered 401.+ stub.fault(Fault::Status(401));+ let (st, _) = node+ .search("/kb/_search", json!({"query": match_q(), "rerank": {}}))+ .await;+ assert_eq!(st, StatusCode::BAD_GATEWAY);+ // degraded: the deadline ran out.+ stub.fault(Fault::Delay(Duration::from_secs(2)));+ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "rerank": {"timeout_ms": 100}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["applied"], false);+ // A search with no `rerank` block is not counted at all.+ let (st, _) = node+ .search("/kb/_search", json!({"query": match_q()}))+ .await;+ assert_eq!(st, StatusCode::OK);++ let (st, text) = node.raw("GET", "/v1/metrics", String::new()).await;+ assert_eq!(st, StatusCode::OK);+ let value = |needle: &str| -> Option<f64> {+ text.lines()+ .find(|l| l.starts_with(needle))+ .and_then(|l| l.rsplit(' ').next())+ .and_then(|v| v.parse().ok())+ };+ for (series, want) in [+ (r#"xerj_rerank_requests_total{outcome="applied"}"#, 1.0),+ (r#"xerj_rerank_requests_total{outcome="refused"}"#, 1.0),+ (r#"xerj_rerank_requests_total{outcome="failed"}"#, 1.0),+ (r#"xerj_rerank_requests_total{outcome="degraded"}"#, 1.0),+ ("xerj_rerank_documents_judged_total", 4.0),+ (r#"xerj_rerank_provider_tokens_total{kind="input"}"#, 100.0),+ (r#"xerj_rerank_provider_tokens_total{kind="output"}"#, 10.0),+ ] {+ assert_eq!(value(series), Some(want), "{series}\n{text}");+ }+}++/// `docvalue_fields` also puts values on a hit, so the pre-flight prediction+/// must stand aside for it rather than refuse a request that works.+#[tokio::test]+async fn text_returned_through_docvalue_fields_is_judgeable() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;++ let (st, r) = node+ .search(+ "/kb/_search",+ json!({+ "query": match_q(), "size": 4,+ "_source": false, "docvalue_fields": ["cat"],+ "rerank": {"fields": ["cat"], "query": "which category is a trial?"}+ }),+ )+ .await;+ // The engine returns `cat` under `fields`, so it is judged — on exactly the+ // text that came back. The first draft of `prepare` gated on `fields`+ // alone and refused this pre-flight, with a message naming `fields` as the+ // only way; this test accepted that refusal and so passed vacuously.+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["applied"], true, "{r}");+ assert_eq!(r["_rerank"]["judged"], 4, "{r}");+ let sent = stub.everything_sent();+ assert!(sent.contains("trial"), "{sent}");+ assert!(+ !sent.contains("supplementation"),+ "`_source: false` returned no body, so no body was sent: {sent}"+ );+}++/// The provider is a third party. A verdict it returns for a document XERJ+/// never sent must not score that document, and must not move the meter.+#[tokio::test]+async fn a_verdict_for_a_document_that_was_never_sent_is_ignored() {+ let stub = Stub::start().await;+ stub.fault(Fault::AnswerUnsent);+ let node = node(&stub).await;+ node.seed_kb().await;+ // A fifth hit with no prose: skipped, so never sent.+ let (st, b) = node.call("PUT", "/kb/_doc/9", json!({"n": 9})).await;+ assert!(st.is_success(), "{b}");+ node.call("POST", "/kb/_refresh", json!({})).await;++ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": {"match_all": {}}, "size": 5, "rerank": {"query": "bone density"}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["skipped_no_text"], 1, "{r}");+ assert_eq!(+ r["_rerank"]["judged"], 4,+ "four documents were sent; the stub answered for ten: {r}"+ );+ let nine = r["hits"]["hits"]+ .as_array()+ .unwrap()+ .iter()+ .find(|h| h["_id"] == "9")+ .expect("unjudged, not dropped")+ .clone();+ assert!(+ nine["_score"].is_null(),+ "doc 9 was never sent, so the stub's 0.99 for it is not its score: {r}"+ );+ assert_eq!(ids(&r).last().map(String::as_str), Some("9"), "{r}");+ assert_eq!(r["_rerank"]["unjudged"], 1, "{r}");+}++/// `fields: ["_passage"]` returns the matching slice of a document. It is an+/// array of OBJECTS, so it needs reading deliberately — and it is the best thing+/// to send: the part that matched, rather than the first `max_doc_chars` of a+/// long document, and less text leaving the machine.+#[tokio::test]+async fn the_matching_passage_is_judgeable_and_goes_first() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;+ let bodies = [+ "vitamin supplements are popular. vitamin vitamin vitamin vitamin.",+ "vitamin d supplementation improved bone density in the treatment group",+ "vitamin rich vegetables for dinner",+ "bone china has high density",+ ];++ // Named explicitly: the passage and nothing else.+ let (st, r) = node+ .search(+ "/kb/_search",+ json!({+ "query": match_q(), "size": 4,+ "_source": ["title"], "fields": ["_passage"],+ "rerank": {"fields": ["_passage"]}+ }),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["applied"], true, "{r}");+ assert!(r["_rerank"].get("fields_without_text").is_none(), "{r}");+ let sent = stub.calls().last().unwrap().body.clone();+ for (key, doc) in sent["state"]["documents"].as_object().unwrap() {+ assert!(+ doc.get("title").is_none(),+ "{key}: the title was not named: {doc}"+ );+ let text = doc["text"].as_str().unwrap();+ assert!(!text.trim().is_empty(), "{key}: {doc}");+ assert!(+ bodies.iter().any(|b| b.contains(text.trim())),+ "{key}: a passage is a slice of the body it came from: {text:?}"+ );+ }++ // Not named: the passage still leads, ahead of the rest of `_source`, so a+ // clip at `max_doc_chars` cuts the tail and never the part that matched.+ let (st, r) = node+ .search(+ "/kb/_search",+ json!({+ "query": match_q(), "size": 4,+ "_source": ["title", "cat"], "fields": ["_passage"],+ "rerank": {}+ }),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ let sent = stub.calls().last().unwrap().body.clone();+ for (key, doc) in sent["state"]["documents"].as_object().unwrap() {+ let text = doc["text"].as_str().unwrap();+ let first_line = text.lines().next().unwrap_or_default();+ assert!(+ bodies.iter().any(|b| b.contains(first_line.trim())) && !first_line.trim().is_empty(),+ "{key}: the passage comes first: {text:?}"+ );+ assert!(+ ["health", "trial", "food", "materials"].contains(&text.lines().last().unwrap()),+ "{key}: then the returned `_source` strings: {text:?}"+ );+ }+}++/// The large-response hint offers a "ready-to-send" corrected request. For a+/// caller that asked for `rerank`, a suggestion without it is a different+/// search: following it would silently return the engine's order.+#[tokio::test]+async fn the_response_hint_keeps_the_rerank_block() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ let filler = "lorem ipsum dolor sit amet ".repeat(120); // ~3.2 KB each+ let big: Vec<(String, String)> = (0..3)+ .map(|i| (format!("{i}"), format!("shared keyword {filler} {i}")))+ .collect();+ let seeded: Vec<(&str, &str, &str, &str, i64)> = big+ .iter()+ .map(|(id, body)| (id.as_str(), "a long one", body.as_str(), "c", 1))+ .collect();+ node.seed("big", &seeded).await;++ let rerank = json!({"min_score": 0.0, "query": "which one is shared?"});+ let (st, r) = node+ .search(+ "/big/_search",+ json!({"query": {"match": {"body": "shared"}}, "size": 3, "rerank": rerank}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ // Several diagnostics can share the `hints` array; find the payload one.+ let payload_hint = |resp: &Value| -> Option<Value> {+ resp["_xerj"]["hints"]+ .as_array()?+ .iter()+ .find(|h| h["try"]["body"]["fields"] == json!(["_passage"]))+ .cloned()+ };+ let hint = payload_hint(&r)+ .unwrap_or_else(|| panic!("an unprojected ~10 KB response carries the hint: {r}"));+ assert_eq!(+ hint["try"]["body"]["rerank"], rerank,+ "the suggested request is still the caller's search: {hint}"+ );++ // Without `rerank` the suggestion does not grow one.+ let (_, plain) = node+ .search(+ "/big/_search",+ json!({"query": {"match": {"body": "shared"}}, "size": 3}),+ )+ .await;+ let hint = payload_hint(&plain).expect("same payload, same hint");+ assert!(hint["try"]["body"].get("rerank").is_none(), "{hint}");+}++/// The 502's `reason` carries the provider's error body to whoever ran the+/// search — not necessarily the operator who owns the key. A provider that+/// echoes the credential in its auth error must not turn every search caller+/// into someone who can read it.+#[tokio::test]+async fn a_provider_error_that_echoes_the_key_never_reaches_the_caller() {+ let stub = Stub::start().await;+ stub.fault(Fault::EchoKey);+ let node = node_with(ProviderSettings::with_key_and_endpoint(+ "sk-operator-secret-do-not-leak",+ &stub.endpoint,+ ))+ .await;+ node.seed_kb().await;++ let (st, text) = node+ .raw(+ "POST",+ "/kb/_search",+ json!({"query": match_q(), "rerank": {}}).to_string(),+ )+ .await;+ assert_eq!(st, StatusCode::BAD_GATEWAY, "{text}");+ assert!(+ stub.calls()[0]+ .authorization+ .as_deref()+ .is_some_and(|a| a.contains("sk-operator-secret-do-not-leak")),+ "the fixture is only a test if the stub really received the key"+ );+ assert!(+ !text.contains("sk-operator-secret-do-not-leak"),+ "the operator's key leaked to the search caller: {text}"+ );+ assert!(text.contains("<redacted>"), "{text}");+ assert!(+ text.contains("401"),+ "the status still gets through: {text}"+ );+}++// ─────────────────────────────────────────────────────────────────────────────+// 4. Review remediation (PR #946): provider answers that are partial, wrong or+// duplicated must never produce a response that contradicts itself.+// ─────────────────────────────────────────────────────────────────────────────++/// Twelve documents with distinct titles and one common body term, so the+/// engine returns all of them and the judge can tell them apart.+async fn seed_dozen(node: &Node) {+ let docs: Vec<(String, String)> = (0..12)+ .map(|i| (format!("{i}"), format!("doc number {i}")))+ .collect();+ let seeded: Vec<(&str, &str, &str, &str, i64)> = docs+ .iter()+ .map(|(id, title)| (id.as_str(), title.as_str(), "common term here", "c", 1))+ .collect();+ node.seed("dozen", &seeded).await;+}++fn scores_of(r: &Value) -> Vec<Option<f64>> {+ r["hits"]["hits"]+ .as_array()+ .unwrap()+ .iter()+ .map(|h| h["_score"].as_f64())+ .collect()+}++/// The provider answers half the window. The response must still be one a+/// client can read without contradiction: every `_score` is a probability or+/// `null`, never the engine's BM25 next to probabilities; `max_score` is the+/// maximum of every score on the page; scores descend; unjudged hits are+/// counted and sort last.+#[tokio::test]+async fn a_partial_verdict_set_keeps_score_single_kind_on_the_wire() {+ let stub = Stub::start().await;+ stub.fault(Fault::Partial);+ stub.by_title(&[+ ("doc number 0", 0.9),+ ("doc number 1", 0.8),+ ("doc number 2", 0.7),+ ("doc number 3", 0.6),+ ("doc number 4", 0.5),+ ("doc number 5", 0.4),+ ("doc number 6", 0.3),+ ("doc number 7", 0.2),+ ("doc number 8", 0.1),+ ]);+ let node = node(&stub).await;+ seed_dozen(&node).await;++ let (st, r) = node+ .search(+ "/dozen/_search",+ json!({"query": {"match": {"body": "common"}}, "size": 12, "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ let rr = &r["_rerank"];+ assert_eq!(rr["applied"], true, "{rr}");+ assert_eq!(rr["score_kind"], "probability", "{rr}");+ assert_eq!(rr["window"], 12, "{rr}");+ assert_eq!(rr["judged"], 6, "{rr}");+ assert_eq!(rr["unjudged"], 6, "{rr}");+ assert_eq!(rr["skipped_no_text"], 0, "{rr}");++ let scores = scores_of(&r);+ assert_eq!(scores.len(), 12, "{r}");+ let judged: Vec<f64> = scores.iter().flatten().copied().collect();+ assert_eq!(judged.len(), 6, "six probabilities, six nulls: {scores:?}");+ assert!(+ scores[..6].iter().all(Option::is_some) && scores[6..].iter().all(Option::is_none),+ "judged hits first, unjudged after, nothing interleaved: {scores:?}"+ );+ for p in &judged {+ assert!(+ (0.0..=1.0).contains(p),+ "a BM25 value leaked into _score: {scores:?}"+ );+ }+ assert!(+ judged.windows(2).all(|w| w[0] >= w[1]),+ "_score must not rise down the page: {scores:?}"+ );+ let max = r["hits"]["max_score"]+ .as_f64()+ .expect("max_score is a number");+ assert!(+ judged.iter().all(|p| *p <= max),+ "max_score {max} is below a later hit's _score: {scores:?}"+ );+ assert_eq!(+ Some(max),+ scores[0],+ "max_score is the top hit's score: {r}"+ );+ // `hits.total` is still the engine's: twelve matched, whatever was judged.+ assert_eq!(r["hits"]["total"]["value"], 12, "{r}");+}++/// The provider answers, but only about documents nobody sent. That is not a+/// reranked order, and the response must not say it is — nor may the+/// operator's meter count it as applied.+#[tokio::test]+async fn verdicts_only_for_unknown_keys_degrade_instead_of_claiming_the_judges_order() {+ let stub = Stub::start().await;+ stub.fault(Fault::WrongKeys);+ let node = node(&stub).await;+ node.seed_kb().await;++ let (_, base) = node+ .search("/kb/_search", json!({"query": match_q(), "size": 3}))+ .await;+ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 3, "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ let rr = &r["_rerank"];+ assert_eq!(+ rr["applied"], false,+ "verdicts for d999 and x are not a reranking: {rr}"+ );+ assert_eq!(rr["score_kind"], "engine", "{rr}");+ let why = rr["reason"].as_str().unwrap_or_default();+ assert!(+ why.contains("no scores") || why.contains("no verdict"),+ "the reason says nothing came back for what was sent: {rr}"+ );+ assert!(+ rr.get("judged").is_none(),+ "a degraded block has no `judged`: {rr}"+ );+ assert_eq!(ids(&r), ids(&base), "the engine's order, untouched: {r}");+ assert_eq!(+ scores_of(&r),+ scores_of(&base),+ "and the engine's scores: {r}"+ );+ assert_eq!(stub.calls().len(), 1);++ let (_, text) = node.raw("GET", "/v1/metrics", String::new()).await;+ let value = |needle: &str| -> Option<f64> {+ text.lines()+ .find(|l| l.starts_with(needle))+ .and_then(|l| l.rsplit(' ').next())+ .and_then(|v| v.parse().ok())+ };+ assert_eq!(+ value(r#"xerj_rerank_requests_total{outcome="degraded"}"#),+ Some(1.0),+ "{text}"+ );+ assert_ne!(+ value(r#"xerj_rerank_requests_total{outcome="applied"}"#),+ Some(1.0),+ "must not be counted as applied: {text}"+ );+ assert_eq!(+ value("xerj_rerank_documents_judged_total").unwrap_or(0.0),+ 0.0,+ "nothing was judged, nothing is billed: {text}"+ );+}++/// A provider that echoes `d0..d59` back on every call must not turn a+/// 35-document window into 105 judged documents (three verdicts per hit, the+/// last one winning) or triple the billing meter. One verdict per document,+/// from the batch that sent it.+#[tokio::test]+async fn a_verdict_echoed_from_another_batch_is_not_counted_twice() {+ let stub = Stub::start().await;+ stub.fault(Fault::EchoOtherBatches);+ let node = node(&stub).await;+ let docs: Vec<(String, String)> = (0..35)+ .map(|i| (format!("{i}"), format!("doc number {i}")))+ .collect();+ let seeded: Vec<(&str, &str, &str, &str, i64)> = docs+ .iter()+ .map(|(id, title)| (id.as_str(), title.as_str(), "common term here", "c", 1))+ .collect();+ node.seed("many", &seeded).await;+ stub.by_title(&[("doc number 33", 0.95)]);++ let (st, r) = node+ .search(+ "/many/_search",+ json!({"query": {"match": {"body": "common"}}, "size": 35,+ "rerank": {"window": 35, "batch": 10}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(stub.calls().len(), 4, "35 documents in batches of 10");+ let rr = &r["_rerank"];+ assert_eq!(rr["applied"], true, "{rr}");+ assert_eq!(rr["window"], 35, "{rr}");+ assert_eq!(+ rr["judged"], 35,+ "one verdict per document, not per echo: {rr}"+ );+ assert_eq!(rr["unjudged"], 0, "{rr}");+ assert_eq!(+ ids(&r)[0],+ "33",+ "the real verdict wins over the echoed 0.5: {r}"+ );+ let scores = scores_of(&r);+ assert!(scores.iter().all(Option::is_some), "{scores:?}");+ assert_eq!(scores[0], Some(0.95), "{scores:?}");++ let (_, text) = node.raw("GET", "/v1/metrics", String::new()).await;+ let judged_total = text+ .lines()+ .find(|l| l.starts_with("xerj_rerank_documents_judged_total"))+ .and_then(|l| l.rsplit(' ').next())+ .and_then(|v| v.parse::<f64>().ok());+ assert_eq!(+ judged_total,+ Some(35.0),+ "the meter counts documents: {text}"+ );+}++/// The deadline stops batches from ever being sent. Those documents are as+/// unjudged as the ones in a batch that failed, and the block must account+/// for the whole window: `judged` plus `partial_failures` batches.+#[tokio::test]+async fn batches_the_deadline_never_dispatched_count_as_partial_failures() {+ let stub = Stub::start().await;+ stub.fault(Fault::Delay(Duration::from_millis(1000)));+ let node = node(&stub).await;+ let docs: Vec<(String, String)> = (0..35)+ .map(|i| (format!("{i}"), format!("doc number {i}")))+ .collect();+ let seeded: Vec<(&str, &str, &str, &str, i64)> = docs+ .iter()+ .map(|(id, title)| (id.as_str(), title.as_str(), "common term here", "c", 1))+ .collect();+ node.seed("many", &seeded).await;++ // Batch 1 answers at ~1.0 s; batch 2 is sent with ~0.8 s left and times+ // out; batches 3 and 4 are never sent.+ let (st, r) = node+ .search(+ "/many/_search",+ json!({"query": {"match": {"body": "common"}}, "size": 5,+ "rerank": {"window": 35, "batch": 10, "max_concurrency": 1,+ "timeout_ms": 1800}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ let rr = &r["_rerank"];+ assert_eq!(+ rr["applied"], true,+ "one batch answered, so its order stands: {rr}"+ );+ assert_eq!(rr["window"], 35, "{rr}");+ assert_eq!(rr["judged"], 10, "{rr}");+ assert_eq!(rr["unjudged"], 25, "{rr}");+ assert_eq!(+ rr["partial_failures"], 3,+ "one batch timed out and two were never dispatched: {rr}"+ );+ assert_eq!(stub.calls().len(), 2, "batches 3 and 4 were never sent");+}++/// A scroll continuation that carries `rerank` is refused like the `?scroll=`+/// open is; and `"rerank": null` means "no rerank" on every surface, so an+/// `_msearch` item and a `_search` body with it behave the same way.+#[tokio::test]+async fn scroll_continuation_refuses_rerank_and_null_is_absent_everywhere() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;++ let (st, opened) = node+ .search(+ "/kb/_search?scroll=1m",+ json!({"query": match_q(), "size": 2}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{opened}");+ let scroll_id = opened["_scroll_id"]+ .as_str()+ .expect("scroll id")+ .to_string();+ let (st, r) = node+ .search(+ "/_search/scroll",+ json!({"scroll": "1m", "scroll_id": scroll_id, "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::BAD_REQUEST, "_search/scroll: {r}");+ assert!(reason(&r).contains("_search/scroll"), "{r}");+ let (st, r) = node+ .search(+ "/_search/scroll",+ json!({"scroll": "1m", "scroll_id": scroll_id}),+ )+ .await;+ assert_eq!(+ st,+ StatusCode::OK,+ "the continuation itself still works: {r}"+ );++ let (st, r) = node+ .search("/kb/_search", json!({"query": match_q(), "rerank": null}))+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert!(r.get("_rerank").is_none(), "null is absent: {r}");++ let ndjson = format!(+ "{}\n{}\n",+ json!({"index": "kb"}),+ json!({"query": match_q(), "size": 2, "rerank": null}),+ );+ let (st, text) = node.raw("POST", "/_msearch", ndjson).await;+ assert_eq!(st, StatusCode::OK, "{text}");+ let r: Value = serde_json::from_str(&text).unwrap();+ assert_eq!(+ r["responses"][0]["hits"]["hits"].as_array().map(Vec::len),+ Some(2),+ "null is absent on _msearch too, not a refused block: {r}"+ );+ assert!(stub.calls().is_empty());+}++/// A point-in-time search has no `sort` or `search_after` of its own, so the+/// stage runs on it; paging it with `search_after` is refused like any other.+#[tokio::test]+async fn a_pit_search_runs_the_stage() {+ let stub = Stub::start().await;+ inverted(&stub);+ let node = node(&stub).await;+ node.seed_kb().await;++ let (st, pit) = node.call("POST", "/kb/_pit?keep_alive=1m", json!({})).await;+ assert_eq!(st, StatusCode::OK, "{pit}");+ let id = pit["id"].as_str().expect("pit id").to_string();++ let (st, r) = node+ .search(+ "/_search",+ json!({"pit": {"id": id, "keep_alive": "1m"}, "query": match_q(), "size": 4,+ "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["applied"], true, "{r}");+ assert_eq!(ids(&r), INVERTED_IDS, "{r}");+}++/// The stage widens the engine's page to `rerank.window`. On an index whose+/// `index.max_result_window` is smaller, the engine's own check fired with+/// "from + size > max_result_window" — numbers the caller never sent. The+/// refusal is now in the caller's terms, per index, and a wildcard search that+/// touches such an index names it.+#[tokio::test]+async fn a_small_max_result_window_is_refused_in_the_callers_terms() {+ let stub = Stub::start().await;+ inverted(&stub);+ let node = node(&stub).await;+ node.seed_kb().await;+ let (st, b) = node+ .call(+ "PUT",+ "/kbsmall",+ json!({"settings": {"index": {"max_result_window": 20}},+ "mappings": {"properties": {"title": {"type": "text"}, "body": {"type": "text"}}}}),+ )+ .await;+ assert!(st.is_success(), "{b}");+ node.call(+ "PUT",+ "/kbsmall/_doc/1",+ json!({"title": "t", "body": "vitamin d"}),+ )+ .await;+ node.call("POST", "/kbsmall/_refresh", json!({})).await;++ let (st, r) = node+ .search(+ "/kbsmall/_search",+ json!({"query": match_q(), "size": 5, "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::BAD_REQUEST, "{r}");+ let why = reason(&r);+ assert!(why.contains("`rerank.window` (30)"), "{why}");+ assert!(why.contains("`index.max_result_window` (20)"), "{why}");+ assert!(why.contains("kbsmall"), "{why}");+ assert!(+ !why.contains("from + size"),+ "not the engine's page arithmetic: {why}"+ );++ // The wildcard search names the index that cannot serve the window.+ let (st, r) = node+ .search(+ "/kb*/_search",+ json!({"query": match_q(), "size": 5, "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::BAD_REQUEST, "{r}");+ assert!(reason(&r).contains("kbsmall"), "{r}");++ // A window that fits works, on the small index and across the wildcard.+ let (st, r) = node+ .search(+ "/kb*/_search",+ json!({"query": match_q(), "size": 5, "rerank": {"window": 20}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["applied"], true, "{r}");+ assert!(+ stub.calls().len() == 1,+ "only the fitting request reached the provider"+ );+}++// ─────────────────────────────────────────────────────────────────────────────+// 6. Second review round (PR #946 at 2cc53f60)+// ─────────────────────────────────────────────────────────────────────────────++/// Forty matching documents, for windows wider than one provider call.+async fn seed_forty(node: &Node) {+ let docs: Vec<(String, String, String)> = (0..40)+ .map(|i| {+ (+ i.to_string(),+ format!("Doc {i}"),+ format!("vitamin d supplementation bone density note {i}"),+ )+ })+ .collect();+ let refs: Vec<(&str, &str, &str, &str, i64)> = docs+ .iter()+ .enumerate()+ .map(|(i, (id, t, b))| (id.as_str(), t.as_str(), b.as_str(), "c", i as i64))+ .collect();+ node.seed("kb40", &refs).await;+}++/// MAJOR (review round 2): `rerank.instructions` had no ceiling, and the wire+/// format repeats it inside every per-document question. One ~1 MB string at+/// `window: 40`, `max_doc_chars: 1` put 40 MB on the wire to the provider for a+/// `size: 1` search. It is a caller-chosen cost knob like the window, so it+/// gets a server-side ceiling like the window: a 400 that names the limit, and+/// nothing sent.+#[tokio::test]+async fn oversized_instructions_query_and_model_are_refused_and_send_nothing() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ seed_forty(&node).await;++ let big = "y".repeat(1_000_000);+ for (name, body, needle) in [+ (+ "instructions",+ json!({"query": match_q(), "size": 1,+ "rerank": {"window": 40, "max_doc_chars": 1, "instructions": big}}),+ "rerank.instructions",+ ),+ (+ "rerank.query",+ json!({"query": match_q(), "size": 1, "rerank": {"query": big}}),+ "rerank.query",+ ),+ (+ "model",+ json!({"query": match_q(), "size": 1, "rerank": {"model": big}}),+ "rerank.model",+ ),+ (+ // The question read out of the search query is the same string on+ // the same wire, so it has the same ceiling — and the refusal says+ // what to do about it.+ "inferred question",+ json!({"query": {"match": {"body": big}}, "size": 1, "rerank": {}}),+ "rerank.query",+ ),+ ] {+ let (st, r) = node.search("/kb40/_search", body).await;+ assert_eq!(st, StatusCode::BAD_REQUEST, "{name}: {}", reason(&r));+ let why = reason(&r);+ assert!(why.contains(needle), "{name}: {why}");+ assert!(+ why.contains("characters"),+ "{name}: the refusal names the limit: {why}"+ );+ assert!(+ why.len() < 600,+ "{name}: the refusal must not echo the oversized value ({} bytes)",+ why.len()+ );+ }+ assert!(+ stub.calls().is_empty(),+ "a refused request must not have sent a byte to the provider"+ );++ // At the ceiling it runs, and what one search can put on the wire is+ // bounded by the documented ceilings alone: 40 documents of 1 character+ // each, under instructions at the limit.+ let (st, r) = node+ .search(+ "/kb40/_search",+ json!({"query": match_q(), "size": 1,+ "rerank": {"window": 40, "max_doc_chars": 1,+ "instructions": "y".repeat(xerj_rerank::MAX_INSTRUCTIONS_CHARS)}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["applied"], true, "{r}");+ assert_eq!(r["_rerank"]["judged"], 40, "{r}");+ let sent: usize = stub.calls().iter().map(|c| c.body.to_string().len()).sum();+ let bound = 40 * (xerj_rerank::MAX_INSTRUCTIONS_CHARS + 2 + 512) + 2 * 1_024;+ assert!(+ sent <= bound,+ "{sent} bytes left the node for a 40-document window of 1-character documents; \+ the ceilings allow {bound}"+ );+}++/// `_rank_eval` builds its own search from `requests[].request` and the parser+/// ignores keys it does not know, so a `rerank` block vanished and the metric+/// was computed over the engine's order — "reranking does not help", under a+/// 200, on the one endpoint whose job is measuring ranking quality. It is+/// recorded per request in `failures` (the channel `_rank_eval` has for a+/// request it cannot run), the rest of the batch is evaluated, and nothing is+/// sent to the provider.+#[tokio::test]+async fn rank_eval_refuses_a_rerank_block_instead_of_scoring_the_engines_order() {+ let stub = Stub::start().await;+ inverted(&stub);+ let node = node(&stub).await;+ node.seed_kb().await;++ let ratings = json!([{"_index": "kb", "_id": "3", "rating": 1}]);+ let (st, r) = node+ .call(+ "POST",+ "/kb/_rank_eval",+ json!({+ "requests": [+ {"id": "with_rerank",+ "request": {"query": match_q(), "rerank": {"window": 30}},+ "ratings": ratings},+ {"id": "plain", "request": {"query": match_q()}, "ratings": ratings},+ {"id": "null_is_absent",+ "request": {"query": match_q(), "rerank": null},+ "ratings": ratings}+ ],+ "metric": {"precision": {"k": 1}}+ }),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ let failure = &r["failures"]["with_rerank"];+ assert!(+ failure.is_object(),+ "a request carrying `rerank` must be reported in `failures`, not scored: {r}"+ );+ let why = failure["reason"].as_str().unwrap_or_default();+ assert!(why.contains("rerank"), "{r}");+ assert!(why.contains("_rank_eval"), "{r}");+ assert_eq!(+ failure["type"], "illegal_argument_exception",+ "the same refusal every other surface gives, not a failed search: {r}"+ );+ assert!(+ r["details"]["with_rerank"].is_null(),+ "no metric may be published for it: {r}"+ );+ // The siblings ran, on the engine's order, and say so by being in details.+ assert!(r["details"]["plain"].is_object(), "{r}");+ assert!(r["details"]["null_is_absent"].is_object(), "{r}");+ assert!(r["failures"]["plain"].is_null(), "{r}");+ assert!(r["failures"]["null_is_absent"].is_null(), "{r}");+ assert!(+ stub.calls().is_empty(),+ "_rank_eval does not run the stage, so nothing may reach the provider"+ );+}++/// The interaction table says the judge reads what the response returns, and+/// `prepare`'s own refusal says "return the text through `fields`". In the+/// default mode (no `rerank.fields`) the stage read `_source` and `_passage`+/// only, so following that advice produced a second 400.+#[tokio::test]+async fn text_returned_only_through_fields_is_judged_without_naming_it() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ node.seed_kb().await;++ // `fields`+ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "_source": false,+ "fields": ["body"], "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["applied"], true, "{r}");+ assert_eq!(r["_rerank"]["judged"], 4, "{r}");+ let sent = stub.everything_sent();+ assert!(sent.contains("supplementation"), "{sent}");+ assert!(+ !sent.contains("Trial results"),+ "the title was not returned, so it was not sent: {sent}"+ );++ // `docvalue_fields`: the keyword value is the only text on the hit.+ let stub2 = Stub::start().await;+ let node2 = crate::node(&stub2).await;+ node2.seed_kb().await;+ let (st, r) = node2+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "_source": false,+ "docvalue_fields": ["cat"],+ "rerank": {"query": "which category is a trial?"}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["judged"], 4, "{r}");+ let sent = stub2.everything_sent();+ assert!(sent.contains("trial"), "{sent}");+ assert!(!sent.contains("supplementation"), "{sent}");++ // A field that is in BOTH `_source` and `fields` is sent once, not twice.+ let stub3 = Stub::start().await;+ let node3 = crate::node(&stub3).await;+ node3.seed_kb().await;+ let (st, r) = node3+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "fields": ["body"], "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ for call in stub3.calls() {+ for (_, doc) in call.body["state"]["documents"].as_object().unwrap() {+ let text = doc["text"].as_str().unwrap_or_default();+ assert!(+ text.matches("vitamin rich vegetables").count() <= 1+ && text.matches("bone china").count() <= 1,+ "a value present in `_source` and `fields` was sent twice: {text}"+ );+ }+ }+}++/// `usage` is advisory metering. A provider that reports a negative or+/// otherwise odd token count must not cost the caller a ranking whose verdicts+/// were all valid (it used to be a 502 with no hits).+#[tokio::test]+async fn odd_provider_usage_does_not_veto_a_valid_ranking() {+ let stub = Stub::start().await;+ inverted(&stub);+ stub.fault(Fault::OddUsage);+ let node = node(&stub).await;+ node.seed_kb().await;++ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ assert_eq!(r["_rerank"]["applied"], true, "{r}");+ assert_eq!(r["_rerank"]["judged"], 4, "{r}");+ assert_eq!(ids(&r), INVERTED_IDS, "{r}");+ // Neither count is one a call can produce (`-5`; 2^63, far past what 30+ // documents at the ceilings hold), so both meter as zero — and the+ // operator's Prometheus counter is not poisoned by one bad response.+ assert_eq!(r["_rerank"]["usage"]["output_tokens"], 0, "{r}");+ assert_eq!(r["_rerank"]["usage"]["input_tokens"], 0, "{r}");+ let (_, metrics) = node.raw("GET", "/v1/metrics", String::new()).await;+ let input_line = metrics+ .lines()+ .find(|l| l.starts_with("xerj_rerank_provider_tokens_total{kind=\"input\"}"))+ .unwrap_or_default()+ .to_string();+ assert!(+ input_line.is_empty() || input_line.ends_with(" 0"),+ "an implausible token count reached the operator's meter: {input_line}"+ );+}++/// The provider is a third party. A legitimate answer for 30 documents is a+/// few kilobytes; a response of megabytes is not one, and reading it whole+/// would let whatever answers at the endpoint choose how much memory a search+/// allocates. It is a contract break — 502, no hits — not an allocation.+#[tokio::test]+async fn an_oversized_provider_response_is_a_502_not_an_allocation() {+ let stub = Stub::start().await;+ stub.fault(Fault::HugeBody);+ let node = node(&stub).await;+ node.seed_kb().await;++ let (st, r) = node+ .search(+ "/kb/_search",+ json!({"query": match_q(), "size": 4, "rerank": {}}),+ )+ .await;+ assert_eq!(st, StatusCode::BAD_GATEWAY, "{r}");+ assert_eq!(r["error"]["type"], "rerank_exception", "{r}");+ let why = reason(&r);+ assert!(why.contains("larger than"), "{why}");+ assert!(r["hits"].is_null(), "a surfaced fault returns no hits: {r}");+}++/// Each page request judges the window again — there is no verdict cache. The+/// docs say so; this pins the fact they state, so a future cache changes both.+#[tokio::test]+async fn every_page_request_judges_the_whole_window_again() {+ let stub = Stub::start().await;+ let node = node(&stub).await;+ seed_forty(&node).await;++ let mut judged = 0u64;+ for from in [0, 10, 20] {+ let (st, r) = node+ .search(+ "/kb40/_search",+ json!({"query": match_q(), "size": 10, "from": from, "rerank": {"window": 30}}),+ )+ .await;+ assert_eq!(st, StatusCode::OK, "{r}");+ judged += r["_rerank"]["judged"].as_u64().unwrap_or(0);+ }+ assert_eq!(stub.calls().len(), 3, "one provider call per page request");+ assert_eq!(+ judged, 90,+ "three pages of one 30-document window pay for 90"+ );+}diff --git a/engine/crates/xerj-common/src/config.rs b/engine/crates/xerj-common/src/config.rsindex 8e3eaf264..fdb3ed173 100644--- a/engine/crates/xerj-common/src/config.rs+++ b/engine/crates/xerj-common/src/config.rs@@ -1,6 +1,6 @@//! xerj configuration system.//!-//! Configuration is intentionally minimal: **117 settings** versus+//! Configuration is intentionally minimal: **120 settings** versus//! Elasticsearch's 3000+. Every option is named, documented, and has a sensible//! production-ready default. The format is TOML, loaded from a single file.//!@@ -95,9 +95,11 @@ pub struct Config {/// Single-node WAL tap: push a filtered index subset to an external/// ES-compatible target — 10 settings. Off by default.pub wal_tap: WalTapConfig,+ /// Second-stage reranking provider — 3 settings. Inert until a key is set.+ pub rerank: RerankProviderConfig,}-// 21 sub-configs, 117 leaf settings in total. Do not maintain that sum by hand+// 22 sub-configs, 120 leaf settings in total. Do not maintain that sum by hand// — `journey_zero_config` in xerj-engine/tests/product_experience.rs counts a// serialised `Config::default()` and fails if this comment and the module// header stop matching. `Default` is derived: every field is a sub-config that@@ -207,6 +209,12 @@ impl Config {return Err(XerjError::config(reason));}+ // Rerank: the endpoint is where document text gets POSTed, and it is+ // echoed by `GET /_xerj/rerank`. Same two rules as the WAL tap target.+ if let Err(reason) = RerankProviderConfig::check_endpoint(&self.rerank.endpoint) {+ return Err(XerjError::config(reason));+ }+// Compression: block_size_docs is documented as 16–4096 and was// accepted at any value (#318 got `999999` past startup in silence).// Range-check it even though the knob is dormant: an out-of-range@@ -455,7 +463,7 @@ impl Config {}// ═════════════════════════════════════════════════════════════════════════════-// Sub-configs (117 user-facing settings total; counted by+// Sub-configs (120 user-facing settings total; counted by// `journey_zero_config`, not by hand)// ═════════════════════════════════════════════════════════════════════════════@@ -2133,6 +2141,153 @@ impl Default for SearchContextConfig {}}+// ─────────────────────────────────────────────────────────────────────────────+// Rerank provider+// ─────────────────────────────────────────────────────────────────────────────++/// Second-stage reranking provider (`"rerank": {...}` on `_search`).+///+/// **3 settings.**+///+/// Reranking is the only search-time feature that sends document text off+/// this node: the text of the top-N hits is POSTed to a third-party judge.+/// (Two other features send text off the node, both operator configuration:+/// `[embedding] default_endpoint` — document text at write time, query text at+/// search time — and the WAL tap. The complete list of outbound connections,+/// including the neural model download and cluster Raft traffic, is in+/// `docs/RERANK.md`, "Every way data leaves a XERJ node".)+/// It therefore does nothing until an operator supplies a key, and it can be+/// forbidden outright.+///+/// The key can come from here or from the `TYPESAFE_API_KEY` environment+/// variable; the endpoint from here or `TYPESAFE_ENDPOINT`. **An explicit value+/// in this file wins over the environment** — the same rule as+/// `cluster.auth_secret`, for the same reason: a stray variable must not+/// silently re-point where document text goes. The resolution itself lives in+/// `xerj_rerank::ProviderSettings::resolve`, which is a pure function so it can+/// be tested without touching process-wide environment state.+///+/// The endpoint is validated wherever it came from: the config value in+/// [`Config::validate`], the environment value in [`Self::check_environment`]+/// at boot. The first draft validated only the file, so+/// `TYPESAFE_ENDPOINT=http://user:pw@host/…` started a node whose docs said it+/// would refuse — and `reqwest` would have sent that userinfo as Basic auth+/// beside the bearer key.+///+/// There is deliberately no per-request key. A key in a search body would land+/// in slow-query logs, audit trails and client-side request dumps.+#[derive(Clone, Serialize, Deserialize)]+#[serde(default, deny_unknown_fields)]+pub struct RerankProviderConfig {+ /// `false` refuses every `rerank` request with a 403 and guarantees no+ /// document text leaves the host through this feature, whatever the+ /// environment holds (default: `true` — but inert without a key).+ pub enabled: bool,+ /// Provider API key (default: empty → fall back to `TYPESAFE_API_KEY`).+ /// Never echoed by any endpoint; `GET /_xerj/rerank` reports only whether+ /// one is set and where it came from.+ pub api_key: String,+ /// Provider endpoint (default: empty → `TYPESAFE_ENDPOINT`, then+ /// `https://api.typesafe.ai/v1/systemone`). Must be an absolute+ /// `http(s)://` URL without credentials in it.+ pub endpoint: String,+}++impl Default for RerankProviderConfig {+ fn default() -> Self {+ Self {+ enabled: true,+ api_key: String::new(),+ endpoint: String::new(),+ }+ }+}++/// Hand-written so `{:?}` on a `Config` can never put the key in a log line.+impl std::fmt::Debug for RerankProviderConfig {+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {+ f.debug_struct("RerankProviderConfig")+ .field("enabled", &self.enabled)+ .field(+ "api_key",+ &if self.api_key.trim().is_empty() {+ "<unset>"+ } else {+ "<redacted>"+ },+ )+ .field("endpoint", &redact_url_userinfo(&self.endpoint))+ .finish()+ }+}++impl RerankProviderConfig {+ /// The environment variable the endpoint falls back to when the config+ /// field is empty. Mirrors `xerj_rerank::JevProvider::ENV_ENDPOINT`; this+ /// crate does not link `xerj-rerank`, and the two are pinned equal by a+ /// test in `xerj-api`, which links both.+ pub const ENV_ENDPOINT: &'static str = "TYPESAFE_ENDPOINT";++ /// The endpoint the node will actually use, given the config field and the+ /// environment fallback — the same precedence `ProviderSettings::resolve`+ /// applies — checked by [`Self::check_endpoint`], with the failure naming+ /// where the bad value came from. Pure, so it is testable without+ /// mutating the process environment.+ pub fn check_resolved_endpoint(+ config_endpoint: &str,+ env_endpoint: Option<&str>,+ ) -> Result<(), String> {+ if !config_endpoint.trim().is_empty() {+ return Self::check_endpoint(config_endpoint);+ }+ match env_endpoint {+ Some(v) if !v.trim().is_empty() => Self::check_endpoint(v)+ .map_err(|reason| format!("{}: {reason}", Self::ENV_ENDPOINT)),+ _ => Ok(()),+ }+ }++ /// Boot-time check of the environment fallback. Separate from+ /// [`Config::validate`], which must stay a pure function of the file so+ /// config tests do not depend on the developer's shell.+ pub fn check_environment(&self) -> Result<(), String> {+ let env = std::env::var(Self::ENV_ENDPOINT).ok();+ Self::check_resolved_endpoint(&self.endpoint, env.as_deref())+ }++ /// An absolute `http(s)://` URL with no userinfo, or empty.+ ///+ /// Userinfo is refused for the reason `wal_tap.target_url` refuses it:+ /// `reqwest` turns `user:pass@host` into an `Authorization` header, so it is+ /// a credential in a field that a status endpoint echoes.+ pub fn check_endpoint(url: &str) -> Result<(), String> {+ let trimmed = url.trim();+ if trimmed.is_empty() {+ return Ok(());+ }+ let Some(rest) = trimmed+ .strip_prefix("http://")+ .or_else(|| trimmed.strip_prefix("https://"))+ else {+ return Err(+ "rerank.endpoint must be an absolute http:// or https:// URL, e.g. \+ \"https://api.typesafe.ai/v1/systemone\""+ .to_string(),+ );+ };+ let authority = rest.split(['/', '?', '#']).next().unwrap_or_default();+ if authority.contains('@') {+ return Err(+ "rerank.endpoint must not carry credentials in the URL (user:password@host): \+ the endpoint is echoed by GET /_xerj/rerank. Put the credential in \+ rerank.api_key, which is never echoed."+ .to_string(),+ );+ }+ Ok(())+ }+}+// ═════════════════════════════════════════════════════════════════════════════// Tests// ═════════════════════════════════════════════════════════════════════════════@@ -2141,6 +2296,26 @@ impl Default for SearchContextConfig {mod tests {use super::*;+ /// The documented refusal applies to the value the node will USE, from+ /// whichever place it came. Config wins, so a good config value shadows+ /// a bad environment one; with no config value the environment is checked.+ #[test]+ fn rerank_endpoint_is_checked_from_the_environment_too() {+ let bad = "http://user:[email protected]:9/v1/systemone";+ let good = "https://api.typesafe.ai/v1/systemone";+ let err = RerankProviderConfig::check_resolved_endpoint("", Some(bad)).unwrap_err();+ assert!(err.starts_with("TYPESAFE_ENDPOINT: "), "{err}");+ assert!(err.contains("must not carry credentials"), "{err}");+ assert!(RerankProviderConfig::check_resolved_endpoint("", Some("ftp://x")).is_err());+ assert!(RerankProviderConfig::check_resolved_endpoint("", Some(good)).is_ok());+ assert!(RerankProviderConfig::check_resolved_endpoint("", Some(" ")).is_ok());+ assert!(RerankProviderConfig::check_resolved_endpoint("", None).is_ok());+ // The config value is in force, so the environment value is not used+ // and not judged.+ assert!(RerankProviderConfig::check_resolved_endpoint(good, Some(bad)).is_ok());+ assert!(RerankProviderConfig::check_resolved_endpoint(bad, Some(good)).is_err());+ }+#[test]fn default_config_is_valid() {Config::default()@@ -2400,7 +2575,7 @@ mod tests {drift.join("\n "));- // …and the file's own header quotes how many of the 117 it sets. That+ // …and the file's own header quotes how many of the 120 it sets. That// number was 38, then 56, and never once the truth (#207), so count the// assignments instead of trusting the sentence.let set_here = toml_src@@ -2854,6 +3029,7 @@ mod tests {("compat", 2),("lifecycle", 1),("wal_tap", 10),+ ("rerank", 3),];/// Count the settings by *counting them*.@@ -2896,7 +3072,7 @@ mod tests {"the section table must sum to the whole config");assert_eq!(- total, 117,+ total, 120,"the total settings count changed. It is quoted in this module's \header, in xerj-common/src/lib.rs, in engine/README.md, in \xerj.default.toml and in EXPECTED_SETTINGS in \diff --git a/engine/crates/xerj-common/src/lib.rs b/engine/crates/xerj-common/src/lib.rsindex e60f4d636..e6887a416 100644--- a/engine/crates/xerj-common/src/lib.rs+++ b/engine/crates/xerj-common/src/lib.rs@@ -7,13 +7,13 @@//! ## Design philosophy//!//! Unlike Elasticsearch's 3000+ configuration knobs, xerj deliberately exposes-//! **115 settings**, each meaningful and production-tested. Every default+//! **120 settings**, each meaningful and production-tested. Every default//! is chosen so that a fresh deployment with zero configuration changes performs//! well for the majority of workloads.//!//! ## Modules//!-//! - [`config`] — TOML-based configuration (115 settings)+//! - [`config`] — TOML-based configuration (120 settings)//! - [`feedback`] — the bug/UX-report invitation shared by every `--help`//! - [`error`] — Unified error type ([`XerjError`])//! - [`types`] — Core domain types (documents, fields, IDs)diff --git a/engine/crates/xerj-common/src/metrics.rs b/engine/crates/xerj-common/src/metrics.rsindex 1c267f582..229652c1e 100644--- a/engine/crates/xerj-common/src/metrics.rs+++ b/engine/crates/xerj-common/src/metrics.rs@@ -120,6 +120,20 @@ pub struct Metrics {pub query_cache_hits: IntGauge,/// Cumulative internal query-result cache misses across all indices.pub query_cache_misses: IntGauge,++ // ── Rerank stage ──────────────────────────────────────────────────────────+ // Reranking is a search feature the operator pays a third party for, per+ // judged document, on a volume the CALLER chooses. These are the meter.+ /// `rerank` requests by outcome: `applied` (the judge's order was served),+ /// `degraded` (deadline missed, engine order served under a 200),+ /// `refused` (400 / 403 / 503 — nothing was sent to the provider) and+ /// `failed` (502 — the provider was called and the contract broke).+ pub rerank_requests: IntCounterVec,+ /// Documents the provider returned a verdict for. The billing unit.+ pub rerank_documents_judged: IntCounter,+ /// Tokens the provider reported, by `kind` (`input` / `output`). Zero when+ /// the provider omits usage — this is its meter reading, not an estimate.+ pub rerank_provider_tokens: IntCounterVec,}impl Metrics {@@ -278,6 +292,31 @@ impl Metrics {)).map_err(|e| XerjError::internal(format!("metrics: {e}")))?;+ let rerank_requests = IntCounterVec::new(+ Opts::new(+ "xerj_rerank_requests_total",+ "Searches that carried a `rerank` block, by outcome \+ (applied, degraded, refused, failed)",+ ),+ &["outcome"],+ )+ .map_err(|e| XerjError::internal(format!("metrics: {e}")))?;++ let rerank_documents_judged = IntCounter::with_opts(Opts::new(+ "xerj_rerank_documents_judged_total",+ "Documents the rerank provider returned a verdict for",+ ))+ .map_err(|e| XerjError::internal(format!("metrics: {e}")))?;++ let rerank_provider_tokens = IntCounterVec::new(+ Opts::new(+ "xerj_rerank_provider_tokens_total",+ "Tokens the rerank provider reported, by kind (input, output)",+ ),+ &["kind"],+ )+ .map_err(|e| XerjError::internal(format!("metrics: {e}")))?;+// ── Register everything ───────────────────────────────────────────────macro_rules! reg {($metric:expr) => {@@ -306,6 +345,9 @@ impl Metrics {reg!(wal_size_bytes);reg!(query_cache_hits);reg!(query_cache_misses);+ reg!(rerank_requests);+ reg!(rerank_documents_judged);+ reg!(rerank_provider_tokens);Ok(Self {registry,@@ -328,6 +370,9 @@ impl Metrics {wal_size_bytes,query_cache_hits,query_cache_misses,+ rerank_requests,+ rerank_documents_judged,+ rerank_provider_tokens,})}diff --git a/engine/crates/xerj-engine/tests/product_experience.rs b/engine/crates/xerj-engine/tests/product_experience.rsindex 295ca8c4e..ce82bc807 100644--- a/engine/crates/xerj-engine/tests/product_experience.rs+++ b/engine/crates/xerj-engine/tests/product_experience.rs@@ -753,7 +753,7 @@ fn count_settings(value: &serde_json::Value) -> usize {/// Settings in a default `Config`, measured by `count_settings`. Quoted in/// `xerj-common/src/config.rs`; keep the two in step.-const EXPECTED_SETTINGS: usize = 117;+const EXPECTED_SETTINGS: usize = 120;#[tokio::test]async fn journey_zero_config() {diff --git a/engine/crates/xerj-mcp/Cargo.toml b/engine/crates/xerj-mcp/Cargo.tomlindex fdb4a9ef3..939c338e6 100644--- a/engine/crates/xerj-mcp/Cargo.toml+++ b/engine/crates/xerj-mcp/Cargo.toml@@ -33,3 +33,8 @@ anyhow = { workspace = true }# The feedback invitation, shared with every other --help surface so the five# cannot drift apart.xerj-common = { path = "../xerj-common" }+# The `rerank` argument's limits and its validator. Depending on the crate the+# node itself validates with means the tool description quotes the real+# ceilings and the builder refuses exactly what the node would refuse — the+# two cannot drift. A light leaf: it adds no dependency this crate lacks.+xerj-rerank = { path = "../xerj-rerank" }diff --git a/engine/crates/xerj-mcp/src/lib.rs b/engine/crates/xerj-mcp/src/lib.rsindex 28c042141..e2b71a691 100644--- a/engine/crates/xerj-mcp/src/lib.rs+++ b/engine/crates/xerj-mcp/src/lib.rs@@ -35,6 +35,13 @@//! 1.00 on the official bench query); filtered kNN and other ineligible shapes//! (non-cosine, SQ8, small indexes) run the exact brute-force scan. The tool//! description says so.+//! * `rerank` on `xerj_search` / `xerj_hybrid_search` is **opt-in and needs a+//! provider the node's operator configured**; this server cannot supply the+//! key. It is also the one argument that makes the node send document text to+//! a third party, and the tool description says so in those words. Ranking+//! quality with the real provider model has **not been verified by this+//! project** — the description promises the mechanism (a calibrated+//! probability replaces `_score`), never an accuracy figure.//! * `hybrid` supports `fusion: "rrf"` and `"linear"` only. `"learned"` is//! forwarded verbatim and the engine rejects it loudly; the schema therefore//! advertises only `rrf`/`linear`.@@ -332,6 +339,12 @@ fn initialize_result(msg: &Value) -> Value {kNN over a dense_vector field, xerj_hybrid_search to fuse \lexical + vector results (rrf|linear), and xerj_memory_store / \xerj_memory_recall for durable agent memory recalled by meaning. \+ xerj_search and xerj_hybrid_search take an optional `rerank` \+ argument: a second stage that re-judges the top hits with an \+ external relevance model. It only works if the node's operator \+ configured a provider key, and it sends the returned document \+ text off the machine — leave it out unless first-stage order is \+ the problem and the user is fine with that. \The xerj_brain_* tools work a second brain (a bi-temporal, \evidence-carrying link index built by `xerj brain <folder>`): \xerj_brain_overview to orient, then xerj_brain_ego for one node's \@@ -371,6 +384,68 @@ const BRAIN_HONESTY: &str = " Honesty: links come from deterministic lexical \search engine with a graph-shaped index over its own documents, not a \graph database.";+/// JSON-schema fragment for the optional `rerank` argument, shared by+/// `xerj_search` and `xerj_hybrid_search` so the two descriptions cannot drift.+///+/// The description is written for the reader that will act on it — an agent+/// deciding whether to spend a paid third-party call. It therefore says three+/// things a schema normally would not: when the stage is worth it, that it+/// needs a provider only the node's operator can configure, and that it is the+/// only search-time feature that sends document text off the node.+fn rerank_arg_schema(query_required: bool) -> Value {+ let defaults = xerj_rerank::RerankConfig::default();+ let question = if query_required {+ "`query` (string, REQUIRED for this tool): the natural-language question to \+ judge documents against."+ } else {+ "`query` (string): the natural-language question to judge documents against. \+ Filled in for you when the tool's `query` is a plain string (that mode also \+ returns the matching passage, which leads the text the judge reads); REQUIRED when \+ the tool's `query` is a `bool`, a `knn`, or anything else with no single \+ question in it (the node refuses rather than guess)."+ };+ let description = format!(+ "Optional second-stage reranking. XERJ sends the text of the top `window` hits \+ plus the question to an external relevance model, which returns a calibrated \+ probability (0..1) per document. Hits are reordered by it, that probability \+ REPLACES `_score`, and `min_score` prunes by it — an absolute relevance cut, \+ which a BM25 score cannot give you. \+ WORTH IT WHEN: a natural-language question over prose where the right document \+ is probably in the top {default_window} but not at the top; or you need a yes/no relevance \+ threshold rather than a relative rank. \+ NOT WORTH IT FOR: exact identifier or symbol lookups (the first stage already \+ ranks the definition first), filters, sorted or aggregation-only requests — it \+ adds a paid network round-trip per search. \+ NEEDS A CONFIGURED PROVIDER: the node's operator must have set a provider key \+ (`[rerank] api_key` or TYPESAFE_API_KEY). You cannot supply one in this call. \+ Without it the search fails with HTTP 503 `rerank_exception`; do not retry, \+ repeat the search without `rerank`. HTTP 403 means the operator disabled it. \+ PRIVACY: this is the only search-time feature that sends document text off the \+ node. Proxy embeddings (`[embedding] default_endpoint`) and the WAL tap also send \+ text off the node when an operator configures them, so never tell a user that \+ nothing else leaves the machine — do not use rerank on data that must stay local \+ unless the user has agreed. Only fields \+ the response returns are sent, so `_source` filtering also limits what leaves. \+ Pass true or {{}} for defaults, or an object with: {question} \+ `window` (int, default {default_window}, max {max_window}): how many top hits \+ are judged — every one is a paid judgement; `from`+`size` must fit inside it. \+ `min_score` (0..1): drop hits judged below this probability. \+ `fields` (string[]): which returned fields to send; default is every returned \+ string field. `instructions` (string): override what \"relevant\" means for \+ this corpus. `timeout_ms` (int, default {default_timeout_ms}). `model` (string). \+ Cannot be combined with `sort`. \+ READ THE RESPONSE'S `_rerank` BLOCK: `applied: true` means the order is the \+ judge's and every `_score` is a probability or null (a hit with no verdict, \+ counted in `unjudged`, sorts last); `applied: false` means the provider missed \+ the deadline or answered nothing usable, and you are looking at the engine's own \+ order with engine scores, stated in `reason`.",+ default_window = defaults.window,+ max_window = xerj_rerank::MAX_WINDOW,+ default_timeout_ms = defaults.timeout.as_millis(),+ );+ json!({ "type": ["object", "boolean"], "description": description })+}+/// The ten tool specifications advertised via `tools/list`. Input schemas are/// plain JSON Schema; every property maps onto a field the engine accepts.///@@ -389,7 +464,11 @@ pub fn tool_specs() -> Value {definition-first code search (ranks the file that defines a symbol \above files that merely mention it — the right default for code), \or as an ES query-DSL object (e.g. {\"term\":{\"status\":\"open\"}}, \- or a bool clause) for structured queries. Omit `query` for match_all.",+ or a bool clause) for structured queries. Omit `query` for match_all. \+ Optional `rerank` adds a second stage that re-judges the top hits with \+ an external relevance model — read that argument's description before \+ using it: it needs a provider the operator configured, and it sends \+ document text off the machine.","inputSchema": {"type": "object","properties": {@@ -401,7 +480,8 @@ pub fn tool_specs() -> Value {"size": { "type": "integer", "description": "Max hits to return (default engine value)." },"from": { "type": "integer", "description": "Offset for pagination." },"sort": { "description": "ES sort clause (array or object)." },- "_source": { "description": "Source filtering (bool, field, or {includes,excludes})." }+ "_source": { "description": "Source filtering (bool, field, or {includes,excludes})." },+ "rerank": rerank_arg_schema(false)},"required": ["index"]}@@ -458,7 +538,10 @@ pub fn tool_specs() -> Value {"Hybrid search: fuse several sub-queries (e.g. a lexical `match` plus a \vector `knn`) into one ranked list. Fusion is `rrf` (reciprocal-rank) or \`linear` (weighted). Proxies POST /{index}/_search with \- {\"query\":{\"hybrid\":{\"queries\":[...],\"fusion\":...}}}.",+ {\"query\":{\"hybrid\":{\"queries\":[...],\"fusion\":...}}}. \+ Optional `rerank` re-judges the fused top hits with an external \+ relevance model; here `rerank.query` is REQUIRED, because a hybrid \+ search has several sub-queries and no single question to read.","inputSchema": {"type": "object","properties": {@@ -482,7 +565,8 @@ pub fn tool_specs() -> Value {"enum": ["rrf", "linear"],"description": "Fusion strategy (default rrf)."},- "size": { "type": "integer", "description": "Max fused hits to return." }+ "size": { "type": "integer", "description": "Max fused hits to return." },+ "rerank": rerank_arg_schema(true)},"required": ["index", "queries"]}@@ -810,6 +894,71 @@ fn opt_typed<'a, T>(// ── Per-tool request builders → (method, path, body) ────────────────────────+/// Normalise the optional `rerank` argument into the `_search` body.+///+/// `plain_query` is the tool's `query` when the caller passed a plain string.+/// It matters because the string path expands into the definition-first `bool`+/// shape, and the node refuses to read a question out of a `bool` tree — so+/// without this every plain-string search that asked for reranking would come+/// back 400. The caller's words ARE the question; they are carried across.+///+/// Refusals here mirror the node's, using the node's own validator+/// ([`xerj_rerank::RerankConfig::from_json`]), so an agent is told about a+/// misspelt key without paying for a round trip. Present-but-mistyped is an+/// error, never a silent drop (the `opt_typed` policy): an agent that asked for+/// reranking and silently did not get it would trust an order nobody judged.+fn apply_rerank(+ args: &Value,+ body: &mut serde_json::Map<String, Value>,+ plain_query: Option<&str>,+ query_required: bool,+) -> Result<(), String> {+ let mut block = match args.get("rerank") {+ None | Some(Value::Null) | Some(Value::Bool(false)) => return Ok(()),+ Some(Value::Bool(true)) => serde_json::Map::new(),+ Some(Value::Object(o)) => o.clone(),+ Some(_) => {+ return Err(+ "`rerank` must be true, or an object such as {\"min_score\": 0.5} \+ (pass {} for defaults)"+ .to_string(),+ )+ }+ };+ if body.contains_key("sort") {+ return Err(+ "`rerank` cannot be combined with `sort`: an explicit sort already fixes the \+ order, and reranking would discard it. Drop one of them"+ .to_string(),+ );+ }+ if !block.contains_key("query") {+ match plain_query {+ Some(q) => {+ block.insert("query".into(), json!(q));+ }+ None if query_required => {+ return Err(+ "`rerank.query` is required for a hybrid search: it fuses several \+ sub-queries, so there is no single question to judge documents \+ against. Pass the natural-language question, e.g. \+ {\"rerank\": {\"query\": \"why did the vpn drop?\"}}"+ .to_string(),+ )+ }+ // A DSL object: the node reads the question from `match`,+ // `match_phrase`, `multi_match`, `semantic` and `simple_query_string`,+ // and refuses by name for anything else. Left to it, so there is+ // one implementation of that rule.+ None => {}+ }+ }+ let block = Value::Object(block);+ xerj_rerank::RerankConfig::from_json(&block).map_err(|e| e.to_string())?;+ body.insert("rerank".into(), block);+ Ok(())+}+fn build_search(args: &Value) -> Result<BuiltRequest, String> {let index = req_str(args, "index")?;let mut body = serde_json::Map::new();@@ -845,6 +994,11 @@ fn build_search(args: &Value) -> Result<BuiltRequest, String> {for k in ["size", "from", "sort", "_source"] {copy_opt(args, &mut body, k);}+ let plain_query = match args.get("query") {+ Some(Value::String(q)) if plain_string => Some(q.as_str()),+ _ => None,+ };+ apply_rerank(args, &mut body, plain_query, false)?;// String-mode defaults, only where the caller left them unset: project// `_source` down to the citation fields and ask for `_passage` so the// response is the matching snippet, not the whole file body (measured ~9x@@ -932,6 +1086,7 @@ fn build_hybrid(args: &Value) -> Result<BuiltRequest, String> {let mut body = serde_json::Map::new();body.insert("query".into(), json!({ "hybrid": Value::Object(hybrid) }));copy_opt(args, &mut body, "size");+ apply_rerank(args, &mut body, None, true)?;Ok((Method::Post,format!("/{index}/_search"),@@ -1210,6 +1365,156 @@ mod tests {assert_eq!(body["size"], 10);}+ // ── `rerank` argument ───────────────────────────────────────────────────++ #[test]+ fn rerank_is_absent_unless_asked_for() {+ for args in [+ json!({ "index": "d", "query": "how do refunds work" }),+ json!({ "index": "d", "query": "how do refunds work", "rerank": null }),+ json!({ "index": "d", "query": "how do refunds work", "rerank": false }),+ ] {+ let (_, _, body) = built(build_search(&args));+ assert!(+ body.get("rerank").is_none(),+ "no `rerank` block means no provider call and no egress: {body}"+ );+ }+ }++ #[test]+ fn a_plain_string_query_becomes_the_rerank_question() {+ // The string path expands to a `bool`, which the node refuses to read a+ // question from. Without carrying the string across, every plain-string+ // search that asked for reranking would be a 400.+ let (_, _, body) = built(build_search(&json!({+ "index": "d", "query": "how do refunds work", "rerank": true+ })));+ assert!(body["query"]["bool"].is_object(), "still definition-first");+ assert_eq!(body["rerank"]["query"], "how do refunds work");++ let (_, _, body) = built(build_search(&json!({+ "index": "d", "query": "how do refunds work", "rerank": { "min_score": 0.5 }+ })));+ assert_eq!(body["rerank"]["query"], "how do refunds work");+ assert_eq!(body["rerank"]["min_score"], 0.5);+ }++ #[test]+ fn an_explicit_rerank_query_is_never_overwritten() {+ let (_, _, body) = built(build_search(&json!({+ "index": "d", "query": "refund",+ "rerank": { "query": "What is the refund window for annual plans?" }+ })));+ assert_eq!(+ body["rerank"]["query"],+ "What is the refund window for annual plans?"+ );+ }++ #[test]+ fn a_dsl_query_leaves_question_inference_to_the_node() {+ let (_, _, body) = built(build_search(&json!({+ "index": "d", "query": { "match": { "body": "refund window" } }, "rerank": {}+ })));+ assert_eq!(+ body["rerank"],+ json!({}),+ "one implementation of inference: the node's"+ );+ }++ #[test]+ fn rerank_refusals_happen_before_the_round_trip() {+ let bad = |args: Value| build_search(&args).expect_err("must be refused");++ let e = bad(json!({ "index": "d", "query": "q", "rerank": "yes" }));+ assert!(e.contains("must be true, or an object"), "{e}");++ let e = bad(json!({ "index": "d", "query": "q", "sort": ["_doc"], "rerank": true }));+ assert!(e.contains("sort"), "{e}");++ // The node's own validator: a misspelt key is named, not dropped.+ let e = bad(json!({ "index": "d", "query": "q", "rerank": { "treshold": 0.5 } }));+ assert!(e.contains("treshold"), "{e}");++ let e = bad(json!({ "index": "d", "query": "q", "rerank": { "min_score": 7 } }));+ assert!(e.contains("probability"), "{e}");++ let e = bad(json!({ "index": "d", "query": "q", "rerank": { "window": 100000 } }));+ assert!(e.contains("rerank.window"), "{e}");+ }++ #[test]+ fn hybrid_rerank_requires_the_question() {+ let queries = json!([{ "query": { "match": { "body": "vpn" } } }]);+ let e = build_hybrid(&json!({ "index": "h", "queries": queries, "rerank": true }))+ .expect_err("a hybrid has no single question");+ assert!(e.contains("rerank.query"), "{e}");++ let (_, path, body) = built(build_hybrid(&json!({+ "index": "h", "queries": queries,+ "rerank": { "query": "why did the vpn drop?", "window": 20 }+ })));+ assert_eq!(path, "/h/_search");+ assert_eq!(body["rerank"]["query"], "why did the vpn drop?");+ assert_eq!(body["rerank"]["window"], 20);+ assert!(body["query"]["hybrid"].is_object());+ }++ #[test]+ fn the_rerank_description_tells_an_agent_what_it_needs_to_decide() {+ let specs = tool_specs();+ for tool in ["xerj_search", "xerj_hybrid_search"] {+ let spec = specs+ .as_array()+ .unwrap()+ .iter()+ .find(|t| t["name"] == tool)+ .unwrap();+ let d = spec["inputSchema"]["properties"]["rerank"]["description"]+ .as_str()+ .unwrap_or_else(|| panic!("{tool} must describe `rerank`"));+ for needle in [+ "WORTH IT WHEN",+ "NOT WORTH IT FOR",+ "NEEDS A CONFIGURED PROVIDER",+ "503",+ // The honest egress statement: rerank is the only SEARCH-TIME+ // path, and the description names the other two that send text+ // so an agent never repeats "nothing else leaves the machine"+ // on a node running proxy embeddings or a WAL tap.+ "only search-time feature that sends document text off the node",+ "default_endpoint",+ "WAL tap",+ "never tell a user that nothing else leaves the machine",+ "_rerank",+ ] {+ assert!(d.contains(needle), "{tool}: description lacks `{needle}`");+ }+ // The first corrected wording ("two other ... paths") was incomplete: the neural model+ // download and cluster Raft traffic are outbound too (they carry no+ // document text). docs/RERANK.md holds the complete list.+ assert!(+ !d.to_lowercase().contains("other outbound paths"),+ "{tool}: {d}"+ );+ // The quoted ceilings are the node's, not a copy of them.+ assert!(+ d.contains(&format!("max {}", xerj_rerank::MAX_WINDOW)),+ "{d}"+ );+ assert!(+ !spec["inputSchema"]["required"]+ .as_array()+ .unwrap()+ .iter()+ .any(|r| r == "rerank"),+ "{tool}: `rerank` is optional"+ );+ }+ }+#[test]fn memory_store_requires_text_or_vector() {assert!(build_memory_store(&json!({ "namespace": "n" })).is_err());diff --git a/engine/crates/xerj-rerank/Cargo.toml b/engine/crates/xerj-rerank/Cargo.tomlnew file mode 100644index 000000000..afac4f13c--- /dev/null+++ b/engine/crates/xerj-rerank/Cargo.toml@@ -0,0 +1,18 @@+[package]+name = "xerj-rerank"+version.workspace = true+edition.workspace = true+license.workspace = true+authors.workspace = true+description = "Pluggable relevance reranking for xerj — calibrated second-stage scoring over retrieved candidates"++[dependencies]+reqwest.workspace = true+serde.workspace = true+serde_json.workspace = true+thiserror.workspace = true+tokio.workspace = true+tracing.workspace = true++[dev-dependencies]+tokio = { workspace = true }diff --git a/engine/crates/xerj-rerank/src/lib.rs b/engine/crates/xerj-rerank/src/lib.rsnew file mode 100644index 000000000..babda47db--- /dev/null+++ b/engine/crates/xerj-rerank/src/lib.rs@@ -0,0 +1,2153 @@+//! Second-stage relevance reranking over candidates the engine already retrieved.+//!+//! # Why this is not in the engine+//!+//! Reranking runs *after* hits are collected and hydrated, and it makes an+//! outbound network call. Neither property belongs in the segment loop, so this+//! crate is a leaf the API layer calls once per search, never something+//! `xerj-engine` links. That keeps the read hot path and the ES-compat+//! conformance surface untouched.+//!+//! # Why a second stage at all+//!+//! XERJ ranks with BM25 and fuses with RRF or a linear combination. Those scores+//! order results but carry no absolute meaning: a `_score` of 7.2 is not+//! comparable across queries and cannot be thresholded. A calibrated reranker+//! returns a *probability* per document, which is both a sort key and an+//! absolute cut — so `min_score` becomes a meaningful knob for the first time.+//!+//! This matters more than ordering quality for XERJ specifically: the default+//! embedder is lexical feature hashing, so the engine has no semantic signal of+//! its own to offer. A reranker supplies one without the project claiming a+//! neural embedder it does not ship.+//!+//! # Failure policy: degrade on time, surface on contract+//!+//! Not every failure deserves the same answer, and a blanket "always fall back+//! to BM25" is the wrong default: a caller who asked for calibrated relevance+//! and silently got lexical order has been misled, which is the silent-fake+//! class this project treats as a defect (see the `learned` fusion arm in+//! `xerj-query`, which fails loud rather than substituting RRF).+//!+//! So the policy is split, following the one Meilisearch arrived at for its+//! Cohere reranker (`crates/meilisearch/src/personalization/mod.rs:74-135`,+//! approach adapted, not copied):+//!+//! * **Deadline exceeded → degrade.** Return the engine's own order and say so.+//! A slow third party is not a reason to deny the caller results it already+//! has.+//! * **Auth, bad request, unknown provider, malformed response, rate limit that+//! survives back-off → surface.** These are contract and configuration+//! errors. They do not fix themselves and hiding them wastes the operator's+//! time.+//!+//! Either way the outcome is explicit in [`RerankOutcome`], so the API layer can+//! tell the caller which ranking it is actually looking at.+//!+//! # Providers+//!+//! Dispatch is an enum rather than a `dyn` trait: the set is closed, small, and+//! every arm is async, which would otherwise require `async-trait` and a boxed+//! future per call. Add an arm to extend it. A `Disabled` arm exists so an empty+//! API key is a configured no-op rather than a hard error — also Meilisearch's+//! choice (`personalization/mod.rs:333-345`).+//!+//! The Jev arm speaks TypeSafe AI's System One API+//! (`POST https://api.typesafe.ai/v1/systemone`, documented at+//! <https://docs.typesafe.ai/api>). The one-`noul`-per-document shape is the+//! approach used by `hev/jev-rerank` (Apache-2.0); asking a separate yes/no per+//! document rather than one `choice` across them is deliberate, because real+//! corpora have more than one relevant document and a `choice` forces a single+//! winner.++use std::time::Duration;++use serde::Deserialize;+use serde_json::{json, Map, Value};++/// Hard ceiling on documents per Jev request.+///+/// The `hev/jev-rerank` README reports a ~32k-token request budget and+/// "~30–50 typical passages per call", and uses 30; this project has not+/// measured the provider's limit itself. Longer candidate lists are split+/// into several calls, and the scores stay comparable across them because each+/// answer is an absolute probability rather than a rank within its batch.+pub const JEV_MAX_DOCS_PER_CALL: usize = 30;++/// Default concurrent in-flight provider calls.+///+/// Conservative on purpose. The provider's rate limits are undocumented; the+/// `hev/jev-rerank` README reports sustained 429s at about 24 requests in+/// flight (their observation, not ours). A search path that provokes rate+/// limiting is worse than a slightly slower one.+pub const DEFAULT_MAX_CONCURRENCY: usize = 8;++/// Ceiling on `rerank.window`.+///+/// The operator pays per judged document, and the caller picks the window — so+/// the ceiling is the server's, not the caller's. 300 is ten full provider+/// calls, two waves at the default concurrency.+pub const MAX_WINDOW: usize = 300;++/// Ceiling on `rerank.max_concurrency`. Kept under the ~24 in flight at which+/// `hev/jev-rerank` reports sustained 429s: a caller must not be able to push+/// the server's key into rate limiting.+pub const MAX_CONCURRENCY: usize = 16;++/// Ceiling on `rerank.max_doc_chars`. 30 documents at this size is already past+/// what a ~32k-token context holds; anything larger only buys a provider 4xx.+pub const MAX_DOC_CHARS: usize = 8_000;++/// Ceiling on `rerank.timeout_ms`: a search must not be able to pin a+/// connection for minutes on a third party's behalf.+pub const MAX_TIMEOUT_MS: u64 = 60_000;++/// How much of each document body to send.+///+/// Reranking reads titles and leading text; shipping whole documents burns the+/// context budget that limits how many candidates fit per call.+pub const DEFAULT_MAX_DOC_CHARS: usize = 1200;++/// Ceiling on `rerank.instructions`, in characters.+///+/// The instructions are not sent once: the System One wire format carries them+/// inside EVERY per-document question, so whatever the caller writes here is+/// multiplied by the window. Uncapped, a 1 MB string at `window: 40` and+/// `max_doc_chars: 1` put 40 MB on the wire to the provider (measured in+/// review), and an 8 MB one grew the node's peak RSS by 700 MB for a single+/// `size: 1` search — every other cost knob had a ceiling and this, the one+/// multiplied by the window, did not. 2,000 characters is a dozen sentences of+/// domain guidance; at the largest window (300) it is 600 KB of instructions,+/// well under the 4.8 MB of document text the same window may already carry.+pub const MAX_INSTRUCTIONS_CHARS: usize = 2_000;++/// Ceiling on the question — `rerank.query`, or the one read from the search+/// query. Sent once per provider call (at most ten calls per search), and+/// echoed in `_rerank.query`. A question is a sentence or a short paragraph;+/// anything longer is a document, and the judge's context is the scarce thing.+pub const MAX_QUERY_CHARS: usize = 4_000;++/// Ceiling on `rerank.model` and `rerank.provider`: identifiers, not prose.+pub const MAX_MODEL_CHARS: usize = 128;++/// Ceiling on the number of `rerank.fields` entries. Every entry is looked up+/// on every hit in the window, and entries that returned no text are echoed in+/// `_rerank.fields_without_text`.+pub const MAX_FIELDS: usize = 64;++/// Ceiling on one `rerank.fields` entry, in characters — a field path.+pub const MAX_FIELD_NAME_CHARS: usize = 256;++/// What `GET /_xerj/rerank` says leaves the node, as `data_egress`.+///+/// A privacy statement an operator — or an agent quoting it to one — will rely+/// on, so it is complete rather than flattering: reranking is the only+/// SEARCH-TIME feature that sends document text off the node, it is not the+/// only feature that sends text, and it is not the only outbound connection.+/// `tests/egress_inventory.rs` checks it names every path the engine source+/// contains, and that `docs/RERANK.md` quotes it verbatim.+pub const DATA_EGRESS: &str = "A search that carries a `rerank` block sends the text of up to \+ `window` hits, and the query, to the endpoint above. It is the only search-time feature \+ that sends document text off the node. Two other features send text off the node, both \+ operator configuration and off by default: `[embedding] default_endpoint` \+ (`--embed-mode proxy`) sends document text at write time and query text at search time \+ to an external embeddings API, and the WAL tap (`PUT /_xerj/wal_tap`) replays every \+ write on tapped indices to an external `_bulk` endpoint. The node's other outbound \+ connections carry no document or query text: the one-time HuggingFace model download \+ for `--embed-mode neural`, Raft messages (index names, mappings, shard assignments) \+ to the configured peers in cluster mode, and object storage — the `S3Backend` client, \+ which nothing on the segment path constructs today, and `xerj autoindex s3://`, which \+ names buckets and keys to the endpoint you configure and reads objects IN.";++/// Ceiling on one provider response body, in bytes.+///+/// A legitimate System One answer for a full 30-document call is one small+/// object per document — a few kilobytes. The endpoint is a third party (or+/// whatever answers at the configured URL), and reading its body whole let it+/// choose how much memory one search allocates: `max_concurrency` calls in+/// flight, each as large as the peer cared to make it. 2 MiB is hundreds of+/// times a real answer; a larger body is a contract break (502), not an+/// allocation.+pub const MAX_RESPONSE_BYTES: usize = 2 * 1024 * 1024;++/// How much of a non-2xx response body is read. Only its first 400 characters+/// reach the error message, so there is no reason to read more.+const ERROR_BODY_READ_BYTES: usize = 4 * 1024;++#[derive(Debug, thiserror::Error)]+pub enum RerankError {+ #[error(+ "no rerank API key is configured on this server: set `api_key` under `[rerank]` in the \+ config file, or the {0} environment variable, and restart"+ )]+ MissingKey(&'static str),+ #[error("provider `{0}` is not supported")]+ UnknownProvider(String),+ #[error("transport: {0}")]+ Transport(String),+ #[error("provider returned {status}: {body}")]+ Status { status: u16, body: String },+ #[error("malformed provider response: {0}")]+ Malformed(String),+ #[error("invalid configuration: {0}")]+ Config(String),+ #[error("rerank deadline exceeded after {elapsed_ms}ms")]+ Deadline { elapsed_ms: u128 },+ /// The operator switched reranking off (`[rerank] enabled = false`).+ /// Distinct from [`Self::MissingKey`]: that one is "not set up yet", this+ /// one is "deliberately never" — reranking sends document text to a third+ /// party, and an operator must be able to forbid that outright.+ #[error(+ "reranking is disabled on this server (`[rerank] enabled = false`): it would send \+ document text to a third-party provider, and the operator has forbidden that"+ )]+ DisabledByOperator,+}++/// What the caller should do with a failure.+#[derive(Debug, Clone, Copy, PartialEq, Eq)]+pub enum Policy {+ /// Keep the engine's own ranking and report that reranking did not run.+ Degrade,+ /// Return the error to the caller. Configuration and contract faults do not+ /// fix themselves, and masking them costs the operator more than it saves.+ Surface,+}++impl RerankError {+ pub fn policy(&self) -> Policy {+ match self {+ // Ran out of time. The caller still has usable results.+ Self::Deadline { .. } => Policy::Degrade,+ // A transport fault that outlived the retries is indistinguishable+ // from "the provider is down"; the operator needs to know.+ Self::Transport(_) => Policy::Surface,+ Self::MissingKey(_)+ | Self::UnknownProvider(_)+ | Self::Config(_)+ | Self::Malformed(_)+ | Self::DisabledByOperator+ | Self::Status { .. } => Policy::Surface,+ }+ }++ /// Whether another attempt could plausibly succeed.+ ///+ /// 429 and 529 are the two codes TypeSafe documents as "back off and retry";+ /// 5xx is retried as a courtesy. A 4xx other than 429 is a contract error+ /// and retrying it only burns the deadline.+ pub fn retryable(&self) -> bool {+ match self {+ Self::Transport(_) => true,+ Self::Status { status, .. } => {+ *status == 429 || *status == 529 || (500..600).contains(status)+ }+ _ => false,+ }+ }+}++/// A wall-clock budget for the whole reranking stage.+///+/// Reranking sits inside a search request that the caller is waiting on, so the+/// stage needs its own ceiling independent of any single HTTP timeout: three+/// sequential batches each finishing just inside a 10s timeout is a 30s search.+#[derive(Debug, Clone, Copy)]+pub struct Deadline {+ started: std::time::Instant,+ budget: Duration,+}++impl Deadline {+ pub fn new(budget: Duration) -> Self {+ Self {+ started: std::time::Instant::now(),+ budget,+ }+ }+ pub fn exceeded(&self) -> bool {+ self.started.elapsed() >= self.budget+ }+ pub fn remaining(&self) -> Duration {+ self.budget.saturating_sub(self.started.elapsed())+ }+ fn err(&self) -> RerankError {+ RerankError::Deadline {+ elapsed_ms: self.started.elapsed().as_millis(),+ }+ }+}++/// What actually happened, so the API layer never has to guess.+#[derive(Debug)]+pub enum RerankOutcome {+ /// Provider scored the window; `scores` is the new order.+ Reordered {+ scores: Vec<Scored>,+ /// Batches that failed while others succeeded. Ordering among scored+ /// documents is still correct — the probabilities are absolute — but the+ /// caller should know the window was not fully covered.+ partial_failures: usize,+ /// Tokens the provider billed for, summed over every batch that+ /// answered. Reranking is a paid call inside a search, so the caller+ /// gets the meter reading rather than having to infer it.+ usage: Usage,+ },+ /// Reranking did not run. The engine's order stands.+ Degraded { reason: String },+}++/// Token usage as the provider reports it. Zero when the provider omits it.+#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]+pub struct Usage {+ pub input_tokens: u64,+ pub output_tokens: u64,+}++impl Usage {+ fn add(&mut self, other: Usage) {+ self.input_tokens = self.input_tokens.saturating_add(other.input_tokens);+ self.output_tokens = self.output_tokens.saturating_add(other.output_tokens);+ }+}++/// One candidate handed to the reranker.+#[derive(Debug, Clone)]+pub struct Candidate {+ /// Caller-side position. Returned untouched so the caller can reorder its+ /// own hit array without matching on document contents.+ pub ordinal: usize,+ pub title: Option<String>,+ pub text: String,+}++/// A relevance probability for one candidate, in `0.0..=1.0`.+///+/// Carried as `f64` end to end. The provider answers a JSON number, the hit's+/// `_score` is an `f64` on the wire, and a threshold the caller wrote as `0.9`+/// has to compare equal to the `0.9` the response prints: an `f32` in the+/// middle turned that into `0.8999999761581421` and made a client-side+/// `_score >= 0.9` disagree with the server's own `rerank.min_score`.+#[derive(Debug, Clone, Copy, PartialEq)]+pub struct Scored {+ pub ordinal: usize,+ pub score: f64,+}++/// Parsed `rerank` block from a search body.+#[derive(Debug, Clone)]+pub struct RerankConfig {+ pub provider: String,+ pub model: String,+ /// How many of the engine's top hits to rerank. The rest keep their order+ /// and sort below the reranked window.+ pub window: usize,+ /// Documents per provider call, capped at [`JEV_MAX_DOCS_PER_CALL`].+ pub batch: usize,+ /// Drop candidates scoring below this. `None` keeps all and only reorders.+ pub min_score: Option<f64>,+ pub max_concurrency: usize,+ pub max_doc_chars: usize,+ pub timeout: Duration,+ /// Natural-language relevance question. Overridable because "relevant" is+ /// domain-specific — a support corpus and a code corpus do not mean the+ /// same thing by it.+ pub instructions: Option<String>,+ /// The question to judge documents against. Required unless the caller's+ /// search query is a shape the API layer can read a plain string out of —+ /// guessing at the intent behind a `bool` tree would rank against the+ /// wrong question without anyone noticing.+ pub query: Option<String>,+ /// Returned fields sent to the provider. An EXHAUSTIVE allow-list: it is an+ /// egress control, so `["body"]` sends the body and not even the title.+ /// `None` sends the title, then the matching `_passage` when the response+ /// carries one, then every other returned top-level string field — right+ /// for small documents and wasteful for wide ones. The API layer reads the+ /// values; this crate only carries the names.+ pub fields: Option<Vec<String>>,+}++impl Default for RerankConfig {+ fn default() -> Self {+ Self {+ provider: "jev".to_string(),+ model: "jev-latest".to_string(),+ window: JEV_MAX_DOCS_PER_CALL,+ batch: JEV_MAX_DOCS_PER_CALL,+ min_score: None,+ max_concurrency: DEFAULT_MAX_CONCURRENCY,+ max_doc_chars: DEFAULT_MAX_DOC_CHARS,+ timeout: Duration::from_secs(10),+ instructions: None,+ query: None,+ fields: None,+ }+ }+}++impl RerankConfig {+ /// Parse the `rerank` object from a search body.+ ///+ /// Unknown keys are rejected rather than ignored: a caller who misspells+ /// `min_score` and silently gets unpruned results has been misled, and that+ /// is the failure mode this project treats as a defect.+ pub fn from_json(v: &Value) -> Result<Self, RerankError> {+ let obj = v+ .as_object()+ .ok_or_else(|| RerankError::Config("`rerank` must be an object".into()))?;++ const KNOWN: &[&str] = &[+ "provider",+ "model",+ "window",+ "batch",+ "min_score",+ "max_concurrency",+ "max_doc_chars",+ "timeout_ms",+ "instructions",+ "query",+ "fields",+ ];+ for k in obj.keys() {+ if !KNOWN.contains(&k.as_str()) {+ // Named, but never echoed whole: the key is the caller's, and a+ // megabyte-long one must not come back in the 400.+ let shown = clip(k, MAX_ECHOED_NAME_CHARS);+ let ellipsis = if shown.len() < k.len() { "…" } else { "" };+ return Err(RerankError::Config(format!(+ "unknown `rerank` field `{shown}{ellipsis}`; supported: {}",+ KNOWN.join(", ")+ )));+ }+ }++ let mut cfg = Self::default();++ if let Some(p) = obj.get("provider") {+ cfg.provider = bounded_string("provider", p, MAX_MODEL_CHARS)?;+ }+ if let Some(m) = obj.get("model") {+ cfg.model = bounded_string("model", m, MAX_MODEL_CHARS)?;+ }+ if let Some(w) = obj.get("window") {+ let w = w+ .as_u64()+ .ok_or_else(|| RerankError::Config("`rerank.window` must be a number".into()))?;+ if w == 0 {+ return Err(RerankError::Config("`rerank.window` must be > 0".into()));+ }+ if w as usize > MAX_WINDOW {+ return Err(RerankError::Config(format!(+ "`rerank.window` must be <= {MAX_WINDOW}: every document in the window is \+ a paid provider judgement"+ )));+ }+ cfg.window = w as usize;+ }+ if let Some(b) = obj.get("batch") {+ let b = b+ .as_u64()+ .ok_or_else(|| RerankError::Config("`rerank.batch` must be a number".into()))?;+ if b == 0 {+ return Err(RerankError::Config("`rerank.batch` must be > 0".into()));+ }+ // Clamp rather than reject: the ceiling is a property of the+ // provider, not a mistake by the caller.+ cfg.batch = (b as usize).min(JEV_MAX_DOCS_PER_CALL);+ }+ if let Some(t) = obj.get("min_score") {+ let t = t+ .as_f64()+ .ok_or_else(|| RerankError::Config("`rerank.min_score` must be a number".into()))?;+ if !(0.0..=1.0).contains(&t) {+ return Err(RerankError::Config(+ "`rerank.min_score` must be between 0 and 1 — it is a probability".into(),+ ));+ }+ cfg.min_score = Some(t);+ }+ if let Some(c) = obj.get("max_concurrency") {+ let c = c.as_u64().ok_or_else(|| {+ RerankError::Config("`rerank.max_concurrency` must be a number".into())+ })?;+ if c == 0 || c as usize > MAX_CONCURRENCY {+ return Err(RerankError::Config(format!(+ "`rerank.max_concurrency` must be between 1 and {MAX_CONCURRENCY}"+ )));+ }+ cfg.max_concurrency = c as usize;+ }+ if let Some(c) = obj.get("max_doc_chars") {+ let c = c.as_u64().ok_or_else(|| {+ RerankError::Config("`rerank.max_doc_chars` must be a number".into())+ })?;+ if c == 0 || c as usize > MAX_DOC_CHARS {+ return Err(RerankError::Config(format!(+ "`rerank.max_doc_chars` must be between 1 and {MAX_DOC_CHARS}"+ )));+ }+ cfg.max_doc_chars = c as usize;+ }+ if let Some(t) = obj.get("timeout_ms") {+ let t = t.as_u64().ok_or_else(|| {+ RerankError::Config("`rerank.timeout_ms` must be a number".into())+ })?;+ if t == 0 || t > MAX_TIMEOUT_MS {+ return Err(RerankError::Config(format!(+ "`rerank.timeout_ms` must be between 1 and {MAX_TIMEOUT_MS}"+ )));+ }+ cfg.timeout = Duration::from_millis(t);+ }+ if let Some(i) = obj.get("instructions") {+ // The one string that is multiplied by the window: it rides inside+ // every per-document question (see `build_jev_request`).+ cfg.instructions = Some(bounded_string("instructions", i, MAX_INSTRUCTIONS_CHARS)?);+ }++ if let Some(q) = obj.get("query") {+ let q = bounded_string("query", q, MAX_QUERY_CHARS)?;+ if q.trim().is_empty() {+ return Err(RerankError::Config(+ "`rerank.query` must not be empty".into(),+ ));+ }+ cfg.query = Some(q);+ }+ if let Some(f) = obj.get("fields") {+ let arr = f.as_array().ok_or_else(|| {+ RerankError::Config("`rerank.fields` must be an array of field names".into())+ })?;+ if arr.len() > MAX_FIELDS {+ return Err(RerankError::Config(format!(+ "`rerank.fields` must name at most {MAX_FIELDS} fields (got {}): every name \+ is looked up on every hit in the window",+ arr.len()+ )));+ }+ let mut fields = Vec::with_capacity(arr.len());+ for v in arr {+ let name = v.as_str().ok_or_else(|| {+ RerankError::Config("`rerank.fields` entries must be strings".into())+ })?;+ if longer_than(name, MAX_FIELD_NAME_CHARS) {+ return Err(RerankError::Config(format!(+ "`rerank.fields` entries must be at most {MAX_FIELD_NAME_CHARS} \+ characters — they are field paths (one entry is {} bytes)",+ name.len()+ )));+ }+ fields.push(name.to_string());+ }+ if fields.is_empty() {+ return Err(RerankError::Config(+ "`rerank.fields` must not be empty".into(),+ ));+ }+ cfg.fields = Some(fields);+ }++ Ok(cfg)+ }++ fn question(&self) -> &str {+ self.instructions.as_deref().unwrap_or(+ "Does this document contain information that answers the query? \+ Judge only whether it is relevant to the query, not whether it is \+ well written.",+ )+ }+}++/// Truncate on a char boundary. Byte slicing a `&str` here would panic on any+/// multi-byte character — the exact defect that crash-looped `autoindex` on+/// non-ASCII input, so it is not repeated.+pub fn clip(s: &str, max_chars: usize) -> &str {+ match s.char_indices().nth(max_chars) {+ Some((byte_idx, _)) => &s[..byte_idx],+ None => s,+ }+}++/// Whether `s` is longer than `max` characters, without walking all of it: a+/// caller can send a 100 MB string, and the answer is known after `max` of+/// them. Byte length is a free first test — `max` bytes cannot hold more than+/// `max` characters.+pub fn longer_than(s: &str, max: usize) -> bool {+ s.len() > max && s.char_indices().nth(max).is_some()+}++/// How much of a caller-supplied NAME (an unknown key) a refusal echoes back.+const MAX_ECHOED_NAME_CHARS: usize = 64;++/// A `rerank.<key>` string under its server-side ceiling.+///+/// The refusal names the field and the limit and reports the size in bytes+/// (known without reading the string); it never echoes the value, which is the+/// caller's and may be megabytes.+fn bounded_string(key: &str, v: &Value, max_chars: usize) -> Result<String, RerankError> {+ let s = v+ .as_str()+ .ok_or_else(|| RerankError::Config(format!("`rerank.{key}` must be a string")))?;+ if longer_than(s, max_chars) {+ return Err(RerankError::Config(format!(+ "`rerank.{key}` must be at most {max_chars} characters (this one is {} bytes): \+ it is sent to the provider on the operator's key, so the ceiling is the server's",+ s.len()+ )));+ }+ Ok(s.to_string())+}++/// The ceiling on the question, for a question the API layer read out of the+/// search query rather than out of `rerank.query`. Same string, same wire, same+/// limit — and the refusal says what to do about it.+pub fn check_inferred_question(q: &str) -> Result<(), RerankError> {+ if longer_than(q, MAX_QUERY_CHARS) {+ return Err(RerankError::Config(format!(+ "the question read from the search query is longer than the {MAX_QUERY_CHARS} \+ characters `rerank.query` allows (this one is {} bytes). Pass a shorter question \+ as `rerank.query`: it is sent to the provider with every call, and the judge's \+ context is what limits how many documents fit in one",+ q.len()+ )));+ }+ Ok(())+}++/// Remove the provider key from text that is about to reach a caller or a log.+///+/// A provider's error body is returned to whoever ran the search (the 502's+/// `reason`) and written to the log, and that caller is not necessarily the+/// operator who owns the key. Some APIs echo the credential in an auth error+/// ("invalid key: sk-…"). Without this, any principal allowed to search could+/// read the operator's key by provoking one.+fn redact_key(text: &str, api_key: &str) -> String {+ if api_key.is_empty() {+ return text.to_string();+ }+ text.replace(api_key, "<redacted>")+}++/// The part of a provider's error body that reaches the caller: the key+/// redacted FIRST, then cut to 400 characters. Cutting first would leave a key+/// that straddles the cut as a prefix `redact_key` no longer recognises — the+/// first characters of the operator's key, in a search response.+fn error_body(text: &str, api_key: &str) -> String {+ clip(&redact_key(text, api_key), 400).to_string()+}++/// Build one System One request body for a batch of candidates.+///+/// Split out from the HTTP call so the wire format is testable without a network+/// or an API key.+///+/// Every caller-supplied string is cut to its ceiling HERE as well as refused+/// in [`RerankConfig::from_json`]. `RerankConfig`'s fields are public, so the+/// parser is not the only way to build one, and what one call can put on the+/// wire — on the operator's key, to a third party — should be bounded by+/// construction rather than by every caller remembering to validate. For a+/// request that came through the parser the cuts change nothing.+pub fn build_jev_request(query: &str, batch: &[Candidate], cfg: &RerankConfig) -> Value {+ let mut documents = Map::new();+ let mut questions = Map::new();+ let instructions = clip(cfg.question(), MAX_INSTRUCTIONS_CHARS);+ let max_doc_chars = cfg.max_doc_chars.min(MAX_DOC_CHARS);++ for cand in batch {+ // Keyed by the caller's ordinal, so a reordered or partial response+ // still maps back to the right hit.+ let key = format!("d{}", cand.ordinal);+ let mut doc = Map::new();+ if let Some(t) = &cand.title {+ doc.insert("title".into(), json!(clip(t, max_doc_chars)));+ }+ doc.insert("text".into(), json!(clip(&cand.text, max_doc_chars)));+ documents.insert(key.clone(), Value::Object(doc));++ questions.insert(+ key,+ json!({+ "type": "noul",+ "instructions": instructions,+ "criteria": {+ "true": "The document is relevant to the query.",+ "false": "The document is not relevant to the query."+ }+ }),+ );+ }++ json!({+ "state": {+ "query": clip(query, MAX_QUERY_CHARS),+ "documents": Value::Object(documents),+ },+ "model": clip(&cfg.model, MAX_MODEL_CHARS),+ "questions": Value::Object(questions),+ })+}++#[derive(Deserialize)]+struct JevAnswer {+ #[serde(default)]+ noul: Option<f64>,+}++#[derive(Deserialize)]+struct JevResponse {+ #[serde(default)]+ answers: std::collections::HashMap<String, JevAnswer>,+ /// Read as loose JSON on purpose. `usage` is advisory metering and not part+ /// of the ranking contract: typed as `{input_tokens: u64, output_tokens:+ /// u64}` it made a provider reporting `-5` (or a float, a string, `null`)+ /// fail the whole call with a 502 and no hits, although every verdict in+ /// `answers` was valid. `answers` stays strict — that IS the contract.+ #[serde(default)]+ usage: Value,+}++/// The most tokens one provider call can plausibly bill for.+///+/// One call carries at most 30 documents of `2 × MAX_DOC_CHARS` characters plus+/// the instructions, and the question once: about 544,000 characters. Even at+/// several tokens per character that is a few million, so ten million is past+/// anything a call can produce. A larger figure is not metering, it is a+/// provider bug (review's test value was 2^63) — and because the counts feed a+/// Prometheus counter that only ever goes up, one such response would poison+/// `xerj_rerank_provider_tokens_total` until the node restarts.+pub const MAX_PLAUSIBLE_TOKENS_PER_CALL: u64 = 10_000_000;++/// One token count out of a provider `usage` block: a non-negative number, or+/// zero. Zero rather than a guess — the meter under-reads instead of inventing.+///+/// A count above [`MAX_PLAUSIBLE_TOKENS_PER_CALL`] is also zero: it is not one a+/// call can produce, and the counter it would feed never goes back down.+fn token_count(usage: &Value, key: &str) -> u64 {+ let n = match usage.get(key) {+ Some(n) => n.as_u64().unwrap_or_else(|| {+ n.as_f64()+ .filter(|f| f.is_finite() && *f >= 0.0)+ .map(|f| f as u64)+ .unwrap_or(0)+ }),+ None => 0,+ };+ if n > MAX_PLAUSIBLE_TOKENS_PER_CALL {+ 0+ } else {+ n+ }+}++/// Read a provider response body, stopping at `cap` bytes.+///+/// Returns the text read and whether the body went past `cap` (either by its+/// declared `Content-Length`, checked before reading anything, or by what+/// arrived). Past the cap nothing more is read: the connection is dropped with+/// the response. Invalid UTF-8 is replaced rather than refused — the JSON parse+/// that follows is what judges the content.+async fn read_capped(+ mut resp: reqwest::Response,+ cap: usize,+ deadline: &Deadline,+) -> Result<(String, bool), RerankError> {+ let io_err = |e: reqwest::Error| {+ if e.is_timeout() {+ deadline.err()+ } else {+ RerankError::Transport(e.to_string())+ }+ };+ if resp.content_length().is_some_and(|n| n > cap as u64) {+ return Ok((String::new(), true));+ }+ let mut buf: Vec<u8> = Vec::new();+ let mut oversized = false;+ while let Some(chunk) = resp.chunk().await.map_err(io_err)? {+ let room = cap - buf.len();+ if chunk.len() > room {+ buf.extend_from_slice(&chunk[..room]);+ oversized = true;+ break;+ }+ buf.extend_from_slice(&chunk);+ }+ Ok((String::from_utf8_lossy(&buf).into_owned(), oversized))+}++/// Parse a System One response into scores keyed by the caller's ordinals.+///+/// A missing or non-`noul` answer is skipped rather than defaulted to zero:+/// inventing a "not relevant" verdict the model never gave would silently+/// demote a document.+pub fn parse_jev_response(body: &str) -> Result<Vec<Scored>, RerankError> {+ parse_jev_response_with_usage(body).map(|(scores, _)| scores)+}++/// [`parse_jev_response`], plus the `usage` block the provider bills against.+pub fn parse_jev_response_with_usage(body: &str) -> Result<(Vec<Scored>, Usage), RerankError> {+ let parsed: JevResponse =+ serde_json::from_str(body).map_err(|e| RerankError::Malformed(e.to_string()))?;+ let usage = Usage {+ input_tokens: token_count(&parsed.usage, "input_tokens"),+ output_tokens: token_count(&parsed.usage, "output_tokens"),+ };++ let mut out = Vec::with_capacity(parsed.answers.len());+ for (key, ans) in parsed.answers {+ let Some(ordinal) = key.strip_prefix('d').and_then(|n| n.parse::<usize>().ok()) else {+ continue;+ };+ let Some(noul) = ans.noul else { continue };+ // A probability outside 0..=1 is not a probability. Clamp rather than+ // reject the whole batch: the ordering signal is still usable, and the+ // caller's `min_score` contract (0..=1) must keep meaning something.+ out.push(Scored {+ ordinal,+ score: noul.clamp(0.0, 1.0),+ });+ }+ Ok((out, usage))+}++/// Keep only the verdicts for documents that were in `batch`.+///+/// The provider is a third party, and a call is answered per batch: an answer+/// keyed to a document another batch sent, or to a key nobody sent, is not a+/// verdict on anything this call asked about. Without this filter a provider+/// that echoed `d0..d59` on every call scored a 30-document window as 90+/// judged documents — three verdicts per hit, the last one winning, and the+/// operator's billing meter (`judged`) tripled. One verdict per ordinal per+/// batch, in the batch's own order, so `judged` counts documents rather than+/// answers.+fn keep_verdicts_for(batch: &[Candidate], scores: Vec<Scored>) -> Vec<Scored> {+ let mut by_ordinal: std::collections::HashMap<usize, f64> =+ std::collections::HashMap::with_capacity(scores.len());+ for s in scores {+ by_ordinal.insert(s.ordinal, s.score);+ }+ batch+ .iter()+ .filter_map(|c| {+ by_ordinal.get(&c.ordinal).map(|&score| Scored {+ ordinal: c.ordinal,+ score,+ })+ })+ .collect()+}++/// Reorder `candidates` by provider score.+///+/// Candidates the provider did not score keep their original relative order and+/// sort *below* every scored one — an unscored document is not evidence of+/// irrelevance, but it cannot be ranked against ones that were scored.+pub fn apply_scores(+ candidates: &[Candidate],+ scores: &[Scored],+ min_score: Option<f64>,+) -> Vec<Scored> {+ let mut by_ordinal: std::collections::HashMap<usize, f64> =+ std::collections::HashMap::with_capacity(scores.len());+ for s in scores {+ by_ordinal.insert(s.ordinal, s.score);+ }++ let mut scored: Vec<Scored> = Vec::with_capacity(candidates.len());+ for (rank, cand) in candidates.iter().enumerate() {+ if let Some(&score) = by_ordinal.get(&cand.ordinal) {+ if let Some(min) = min_score {+ if score < min {+ continue;+ }+ }+ scored.push(Scored {+ ordinal: cand.ordinal,+ score,+ });+ } else if min_score.is_none() {+ // Negative score keeps unscored hits after every scored one while+ // preserving their engine order via the stable sort below.+ scored.push(Scored {+ ordinal: cand.ordinal,+ score: -1.0 - (rank as f64),+ });+ }+ }++ // Stable, so equal probabilities keep the engine's ordering — which is the+ // right tie-break: BM25 already ranked them.+ scored.sort_by(|a, b| {+ b.score+ .partial_cmp(&a.score)+ .unwrap_or(std::cmp::Ordering::Equal)+ });+ scored+}++/// Where a resolved setting came from. Reported by `GET /_xerj/rerank` so an+/// operator can tell which of two places is actually in force — without the+/// endpoint ever having to print the value itself.+#[derive(Debug, Clone, Copy, PartialEq, Eq)]+pub enum SettingSource {+ /// The server's config file (`[rerank]`).+ Config,+ /// The process environment.+ Env,+ /// Neither — the built-in default (endpoint) or nothing at all (key).+ Default,+}++impl SettingSource {+ pub fn as_str(self) -> &'static str {+ match self {+ Self::Config => "config",+ Self::Env => "env",+ Self::Default => "default",+ }+ }+}++/// The server's resolved provider settings: the injection seam between+/// configuration and a [`Provider`].+///+/// Resolved ONCE at startup and carried in the API layer's shared state. Two+/// reasons it is not read from the environment per request, which is what the+/// first draft did:+///+/// * tests run in parallel threads of one process, and a test that has to+/// `set_var("TYPESAFE_API_KEY")` races every other test that reads it;+/// * one `reqwest::Client` lives here, so consecutive searches share a+/// connection pool instead of paying a TLS handshake per rerank.+///+/// `Debug` is written by hand so the key can never reach a log line.+#[derive(Clone)]+pub struct ProviderSettings {+ /// `false` refuses every rerank request ([`RerankError::DisabledByOperator`]).+ pub enabled: bool,+ api_key: Option<String>,+ key_source: SettingSource,+ endpoint: String,+ endpoint_source: SettingSource,+ client: reqwest::Client,+}++impl std::fmt::Debug for ProviderSettings {+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {+ f.debug_struct("ProviderSettings")+ .field("enabled", &self.enabled)+ .field("api_key", &self.api_key.as_ref().map(|_| "<redacted>"))+ .field("key_source", &self.key_source)+ .field("endpoint", &self.endpoint_for_display())+ .field("endpoint_source", &self.endpoint_source)+ .finish()+ }+}++impl Default for ProviderSettings {+ /// No key, the public endpoint, enabled: a server that has not been set up.+ fn default() -> Self {+ Self::resolve(true, "", "", None, None)+ }+}++impl ProviderSettings {+ /// Pure resolution rule, split from the environment read so it can be+ /// tested without mutating process-wide state.+ ///+ /// Precedence for both values: a non-empty config field wins, then the+ /// environment, then the default. Explicit configuration beats the ambient+ /// environment for the reason `cluster.auth_secret` does — a stray variable+ /// in the service's environment must not silently re-point where document+ /// text is sent.+ pub fn resolve(+ enabled: bool,+ config_key: &str,+ config_endpoint: &str,+ env_key: Option<String>,+ env_endpoint: Option<String>,+ ) -> Self {+ let non_empty =+ |v: Option<String>| v.map(|s| s.trim().to_string()).filter(|s| !s.is_empty());+ let (api_key, key_source) =+ match (non_empty(Some(config_key.to_string())), non_empty(env_key)) {+ (Some(k), _) => (Some(k), SettingSource::Config),+ (None, Some(k)) => (Some(k), SettingSource::Env),+ (None, None) => (None, SettingSource::Default),+ };+ let (endpoint, endpoint_source) = match (+ non_empty(Some(config_endpoint.to_string())),+ non_empty(env_endpoint),+ ) {+ (Some(e), _) => (e, SettingSource::Config),+ (None, Some(e)) => (e, SettingSource::Env),+ (None, None) => (+ JevProvider::DEFAULT_ENDPOINT.to_string(),+ SettingSource::Default,+ ),+ };+ Self {+ enabled,+ api_key,+ key_source,+ endpoint,+ endpoint_source,+ client: reqwest::Client::new(),+ }+ }++ /// Config fields first, then `TYPESAFE_API_KEY` / `TYPESAFE_ENDPOINT`.+ pub fn from_config_and_env(enabled: bool, config_key: &str, config_endpoint: &str) -> Self {+ Self::resolve(+ enabled,+ config_key,+ config_endpoint,+ std::env::var(JevProvider::ENV_KEY).ok(),+ std::env::var(JevProvider::ENV_ENDPOINT).ok(),+ )+ }++ /// Settings for a known key and endpoint — what a test points at its stub.+ pub fn with_key_and_endpoint(api_key: &str, endpoint: &str) -> Self {+ Self::resolve(true, api_key, endpoint, None, None)+ }++ /// Whether a key is in force. Never the key.+ pub fn has_key(&self) -> bool {+ self.api_key.is_some()+ }+ pub fn key_source(&self) -> Option<SettingSource> {+ self.api_key.as_ref().map(|_| self.key_source)+ }+ pub fn endpoint_source(&self) -> SettingSource {+ self.endpoint_source+ }++ /// The endpoint with anything secret-shaped removed: userinfo, query+ /// string and fragment. An operator who put a token in the URL should not+ /// find it echoed by a status endpoint.+ pub fn endpoint_for_display(&self) -> String {+ match reqwest::Url::parse(&self.endpoint) {+ Ok(mut u) => {+ let _ = u.set_username("");+ let _ = u.set_password(None);+ u.set_query(None);+ u.set_fragment(None);+ u.to_string()+ }+ Err(_) => "<unparseable endpoint>".to_string(),+ }+ }+}++/// A configured reranking backend.+pub enum Provider {+ Jev(JevProvider),+ /// Configured but inert. An empty API key lands here rather than erroring,+ /// so a deployment that has not set one yet still serves searches.+ Disabled,+}++impl Provider {+ /// Build a provider from config plus the ambient environment.+ ///+ /// Kept for callers without server state (a CLI, a benchmark). A server+ /// uses [`Self::from_settings`], which never touches the environment.+ pub fn from_config(cfg: &RerankConfig) -> Result<Self, RerankError> {+ Self::from_settings(cfg, &ProviderSettings::from_config_and_env(true, "", ""))+ }++ /// Build a provider from the request's `rerank` block and the server's+ /// resolved settings. The environment is not consulted.+ pub fn from_settings(+ cfg: &RerankConfig,+ settings: &ProviderSettings,+ ) -> Result<Self, RerankError> {+ match cfg.provider.as_str() {+ "none" | "disabled" => Ok(Self::Disabled),+ "jev" | "typesafe" => {+ if !settings.enabled {+ return Err(RerankError::DisabledByOperator);+ }+ JevProvider::from_settings(settings).map(Self::Jev)+ }+ other => Err(RerankError::UnknownProvider(other.to_string())),+ }+ }++ /// Score and reorder `candidates`, honouring `deadline`.+ ///+ /// Never returns `Err` for a degradable cause — those come back as+ /// [`RerankOutcome::Degraded`] so the caller does not have to re-derive the+ /// policy from the error type.+ pub async fn rerank(+ &self,+ query: &str,+ candidates: &[Candidate],+ cfg: &RerankConfig,+ deadline: &Deadline,+ ) -> Result<RerankOutcome, RerankError> {+ if candidates.is_empty() {+ return Ok(RerankOutcome::Reordered {+ scores: Vec::new(),+ partial_failures: 0,+ usage: Usage::default(),+ });+ }+ if deadline.exceeded() {+ return Ok(RerankOutcome::Degraded {+ reason: "deadline exceeded before the provider was called".into(),+ });+ }+ match self {+ Self::Disabled => Ok(RerankOutcome::Degraded {+ reason: "rerank provider disabled (`rerank.provider` is `none`)".into(),+ }),+ Self::Jev(p) => match p.rerank(query, candidates, cfg, deadline).await {+ Ok(outcome) => Ok(outcome),+ Err(e) if e.policy() == Policy::Degrade => Ok(RerankOutcome::Degraded {+ reason: e.to_string(),+ }),+ Err(e) => Err(e),+ },+ }+ }+}++/// TypeSafe AI System One client.+pub struct JevProvider {+ client: reqwest::Client,+ endpoint: String,+ api_key: String,+}++impl JevProvider {+ pub const ENV_KEY: &'static str = "TYPESAFE_API_KEY";+ pub const ENV_ENDPOINT: &'static str = "TYPESAFE_ENDPOINT";+ pub const DEFAULT_ENDPOINT: &'static str = "https://api.typesafe.ai/v1/systemone";++ /// Shares the settings' HTTP client, so the pool outlives this request.+ pub fn from_settings(settings: &ProviderSettings) -> Result<Self, RerankError> {+ let api_key = settings+ .api_key+ .clone()+ .ok_or(RerankError::MissingKey(Self::ENV_KEY))?;+ Ok(Self {+ client: settings.client.clone(),+ endpoint: settings.endpoint.clone(),+ api_key,+ })+ }++ pub fn new(api_key: String, endpoint: String, timeout: Duration) -> Self {+ let client = reqwest::Client::builder()+ .timeout(timeout)+ .build()+ .unwrap_or_else(|_| reqwest::Client::new());+ Self {+ client,+ endpoint,+ api_key,+ }+ }++ /// One batch, retried while the deadline allows.+ ///+ /// Back-off shape follows Meilisearch's Cohere client+ /// (`personalization/mod.rs:308-321`): exponential in the attempt, with a+ /// flat additional penalty for rate limiting, because a 429 means the+ /// service wants distance rather than a slightly later retry.+ async fn call_batch_retrying(+ &self,+ query: &str,+ batch: &[Candidate],+ cfg: &RerankConfig,+ deadline: &Deadline,+ ) -> Result<(Vec<Scored>, Usage), RerankError> {+ const MAX_ATTEMPTS: u32 = 3;+ let mut attempt = 0;+ loop {+ if deadline.exceeded() {+ return Err(deadline.err());+ }+ match self.call_batch(query, batch, cfg, deadline).await {+ Ok(scores) => return Ok(scores),+ Err(e) => {+ attempt += 1;+ if attempt >= MAX_ATTEMPTS || !e.retryable() {+ return Err(e);+ }+ let base = Duration::from_millis(10u64.pow(attempt));+ let backoff = match &e {+ RerankError::Status { status: 429, .. } => {+ base + Duration::from_millis(100)+ }+ _ => base,+ };+ // Never sleep past the budget: a deadline that expires mid+ // back-off should degrade, not wake up to fail.+ if backoff >= deadline.remaining() {+ return Err(deadline.err());+ }+ tracing::debug!(+ attempt,+ backoff_ms = backoff.as_millis(),+ error = %e,+ "retrying rerank batch"+ );+ tokio::time::sleep(backoff).await;+ }+ }+ }+ }++ async fn call_batch(+ &self,+ query: &str,+ batch: &[Candidate],+ cfg: &RerankConfig,+ deadline: &Deadline,+ ) -> Result<(Vec<Scored>, Usage), RerankError> {+ let body = build_jev_request(query, batch, cfg);+ // Per request, not per client: the client is shared across searches+ // (see `ProviderSettings`), and each search brings its own budget. The+ // HTTP timeout is whatever is left of the stage deadline, so a call+ // can never outlive the search that is waiting on it.+ let http_timeout = cfg+ .timeout+ .min(deadline.remaining())+ .max(Duration::from_millis(1));+ let resp = self+ .client+ .post(&self.endpoint)+ .timeout(http_timeout)+ .bearer_auth(&self.api_key)+ .json(&body)+ .send()+ .await+ .map_err(|e| {+ // A timeout is the deadline speaking, not the network: report+ // it as one so the policy degrades instead of surfacing.+ if e.is_timeout() {+ deadline.err()+ } else {+ RerankError::Transport(e.to_string())+ }+ })?;++ let status = resp.status();+ let cap = if status.is_success() {+ MAX_RESPONSE_BYTES+ } else {+ ERROR_BODY_READ_BYTES+ };+ let (text, oversized) = read_capped(resp, cap, deadline).await?;++ if status.is_success() && oversized {+ return Err(RerankError::Malformed(format!(+ "response body larger than {MAX_RESPONSE_BYTES} bytes; a verdict for \+ {JEV_MAX_DOCS_PER_CALL} documents is a few kilobytes"+ )));+ }+ if !status.is_success() {+ // 429 and 529 are the documented back-off codes. Surfaced rather+ // than retried here: the caller is inside a search request with a+ // deadline, and fail-open beats spending it on retries.+ return Err(RerankError::Status {+ status: status.as_u16(),+ body: error_body(&text, &self.api_key),+ });+ }+ let (scores, usage) = parse_jev_response_with_usage(&text)?;+ Ok((keep_verdicts_for(batch, scores), usage))+ }++ pub async fn rerank(+ &self,+ query: &str,+ candidates: &[Candidate],+ cfg: &RerankConfig,+ deadline: &Deadline,+ ) -> Result<RerankOutcome, RerankError> {+ if candidates.is_empty() {+ return Ok(RerankOutcome::Reordered {+ scores: Vec::new(),+ partial_failures: 0,+ usage: Usage::default(),+ });+ }+ let batch_size = cfg.batch.clamp(1, JEV_MAX_DOCS_PER_CALL);+ let batches: Vec<Vec<Candidate>> =+ candidates.chunks(batch_size).map(|c| c.to_vec()).collect();++ let mut scores = Vec::with_capacity(candidates.len());+ let mut usage = Usage::default();+ let mut partial_failures = 0usize;+ let mut first_hard_error: Option<RerankError> = None;++ // Bounded waves rather than unbounded fan-out: the provider rate-limits,+ // and a search that trips a 429 is slower than one that paced itself.+ let mut dispatched = 0usize;+ for wave in batches.chunks(cfg.max_concurrency.max(1)) {+ if deadline.exceeded() {+ break;+ }+ dispatched += wave.len();+ let mut set = tokio::task::JoinSet::new();+ for batch in wave {+ let this = self.clone_for_task();+ let q = query.to_string();+ let c = cfg.clone();+ let b = batch.clone();+ let d = *deadline;+ set.spawn(async move { this.call_batch_retrying(&q, &b, &c, &d).await });+ }+ while let Some(joined) = set.join_next().await {+ match joined {+ Ok(Ok((mut part, used))) => {+ scores.append(&mut part);+ usage.add(used);+ }+ // One failed batch must not discard the batches that+ // succeeded: their scores are absolute probabilities, so a+ // partial result is still correctly ordered.+ Ok(Err(e)) => {+ partial_failures += 1;+ tracing::warn!(error = %e, "rerank batch failed");+ if first_hard_error.is_none() && e.policy() == Policy::Surface {+ first_hard_error = Some(e);+ }+ }+ Err(e) => {+ partial_failures += 1;+ tracing::warn!(error = %e, "rerank batch task panicked");+ }+ }+ }+ }++ // A batch the deadline stopped from ever being sent left its documents+ // exactly as unjudged as a batch that failed. Counting it makes the+ // block auditable: `judged` plus `partial_failures` batches accounts+ // for the whole window, instead of a window of 36 with 10 judged and+ // one failure and 16 documents unexplained.+ partial_failures += batches.len() - dispatched;++ // Nothing came back at all. Report why rather than silently handing back+ // the engine's order as if it had been reranked.+ if scores.is_empty() {+ if let Some(e) = first_hard_error {+ return Err(e);+ }+ return Ok(RerankOutcome::Degraded {+ reason: if deadline.exceeded() {+ format!("deadline exceeded after {}ms", deadline.budget.as_millis())+ } else {+ "provider returned no scores".to_string()+ },+ });+ }++ Ok(RerankOutcome::Reordered {+ scores,+ partial_failures,+ usage,+ })+ }++ fn clone_for_task(&self) -> Self {+ // `reqwest::Client` is an Arc internally; cloning shares the pool.+ Self {+ client: self.client.clone(),+ endpoint: self.endpoint.clone(),+ api_key: self.api_key.clone(),+ }+ }+}++#[cfg(test)]+mod tests {+ use super::*;++ fn cands(n: usize) -> Vec<Candidate> {+ (0..n)+ .map(|i| Candidate {+ ordinal: i,+ title: Some(format!("title {i}")),+ text: format!("body {i}"),+ })+ .collect()+ }++ #[test]+ fn a_provider_error_that_echoes_the_key_is_redacted() {+ let body = r#"{"error":"invalid api key: sk-operator-secret (Bearer sk-operator-secret)"}"#;+ let out = redact_key(body, "sk-operator-secret");+ assert!(!out.contains("sk-operator-secret"), "{out}");+ assert_eq!(out.matches("<redacted>").count(), 2, "{out}");+ // Nothing to redact leaves the text alone, and an empty key is not a+ // pattern that matches between every character.+ assert_eq!(redact_key("rate limited", "sk-x"), "rate limited");+ assert_eq!(redact_key("rate limited", ""), "rate limited");+ }++ /// The error body is cut to 400 characters for the caller. A key that+ /// straddles that cut must not survive as a prefix: cutting first left+ /// `sk-operator-sec…` for `redact_key` to miss, in a search response.+ #[test]+ fn a_key_straddling_the_error_cut_leaks_no_prefix() {+ let key = "sk-operator-secret-0123456789";+ for pad in 380..400 {+ let body = format!("{}{key} trailing", "x".repeat(pad));+ let out = error_body(&body, key);+ assert!(out.chars().count() <= 400, "pad {pad}: {out}");+ assert!(+ !out.contains("sk-op"),+ "pad {pad} leaked a key prefix: {out}"+ );+ }+ }++ /// A 2xx body past [`MAX_RESPONSE_BYTES`] is a contract break, not an+ /// allocation: whatever answers at the endpoint does not get to choose how+ /// much memory a search uses. A body inside the ceiling still parses.+ #[test]+ fn an_oversized_provider_response_is_refused_not_read_whole() {+ use std::io::{Read, Write};+ fn serve(body: String, chunked: bool) -> String {+ let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();+ let addr = listener.local_addr().unwrap();+ std::thread::spawn(move || {+ let (mut s, _) = listener.accept().unwrap();+ let mut buf = [0u8; 65536];+ let _ = s.read(&mut buf);+ let head = if chunked {+ "HTTP/1.1 200 OK\r\ncontent-type: application/json\r\n\+ transfer-encoding: chunked\r\nconnection: close\r\n\r\n"+ .to_string()+ } else {+ format!(+ "HTTP/1.1 200 OK\r\ncontent-type: application/json\r\n\+ content-length: {}\r\nconnection: close\r\n\r\n",+ body.len()+ )+ };+ let _ = s.write_all(head.as_bytes());+ if chunked {+ // No Content-Length to refuse up front: the cap must hold+ // while the body streams in.+ for piece in body.as_bytes().chunks(64 * 1024) {+ let _ = s.write_all(format!("{:x}\r\n", piece.len()).as_bytes());+ let _ = s.write_all(piece);+ let _ = s.write_all(b"\r\n");+ }+ let _ = s.write_all(b"0\r\n\r\n");+ } else {+ let _ = s.write_all(body.as_bytes());+ }+ });+ format!("http://{addr}/v1/systemone")+ }+ let huge = format!(+ r#"{{"answers":{{"d0":{{"noul":0.9}}}},"padding":"{}"}}"#,+ "x".repeat(MAX_RESPONSE_BYTES)+ );+ let small = r#"{"answers":{"d0":{"noul":0.9}}}"#.to_string();+ let cfg = RerankConfig::default();+ let rt = rt();+ for chunked in [false, true] {+ let p = JevProvider::new(+ "k".into(),+ serve(huge.clone(), chunked),+ Duration::from_secs(10),+ );+ let d = Deadline::new(Duration::from_secs(10));+ let err = rt+ .block_on(p.call_batch("q", &cands(1), &cfg, &d))+ .expect_err("an oversized body must not parse");+ assert!(+ matches!(&err, RerankError::Malformed(m) if m.contains("larger than")),+ "chunked={chunked}: {err}"+ );+ assert_eq!(err.policy(), Policy::Surface, "chunked={chunked}");++ let p = JevProvider::new(+ "k".into(),+ serve(small.clone(), chunked),+ Duration::from_secs(10),+ );+ let (scores, _) = rt+ .block_on(p.call_batch("q", &cands(1), &cfg, &d))+ .expect("a body inside the ceiling parses");+ assert_eq!(scores.len(), 1, "chunked={chunked}");+ }+ }++ #[test]+ fn request_matches_documented_wire_format() {+ let cfg = RerankConfig::default();+ let body = build_jev_request("vitamin d bone density", &cands(2), &cfg);++ assert_eq!(body["model"], "jev-latest");+ assert_eq!(body["state"]["query"], "vitamin d bone density");+ assert_eq!(body["state"]["documents"]["d0"]["text"], "body 0");+ // One noul question per document, keyed identically to the document.+ assert_eq!(body["questions"]["d0"]["type"], "noul");+ assert_eq!(body["questions"]["d1"]["type"], "noul");+ assert!(body["questions"]["d0"]["criteria"]["true"].is_string());+ assert_eq!(body["questions"].as_object().unwrap().len(), 2);+ }++ #[test]+ fn parses_documented_response() {+ let body = r#"{"model":"jev-latest","answers":{+ "d0":{"type":"noul","noul":0.92},+ "d1":{"type":"noul","noul":0.03}},+ "usage":{"input_tokens":312,"output_tokens":48}}"#;+ let mut got = parse_jev_response(body).unwrap();+ got.sort_by_key(|s| s.ordinal);+ assert_eq!(got.len(), 2);+ assert_eq!(got[0].ordinal, 0);+ assert!((got[0].score - 0.92).abs() < 1e-6);+ assert!((got[1].score - 0.03).abs() < 1e-6);+ }++ #[test]+ fn unanswered_documents_are_skipped_not_zeroed() {+ let body = r#"{"answers":{"d0":{"type":"noul","noul":0.5},"d1":{"type":"noul"}}}"#;+ let got = parse_jev_response(body).unwrap();+ assert_eq!(got.len(), 1, "d1 has no noul and must not become 0.0");+ assert_eq!(got[0].ordinal, 0);+ }++ #[test]+ fn reorders_by_probability() {+ let c = cands(3);+ let scores = vec![+ Scored {+ ordinal: 0,+ score: 0.10,+ },+ Scored {+ ordinal: 1,+ score: 0.90,+ },+ Scored {+ ordinal: 2,+ score: 0.50,+ },+ ];+ let out = apply_scores(&c, &scores, None);+ assert_eq!(+ out.iter().map(|s| s.ordinal).collect::<Vec<_>>(),+ vec![1, 2, 0]+ );+ }++ #[test]+ fn min_score_prunes() {+ let c = cands(3);+ let scores = vec![+ Scored {+ ordinal: 0,+ score: 0.10,+ },+ Scored {+ ordinal: 1,+ score: 0.90,+ },+ Scored {+ ordinal: 2,+ score: 0.50,+ },+ ];+ let out = apply_scores(&c, &scores, Some(0.4));+ assert_eq!(out.len(), 2);+ assert_eq!(out[0].ordinal, 1);+ assert_eq!(out[1].ordinal, 2);+ }++ #[test]+ fn unscored_hits_sort_below_scored_and_keep_engine_order() {+ let c = cands(4);+ // Only d2 came back scored, and with a low probability at that.+ let scores = vec![Scored {+ ordinal: 2,+ score: 0.01,+ }];+ let out = apply_scores(&c, &scores, None);+ assert_eq!(out[0].ordinal, 2, "the only scored hit leads");+ assert_eq!(+ out[1..].iter().map(|s| s.ordinal).collect::<Vec<_>>(),+ vec![0, 1, 3],+ "unscored hits keep the engine's relative order"+ );+ }++ #[test]+ fn ties_keep_engine_order() {+ let c = cands(3);+ let scores = vec![+ Scored {+ ordinal: 0,+ score: 0.7,+ },+ Scored {+ ordinal: 1,+ score: 0.7,+ },+ Scored {+ ordinal: 2,+ score: 0.7,+ },+ ];+ let out = apply_scores(&c, &scores, None);+ assert_eq!(+ out.iter().map(|s| s.ordinal).collect::<Vec<_>>(),+ vec![0, 1, 2]+ );+ }++ /// A provider that answers for documents this batch never sent — another+ /// batch's keys, or keys nobody sent — must not score them here, and must+ /// not count them. `judged` is the operator's billing meter.+ #[test]+ fn verdicts_are_kept_per_batch_one_per_document() {+ let batch = cands(3); // ordinals 0, 1, 2+ let answers = vec![+ Scored {+ ordinal: 2,+ score: 0.2,+ },+ Scored {+ ordinal: 0,+ score: 0.9,+ },+ // Another batch's documents, echoed back.+ Scored {+ ordinal: 7,+ score: 0.99,+ },+ Scored {+ ordinal: 30,+ score: 0.99,+ },+ // A key nobody sent.+ Scored {+ ordinal: 999,+ score: 0.99,+ },+ ];+ let kept = keep_verdicts_for(&batch, answers);+ assert_eq!(kept.len(), 2, "{kept:?}");+ assert_eq!(+ kept[0],+ Scored {+ ordinal: 0,+ score: 0.9+ }+ );+ assert_eq!(+ kept[1],+ Scored {+ ordinal: 2,+ score: 0.2+ }+ );++ // Only unknown keys: nothing was judged, and the caller must not be+ // told the order is the judge's.+ let none = keep_verdicts_for(+ &batch,+ vec![Scored {+ ordinal: 999,+ score: 0.99,+ }],+ );+ assert!(none.is_empty(), "{none:?}");+ }++ /// The probability the provider sent is the probability the caller gets,+ /// to the digit: `0.9` in, `0.9` out — not `0.8999999761581421`.+ #[test]+ fn probabilities_keep_double_precision() {+ let body =+ r#"{"answers":{"d0":{"type":"noul","noul":0.9},"d1":{"type":"noul","noul":0.6}}}"#;+ let scores = parse_jev_response(body).unwrap();+ let of = |o: usize| scores.iter().find(|s| s.ordinal == o).unwrap().score;+ assert_eq!(of(0), 0.9);+ assert_eq!(of(1), 0.6);+ assert_eq!(format!("{}", of(0)), "0.9");+ // The threshold a caller writes is compared without a narrowing step.+ let kept = apply_scores(&cands(2), &scores, Some(0.9));+ assert_eq!(kept.len(), 1, "{kept:?}");+ assert_eq!(kept[0].ordinal, 0);+ }++ #[test]+ fn batch_is_clamped_to_provider_ceiling() {+ let cfg = RerankConfig::from_json(&json!({"batch": 500})).unwrap();+ assert_eq!(cfg.batch, JEV_MAX_DOCS_PER_CALL);+ }++ #[test]+ fn unknown_field_is_rejected_not_ignored() {+ let err = RerankConfig::from_json(&json!({"treshold": 0.5})).unwrap_err();+ assert!(matches!(err, RerankError::Config(_)));+ assert!(err.to_string().contains("treshold"));+ }++ #[test]+ fn min_score_outside_probability_range_is_rejected() {+ assert!(RerankConfig::from_json(&json!({"min_score": 7.0})).is_err());+ assert!(RerankConfig::from_json(&json!({"min_score": -0.1})).is_err());+ assert!(RerankConfig::from_json(&json!({"min_score": 0.0})).is_ok());+ assert!(RerankConfig::from_json(&json!({"min_score": 1.0})).is_ok());+ }++ #[test]+ fn unknown_provider_is_rejected() {+ let cfg = RerankConfig::from_json(&json!({"provider": "voyage"})).unwrap();+ assert!(matches!(+ Provider::from_config(&cfg),+ Err(RerankError::UnknownProvider(_))+ ));+ }++ #[test]+ fn clip_never_splits_a_multibyte_char() {+ // The byte-slice version of this panicked and aborted a whole+ // autoindex run on exactly these characters.+ assert_eq!(clip("اختبار", 3).chars().count(), 3);+ assert_eq!(clip("设计文档", 2), "设计");+ assert_eq!(clip("ascii", 99), "ascii");+ }++ #[test]+ fn caller_chosen_cost_knobs_have_server_side_ceilings() {+ for (body, needle) in [+ (json!({"window": MAX_WINDOW + 1}), "rerank.window"),+ (+ json!({"max_concurrency": MAX_CONCURRENCY + 1}),+ "max_concurrency",+ ),+ (json!({"max_doc_chars": MAX_DOC_CHARS + 1}), "max_doc_chars"),+ (json!({"timeout_ms": MAX_TIMEOUT_MS + 1}), "timeout_ms"),+ ] {+ let err = RerankConfig::from_json(&body).expect_err("over the ceiling");+ assert!(err.to_string().contains(needle), "{err}");+ }+ assert!(RerankConfig::from_json(&json!({"window": MAX_WINDOW})).is_ok());+ }++ /// The strings a caller writes are cost knobs too — `instructions` most of+ /// all, because the wire format repeats it once per judged document.+ #[test]+ fn caller_written_strings_have_server_side_ceilings() {+ let long = |n: usize| "y".repeat(n);+ for (body, needle) in [+ (+ json!({"instructions": long(MAX_INSTRUCTIONS_CHARS + 1)}),+ "rerank.instructions",+ ),+ (json!({"query": long(MAX_QUERY_CHARS + 1)}), "rerank.query"),+ (json!({"model": long(MAX_MODEL_CHARS + 1)}), "rerank.model"),+ (+ json!({"provider": long(MAX_MODEL_CHARS + 1)}),+ "rerank.provider",+ ),+ (+ json!({"fields": vec!["f"; MAX_FIELDS + 1]}),+ "rerank.fields",+ ),+ (+ json!({"fields": [long(MAX_FIELD_NAME_CHARS + 1)]}),+ "rerank.fields",+ ),+ ] {+ let err = match RerankConfig::from_json(&body) {+ Err(e) => e.to_string(),+ Ok(_) => panic!("over the ceiling must be refused: needle {needle}"),+ };+ assert!(err.contains(needle), "{err}");+ // The refusal names the limit and never echoes the oversized value.+ assert!(+ err.len() < 400,+ "refusal must not echo the value: {} bytes",+ err.len()+ );+ }+ // Exactly at the ceiling is accepted, and the ceiling counts+ // characters, not bytes: a multi-byte string of the same length fits.+ assert!(RerankConfig::from_json(&json!({+ "instructions": long(MAX_INSTRUCTIONS_CHARS),+ "query": long(MAX_QUERY_CHARS),+ "model": long(MAX_MODEL_CHARS),+ "fields": vec![long(MAX_FIELD_NAME_CHARS); MAX_FIELDS],+ }))+ .is_ok());+ assert!(RerankConfig::from_json(&json!({+ "instructions": "é".repeat(MAX_INSTRUCTIONS_CHARS),+ }))+ .is_ok());+ }++ /// An unknown key or provider is named in the refusal, but a caller must+ /// not be able to make the node echo megabytes back by naming a huge one.+ #[test]+ fn refusals_do_not_echo_oversized_names() {+ let huge = "k".repeat(1_000_000);+ let mut block = Map::new();+ block.insert(huge, json!(1));+ let err = RerankConfig::from_json(&Value::Object(block))+ .expect_err("unknown key")+ .to_string();+ assert!(+ err.contains("unknown `rerank` field"),+ "{}",+ &err[..200.min(err.len())]+ );+ assert!(err.len() < 1_000, "echoed {} bytes", err.len());+ }++ /// The wire is bounded by construction, whatever built the `RerankConfig`:+ /// the fields are public, so the parser's refusal is not the only guard.+ #[test]+ fn one_provider_call_is_bounded_whatever_the_config_holds() {+ let cfg = RerankConfig {+ instructions: Some("i".repeat(5_000_000)),+ model: "m".repeat(1_000_000),+ max_doc_chars: MAX_DOC_CHARS,+ ..RerankConfig::default()+ };+ let batch: Vec<Candidate> = (0..JEV_MAX_DOCS_PER_CALL)+ .map(|i| Candidate {+ ordinal: i,+ title: Some("t".repeat(100_000)),+ text: "x".repeat(100_000),+ })+ .collect();+ let question = "q".repeat(3_000_000);+ let body = build_jev_request(&question, &batch, &cfg).to_string();+ // 30 documents x (title + text at the ceiling + instructions at the+ // ceiling) + the question + the model + JSON framing.+ let bound = JEV_MAX_DOCS_PER_CALL * (2 * MAX_DOC_CHARS + MAX_INSTRUCTIONS_CHARS + 512)+ + MAX_QUERY_CHARS+ + MAX_MODEL_CHARS+ + 1_024;+ assert!(+ body.len() <= bound,+ "one call put {} bytes on the wire; the bound is {bound}",+ body.len()+ );+ }++ /// `usage` is advisory metering. It must never veto a ranking whose+ /// verdicts all parsed — a provider that reports `-5`, a float, a string+ /// or `null` there used to fail the whole call with a 502 and no hits.+ #[test]+ fn odd_usage_never_discards_valid_verdicts() {+ for usage in [+ r#"{"input_tokens":9223372036854775808,"output_tokens":-5}"#,+ r#"{"input_tokens":"many","output_tokens":null}"#,+ r#"{"input_tokens":12.0,"output_tokens":3.9}"#,+ r#""n/a""#,+ r#"null"#,+ r#"[1,2]"#,+ ] {+ let body = format!(+ r#"{{"answers":{{"d0":{{"type":"noul","noul":0.5}},"d1":{{"type":"noul","noul":0.25}}}},"usage":{usage}}}"#+ );+ let (mut scores, u) = match parse_jev_response_with_usage(&body) {+ Ok(v) => v,+ Err(e) => panic!("usage {usage} vetoed two valid verdicts: {e}"),+ };+ scores.sort_by_key(|s| s.ordinal);+ assert_eq!(scores.len(), 2, "usage {usage}");+ assert_eq!(scores[0].score, 0.5);+ // A number that is a whole, non-negative count is kept; anything+ // else meters as zero rather than as a guess.+ if usage.contains("12.0") {+ assert_eq!((u.input_tokens, u.output_tokens), (12, 3), "usage {usage}");+ }+ if usage.contains("-5") {+ // 2^63 input tokens is a number, but not one a call of 30+ // documents can produce: it must not reach the operator's+ // counter, where one such response would poison the series.+ assert_eq!(u, Usage::default(), "usage {usage}");+ }+ }+ // The largest count one call could plausibly report is kept.+ let (_, u) = parse_jev_response_with_usage(&format!(+ r#"{{"answers":{{}},"usage":{{"input_tokens":{MAX_PLAUSIBLE_TOKENS_PER_CALL},"output_tokens":{}}}}}"#,+ MAX_PLAUSIBLE_TOKENS_PER_CALL + 1+ ))+ .unwrap();+ assert_eq!(+ (u.input_tokens, u.output_tokens),+ (MAX_PLAUSIBLE_TOKENS_PER_CALL, 0)+ );+ // Malformed ANSWERS are still a contract break.+ assert!(parse_jev_response_with_usage(r#"{"answers":[1,2]}"#).is_err());+ assert!(parse_jev_response_with_usage(+ r#"{"answers":{"d0":{"type":"noul","noul":"high"}}}"#+ )+ .is_err());+ }++ // ── server-side settings: the env-free seam ─────────────────────────────++ #[test]+ fn config_beats_env_and_env_beats_default() {+ let s = ProviderSettings::resolve(+ true,+ "from-config",+ "http://config.example/v1",+ Some("from-env".into()),+ Some("http://env.example/v1".into()),+ );+ assert_eq!(s.key_source(), Some(SettingSource::Config));+ assert_eq!(s.endpoint_source(), SettingSource::Config);+ assert_eq!(s.endpoint_for_display(), "http://config.example/v1");++ let s = ProviderSettings::resolve(true, "", " ", Some("from-env".into()), None);+ assert_eq!(s.key_source(), Some(SettingSource::Env));+ assert_eq!(s.endpoint_source(), SettingSource::Default);+ assert_eq!(s.endpoint_for_display(), JevProvider::DEFAULT_ENDPOINT);++ // Whitespace is not a key.+ let s = ProviderSettings::resolve(true, "", "", Some(" ".into()), None);+ assert!(!s.has_key());+ assert_eq!(s.key_source(), None);+ }++ #[test]+ fn debug_output_never_contains_the_key() {+ let s = ProviderSettings::with_key_and_endpoint(+ "sk-very-secret",+ "https://user:[email protected]/v1/systemone?token=abc#frag",+ );+ let dbg = format!("{s:?}");+ assert!(!dbg.contains("sk-very-secret"), "{dbg}");+ assert!(!dbg.contains("hunter2"), "{dbg}");+ assert!(!dbg.contains("token=abc"), "{dbg}");+ assert!(dbg.contains("<redacted>"), "{dbg}");+ assert_eq!(+ s.endpoint_for_display(),+ "https://api.example/v1/systemone",+ "userinfo, query and fragment are all places a secret hides"+ );+ }++ #[test]+ fn missing_key_is_a_surfaced_error_not_a_silent_degrade() {+ let settings = ProviderSettings::resolve(true, "", "", None, None);+ let err = match Provider::from_settings(&RerankConfig::default(), &settings) {+ Err(e) => e,+ Ok(_) => panic!("no key must not build a working provider"),+ };+ assert!(matches!(err, RerankError::MissingKey(_)));+ assert_eq!(err.policy(), Policy::Surface);+ // The message must not tell callers to put a key in the request: there+ // is no such field, and a key in a search body ends up in slow-query+ // logs and audit trails.+ assert!(!err.to_string().contains("in the request"), "{err}");+ }++ #[test]+ fn an_operator_can_forbid_reranking_outright() {+ let settings = ProviderSettings::resolve(false, "a-key", "", None, None);+ let err = match Provider::from_settings(&RerankConfig::default(), &settings) {+ Err(e) => e,+ Ok(_) => panic!("enabled = false must refuse even with a key"),+ };+ assert!(matches!(err, RerankError::DisabledByOperator));+ assert_eq!(err.policy(), Policy::Surface);+ }++ #[test]+ fn usage_is_parsed_and_absent_usage_is_zero() {+ let (_, u) = parse_jev_response_with_usage(+ r#"{"answers":{"d0":{"type":"noul","noul":0.5}},+ "usage":{"input_tokens":312,"output_tokens":48}}"#,+ )+ .unwrap();+ assert_eq!((u.input_tokens, u.output_tokens), (312, 48));+ let (_, u) =+ parse_jev_response_with_usage(r#"{"answers":{"d0":{"type":"noul","noul":0.5}}}"#)+ .unwrap();+ assert_eq!(u, Usage::default());+ }++ #[test]+ fn out_of_range_probabilities_are_clamped() {+ let mut got = parse_jev_response(+ r#"{"answers":{"d0":{"type":"noul","noul":1.7},"d1":{"type":"noul","noul":-0.2}}}"#,+ )+ .unwrap();+ got.sort_by_key(|s| s.ordinal);+ assert_eq!(got[0].score, 1.0);+ assert_eq!(got[1].score, 0.0);+ }++ fn rt() -> tokio::runtime::Runtime {+ tokio::runtime::Builder::new_current_thread()+ .enable_time()+ .enable_io()+ .build()+ .unwrap()+ }++ /// Unroutable on purpose — port 1 refuses immediately.+ fn dead_provider() -> JevProvider {+ JevProvider::new(+ "k".into(),+ "http://127.0.0.1:1/never".into(),+ Duration::from_millis(50),+ )+ }++ #[test]+ fn empty_candidate_list_makes_no_request() {+ let out = rt()+ .block_on(dead_provider().rerank(+ "q",+ &[],+ &RerankConfig::default(),+ &Deadline::new(Duration::from_secs(1)),+ ))+ .unwrap();+ assert!(matches!(+ out,+ RerankOutcome::Reordered { ref scores, .. } if scores.is_empty()+ ));+ }++ #[test]+ fn disabled_provider_degrades_it_does_not_error() {+ let out = rt()+ .block_on(Provider::Disabled.rerank(+ "q",+ &cands(3),+ &RerankConfig::default(),+ &Deadline::new(Duration::from_secs(1)),+ ))+ .unwrap();+ assert!(matches!(out, RerankOutcome::Degraded { .. }));+ }++ #[test]+ fn expired_deadline_degrades_without_calling_the_provider() {+ let cfg = RerankConfig::default();+ // Zero budget is already exceeded; a call would hang on the dead port.+ let out = rt()+ .block_on(Provider::Jev(dead_provider()).rerank(+ "q",+ &cands(3),+ &cfg,+ &Deadline::new(Duration::ZERO),+ ))+ .unwrap();+ match out {+ RerankOutcome::Degraded { reason } => assert!(reason.contains("deadline")),+ other => panic!("expected degrade, got {other:?}"),+ }+ }++ #[test]+ fn transport_failure_surfaces_rather_than_pretending_to_rerank() {+ let cfg = RerankConfig {+ // No retry budget to burn; we want the error, not a timeout.+ timeout: Duration::from_millis(50),+ ..RerankConfig::default()+ };+ let err = rt()+ .block_on(Provider::Jev(dead_provider()).rerank(+ "q",+ &cands(2),+ &cfg,+ &Deadline::new(Duration::from_secs(5)),+ ))+ .expect_err("a dead provider must not look like a successful rerank");+ assert_eq!(err.policy(), Policy::Surface);+ }++ #[test]+ fn error_policy_split_matches_the_documented_rule() {+ // Degrade: ran out of time, caller still has results.+ assert_eq!(+ RerankError::Deadline { elapsed_ms: 10 }.policy(),+ Policy::Degrade+ );+ // Surface: configuration and contract faults.+ assert_eq!(+ RerankError::MissingKey("TYPESAFE_API_KEY").policy(),+ Policy::Surface+ );+ assert_eq!(+ RerankError::Status {+ status: 401,+ body: String::new()+ }+ .policy(),+ Policy::Surface+ );+ assert_eq!(+ RerankError::Malformed("bad json".into()).policy(),+ Policy::Surface+ );+ }++ #[test]+ fn only_documented_backoff_codes_are_retried() {+ let retry = |code| {+ RerankError::Status {+ status: code,+ body: String::new(),+ }+ .retryable()+ };+ // TypeSafe documents 429 and 529 as back-off-and-retry.+ assert!(retry(429));+ assert!(retry(529));+ assert!(retry(500));+ // Contract errors: retrying only burns the deadline.+ assert!(!retry(401));+ assert!(!retry(422));+ assert!(!RerankError::Malformed("x".into()).retryable());+ }++ #[test]+ fn deadline_reports_remaining_budget() {+ let d = Deadline::new(Duration::from_millis(500));+ assert!(!d.exceeded());+ assert!(d.remaining() <= Duration::from_millis(500));+ let spent = Deadline::new(Duration::ZERO);+ assert!(spent.exceeded());+ assert_eq!(spent.remaining(), Duration::ZERO);+ }++ #[test]+ fn batching_splits_at_the_provider_ceiling() {+ let cfg = RerankConfig::default();+ let c = cands(70);+ let batches: Vec<_> = c.chunks(cfg.batch).collect();+ assert_eq!(batches.len(), 3, "70 docs at 30/call is 3 calls");+ assert_eq!(batches[0].len(), 30);+ assert_eq!(batches[2].len(), 10);+ }++ #[test]+ fn ordinals_survive_batching() {+ // The response keys carry the caller's ordinal, not a position within+ // the batch, so a second-batch document maps back correctly.+ let cfg = RerankConfig::default();+ let c = cands(40);+ let body = build_jev_request("q", &c[30..], &cfg);+ assert!(body["questions"]["d30"].is_object());+ assert!(body["questions"]["d39"].is_object());+ assert!(body["questions"]["d0"].is_null());+ }+}diff --git a/engine/crates/xerj-rerank/tests/egress_inventory.rs b/engine/crates/xerj-rerank/tests/egress_inventory.rsnew file mode 100644index 000000000..0aab82ab2--- /dev/null+++ b/engine/crates/xerj-rerank/tests/egress_inventory.rs@@ -0,0 +1,398 @@+//! Every outbound connection the engine source can open, checked against the+//! list `docs/RERANK.md` publishes ("Every way data leaves a XERJ node").+//!+//! Why this exists: the first version of the rerank documentation told+//! operators — and, through `llms.txt` and the MCP tool descriptions, the+//! agents that talk to them — that reranking was "the ONE XERJ feature that+//! sends document text off the machine". Proxy embeddings and the WAL tap+//! already did. A privacy statement is only as good as the inventory under it,+//! so the inventory is a test: a new outbound client anywhere under+//! `engine/crates/*/src` fails here until it is classified below AND the+//! published list accounts for it.+//!+//! The scan is textual (it looks for the constructors of every network client+//! the workspace depends on), so it can over-report — that is the safe+//! direction. A file whose only use is test code is classified `TestOnly`, and+//! that claim is itself checked: every marker must sit after the file's first+//! `#[cfg(test)]`, or the whole file must be a module declared under+//! `#[cfg(test)]`.++use std::collections::BTreeSet;+use std::path::{Path, PathBuf};++/// Constructors and entry points of outbound network clients. Types used only+/// for parsing (`reqwest::Url`) are deliberately absent.+const MARKERS: &[&str] = &[+ // Object storage: aws-sdk-s3 is already a workspace dependency and the+ // S3 backend/source PRs put it on the engine path. Without these the+ // inventory would stay green while a whole outbound client went unlisted.+ "aws_sdk_s3::",+ "aws_config::",+ "object_store::",+ "reqwest::Client",+ "reqwest::ClientBuilder",+ "reqwest::blocking::Client",+ "reqwest::get(",+ "reqwest::blocking::get(",+ "hf_hub::",+ "ureq::",+ "TcpStream::connect",+ "TcpSocket::",+ "UdpSocket::",+ "hyper::client",+ "hyper_util::client",+ "tonic::transport::Channel",+ "tonic::transport::Endpoint",+ "tungstenite",+ "lettre::",+ "isahc::",+ "attohttpc::",+ "surf::",+ "Command::new(\"gh\")",+ "Command::new(\"curl\")",+ "Command::new(\"wget\")",+ "Command::new(\"ssh\")",+];++#[derive(Debug, Clone, Copy)]+enum Role {+ /// The running node opens this connection. The string is the row label+ /// the published table must carry.+ Node(&'static str),+ /// A command-line client in the same binary. The string must appear in+ /// the published section.+ Client(&'static str),+ /// Only test code in this file opens a connection.+ TestOnly,+}++/// Paths are relative to `engine/crates`.+const KNOWN: &[(&str, Role)] = &[+ ("xerj-rerank/src/lib.rs", Role::Node("**Rerank provider**")),+ ("xerj-ai/src/embed.rs", Role::Node("**Proxy embeddings**")),+ ("xerj-engine/src/wal_tap.rs", Role::Node("**WAL tap**")),+ (+ "xerj-ai/src/neural.rs",+ Role::Node("**Neural model download**"),+ ),+ (+ "xerj-cluster/src/transport.rs",+ Role::Node("**Cluster transport**"),+ ),+ ("xerj-mcp/src/lib.rs", Role::Client("`xerj mcp`")),+ (+ "xerj-autoindex/src/esclient.rs",+ Role::Client("`xerj autoindex`"),+ ),+ (+ "xerj-autoindex/src/feedback.rs",+ Role::Client("`xerj feedback --open-pr`"),+ ),+ // Object storage, added by #966 (the backend) and #970 (`autoindex s3://`).+ // Both talk to whatever endpoint the operator names, so they belong on the+ // published list even though neither sends DOCUMENT TEXT anywhere: the+ // source reads objects in, and the backend is not on the segment path yet+ // (`storage.backend = "s3"` still refuses to start).+ (+ "xerj-storage/src/s3.rs",+ Role::Node("**Object storage backend**"),+ ),+ (+ "xerj-autoindex/src/objsource.rs",+ Role::Client("`xerj autoindex s3://`"),+ ),+ (+ "xerj-autoindex/src/objsource_minio_tests.rs",+ Role::TestOnly,+ ),+ ("xerj-autoindex/src/objsource_s3_tests.rs", Role::TestOnly),+ ("xerj-api/src/binary_protocol.rs", Role::TestOnly),+ ("xerj-server/src/grpc.rs", Role::TestOnly),+ ("xerj-server/src/main.rs", Role::TestOnly),+ (+ "xerj-autoindex/src/incremental_reconcile_http_tests.rs",+ Role::TestOnly,+ ),+ (+ "xerj-autoindex/src/failure_resume_http_tests.rs",+ Role::TestOnly,+ ),+ ("xerj-autoindex/src/detect/e2e.rs", Role::TestOnly),+];++/// Phrasings that were false and must not come back on a published surface.+const FALSE_CLAIMS: &[&str] = &[+ "the one xerj feature",+ "the one feature that sends",+ "one feature that sends document text",+ "nothing else in xerj sends",+ "no other request sends",+ "every other request — indexing, search, embedding",+ "none of which a search request can trigger",+ "not something a search request can trigger",+ "two other outbound paths",+ "the other two outbound paths",+ "the other outbound paths",+ "other operator-configured outbound paths",+];++/// The surfaces that carry the rerank egress statement.+const SURFACES: &[&str] = &[+ "docs/RERANK.md",+ // ROADMAP.md and ZERO_TOKEN_DIRECTION.md repeated the false claim for a+ // week after every other surface was fixed, because neither was scanned —+ // and landing/llms.txt sends agents straight to ROADMAP.md#the-zero-token-direction.+ "ROADMAP.md",+ "docs/ZERO_TOKEN_DIRECTION.md",+ "docs/ARCHITECTURE.md",+ "docs/recipes/air-gapped-deployment.md",+ "README.md",+ "CHANGELOG.md",+ "content/answers/rerank-search-results-calibrated-judge.md",+ "content/answers/hybrid-search-quality-measured-ndcg.md",+ "landing/answers/rerank-search-results-calibrated-judge.md",+ "landing/answers/rerank-search-results-calibrated-judge.html",+ "landing/llms.txt",+ "landing/llms-full.txt",+ "landing/openapi.json",+ "landing/docs/agents/schemas/mcp-tools.json",+ "landing/docs/agents/schemas/anthropic-tools.json",+ "landing/docs/agents/schemas/openai-tools.json",+ "landing/docs/api-es-compat.html",+ "landing/docs/config.html",+ "landing/docs/env.html",+ "landing/docs/recipes/air-gapped-deployment.html",+ "engine/xerj.default.toml",+ "engine/crates/xerj-common/src/config.rs",+ "engine/crates/xerj-api/src/rerank_stage.rs",+ "engine/crates/xerj-mcp/src/lib.rs",+ "engine/crates/xerj-rerank/src/lib.rs",+];++fn crates_dir() -> PathBuf {+ Path::new(env!("CARGO_MANIFEST_DIR"))+ .parent()+ .expect("crates dir")+ .to_path_buf()+}++fn repo_root() -> PathBuf {+ crates_dir()+ .parent()+ .and_then(Path::parent)+ .expect("repo root")+ .to_path_buf()+}++fn read(path: &Path) -> String {+ std::fs::read_to_string(path).unwrap_or_else(|e| panic!("read {}: {e}", path.display()))+}++fn rust_sources(dir: &Path, out: &mut Vec<PathBuf>) {+ let Ok(entries) = std::fs::read_dir(dir) else {+ return;+ };+ for entry in entries.flatten() {+ let path = entry.path();+ if path.is_dir() {+ rust_sources(&path, out);+ } else if path.extension().is_some_and(|e| e == "rs") {+ out.push(path);+ }+ }+}++/// The file with comment-only lines removed. A marker named in prose — a doc+/// comment saying "in production replace this with `aws_sdk_s3::Client`" — is+/// not an outbound client, and counting it would force a real entry in KNOWN+/// for a file that opens no connection. Lines with code before a trailing `//`+/// are kept whole, so a marker can never hide behind a comment.+fn code_of(text: &str) -> String {+ text.lines()+ .filter(|l| !l.trim_start().starts_with("//"))+ .collect::<Vec<_>>()+ .join("\n")+}++/// Every `.rs` file under `engine/crates/*/src` that names a network client,+/// keyed by its path relative to `engine/crates`.+fn files_with_outbound_clients() -> Vec<(String, String)> {+ let crates = crates_dir();+ let mut found = Vec::new();+ let mut crate_dirs: Vec<PathBuf> = std::fs::read_dir(&crates)+ .expect("read engine/crates")+ .flatten()+ .map(|e| e.path())+ .filter(|p| p.join("src").is_dir())+ .collect();+ crate_dirs.sort();+ for krate in crate_dirs {+ let mut files = Vec::new();+ rust_sources(&krate.join("src"), &mut files);+ files.sort();+ for file in files {+ let text = read(&file);+ if MARKERS.iter().any(|m| code_of(&text).contains(m)) {+ let rel = file+ .strip_prefix(&crates)+ .expect("under engine/crates")+ .to_string_lossy()+ .replace('\\', "/");+ found.push((rel, text));+ }+ }+ }+ found+}++/// Is `file` (relative to `engine/crates`) a module its parent declares under+/// `#[cfg(test)]`?+fn declared_test_only(rel: &str) -> bool {+ let path = crates_dir().join(rel);+ let stem = path.file_stem().unwrap().to_string_lossy().to_string();+ let dir = path.parent().unwrap();+ let parents = [dir.join("mod.rs"), dir.join("lib.rs"), dir.join("main.rs")];+ parents.iter().filter(|p| p.is_file()).any(|p| {+ let lines: Vec<String> = read(p).lines().map(|l| l.trim().to_string()).collect();+ lines.windows(2).any(|w| {+ w[0] == "#[cfg(test)]"+ && (w[1] == format!("mod {stem};")+ || w[1] == format!("pub mod {stem};")+ || w[1] == format!("pub(crate) mod {stem};"))+ })+ })+}++fn published_section() -> String {+ let doc = read(&repo_root().join("docs/RERANK.md"));+ let start = doc+ .find("### Every way data leaves a XERJ node")+ .expect("docs/RERANK.md has the 'Every way data leaves a XERJ node' section");+ let rest = &doc[start..];+ // The section runs to the next heading of the same or a higher level.+ let end = rest[4..]+ .find("\n## ")+ .into_iter()+ .chain(rest[4..].find("\n### "))+ .min()+ .map(|i| i + 4)+ .unwrap_or(rest.len());+ rest[..end].to_string()+}++#[test]+fn every_outbound_client_in_the_engine_is_on_the_published_list() {+ let found = files_with_outbound_clients();+ let found_set: BTreeSet<&str> = found.iter().map(|(p, _)| p.as_str()).collect();+ let known_set: BTreeSet<&str> = KNOWN.iter().map(|(p, _)| *p).collect();++ let unlisted: Vec<&&str> = found_set.difference(&known_set).collect();+ assert!(+ unlisted.is_empty(),+ "new outbound network client(s) in {unlisted:?}. Classify each in KNOWN \+ (node / client / test-only) and, if the node or a client can now send \+ data somewhere, add it to docs/RERANK.md \"Every way data leaves a XERJ \+ node\", to xerj_rerank::DATA_EGRESS and to the air-gapped recipe."+ );+ let stale: Vec<&&str> = known_set.difference(&found_set).collect();+ assert!(+ stale.is_empty(),+ "{stale:?} no longer open(s) a connection: remove the entry from KNOWN and \+ check whether the published list still describes the engine"+ );++ for (rel, text) in &found {+ let role = KNOWN.iter().find(|(p, _)| p == rel).unwrap().1;+ if let Role::TestOnly = role {+ if declared_test_only(rel) {+ continue;+ }+ let test_start = text.find("#[cfg(test)]").unwrap_or_else(|| {+ panic!("{rel} is classified test-only but has no #[cfg(test)] section")+ });+ for m in MARKERS {+ if let Some(at) = text.find(m) {+ assert!(+ at > test_start,+ "{rel} is classified test-only, but `{m}` appears outside its \+ test code: classify it as a node or client path and publish it"+ );+ }+ }+ }+ }+}++#[test]+fn the_published_list_names_every_path() {+ let section = published_section();+ for (rel, role) in KNOWN {+ match role {+ Role::Node(label) | Role::Client(label) => assert!(+ section.contains(label),+ "docs/RERANK.md \"Every way data leaves a XERJ node\" does not mention \+ {label} (from {rel})"+ ),+ Role::TestOnly => {}+ }+ }+ // The air-gapped recipe is the other place an operator looks.+ let recipe = read(&repo_root().join("docs/recipes/air-gapped-deployment.md"));+ for row in [+ "| External proxy |",+ "| WAL tap |",+ "| Neural model |",+ "| Cluster |",+ "| Rerank provider |",+ ] {+ assert!(recipe.contains(row), "air-gapped recipe lost its {row} row");+ }+}++#[test]+fn the_status_statement_names_every_node_path_and_the_docs_quote_it() {+ let s = xerj_rerank::DATA_EGRESS;+ for needle in [+ "only search-time feature",+ "default_endpoint",+ "WAL tap",+ "HuggingFace",+ "cluster mode",+ ] {+ assert!(+ s.contains(needle),+ "DATA_EGRESS does not mention {needle}: {s}"+ );+ }+ let doc = read(&repo_root().join("docs/RERANK.md"));+ assert!(+ doc.contains(s),+ "docs/RERANK.md's GET /_xerj/rerank example must quote DATA_EGRESS verbatim"+ );+}++#[test]+fn no_published_surface_repeats_a_false_egress_claim() {+ let root = repo_root();+ let mut hits = Vec::new();+ for surface in SURFACES {+ let path = root.join(surface);+ let Ok(text) = std::fs::read_to_string(&path) else {+ panic!("surface {surface} is missing — update SURFACES");+ };+ // Folded whitespace and case: the claims were wrapped across lines and+ // written in more than one case.+ let folded = text+ .split_whitespace()+ .collect::<Vec<_>>()+ .join(" ")+ .to_lowercase();+ for claim in FALSE_CLAIMS {+ // This file lists the phrases it forbids.+ if folded.contains(claim) && !surface.ends_with("egress_inventory.rs") {+ hits.push(format!("{surface}: \"{claim}\""));+ }+ }+ }+ assert!(hits.is_empty(), "false egress claims: {hits:#?}");+}diff --git a/engine/crates/xerj-server/src/grpc.rs b/engine/crates/xerj-server/src/grpc.rsindex 7cf6f0a78..4f087a2f0 100644--- a/engine/crates/xerj-server/src/grpc.rs+++ b/engine/crates/xerj-server/src/grpc.rs@@ -128,6 +128,18 @@ impl XerjSearch for GrpcService {Status::invalid_argument(format!("query_json is not valid JSON: {e}"))})?};+ // `query_json` may be a full request body, and `parse_request` ignores+ // keys it does not know — so a `rerank` block vanished here and the+ // caller got the engine's order back, believing it had been reranked.+ // Refused by name instead. Checked before the bare-clause wrap below,+ // which would otherwise bury the key inside `query`.+ // `carries_rerank`, the rule every HTTP surface uses: `"rerank": null`+ // is an absent block, not a rerank request.+ if xerj_api::rerank_stage::carries_rerank(&body) {+ return Err(Status::invalid_argument(+ xerj_api::rerank_stage::unsupported_reason("the gRPC Search RPC"),+ ));+ }if body.get("query").is_none() {body = serde_json::json!({ "query": body });}@@ -586,6 +598,41 @@ mod tests {.into_inner();assert!(!missing.found);+ // ── A `rerank` block is refused by name, never silently dropped ──+ // `query_json` may be a whole request body and `parse_request`+ // ignores unknown keys, so this used to return the engine's order.+ let refused = client+ .search(pb::SearchRequest {+ index: "grpc_test".into(),+ query_json: r#"{"query":{"match":{"title":"hello"}},"rerank":{}}"#.into(),+ size: 10,+ from: 0,+ })+ .await+ .expect_err("a rerank block must be refused on gRPC");+ assert_eq!(refused.code(), tonic::Code::InvalidArgument, "{refused:?}");+ assert!(refused.message().contains("rerank"), "{refused:?}");+ assert!(refused.message().contains("_search"), "{refused:?}");++ // `"rerank": null` is the same as no `rerank` key, here as on every+ // HTTP surface. This check used to be `get("rerank").is_some()`, which+ // is true for JSON null, so gRPC alone refused what the docs call+ // "absent on every surface".+ let null_is_absent = client+ .search(pb::SearchRequest {+ index: "grpc_test".into(),+ query_json: r#"{"query":{"match":{"title":"hello"}},"rerank":null}"#.into(),+ size: 10,+ from: 0,+ })+ .await+ .expect("`rerank: null` is an absent block, not a rerank request")+ .into_inner();+ assert_eq!(+ null_is_absent.hits.first().map(|h| h.id.as_str()),+ Some("doc1")+ );+// ── Search must see the freshly indexed (memtable) doc ───────────let search = client.search(pb::SearchRequest {diff --git a/engine/crates/xerj-server/src/main.rs b/engine/crates/xerj-server/src/main.rsindex 01b7030ee..17107c539 100644--- a/engine/crates/xerj-server/src/main.rs+++ b/engine/crates/xerj-server/src/main.rs@@ -513,6 +513,15 @@ fn load_config(args: &CliArgs) -> Result<Config> {cfg.server.data_dir = dir.clone();}+ // `rerank.endpoint` is where document text gets POSTed. `Config::validate`+ // checked the file's value; the `TYPESAFE_ENDPOINT` fallback is only+ // known here, at boot, and gets the same two rules — the docs promise the+ // node refuses to start on a credentialed or non-http endpoint, whichever+ // place it came from.+ if let Err(reason) = cfg.rerank.check_environment() {+ anyhow::bail!("config error: {reason}");+ }+// Merge settings this build accepts but does not act on. An operator who// throttles merges to protect query latency was getting no throttle and no// signal (#207); the signal is the minimum. Seediff --git a/engine/crates/xerj-server/tests/rerank_env_endpoint_refused_at_boot.rs b/engine/crates/xerj-server/tests/rerank_env_endpoint_refused_at_boot.rsnew file mode 100644index 000000000..67b3ee248--- /dev/null+++ b/engine/crates/xerj-server/tests/rerank_env_endpoint_refused_at_boot.rs@@ -0,0 +1,193 @@+//! `docs/RERANK.md` says a `rerank.endpoint` with `user:password@` in it, or+//! without an `http(s)://` scheme, stops the node from starting — and that the+//! rule holds whichever place the value came from. The first draft validated+//! the config file only: `TYPESAFE_ENDPOINT='http://user:pw@…'` booted fine,+//! `GET /_xerj/rerank` showed the stripped URL, and `reqwest` would have sent+//! the userinfo as Basic auth beside the bearer key. This drives the real+//! binary with the variable set and checks it refuses, and that a clean value+//! still boots.++use std::io::Read;+use std::net::{TcpListener, TcpStream};+use std::process::{Command, Stdio};+use std::sync::{Arc, Mutex};+use std::time::{Duration, Instant};++fn toml_path(p: &std::path::Path) -> String {+ p.to_string_lossy().replace('\\', "/")+}++fn three_free_ports() -> (u16, u16, u16) {+ let held: Vec<TcpListener> = (0..3)+ .map(|_| TcpListener::bind("127.0.0.1:0").unwrap())+ .collect();+ let p: Vec<u16> = held+ .iter()+ .map(|l| l.local_addr().unwrap().port())+ .collect();+ (p[0], p[1], p[2])+}++fn config(dir: &std::path::Path, rest: u16, grpc: u16, es: u16) -> String {+ format!(+ r#"+[server]+bind_address = "127.0.0.1"+rest_port = {rest}+grpc_port = {grpc}+es_compat_port = {es}+data_dir = "{data}"++[limits]+disk_flood_stage_percent = 0+"#,+ data = toml_path(&dir.join("data")),+ )+}++fn drain_into(mut r: impl Read, buf: Arc<Mutex<String>>) {+ let mut chunk = [0u8; 4096];+ loop {+ match r.read(&mut chunk) {+ Ok(0) => break,+ Ok(n) => buf+ .lock()+ .unwrap()+ .push_str(&String::from_utf8_lossy(&chunk[..n])),+ Err(_) => break,+ }+ }+}++struct Boot {+ exited: Option<i32>,+ es_port_open: bool,+ stderr: String,+}++/// Spawn the binary with `TYPESAFE_ENDPOINT` set and no `rerank.endpoint` in+/// the file, and wait until it exits or its ES-compat port accepts a+/// connection — whichever comes first — up to `timeout`.+fn boot_with_env_endpoint(dir: &std::path::Path, endpoint: &str, timeout: Duration) -> Boot {+ let (rest, grpc, es) = three_free_ports();+ let config_path = dir.join("xerj.toml");+ std::fs::write(&config_path, config(dir, rest, grpc, es)).unwrap();++ let mut child = Command::new(env!("CARGO_BIN_EXE_xerj"))+ .arg("--config")+ .arg(&config_path)+ .arg("--insecure")+ .env("TYPESAFE_ENDPOINT", endpoint)+ .env_remove("TYPESAFE_API_KEY")+ .stdout(Stdio::null())+ .stderr(Stdio::piped())+ .spawn()+ .expect("spawn xerj");+ let stderr_buf = Arc::new(Mutex::new(String::new()));+ let err_r = child.stderr.take().expect("stderr piped");+ let err_buf_t = stderr_buf.clone();+ let err_thread = std::thread::spawn(move || drain_into(err_r, err_buf_t));++ let deadline = Instant::now() + timeout;+ let mut exited = None;+ let mut es_port_open = false;+ loop {+ if let Ok(Some(status)) = child.try_wait() {+ exited = status.code();+ break;+ }+ if TcpStream::connect(("127.0.0.1", es)).is_ok() {+ es_port_open = true;+ break;+ }+ if Instant::now() > deadline {+ break;+ }+ std::thread::sleep(Duration::from_millis(100));+ }+ child.kill().ok();+ child.wait().ok();+ err_thread.join().ok();+ let stderr = stderr_buf.lock().unwrap().clone();+ Boot {+ exited,+ es_port_open,+ stderr,+ }+}++#[test]+fn a_credentialed_typesafe_endpoint_in_the_environment_stops_the_node() {+ let dir = tempfile::tempdir().unwrap();+ let boot = boot_with_env_endpoint(+ dir.path(),+ "http://user:[email protected]:9/v1/systemone",+ Duration::from_secs(60),+ );+ assert!(+ !boot.es_port_open,+ "the node came up with a credentialed TYPESAFE_ENDPOINT; stderr:\n{}",+ boot.stderr+ );+ assert_ne!(+ boot.exited,+ Some(0),+ "exit status must be a failure; stderr:\n{}",+ boot.stderr+ );+ assert!(+ boot.exited.is_some(),+ "the node neither exited nor listened within the timeout; stderr:\n{}",+ boot.stderr+ );+ assert!(+ boot.stderr.contains("TYPESAFE_ENDPOINT")+ && boot.stderr.contains("must not carry credentials"),+ "the refusal names the variable and the rule; stderr:\n{}",+ boot.stderr+ );+ assert!(+ !boot.stderr.contains("user:pw@"),+ "the credential must not be echoed; stderr:\n{}",+ boot.stderr+ );+}++#[test]+fn a_non_http_typesafe_endpoint_in_the_environment_stops_the_node() {+ let dir = tempfile::tempdir().unwrap();+ let boot = boot_with_env_endpoint(+ dir.path(),+ "judge.example/v1/systemone",+ Duration::from_secs(60),+ );+ assert!(!boot.es_port_open, "stderr:\n{}", boot.stderr);+ assert!(+ boot.exited.is_some() && boot.exited != Some(0),+ "stderr:\n{}",+ boot.stderr+ );+ assert!(+ boot.stderr.contains("TYPESAFE_ENDPOINT")+ && boot.stderr.contains("absolute http:// or https:// URL"),+ "stderr:\n{}",+ boot.stderr+ );+}++/// The check must not refuse a well-formed value: the same boot with a clean+/// endpoint gets past config loading and starts listening.+#[test]+fn a_clean_typesafe_endpoint_in_the_environment_boots() {+ let dir = tempfile::tempdir().unwrap();+ let boot = boot_with_env_endpoint(+ dir.path(),+ "http://127.0.0.1:9/v1/systemone",+ Duration::from_secs(120),+ );+ assert!(+ boot.es_port_open,+ "the node did not start listening (exit {:?}); stderr:\n{}",+ boot.exited, boot.stderr+ );+}diff --git a/engine/xerj.default.toml b/engine/xerj.default.tomlindex 2a8923ac5..402d6d9c1 100644--- a/engine/xerj.default.toml+++ b/engine/xerj.default.toml@@ -1,6 +1,6 @@# xerj Default Configuration# ============================================================-# This file sets 55 of XERJ's 117 user-facing settings to their+# This file sets 55 of XERJ's 120 user-facing settings to their# production-ready defaults, and describes the rest in comments.# Every value here IS the default, so copying this file changes# nothing until you edit it (a test enforces that). The complete@@ -580,3 +580,25 @@ access_log = false# page later. `min_retained_generations` is capped at 64 because it holds that# many rotated WAL files per shard per index — up to n * storage.wal_max_size_mb# of disk each — whether or not a tap is running.++# ── Rerank provider ─────────────────────────────────────────+# `"rerank": {...}` on _search sends the text of the top-N hits to a+# third-party judge (TypeSafe AI's Jev) and reorders them by the relevance+# probability that comes back. It is the only SEARCH-TIME feature that sends+# document text off this node. Two other features send text off it, both off+# by default: [embedding] default_endpoint (document text at write time, query+# text at search time) and the WAL tap. The node's remaining outbound+# connections (neural model download, cluster Raft) carry no document text;+# docs/RERANK.md lists every one. Reranking does nothing until a key is+# supplied here or in the TYPESAFE_API_KEY environment variable. A value in+# this file wins over the environment. `enabled = false` refuses every rerank request (403) whatever+# the environment holds. The key is never echoed: GET /_xerj/rerank (superuser+# only) reports whether one is set and where it came from, nothing more.+# `endpoint`, from here or from TYPESAFE_ENDPOINT, must be an absolute+# http(s):// URL with no user:password@ in it, or the node refuses to start.+# See docs/RERANK.md.+#+# [rerank]+# enabled = true+# api_key = "ts-..." # not a default ("")+# endpoint = "https://api.typesafe.ai/v1/systemone" # not a default ("")diff --git a/landing/answers/agent-memory-without-postgres-qdrant.html b/landing/answers/agent-memory-without-postgres-qdrant.htmlindex f4cedd174..c510f57e1 100644--- a/landing/answers/agent-memory-without-postgres-qdrant.html+++ b/landing/answers/agent-memory-without-postgres-qdrant.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/audit-php-codebase-with-an-index.html b/landing/answers/audit-php-codebase-with-an-index.htmlindex 37e21ce5d..639dadfa9 100644--- a/landing/answers/audit-php-codebase-with-an-index.html+++ b/landing/answers/audit-php-codebase-with-an-index.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class="active"><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/autoindex-exit-codes.html b/landing/answers/autoindex-exit-codes.htmlindex 18ed16706..3a9544922 100644--- a/landing/answers/autoindex-exit-codes.html+++ b/landing/answers/autoindex-exit-codes.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class="active"><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class="active"><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/autoindex-says-extract-the-archive-first.html b/landing/answers/autoindex-says-extract-the-archive-first.htmlindex 2e1cc6b89..03a046387 100644--- a/landing/answers/autoindex-says-extract-the-archive-first.html+++ b/landing/answers/autoindex-says-extract-the-archive-first.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class="active"><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class="active"><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/catalog-files-with-autoindex-map.html b/landing/answers/catalog-files-with-autoindex-map.htmlindex 41575fa6b..f51520663 100644--- a/landing/answers/catalog-files-with-autoindex-map.html+++ b/landing/answers/catalog-files-with-autoindex-map.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class="active"><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class="active"><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/cheap-low-volume-log-search.html b/landing/answers/cheap-low-volume-log-search.htmlindex 81a6ed0cb..20883343c 100644--- a/landing/answers/cheap-low-volume-log-search.html+++ b/landing/answers/cheap-low-volume-log-search.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class="active"><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class="active"><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/check-codebase-index-is-complete.html b/landing/answers/check-codebase-index-is-complete.htmlindex 43f9fa856..e17e0339d 100644--- a/landing/answers/check-codebase-index-is-complete.html+++ b/landing/answers/check-codebase-index-is-complete.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class="active"><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class="active"><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/classify-text-from-labelled-history-without-llm.html b/landing/answers/classify-text-from-labelled-history-without-llm.htmlnew file mode 100644index 000000000..76b723d79--- /dev/null+++ b/landing/answers/classify-text-from-labelled-history-without-llm.html@@ -0,0 +1,390 @@+<!-- generated by build_articles.py — edit content/answers/classify-text-from-labelled-history-without-llm.md instead -->+<!DOCTYPE html>+<html lang="en" data-theme="day">+<head>+<meta charset="UTF-8">+<meta name="viewport" content="width=device-width, initial-scale=1.0">+<title>Can I route or spam-filter text without an LLM call?</title>+<meta name="description" content="Index labelled examples and classify new text by its nearest neighbours. Measured: 0.933 accuracy on 77-way intent routing, 0.983 on SMS spam, no LLM or API call.">+<link rel="preconnect" href="https://fonts.googleapis.com">+<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>+<link href="https://fonts.googleapis.com/css2?family=Big+Shoulders+Display:wght@400;700;800;900&family=Inter:wght@400;500;600;700&family=IBM+Plex+Sans:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500;700&display=swap" rel="stylesheet">+<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'><rect width='100' height='100' fill='%230b0b0d'/><text x='50' y='68' font-size='60' font-family='Big Shoulders Display,sans-serif' font-weight='900' fill='%23ffc400' text-anchor='middle'>X</text></svg>">+<link rel="stylesheet" href="/style.css?v=90f643e">+<link rel="alternate" type="text/markdown" href="/answers/classify-text-from-labelled-history-without-llm.md">+<!-- seo:begin — generated by scripts/seo/fix_heads.py · do not hand-edit -->+<link rel="canonical" href="https://xerj.org/answers/classify-text-from-labelled-history-without-llm">+<meta property="og:site_name" content="XERJ.ai">+<meta property="og:type" content="article">+<meta property="og:title" content="Can I route or spam-filter text without an LLM call?">+<meta property="og:description" content="Index labelled examples and classify new text by its nearest neighbours. Measured: 0.933 accuracy on 77-way intent routing, 0.983 on SMS spam, no LLM or API call.">+<meta property="og:url" content="https://xerj.org/answers/classify-text-from-labelled-history-without-llm">+<meta property="og:image" content="https://xerj.org/og/xerj-card.png">+<meta property="og:image:width" content="1200">+<meta property="og:image:height" content="630">+<meta property="og:image:alt" content="XERJ.ai — the search engine built for AI agents">+<meta property="og:locale" content="en_US">+<meta name="twitter:card" content="summary_large_image">+<meta name="twitter:title" content="Can I route or spam-filter text without an LLM call?">+<meta name="twitter:description" content="Index labelled examples and classify new text by its nearest neighbours. Measured: 0.933 accuracy on 77-way intent routing, 0.983 on SMS spam, no LLM or API call.">+<meta name="twitter:image" content="https://xerj.org/og/xerj-card.png">+<script type="application/ld+json">+{+ "@context": "https://schema.org",+ "@type": "Organization",+ "@id": "https://xerj.org/#organization",+ "name": "XERJ",+ "alternateName": "XERJ.ai",+ "url": "https://xerj.org/",+ "logo": "https://xerj.org/brandbook/xerj-mark.svg",+ "sameAs": [+ "https://github.com/xerj-org/xerj"+ ]+}+</script>+<script type="application/ld+json">+{+ "@context": "https://schema.org",+ "@type": "WebSite",+ "@id": "https://xerj.org/#website",+ "name": "XERJ.ai",+ "url": "https://xerj.org/",+ "inLanguage": "en",+ "publisher": {+ "@id": "https://xerj.org/#organization"+ }+}+</script>+<script type="application/ld+json">+{+ "@context": "https://schema.org",+ "@type": "TechArticle",+ "headline": "Can I route or spam-filter text without an LLM call?",+ "description": "Index labelled examples and classify new text by its nearest neighbours. Measured: 0.933 accuracy on 77-way intent routing, 0.983 on SMS spam, no LLM or API call.",+ "url": "https://xerj.org/answers/classify-text-from-labelled-history-without-llm",+ "image": "https://xerj.org/og/xerj-card.png",+ "inLanguage": "en",+ "datePublished": "2026-09-18",+ "dateModified": "2026-09-18",+ "author": {+ "@type": "Person",+ "name": "XERJ documentation team"+ },+ "publisher": {+ "@id": "https://xerj.org/#organization"+ },+ "isPartOf": {+ "@id": "https://xerj.org/#website"+ },+ "mainEntityOfPage": {+ "@type": "WebPage",+ "@id": "https://xerj.org/answers/classify-text-from-labelled-history-without-llm"+ },+ "reviewedBy": {+ "@type": "Person",+ "name": "XERJ engineering team"+ }+}+</script>+<script type="application/ld+json">+{+ "@context": "https://schema.org",+ "@type": "BreadcrumbList",+ "itemListElement": [+ {+ "@type": "ListItem",+ "position": 1,+ "name": "Home",+ "item": "https://xerj.org/"+ },+ {+ "@type": "ListItem",+ "position": 2,+ "name": "Answers",+ "item": "https://xerj.org/answers/"+ },+ {+ "@type": "ListItem",+ "position": 3,+ "name": "Can I route tickets or filter spam from labelled history, without an LLM call?"+ }+ ]+}+</script>+<!-- seo:end -->+</head>+<body>+<nav class="nav" aria-label="Primary">+ <a href="/" class="brand">XERJ.AI</a>+ <a href="/">HOME</a>+ <a href="/product">PRODUCT</a>+ <span class="nav-drop">+ <a href="/use-cases">USE CASES</a>+ </span>+ <a href="/case-studies">CASE STUDIES</a>+ <a href="/benchmarks/">BENCHMARKS</a>+ <span class="spacer"></span>+ <a href="/for-agents">FOR AGENTS</a>+ <a href="/docs/" class="active">DOCS</a>+ <a href="/docs/recipes/">RECIPES</a>+ <a href="/playground">DASHBOARDS</a>+ <a href="https://github.com/xerj-org/xerj" rel="external">GITHUB</a>+ <span class="spacer"></span>+ <span class="theme">+ <button type="button" data-theme="day" class="active">DAY</button>+ <span class="dash">·</span>+ <button type="button" data-theme="night">NIGHT</button>+ </span>+</nav>+<div class="docs-shell">+<nav class="docs-nav" aria-label="Article sections">+ <div class="group">+ <span class="label">ANSWERS</span>+ <a href="/answers/"><span class="eyebrow">00</span>All Answers</a>+ <a href="/answers/give-chatgpt-claude-local-file-access" class=""><span class="eyebrow">01</span>Can ChatGPT search a folder on my laptop, or do I need something else?</a>+ <a href="/answers/code-search-mcp-for-claude-code" class=""><span class="eyebrow">02</span>I need my agent to find a passage and cite the file, not dump the whole tree. How?</a>+ <a href="/answers/mcp-api-search-or-vector-search" class=""><span class="eyebrow">03</span>Should an agent use API search or vector search?</a>+ <a href="/answers/knowledge-graphs-for-agent-memory" class=""><span class="eyebrow">04</span>Should agent memory use a knowledge graph?</a>+ <a href="/answers/store-agent-memory-without-llm-call" class=""><span class="eyebrow">05</span>How do I stop an agent memory store from filling with near-duplicate notes?</a>+ <a href="/answers/evidence-for-agent-memory-links" class=""><span class="eyebrow">06</span>How can agent-memory links retain evidence?</a>+ <a href="/answers/private-agent-memory-namespaces" class=""><span class="eyebrow">07</span>How do I give two agents private memory on one laptop?</a>+ <a href="/answers/coding-agent-memory-across-sessions" class=""><span class="eyebrow">08</span>How do people store user preferences so an agent can recall them next week?</a>+ <a href="/answers/set-mcp-memory-storage-path" class=""><span class="eyebrow">09</span>How do I set the MCP memory storage path?</a>+ <a href="/answers/agent-memory-without-postgres-qdrant" class=""><span class="eyebrow">10</span>What's a simple way to give an agent long-term memory on my laptop without Qdrant?</a>+ <a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a>+ <a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a>+ <a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class="active"><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ </div>+ <div class="group">+ <span class="label">COMPARISONS</span>+ <a href="/compare/"><span class="eyebrow">00</span>All Comparisons</a>+ <a href="/compare/xerj-vs-web-agent-search" class=""><span class="eyebrow">01</span>How do I let an agent search a folder on my laptop instead of crawling the web?</a>+ <a href="/compare/xerj-vs-ast-grep" class=""><span class="eyebrow">02</span>ast-grep vs a code index for an agent?</a>+ <a href="/compare/xerj-vs-ripgrep-for-code-agents" class=""><span class="eyebrow">03</span>Is ripgrep enough for a coding agent, or do I need a code index?</a>+ <a href="/compare/xerj-vs-ripgrep-all" class=""><span class="eyebrow">04</span>How do I grep PDFs and Word docs from the command line?</a>+ <a href="/compare/xerj-vs-docfetcher" class=""><span class="eyebrow">05</span>What's a free DocFetcher alternative that an agent can call?</a>+ <a href="/compare/xerj-vs-recoll" class=""><span class="eyebrow">06</span>What's the best local desktop search for a folder of PDFs and docs?</a>+ <a href="/compare/xerj-vs-houdahspot" class=""><span class="eyebrow">07</span>XERJ vs HoudahSpot — Spotlight GUI vs an agent index</a>+ <a href="/compare/xerj-vs-spotlight" class=""><span class="eyebrow">08</span>XERJ vs macOS Spotlight for folder content an agent can query</a>+ <a href="/compare/xerj-vs-localsynapse" class=""><span class="eyebrow">09</span>What's a local MCP search engine for Claude?</a>+ <a href="/compare/xerj-vs-elasticsearch-local" class=""><span class="eyebrow">10</span>Is there an Elasticsearch-compatible engine that isn't Elasticsearch?</a>+ <a href="/compare/xerj-vs-obsidian-omnisearch" class=""><span class="eyebrow">11</span>How do I search an Obsidian vault from outside Obsidian?</a>+ <a href="/compare/xerj-vs-khoj" class=""><span class="eyebrow">12</span>What's a simpler alternative to Khoj if I only need search and memory?</a>+ <a href="/compare/xerj-vs-typesense" class=""><span class="eyebrow">13</span>Do I need Typesense for an agent to search my docs?</a>+ <a href="/compare/xerj-vs-meilisearch" class=""><span class="eyebrow">14</span>What's a local alternative to Meilisearch that indexes a folder by itself?</a>+ <a href="/compare/xerj-vs-pagefind" class=""><span class="eyebrow">15</span>Pagefind vs indexing a local HTML dump for an agent?</a>+ <a href="/compare/xerj-vs-vector-database" class=""><span class="eyebrow">16</span>Should my agent call a search API or stand up Qdrant for persistent memory?</a>+ </div>+</nav>+<main class="docs-content">+ <div class="breadcrumb"><a href="/">HOME</a><span class="sep">·</span><a href="/answers/">ANSWERS</a><span class="sep">·</span>CAN I ROUTE TICKETS OR FILTER SPAM FROM LABELLED HISTORY, WITHOUT AN LLM CALL?</div>+ <div class="kicker"><span>ANSWERS · DECISIONS AS RETRIEVAL</span></div>+ <h1>Can I route tickets or filter spam from labelled history, without an LLM call?</h1>+ <div class="source-note">AUTHOR · XERJ documentation team · REVIEWED BY · XERJ engineering team · PUBLISHED · 2026-09-18 · UPDATED · 2026-09-18</div>+ <p><strong>TL;DR</strong> — If you already have labelled examples, you can route and filter text with no LLM or API call. Index the examples, fetch the 10 nearest for each new item, and vote. Measured: 0.933 accuracy on 77-way intent routing and 0.983 on SMS spam. The 0.933 figure needs <code class="inline">--embed-mode neural</code>, which runs one local MiniLM embedding per item; the spam figure is plain BM25, with no model of any kind.</p>+ <div class="prompt-card">+ <span class="prompt-label">PASTE THIS TO YOUR AI AGENT — IT DOES THE REST</span>+ <code>Act as a coding agent. Read https://xerj.org/llms.txt, index the user's labelled examples into one XERJ index with the label in a keyword field, classify each held-out item by a rank-weighted vote over its 10 nearest neighbours, and report accuracy, the share of items decided at confidence 0.8 or higher, and the accuracy on that share.</code>+ <button type="button" class="prompt-copy" data-copy="Act as a coding agent. Read https://xerj.org/llms.txt, index the user's labelled examples into one XERJ index with the label in a keyword field, classify each held-out item by a rank-weighted vote over its 10 nearest neighbours, and report accuracy, the share of items decided at confidence 0.8 or higher, and the accuracy on that share.">COPY THE PROMPT</button>+ </div>+ <div class="prompt-card">+ <span class="prompt-label">RUN THIS XERJ COMMAND</span>+ <code>xerj --insecure --data-dir ./decisions-data --embed-mode neural</code>+ <button type="button" class="prompt-copy" data-copy="xerj --insecure --data-dir ./decisions-data --embed-mode neural">COPY COMMAND</button>+ <span class="hero-proof">Start a throwaway node. The neural flag is needed for the MiniLM and hybrid rows only. The BM25 row uses no embedder and works on a default node.</span>+ </div>+ <div class="prompt-card">+ <span class="prompt-label">RUN THIS XERJ COMMAND</span>+ <code>curl -so sms.tsv https://raw.githubusercontent.com/justmarkham/pycon-2016-tutorial/master/data/sms.tsv</code>+ <button type="button" class="prompt-copy" data-copy="curl -so sms.tsv https://raw.githubusercontent.com/justmarkham/pycon-2016-tutorial/master/data/sms.tsv">COPY COMMAND</button>+ <span class="hero-proof">Download the public SMS spam dataset used for the yes-or-no run.</span>+ </div>+ <div class="prompt-card">+ <span class="prompt-label">RUN THIS XERJ COMMAND</span>+ <code>curl -s -XPOST 'http://127.0.0.1:9200/sms/_search' -H 'content-type: application/json' -d '{"query":{"match":{"text":"you have won a free prize call now"}},"size":10,"_source":["label"]}'</code>+ <button type="button" class="prompt-copy" data-copy="curl -s -XPOST 'http://127.0.0.1:9200/sms/_search' -H 'content-type: application/json' -d '{"query":{"match":{"text":"you have won a free prize call now"}},"size":10,"_source":["label"]}'">COPY COMMAND</button>+ <span class="hero-proof">Fetch the 10 nearest labelled examples for one new message. The vote over their labels is the decision, and the winning share is the confidence.</span>+ </div>+ <h2>The method</h2>+ <p>A judge model answers typed questions: pick one of N, or yes-or-no with a probability. Where a team already has labelled history, a search index can answer the same shape of question.</p>+ <ol>+ <li>Index every labelled example, with the label in a keyword field.</li>+ <li>For a new item, fetch its 10 nearest neighbours.</li>+ <li>Take a rank-weighted vote over the neighbours' labels.</li>+ <li>Report the winning label, and its share of the vote as the confidence.</li>+ </ol>+ <p>No model runs per decision on the BM25 arm. On the neural arms one local embedding runs per item, on the node, with no API call.</p>+ <h2>Intent routing: Banking77</h2>+ <p>Banking77 is a public dataset of banking support messages with 77 intent labels. The run indexed 10,003 labelled examples and classified 1,000 held-out items.</p>+ <table>+ <thead><tr><th>Arm</th><th>Accuracy</th><th>Calibration error</th><th>Decided at confidence 0.8 or higher</th><th>Accuracy on that share</th><th>ms per item</th></tr></thead>+ <tbody>+ <tr><td>BM25, no model of any kind</td><td>0.819</td><td>0.089</td><td>50.6%</td><td>0.996</td><td>2.2</td></tr>+ <tr><td>MiniLM neighbours</td><td>0.933</td><td>0.012</td><td>86.9%</td><td>0.979</td><td>8.6</td></tr>+ <tr><td>Hybrid RRF</td><td>0.937</td><td>0.052</td><td>79.2%</td><td>0.989</td><td>12.4</td></tr>+ </tbody>+ </table>+ <h2>Spam detection: SMS spam</h2>+ <p>The SMS spam dataset is a public yes-or-no task. The run indexed 4,000 labelled messages and classified 1,000 held-out ones.</p>+ <table>+ <thead><tr><th>Arm</th><th>Accuracy</th><th>Spam F1</th><th>Calibration error</th><th>ms per item</th></tr></thead>+ <tbody>+ <tr><td>BM25, no model of any kind</td><td>0.983</td><td>0.944</td><td>0.017</td><td>0.9</td></tr>+ <tr><td>MiniLM neighbours</td><td>0.982</td><td>0.940</td><td>0.009</td><td>74.5</td></tr>+ <tr><td>Hybrid RRF</td><td>0.989</td><td>0.962</td><td>0.015</td><td>67.2</td></tr>+ </tbody>+ </table>+ <p>Plain BM25 over 4,000 examples reached 0.983 in under a millisecond per message. The neural arm did not improve accuracy here, and it cost far more time.</p>+ <h2>Which embedder these numbers need</h2>+ <p>The BM25 rows use no embedder, so they hold on a default XERJ node. The MiniLM and hybrid rows were measured with <code class="inline">--embed-mode neural</code> and the all-MiniLM-L6-v2 model on CPU.</p>+ <p>XERJ's default embedder is lexical feature hashing. It has no model in it. Do not read the MiniLM or hybrid rows as what a default node scores.</p>+ <h2>The confidence gate is the point</h2>+ <p>An expected calibration error of 0.012 means the reported confidence is usable: averaged over confidence bins, the gap between the confidence the vote reported and the accuracy observed at that confidence was about one point in this run.</p>+ <p>That makes the confidence a gate. On Banking77, the MiniLM arm decided 86.9% of items at confidence 0.8 or higher, and was right on 0.979 of them. Only the remaining items need a judge model or a person.</p>+ <p>This is the economics of a two-stage system. A free local stage takes the bulk, and the paid stage sees only what is uncertain.</p>+ <h2>When to call a judge model instead</h2>+ <p>Call a judge model when there is no labelled history. This method is not zero-shot. A new policy or a new category has nothing to vote from, and a judge model needs no examples.</p>+ <p>Call one also for the low-confidence share. XERJ's <code class="inline">rerank</code> search stage can send documents to an external judge when an operator has set a provider key. XERJ did not run any judge model on these two datasets. It claims nothing about how one would score on them.</p>+ <h2>What this run does not show</h2>+ <p>Two public datasets, one run each, 1,000 held-out items per dataset, k fixed at 10. The held-out split uses a seeded shuffle. The latency was measured on a machine that was also compiling, so treat it as an upper bound.</p>+ <p>The gap between the two datasets' latencies is query length. Short banking messages embed in a few milliseconds, and longer SMS bodies take tens.</p>+ <h2>FAQ</h2>+ <h3>Can I classify text without calling an LLM?</h3>+ <p>Yes, when you have labelled history. Index the labelled examples, fetch the 10 nearest for each new item, and take a rank-weighted vote over their labels. No model call is made per decision.</p>+ <h3>How accurate is nearest-neighbour classification on a real dataset?</h3>+ <p>On Banking77, a 77-way intent dataset, XERJ measured 0.819 with BM25 and 0.933 with MiniLM neighbours. On SMS spam it measured 0.983 with BM25 alone. The MiniLM figures need <code class="inline">--embed-mode neural</code>.</p>+ <h3>Can I trust the confidence value?</h3>+ <p>In this run, yes. Expected calibration error was 0.012 on Banking77 and 0.009 on SMS spam for the MiniLM arm: averaged over confidence bins, the reported confidence and the observed accuracy differed by about one point.</p>+ <h3>Does this work with no labelled examples?</h3>+ <p>No. This is not zero-shot. A new category or a new policy has no history to vote from, and that case needs a judge model or a person.</p>+ <h3>Do I need a neural embedder for this?</h3>+ <p>Not for spam. BM25 alone reached 0.983 on SMS spam with no embedder at all. For 77-way intent routing the neural arm added about 11 points of accuracy over BM25.</p>+ <h3>Did XERJ compare this against an LLM judge?</h3>+ <p>No. XERJ did not run any judge model on these datasets and claims nothing about how one would score.</p>+ <h2>Evidence</h2>+ <ul>+ <li>Banking77, 77-way intent routing, 10,003 labelled examples indexed, 1,000 held-out test items, k = 10: accuracy 0.819 for BM25, 0.933 for MiniLM neighbours and 0.937 for hybrid RRF; expected calibration error 0.089, 0.012 and 0.052; 50.6%, 86.9% and 79.2% of items decided at confidence 0.8 or higher, at accuracy 0.996, 0.979 and 0.989; 2.2, 8.6 and 12.4 ms per item. — <code>benchmarks/decisions-as-retrieval/results/run-2026-09-18.txt</code></li>+ <li>SMS spam, yes-or-no detection, 4,000 labelled examples indexed, 1,000 held-out test items, k = 10: accuracy 0.983 for BM25, 0.982 for MiniLM neighbours and 0.989 for hybrid RRF; spam F1 0.944, 0.940 and 0.962; expected calibration error 0.017, 0.009 and 0.015; 0.9, 74.5 and 67.2 ms per item. — <code>benchmarks/decisions-as-retrieval/results/run-2026-09-18.txt</code></li>+ <li>The run was made on 2026-09-18 with xerj v1.0.0-rc.74 and --embed-mode neural using all-MiniLM-L6-v2 on CPU, with a seeded shuffle for the held-out split, on a machine that was also compiling. — <code>benchmarks/decisions-as-retrieval/README.md</code></li>+ </ul>+ <h2>Related</h2>+ <ul>+ <li><a href="/answers/store-agent-memory-without-llm-call">How do I stop an agent memory store from filling with near-duplicate notes?</a></li>+ <li><a href="/answers/hybrid-search-quality-measured-ndcg">How much better is hybrid search than BM25? Measured nDCG numbers</a></li>+ <li><a href="/answers/rerank-search-results-calibrated-judge">How do I rerank search results with a calibrated relevance judge?</a></li>+ <li><a href="/answers/do-search-embeddings-help">When should I turn on neural embeddings, and when is the lexical default enough?</a></li>+ </ul>+</main>+</div>+<!-- seo:footer-begin — generated by scripts/seo/fix_links.py · do not hand-edit -->+<nav class="footer-map" aria-label="Site map">+ <div class="fm-col">+ <span class="fm-h">PRODUCT</span>+ <a href="/product">PRODUCT</a>+ <a href="/architecture/">ARCHITECTURE</a>+ <a href="/benchmarks/">BENCHMARKS</a>+ <a href="/pricing/">PRICING</a>+ <a href="/security/">SECURITY</a>+ <a href="/playground">DASHBOARDS</a>+ <a href="/demo/">REAL-DATA DEMO</a>+ <a href="/aise-demo">PRODUCT TOUR</a>+ </div>+ <div class="fm-col">+ <span class="fm-h">SOLUTIONS</span>+ <a href="/solutions/">SOLUTIONS</a>+ <a href="/use-cases">USE CASES</a>+ <a href="/case-studies">CASE STUDIES</a>+ <a href="/industries/">INDUSTRIES</a>+ <a href="/industries/finserv">FINANCIAL SERVICES</a>+ <a href="/industries/public-sector">PUBLIC SECTOR</a>+ <a href="/industries/healthcare">HEALTHCARE</a>+ <a href="/industries/retail">RETAIL</a>+ <a href="/use-cases/code-security-audit">AI CODE-SECURITY AUDIT</a>+ <a href="/use-cases/semantic-analytics">SEMANTIC ANALYTICS</a>+ </div>+ <div class="fm-col">+ <span class="fm-h">DOCS</span>+ <a href="/docs/">DOCUMENTATION</a>+ <a href="/docs/quickstart">QUICKSTART</a>+ <a href="/docs/install">INSTALL</a>+ <a href="/docs/migration-from-es">MIGRATE FROM ELASTICSEARCH</a>+ <a href="/docs/recipes/">RECIPES</a>+ <a href="/docs/agents/">DOCS FOR AI</a>+ <a href="/for-agents">FOR AI AGENTS</a>+ <a href="/agent-search/">AGENTIC SEARCH DEMO</a>+ <a href="/answers/">ANSWERS</a>+ <a href="/compare/">COMPARISONS</a>+ </div>+ <div class="fm-col">+ <span class="fm-h">RESOURCES</span>+ <a href="/resources/">SALES KIT</a>+ <a href="/brand">BRAND</a>+ <a href="/brandbook/">BRAND BOOK</a>+ <a href="/llms.txt">LLMS.TXT</a>+ <a href="https://github.com/xerj-org/xerj" rel="external">GITHUB</a>+ </div>+</nav>+<!-- seo:footer-end -->+<footer class="footer">+ <span>XERJ.AI · ANSWERS</span>+ <span>READ THE SOURCE</span>+ <span><a href="/">HOME</a> · <a href="/product">PRODUCT</a> · <a href="/brand">BRAND</a> · <a href="/playground">DASHBOARDS</a> · <a href="/llms.txt">LLMS.TXT</a></span>+</footer>+<script src="/interact.js"></script>+</body>+</html>diff --git a/landing/answers/classify-text-from-labelled-history-without-llm.md b/landing/answers/classify-text-from-labelled-history-without-llm.mdnew file mode 100644index 000000000..a82c06f0b--- /dev/null+++ b/landing/answers/classify-text-from-labelled-history-without-llm.md@@ -0,0 +1,141 @@+<!-- generated by build_articles.py — edit content/answers/classify-text-from-labelled-history-without-llm.md instead -->+---+title: "Can I route or spam-filter text without an LLM call?"+canonical: "https://xerj.org/answers/classify-text-from-labelled-history-without-llm"+updated: "2026-09-18"+source: "content/answers/classify-text-from-labelled-history-without-llm.md"+---++# Can I route tickets or filter spam from labelled history, without an LLM call?++**TL;DR** — If you already have labelled examples, you can route and filter text with no LLM or API call. Index the examples, fetch the 10 nearest for each new item, and vote. Measured: 0.933 accuracy on 77-way intent routing and 0.983 on SMS spam. The 0.933 figure needs `--embed-mode neural`, which runs one local MiniLM embedding per item; the spam figure is plain BM25, with no model of any kind.++## Agent prompt++```text+Act as a coding agent. Read https://xerj.org/llms.txt, index the user's labelled examples into one XERJ index with the label in a keyword field, classify each held-out item by a rank-weighted vote over its 10 nearest neighbours, and report accuracy, the share of items decided at confidence 0.8 or higher, and the accuracy on that share.+```++## Commands++### Command 1++Note: Start a throwaway node. The neural flag is needed for the MiniLM and hybrid rows only. The BM25 row uses no embedder and works on a default node.++```sh+xerj --insecure --data-dir ./decisions-data --embed-mode neural+```++### Command 2++Note: Download the public SMS spam dataset used for the yes-or-no run.++```sh+curl -so sms.tsv https://raw.githubusercontent.com/justmarkham/pycon-2016-tutorial/master/data/sms.tsv+```++### Command 3++Note: Fetch the 10 nearest labelled examples for one new message. The vote over their labels is the decision, and the winning share is the confidence.++```sh+curl -s -XPOST 'http://127.0.0.1:9200/sms/_search' -H 'content-type: application/json' -d '{"query":{"match":{"text":"you have won a free prize call now"}},"size":10,"_source":["label"]}'+```++## The method++A judge model answers typed questions: pick one of N, or yes-or-no with a probability. Where a team already has labelled history, a search index can answer the same shape of question.++1. Index every labelled example, with the label in a keyword field.+2. For a new item, fetch its 10 nearest neighbours.+3. Take a rank-weighted vote over the neighbours' labels.+4. Report the winning label, and its share of the vote as the confidence.++No model runs per decision on the BM25 arm. On the neural arms one local embedding runs per item, on the node, with no API call.++## Intent routing: Banking77++Banking77 is a public dataset of banking support messages with 77 intent labels. The run indexed 10,003 labelled examples and classified 1,000 held-out items.++| Arm | Accuracy | Calibration error | Decided at confidence 0.8 or higher | Accuracy on that share | ms per item |+| --- | --- | --- | --- | --- | --- |+| BM25, no model of any kind | 0.819 | 0.089 | 50.6% | 0.996 | 2.2 |+| MiniLM neighbours | 0.933 | 0.012 | 86.9% | 0.979 | 8.6 |+| Hybrid RRF | 0.937 | 0.052 | 79.2% | 0.989 | 12.4 |++## Spam detection: SMS spam++The SMS spam dataset is a public yes-or-no task. The run indexed 4,000 labelled messages and classified 1,000 held-out ones.++| Arm | Accuracy | Spam F1 | Calibration error | ms per item |+| --- | --- | --- | --- | --- |+| BM25, no model of any kind | 0.983 | 0.944 | 0.017 | 0.9 |+| MiniLM neighbours | 0.982 | 0.940 | 0.009 | 74.5 |+| Hybrid RRF | 0.989 | 0.962 | 0.015 | 67.2 |++Plain BM25 over 4,000 examples reached 0.983 in under a millisecond per message. The neural arm did not improve accuracy here, and it cost far more time.++## Which embedder these numbers need++The BM25 rows use no embedder, so they hold on a default XERJ node. The MiniLM and hybrid rows were measured with `--embed-mode neural` and the all-MiniLM-L6-v2 model on CPU.++XERJ's default embedder is lexical feature hashing. It has no model in it. Do not read the MiniLM or hybrid rows as what a default node scores.++## The confidence gate is the point++An expected calibration error of 0.012 means the reported confidence is usable: averaged over confidence bins, the gap between the confidence the vote reported and the accuracy observed at that confidence was about one point in this run.++That makes the confidence a gate. On Banking77, the MiniLM arm decided 86.9% of items at confidence 0.8 or higher, and was right on 0.979 of them. Only the remaining items need a judge model or a person.++This is the economics of a two-stage system. A free local stage takes the bulk, and the paid stage sees only what is uncertain.++## When to call a judge model instead++Call a judge model when there is no labelled history. This method is not zero-shot. A new policy or a new category has nothing to vote from, and a judge model needs no examples.++Call one also for the low-confidence share. XERJ's `rerank` search stage can send documents to an external judge when an operator has set a provider key. XERJ did not run any judge model on these two datasets. It claims nothing about how one would score on them.++## What this run does not show++Two public datasets, one run each, 1,000 held-out items per dataset, k fixed at 10. The held-out split uses a seeded shuffle. The latency was measured on a machine that was also compiling, so treat it as an upper bound.++The gap between the two datasets' latencies is query length. Short banking messages embed in a few milliseconds, and longer SMS bodies take tens.++## FAQ++### Can I classify text without calling an LLM?++Yes, when you have labelled history. Index the labelled examples, fetch the 10 nearest for each new item, and take a rank-weighted vote over their labels. No model call is made per decision.++### How accurate is nearest-neighbour classification on a real dataset?++On Banking77, a 77-way intent dataset, XERJ measured 0.819 with BM25 and 0.933 with MiniLM neighbours. On SMS spam it measured 0.983 with BM25 alone. The MiniLM figures need `--embed-mode neural`.++### Can I trust the confidence value?++In this run, yes. Expected calibration error was 0.012 on Banking77 and 0.009 on SMS spam for the MiniLM arm: averaged over confidence bins, the reported confidence and the observed accuracy differed by about one point.++### Does this work with no labelled examples?++No. This is not zero-shot. A new category or a new policy has no history to vote from, and that case needs a judge model or a person.++### Do I need a neural embedder for this?++Not for spam. BM25 alone reached 0.983 on SMS spam with no embedder at all. For 77-way intent routing the neural arm added about 11 points of accuracy over BM25.++### Did XERJ compare this against an LLM judge?++No. XERJ did not run any judge model on these datasets and claims nothing about how one would score.++## Evidence++- Banking77, 77-way intent routing, 10,003 labelled examples indexed, 1,000 held-out test items, k = 10: accuracy 0.819 for BM25, 0.933 for MiniLM neighbours and 0.937 for hybrid RRF; expected calibration error 0.089, 0.012 and 0.052; 50.6%, 86.9% and 79.2% of items decided at confidence 0.8 or higher, at accuracy 0.996, 0.979 and 0.989; 2.2, 8.6 and 12.4 ms per item. — `benchmarks/decisions-as-retrieval/results/run-2026-09-18.txt`+- SMS spam, yes-or-no detection, 4,000 labelled examples indexed, 1,000 held-out test items, k = 10: accuracy 0.983 for BM25, 0.982 for MiniLM neighbours and 0.989 for hybrid RRF; spam F1 0.944, 0.940 and 0.962; expected calibration error 0.017, 0.009 and 0.015; 0.9, 74.5 and 67.2 ms per item. — `benchmarks/decisions-as-retrieval/results/run-2026-09-18.txt`+- The run was made on 2026-09-18 with xerj v1.0.0-rc.74 and --embed-mode neural using all-MiniLM-L6-v2 on CPU, with a seeded shuffle for the held-out split, on a machine that was also compiling. — `benchmarks/decisions-as-retrieval/README.md`++## Related++- [How do I stop an agent memory store from filling with near-duplicate notes?](/answers/store-agent-memory-without-llm-call)+- [How much better is hybrid search than BM25? Measured nDCG numbers](/answers/hybrid-search-quality-measured-ndcg)+- [How do I rerank search results with a calibrated relevance judge?](/answers/rerank-search-results-calibrated-judge)+- [When should I turn on neural embeddings, and when is the lexical default enough?](/answers/do-search-embeddings-help)diff --git a/landing/answers/code-search-mcp-for-claude-code.html b/landing/answers/code-search-mcp-for-claude-code.htmlindex 85e954657..69f6ec975 100644--- a/landing/answers/code-search-mcp-for-claude-code.html+++ b/landing/answers/code-search-mcp-for-claude-code.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/coding-agent-memory-across-sessions.html b/landing/answers/coding-agent-memory-across-sessions.htmlindex f9e7d21a1..cbf3e5ed0 100644--- a/landing/answers/coding-agent-memory-across-sessions.html+++ b/landing/answers/coding-agent-memory-across-sessions.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/do-search-embeddings-help.html b/landing/answers/do-search-embeddings-help.htmlindex 92b6a8969..f63c0421c 100644--- a/landing/answers/do-search-embeddings-help.html+++ b/landing/answers/do-search-embeddings-help.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class="active"><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class="active"><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/does-xerj-support-s3-alerting-plugins.html b/landing/answers/does-xerj-support-s3-alerting-plugins.htmlindex 2e2fdf488..5641472e3 100644--- a/landing/answers/does-xerj-support-s3-alerting-plugins.html+++ b/landing/answers/does-xerj-support-s3-alerting-plugins.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class="active"><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/estimate-autoindex-time-before-running.html b/landing/answers/estimate-autoindex-time-before-running.htmlindex 1363a559f..cb12bb853 100644--- a/landing/answers/estimate-autoindex-time-before-running.html+++ b/landing/answers/estimate-autoindex-time-before-running.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class="active"><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class="active"><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/evidence-for-agent-memory-links.html b/landing/answers/evidence-for-agent-memory-links.htmlindex ef55b485f..374130ecf 100644--- a/landing/answers/evidence-for-agent-memory-links.html+++ b/landing/answers/evidence-for-agent-memory-links.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/filter-knn-exact-scan-caveat.html b/landing/answers/filter-knn-exact-scan-caveat.htmlindex a35526296..4f6fac9a2 100644--- a/landing/answers/filter-knn-exact-scan-caveat.html+++ b/landing/answers/filter-knn-exact-scan-caveat.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class="active"><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class="active"><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/full-text-search-browser-history.html b/landing/answers/full-text-search-browser-history.htmlindex 092e06c5c..6b1bf036d 100644--- a/landing/answers/full-text-search-browser-history.html+++ b/landing/answers/full-text-search-browser-history.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class="active"><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class="active"><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/full-text-search-sqlite-database.html b/landing/answers/full-text-search-sqlite-database.htmlindex 2e4f07a5e..41b066fdc 100644--- a/landing/answers/full-text-search-sqlite-database.html+++ b/landing/answers/full-text-search-sqlite-database.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class="active"><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class="active"><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/give-chatgpt-claude-local-file-access.html b/landing/answers/give-chatgpt-claude-local-file-access.htmlindex ad6cae059..a9d120f5e 100644--- a/landing/answers/give-chatgpt-claude-local-file-access.html+++ b/landing/answers/give-chatgpt-claude-local-file-access.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/how-xerj-autoindexes-a-folder.html b/landing/answers/how-xerj-autoindexes-a-folder.htmlindex 575b20563..75ff2a2d8 100644--- a/landing/answers/how-xerj-autoindexes-a-folder.html+++ b/landing/answers/how-xerj-autoindexes-a-folder.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class="active"><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class="active"><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/how-xerj-combines-search.html b/landing/answers/how-xerj-combines-search.htmlindex cd9d2a2b4..db89e4e63 100644--- a/landing/answers/how-xerj-combines-search.html+++ b/landing/answers/how-xerj-combines-search.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class="active"><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class="active"><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/hybrid-search-quality-measured-ndcg.html b/landing/answers/hybrid-search-quality-measured-ndcg.htmlnew file mode 100644index 000000000..491531543--- /dev/null+++ b/landing/answers/hybrid-search-quality-measured-ndcg.html@@ -0,0 +1,400 @@+<!-- generated by build_articles.py — edit content/answers/hybrid-search-quality-measured-ndcg.md instead -->+<!DOCTYPE html>+<html lang="en" data-theme="day">+<head>+<meta charset="UTF-8">+<meta name="viewport" content="width=device-width, initial-scale=1.0">+<title>How much better is hybrid search than BM25? Measured.</title>+<meta name="description" content="XERJ measured BM25, vectors and hybrid RRF on two public BEIR datasets. Hybrid won both: nDCG@10 0.6993 against 0.6572 on SciFact, with a neural embedder.">+<link rel="preconnect" href="https://fonts.googleapis.com">+<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>+<link href="https://fonts.googleapis.com/css2?family=Big+Shoulders+Display:wght@400;700;800;900&family=Inter:wght@400;500;600;700&family=IBM+Plex+Sans:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500;700&display=swap" rel="stylesheet">+<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'><rect width='100' height='100' fill='%230b0b0d'/><text x='50' y='68' font-size='60' font-family='Big Shoulders Display,sans-serif' font-weight='900' fill='%23ffc400' text-anchor='middle'>X</text></svg>">+<link rel="stylesheet" href="/style.css?v=90f643e">+<link rel="alternate" type="text/markdown" href="/answers/hybrid-search-quality-measured-ndcg.md">+<!-- seo:begin — generated by scripts/seo/fix_heads.py · do not hand-edit -->+<link rel="canonical" href="https://xerj.org/answers/hybrid-search-quality-measured-ndcg">+<meta property="og:site_name" content="XERJ.ai">+<meta property="og:type" content="article">+<meta property="og:title" content="How much better is hybrid search than BM25? Measured.">+<meta property="og:description" content="XERJ measured BM25, vectors and hybrid RRF on two public BEIR datasets. Hybrid won both: nDCG@10 0.6993 against 0.6572 on SciFact, with a neural embedder.">+<meta property="og:url" content="https://xerj.org/answers/hybrid-search-quality-measured-ndcg">+<meta property="og:image" content="https://xerj.org/og/xerj-card.png">+<meta property="og:image:width" content="1200">+<meta property="og:image:height" content="630">+<meta property="og:image:alt" content="XERJ.ai — the search engine built for AI agents">+<meta property="og:locale" content="en_US">+<meta name="twitter:card" content="summary_large_image">+<meta name="twitter:title" content="How much better is hybrid search than BM25? Measured.">+<meta name="twitter:description" content="XERJ measured BM25, vectors and hybrid RRF on two public BEIR datasets. Hybrid won both: nDCG@10 0.6993 against 0.6572 on SciFact, with a neural embedder.">+<meta name="twitter:image" content="https://xerj.org/og/xerj-card.png">+<script type="application/ld+json">+{+ "@context": "https://schema.org",+ "@type": "Organization",+ "@id": "https://xerj.org/#organization",+ "name": "XERJ",+ "alternateName": "XERJ.ai",+ "url": "https://xerj.org/",+ "logo": "https://xerj.org/brandbook/xerj-mark.svg",+ "sameAs": [+ "https://github.com/xerj-org/xerj"+ ]+}+</script>+<script type="application/ld+json">+{+ "@context": "https://schema.org",+ "@type": "WebSite",+ "@id": "https://xerj.org/#website",+ "name": "XERJ.ai",+ "url": "https://xerj.org/",+ "inLanguage": "en",+ "publisher": {+ "@id": "https://xerj.org/#organization"+ }+}+</script>+<script type="application/ld+json">+{+ "@context": "https://schema.org",+ "@type": "TechArticle",+ "headline": "How much better is hybrid search than BM25? Measured.",+ "description": "XERJ measured BM25, vectors and hybrid RRF on two public BEIR datasets. Hybrid won both: nDCG@10 0.6993 against 0.6572 on SciFact, with a neural embedder.",+ "url": "https://xerj.org/answers/hybrid-search-quality-measured-ndcg",+ "image": "https://xerj.org/og/xerj-card.png",+ "inLanguage": "en",+ "datePublished": "2026-09-18",+ "dateModified": "2026-09-19",+ "author": {+ "@type": "Person",+ "name": "XERJ documentation team"+ },+ "publisher": {+ "@id": "https://xerj.org/#organization"+ },+ "isPartOf": {+ "@id": "https://xerj.org/#website"+ },+ "mainEntityOfPage": {+ "@type": "WebPage",+ "@id": "https://xerj.org/answers/hybrid-search-quality-measured-ndcg"+ },+ "reviewedBy": {+ "@type": "Person",+ "name": "XERJ engineering team"+ }+}+</script>+<script type="application/ld+json">+{+ "@context": "https://schema.org",+ "@type": "BreadcrumbList",+ "itemListElement": [+ {+ "@type": "ListItem",+ "position": 1,+ "name": "Home",+ "item": "https://xerj.org/"+ },+ {+ "@type": "ListItem",+ "position": 2,+ "name": "Answers",+ "item": "https://xerj.org/answers/"+ },+ {+ "@type": "ListItem",+ "position": 3,+ "name": "How much better is hybrid search than BM25? Measured nDCG numbers"+ }+ ]+}+</script>+<!-- seo:end -->+</head>+<body>+<nav class="nav" aria-label="Primary">+ <a href="/" class="brand">XERJ.AI</a>+ <a href="/">HOME</a>+ <a href="/product">PRODUCT</a>+ <span class="nav-drop">+ <a href="/use-cases">USE CASES</a>+ </span>+ <a href="/case-studies">CASE STUDIES</a>+ <a href="/benchmarks/">BENCHMARKS</a>+ <span class="spacer"></span>+ <a href="/for-agents">FOR AGENTS</a>+ <a href="/docs/" class="active">DOCS</a>+ <a href="/docs/recipes/">RECIPES</a>+ <a href="/playground">DASHBOARDS</a>+ <a href="https://github.com/xerj-org/xerj" rel="external">GITHUB</a>+ <span class="spacer"></span>+ <span class="theme">+ <button type="button" data-theme="day" class="active">DAY</button>+ <span class="dash">·</span>+ <button type="button" data-theme="night">NIGHT</button>+ </span>+</nav>+<div class="docs-shell">+<nav class="docs-nav" aria-label="Article sections">+ <div class="group">+ <span class="label">ANSWERS</span>+ <a href="/answers/"><span class="eyebrow">00</span>All Answers</a>+ <a href="/answers/give-chatgpt-claude-local-file-access" class=""><span class="eyebrow">01</span>Can ChatGPT search a folder on my laptop, or do I need something else?</a>+ <a href="/answers/code-search-mcp-for-claude-code" class=""><span class="eyebrow">02</span>I need my agent to find a passage and cite the file, not dump the whole tree. How?</a>+ <a href="/answers/mcp-api-search-or-vector-search" class=""><span class="eyebrow">03</span>Should an agent use API search or vector search?</a>+ <a href="/answers/knowledge-graphs-for-agent-memory" class=""><span class="eyebrow">04</span>Should agent memory use a knowledge graph?</a>+ <a href="/answers/store-agent-memory-without-llm-call" class=""><span class="eyebrow">05</span>How do I stop an agent memory store from filling with near-duplicate notes?</a>+ <a href="/answers/evidence-for-agent-memory-links" class=""><span class="eyebrow">06</span>How can agent-memory links retain evidence?</a>+ <a href="/answers/private-agent-memory-namespaces" class=""><span class="eyebrow">07</span>How do I give two agents private memory on one laptop?</a>+ <a href="/answers/coding-agent-memory-across-sessions" class=""><span class="eyebrow">08</span>How do people store user preferences so an agent can recall them next week?</a>+ <a href="/answers/set-mcp-memory-storage-path" class=""><span class="eyebrow">09</span>How do I set the MCP memory storage path?</a>+ <a href="/answers/agent-memory-without-postgres-qdrant" class=""><span class="eyebrow">10</span>What's a simple way to give an agent long-term memory on my laptop without Qdrant?</a>+ <a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a>+ <a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a>+ <a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class="active"><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ </div>+ <div class="group">+ <span class="label">COMPARISONS</span>+ <a href="/compare/"><span class="eyebrow">00</span>All Comparisons</a>+ <a href="/compare/xerj-vs-web-agent-search" class=""><span class="eyebrow">01</span>How do I let an agent search a folder on my laptop instead of crawling the web?</a>+ <a href="/compare/xerj-vs-ast-grep" class=""><span class="eyebrow">02</span>ast-grep vs a code index for an agent?</a>+ <a href="/compare/xerj-vs-ripgrep-for-code-agents" class=""><span class="eyebrow">03</span>Is ripgrep enough for a coding agent, or do I need a code index?</a>+ <a href="/compare/xerj-vs-ripgrep-all" class=""><span class="eyebrow">04</span>How do I grep PDFs and Word docs from the command line?</a>+ <a href="/compare/xerj-vs-docfetcher" class=""><span class="eyebrow">05</span>What's a free DocFetcher alternative that an agent can call?</a>+ <a href="/compare/xerj-vs-recoll" class=""><span class="eyebrow">06</span>What's the best local desktop search for a folder of PDFs and docs?</a>+ <a href="/compare/xerj-vs-houdahspot" class=""><span class="eyebrow">07</span>XERJ vs HoudahSpot — Spotlight GUI vs an agent index</a>+ <a href="/compare/xerj-vs-spotlight" class=""><span class="eyebrow">08</span>XERJ vs macOS Spotlight for folder content an agent can query</a>+ <a href="/compare/xerj-vs-localsynapse" class=""><span class="eyebrow">09</span>What's a local MCP search engine for Claude?</a>+ <a href="/compare/xerj-vs-elasticsearch-local" class=""><span class="eyebrow">10</span>Is there an Elasticsearch-compatible engine that isn't Elasticsearch?</a>+ <a href="/compare/xerj-vs-obsidian-omnisearch" class=""><span class="eyebrow">11</span>How do I search an Obsidian vault from outside Obsidian?</a>+ <a href="/compare/xerj-vs-khoj" class=""><span class="eyebrow">12</span>What's a simpler alternative to Khoj if I only need search and memory?</a>+ <a href="/compare/xerj-vs-typesense" class=""><span class="eyebrow">13</span>Do I need Typesense for an agent to search my docs?</a>+ <a href="/compare/xerj-vs-meilisearch" class=""><span class="eyebrow">14</span>What's a local alternative to Meilisearch that indexes a folder by itself?</a>+ <a href="/compare/xerj-vs-pagefind" class=""><span class="eyebrow">15</span>Pagefind vs indexing a local HTML dump for an agent?</a>+ <a href="/compare/xerj-vs-vector-database" class=""><span class="eyebrow">16</span>Should my agent call a search API or stand up Qdrant for persistent memory?</a>+ </div>+</nav>+<main class="docs-content">+ <div class="breadcrumb"><a href="/">HOME</a><span class="sep">·</span><a href="/answers/">ANSWERS</a><span class="sep">·</span>HOW MUCH BETTER IS HYBRID SEARCH THAN BM25? MEASURED NDCG NUMBERS</div>+ <div class="kicker"><span>ANSWERS · HYBRID RETRIEVAL: QUALITY</span></div>+ <h1>How much better is hybrid search than BM25? Measured nDCG numbers</h1>+ <div class="source-note">AUTHOR · XERJ documentation team · REVIEWED BY · XERJ engineering team · PUBLISHED · 2026-09-18 · UPDATED · 2026-09-19</div>+ <p><strong>TL;DR</strong> — On two public BEIR datasets, XERJ's hybrid RRF beat BM25 alone and beat vectors alone: nDCG@10 0.6993 against 0.6572 on SciFact, and 0.3448 against 0.3016 on NFCorpus. These runs used <code class="inline">--embed-mode neural</code> with all-MiniLM-L6-v2. The default embedder is lexical feature hashing, and these vector figures do not apply to it.</p>+ <div class="prompt-card">+ <span class="prompt-label">PASTE THIS TO YOUR AI AGENT — IT DOES THE REST</span>+ <code>Act as a coding agent. Read https://xerj.org/llms.txt, start a throwaway XERJ node with --embed-mode neural, load the BEIR SciFact corpus with benchmarks/beir-hybrid/load.py, run benchmarks/beir-hybrid/eval.py, and report nDCG@10 for the BM25, vector-only, reorder and hybrid RRF arms next to the published figures.</code>+ <button type="button" class="prompt-copy" data-copy="Act as a coding agent. Read https://xerj.org/llms.txt, start a throwaway XERJ node with --embed-mode neural, load the BEIR SciFact corpus with benchmarks/beir-hybrid/load.py, run benchmarks/beir-hybrid/eval.py, and report nDCG@10 for the BM25, vector-only, reorder and hybrid RRF arms next to the published figures.">COPY THE PROMPT</button>+ </div>+ <div class="prompt-card">+ <span class="prompt-label">RUN THIS XERJ COMMAND</span>+ <code>xerj --insecure --data-dir ./beir-data --embed-mode neural</code>+ <button type="button" class="prompt-copy" data-copy="xerj --insecure --data-dir ./beir-data --embed-mode neural">COPY COMMAND</button>+ <span class="hero-proof">Start a throwaway node with the neural embedder. Without this flag the embedder is lexical feature hashing and the vector and hybrid figures on this page do not apply.</span>+ </div>+ <div class="prompt-card">+ <span class="prompt-label">RUN THIS XERJ COMMAND</span>+ <code>curl -LO https://public.ukp.informatik.tu-darmstadt.de/thakur/BEIR/datasets/scifact.zip</code>+ <button type="button" class="prompt-copy" data-copy="curl -LO https://public.ukp.informatik.tu-darmstadt.de/thakur/BEIR/datasets/scifact.zip">COPY COMMAND</button>+ <span class="hero-proof">Download the public BEIR SciFact dataset: 5,183 documents and 300 test queries.</span>+ </div>+ <div class="prompt-card">+ <span class="prompt-label">RUN THIS XERJ COMMAND</span>+ <code>curl -s 'http://127.0.0.1:9200/v1/embedding/identity'</code>+ <button type="button" class="prompt-copy" data-copy="curl -s 'http://127.0.0.1:9200/v1/embedding/identity'">COPY COMMAND</button>+ <span class="hero-proof">Confirm which embedder the node is running before you read any vector or hybrid number.</span>+ </div>+ <h2>The numbers</h2>+ <p>The metric is nDCG@10 on each dataset's BEIR <code class="inline">test</code> split. The evaluation script computes it from live queries against a running node. Nothing is carried over from another run.</p>+ <table>+ <thead><tr><th>Arm</th><th>SciFact (300 queries, 5,183 docs)</th><th>NFCorpus (323 queries, 3,633 docs)</th></tr></thead>+ <tbody>+ <tr><td>BM25 (<code class="inline">multi_match</code> on title and text)</td><td>0.6572</td><td>0.3016</td></tr>+ <tr><td>MiniLM vectors only (<code class="inline">semantic</code>)</td><td>0.6764</td><td>0.3291</td></tr>+ <tr><td>BM25 top 30, reordered by MiniLM</td><td>0.6855</td><td>0.3323</td></tr>+ <tr><td>Hybrid RRF (<code class="inline">hybrid</code>, server-side)</td><td>0.6993</td><td>0.3448</td></tr>+ </tbody>+ </table>+ <p>The run was made on 2026-09-18 with xerj v1.0.0-rc.74, on one node and one shard with default settings. The raw output is committed next to the scripts in <code class="inline">benchmarks/beir-hybrid</code>.</p>+ <h2>Which embedder these numbers need</h2>+ <p>Read this before you quote a row. XERJ embeds with lexical feature hashing by default. That embedder has no model in it and cannot connect synonyms.</p>+ <p>The vector, reorder and hybrid rows were measured with <code class="inline">--embed-mode neural</code>. That flag loads the built-in all-MiniLM-L6-v2 model, 384 dimensions, on CPU. A default node does not score these figures on those three rows. The BM25 row uses no embedder at all, so it holds on any node.</p>+ <h2>Fusion beat reordering</h2>+ <p>The third row is the one that changes a design decision. It takes the BM25 top 30 and reorders them with the same MiniLM model. It scored 0.6855 on SciFact and 0.3323 on NFCorpus.</p>+ <p>Hybrid RRF uses the same model and scored 0.6993 and 0.3448. Fusing with the model beat reranking with it, on both datasets.</p>+ <p>The reason is recall. A reorder can only move documents that BM25 already returned. Fusion also admits documents that only the vector list found. On NFCorpus BM25 returned zero hits for 25 of the 323 queries, and no reorder can fix an empty list.</p>+ <p>Those 25 empty lists are not an engine defect. Each of those queries is a word or phrase that occurs nowhere in the corpus, such as <code class="inline">deafness</code>, <code class="inline">eggnog</code> or <code class="inline">Zoloft</code>. A script in the benchmark folder counts them without a running node, and it reports 0 for SciFact, where BM25 had no empty lists.</p>+ <p>This is why XERJ ships no local MiniLM rerank provider. It would score below the hybrid search that already exists.</p>+ <h2>What it costs</h2>+ <p>Neural mode is not free. These figures come from the same run.</p>+ <table>+ <thead><tr><th>Cost</th><th>Measured</th></tr></thead>+ <tbody>+ <tr><td>Neural indexing speed</td><td>about 3 to 7 documents per second on CPU</td></tr>+ <tr><td>Time to index SciFact</td><td>about 25 minutes</td></tr>+ <tr><td>BM25 query, p50</td><td>about 18 ms</td></tr>+ <tr><td>Hybrid query, p50</td><td>about 220 to 370 ms</td></tr>+ </tbody>+ </table>+ <p>Most of the hybrid latency is one BERT forward pass per query on CPU. The machine was also compiling during the run, so treat the absolute latencies as upper bounds. One vector-arm latency in the SciFact raw log is a cache artefact from an earlier aborted run. It is not quoted here and must not be.</p>+ <h2>How this relates to hosted rerankers</h2>+ <p>The <code class="inline">hev/jev-rerank</code> README publishes nDCG@10 for three hosted rerankers on the same datasets.</p>+ <table>+ <thead><tr><th>System</th><th>SciFact</th><th>NFCorpus</th><th>Run by</th></tr></thead>+ <tbody>+ <tr><td>Jev (TypeSafe AI)</td><td>0.768</td><td>0.358</td><td>that README, not XERJ</td></tr>+ <tr><td>Voyage rerank-3</td><td>0.755</td><td>0.357</td><td>that README, not XERJ</td></tr>+ <tr><td>Cohere rerank-v3.5</td><td>0.745</td><td>0.340</td><td>that README, not XERJ</td></tr>+ <tr><td>XERJ hybrid RRF, local MiniLM</td><td>0.6993</td><td>0.3448</td><td>XERJ, <code class="inline">--embed-mode neural</code></td></tr>+ </tbody>+ </table>+ <p>XERJ did not run those three systems. They rerank that project's own first-stage shortlist, which is not XERJ's. The dataset and the metric match, and nothing else is controlled.</p>+ <p>What the table supports is narrow. On NFCorpus the local hybrid figure sits inside the range of the published reranker figures. On SciFact it is lower than all three. A controlled comparison needs a provider key and XERJ's <code class="inline">rerank</code> stage, and XERJ has not run one.</p>+ <h2>When to choose a hosted reranker instead</h2>+ <p>Choose a hosted reranker when ranking quality on hard queries matters most. You pay for it in latency, in cost, and in text that leaves the machine. The published SciFact figures for all three hosted systems are higher than XERJ's local hybrid figure. A reranker also returns a calibrated probability, which a fused score is not.</p>+ <p>Choose local hybrid when text must stay on the machine, or when a paid call per search is not acceptable. XERJ supports both. <code class="inline">hybrid</code> runs on the node with the built-in lexical or neural embedder. With proxy embeddings, the query text goes to that embeddings API. The <code class="inline">rerank</code> stage calls a judge when an operator has set a key.</p>+ <h2>What this run does not show</h2>+ <p>Two datasets, one model and one run each. Both datasets are scientific and medical text, so the gap may differ on code, logs or chat. The node was a single shard on a shared host. These figures show the ordering of the four arms on this data, not what your corpus will score.</p>+ <h2>FAQ</h2>+ <h3>How much better is hybrid search than BM25?</h3>+ <p>On two public BEIR datasets XERJ's hybrid RRF scored nDCG@10 0.6993 against 0.6572 for BM25 on SciFact, and 0.3448 against 0.3016 on NFCorpus. Both runs used <code class="inline">--embed-mode neural</code>.</p>+ <h3>Do these numbers apply to a default XERJ node?</h3>+ <p>Only the BM25 row does. The vector and hybrid rows need <code class="inline">--embed-mode neural</code>. The default embedder is lexical feature hashing, which has no model in it.</p>+ <h3>Is reranking a BM25 shortlist with the same embedding model a good idea?</h3>+ <p>Not in this run. Reordering the BM25 top 30 by MiniLM scored 0.6855 on SciFact, below 0.6993 for fusing with the same model. Fusion beat reordering on both datasets.</p>+ <h3>How does XERJ hybrid compare to hosted rerankers?</h3>+ <p>XERJ did not run them. Figures published elsewhere put Jev at 0.768 on SciFact and XERJ hybrid at 0.6993, but they use a different first stage, so it is not a controlled comparison.</p>+ <h3>What does hybrid search cost in latency?</h3>+ <p>In this run hybrid queries cost about 220 to 370 ms at p50 against about 18 ms for BM25, on a 5,000-document index on CPU. The machine was also compiling, so read those as upper bounds.</p>+ <h3>How fast is neural indexing?</h3>+ <p>About 3 to 7 documents per second on CPU in this run. SciFact's 5,183 short abstracts took about 25 minutes to index.</p>+ <h2>Evidence</h2>+ <ul>+ <li>On BEIR SciFact (300 test queries, 5,183 documents) nDCG@10 was 0.6572 for BM25, 0.6764 for MiniLM vectors only, 0.6855 for a BM25 top-30 shortlist reordered by MiniLM, and 0.6993 for hybrid RRF. — <code>benchmarks/beir-hybrid/results/scifact.txt</code></li>+ <li>On BEIR NFCorpus (323 test queries, 3,633 documents) nDCG@10 was 0.3016 for BM25, 0.3291 for MiniLM vectors only, 0.3323 for the reorder arm, and 0.3448 for hybrid RRF. BM25 returned zero hits for 25 of the 323 queries. — <code>benchmarks/beir-hybrid/results/nfcorpus.txt</code></li>+ <li>Exactly 25 of the 323 NFCorpus test queries share no token with any document title or text, which accounts for every empty BM25 result list; the same check reports 0 for SciFact. — <code>benchmarks/beir-hybrid/results/nfcorpus-lexical-gap.txt</code></li>+ <li>The run was made on 2026-09-18 with xerj v1.0.0-rc.74 and --embed-mode neural, using the built-in all-MiniLM-L6-v2 model at 384 dimensions on CPU, one node and one shard with default settings. Neural indexing ran at about 3 to 7 documents per second, SciFact took about 25 minutes, and hybrid queries cost about 220 to 370 ms at p50 against about 18 ms for BM25. — <code>benchmarks/beir-hybrid/README.md</code></li>+ <li>The nDCG@10 figures for Jev (0.768 SciFact, 0.358 NFCorpus), Voyage rerank-3 (0.755, 0.357) and Cohere rerank-v3.5 (0.745, 0.340) are published in the hev/jev-rerank README. XERJ did not run them. — <a href="https://github.com/hev/jev-rerank">https://github.com/hev/jev-rerank</a></li>+ </ul>+ <h2>Related</h2>+ <ul>+ <li><a href="/answers/reciprocal-rank-fusion-when-to-use">What is Reciprocal Rank Fusion?</a></li>+ <li><a href="/answers/do-search-embeddings-help">When should I turn on neural embeddings, and when is the lexical default enough?</a></li>+ <li><a href="/answers/rerank-search-results-calibrated-judge">How do I rerank search results with a calibrated relevance judge?</a></li>+ <li><a href="/answers/local-embeddings-without-openai-api">I want local search that works offline, no cloud embeddings. What are my options?</a></li>+ </ul>+</main>+</div>+<!-- seo:footer-begin — generated by scripts/seo/fix_links.py · do not hand-edit -->+<nav class="footer-map" aria-label="Site map">+ <div class="fm-col">+ <span class="fm-h">PRODUCT</span>+ <a href="/product">PRODUCT</a>+ <a href="/architecture/">ARCHITECTURE</a>+ <a href="/benchmarks/">BENCHMARKS</a>+ <a href="/pricing/">PRICING</a>+ <a href="/security/">SECURITY</a>+ <a href="/playground">DASHBOARDS</a>+ <a href="/demo/">REAL-DATA DEMO</a>+ <a href="/aise-demo">PRODUCT TOUR</a>+ </div>+ <div class="fm-col">+ <span class="fm-h">SOLUTIONS</span>+ <a href="/solutions/">SOLUTIONS</a>+ <a href="/use-cases">USE CASES</a>+ <a href="/case-studies">CASE STUDIES</a>+ <a href="/industries/">INDUSTRIES</a>+ <a href="/industries/finserv">FINANCIAL SERVICES</a>+ <a href="/industries/public-sector">PUBLIC SECTOR</a>+ <a href="/industries/healthcare">HEALTHCARE</a>+ <a href="/industries/retail">RETAIL</a>+ <a href="/use-cases/code-security-audit">AI CODE-SECURITY AUDIT</a>+ <a href="/use-cases/semantic-analytics">SEMANTIC ANALYTICS</a>+ </div>+ <div class="fm-col">+ <span class="fm-h">DOCS</span>+ <a href="/docs/">DOCUMENTATION</a>+ <a href="/docs/quickstart">QUICKSTART</a>+ <a href="/docs/install">INSTALL</a>+ <a href="/docs/migration-from-es">MIGRATE FROM ELASTICSEARCH</a>+ <a href="/docs/recipes/">RECIPES</a>+ <a href="/docs/agents/">DOCS FOR AI</a>+ <a href="/for-agents">FOR AI AGENTS</a>+ <a href="/agent-search/">AGENTIC SEARCH DEMO</a>+ <a href="/answers/">ANSWERS</a>+ <a href="/compare/">COMPARISONS</a>+ </div>+ <div class="fm-col">+ <span class="fm-h">RESOURCES</span>+ <a href="/resources/">SALES KIT</a>+ <a href="/brand">BRAND</a>+ <a href="/brandbook/">BRAND BOOK</a>+ <a href="/llms.txt">LLMS.TXT</a>+ <a href="https://github.com/xerj-org/xerj" rel="external">GITHUB</a>+ </div>+</nav>+<!-- seo:footer-end -->+<footer class="footer">+ <span>XERJ.AI · ANSWERS</span>+ <span>READ THE SOURCE</span>+ <span><a href="/">HOME</a> · <a href="/product">PRODUCT</a> · <a href="/brand">BRAND</a> · <a href="/playground">DASHBOARDS</a> · <a href="/llms.txt">LLMS.TXT</a></span>+</footer>+<script src="/interact.js"></script>+</body>+</html>diff --git a/landing/answers/hybrid-search-quality-measured-ndcg.md b/landing/answers/hybrid-search-quality-measured-ndcg.mdnew file mode 100644index 000000000..7a2da72dd--- /dev/null+++ b/landing/answers/hybrid-search-quality-measured-ndcg.md@@ -0,0 +1,153 @@+<!-- generated by build_articles.py — edit content/answers/hybrid-search-quality-measured-ndcg.md instead -->+---+title: "How much better is hybrid search than BM25? Measured."+canonical: "https://xerj.org/answers/hybrid-search-quality-measured-ndcg"+updated: "2026-09-19"+source: "content/answers/hybrid-search-quality-measured-ndcg.md"+---++# How much better is hybrid search than BM25? Measured nDCG numbers++**TL;DR** — On two public BEIR datasets, XERJ's hybrid RRF beat BM25 alone and beat vectors alone: nDCG@10 0.6993 against 0.6572 on SciFact, and 0.3448 against 0.3016 on NFCorpus. These runs used `--embed-mode neural` with all-MiniLM-L6-v2. The default embedder is lexical feature hashing, and these vector figures do not apply to it.++## Agent prompt++```text+Act as a coding agent. Read https://xerj.org/llms.txt, start a throwaway XERJ node with --embed-mode neural, load the BEIR SciFact corpus with benchmarks/beir-hybrid/load.py, run benchmarks/beir-hybrid/eval.py, and report nDCG@10 for the BM25, vector-only, reorder and hybrid RRF arms next to the published figures.+```++## Commands++### Command 1++Note: Start a throwaway node with the neural embedder. Without this flag the embedder is lexical feature hashing and the vector and hybrid figures on this page do not apply.++```sh+xerj --insecure --data-dir ./beir-data --embed-mode neural+```++### Command 2++Note: Download the public BEIR SciFact dataset: 5,183 documents and 300 test queries.++```sh+curl -LO https://public.ukp.informatik.tu-darmstadt.de/thakur/BEIR/datasets/scifact.zip+```++### Command 3++Note: Confirm which embedder the node is running before you read any vector or hybrid number.++```sh+curl -s 'http://127.0.0.1:9200/v1/embedding/identity'+```++## The numbers++The metric is nDCG@10 on each dataset's BEIR `test` split. The evaluation script computes it from live queries against a running node. Nothing is carried over from another run.++| Arm | SciFact (300 queries, 5,183 docs) | NFCorpus (323 queries, 3,633 docs) |+| --- | --- | --- |+| BM25 (`multi_match` on title and text) | 0.6572 | 0.3016 |+| MiniLM vectors only (`semantic`) | 0.6764 | 0.3291 |+| BM25 top 30, reordered by MiniLM | 0.6855 | 0.3323 |+| Hybrid RRF (`hybrid`, server-side) | 0.6993 | 0.3448 |++The run was made on 2026-09-18 with xerj v1.0.0-rc.74, on one node and one shard with default settings. The raw output is committed next to the scripts in `benchmarks/beir-hybrid`.++## Which embedder these numbers need++Read this before you quote a row. XERJ embeds with lexical feature hashing by default. That embedder has no model in it and cannot connect synonyms.++The vector, reorder and hybrid rows were measured with `--embed-mode neural`. That flag loads the built-in all-MiniLM-L6-v2 model, 384 dimensions, on CPU. A default node does not score these figures on those three rows. The BM25 row uses no embedder at all, so it holds on any node.++## Fusion beat reordering++The third row is the one that changes a design decision. It takes the BM25 top 30 and reorders them with the same MiniLM model. It scored 0.6855 on SciFact and 0.3323 on NFCorpus.++Hybrid RRF uses the same model and scored 0.6993 and 0.3448. Fusing with the model beat reranking with it, on both datasets.++The reason is recall. A reorder can only move documents that BM25 already returned. Fusion also admits documents that only the vector list found. On NFCorpus BM25 returned zero hits for 25 of the 323 queries, and no reorder can fix an empty list.++Those 25 empty lists are not an engine defect. Each of those queries is a word or phrase that occurs nowhere in the corpus, such as `deafness`, `eggnog` or `Zoloft`. A script in the benchmark folder counts them without a running node, and it reports 0 for SciFact, where BM25 had no empty lists.++This is why XERJ ships no local MiniLM rerank provider. It would score below the hybrid search that already exists.++## What it costs++Neural mode is not free. These figures come from the same run.++| Cost | Measured |+| --- | --- |+| Neural indexing speed | about 3 to 7 documents per second on CPU |+| Time to index SciFact | about 25 minutes |+| BM25 query, p50 | about 18 ms |+| Hybrid query, p50 | about 220 to 370 ms |++Most of the hybrid latency is one BERT forward pass per query on CPU. The machine was also compiling during the run, so treat the absolute latencies as upper bounds. One vector-arm latency in the SciFact raw log is a cache artefact from an earlier aborted run. It is not quoted here and must not be.++## How this relates to hosted rerankers++The `hev/jev-rerank` README publishes nDCG@10 for three hosted rerankers on the same datasets.++| System | SciFact | NFCorpus | Run by |+| --- | --- | --- | --- |+| Jev (TypeSafe AI) | 0.768 | 0.358 | that README, not XERJ |+| Voyage rerank-3 | 0.755 | 0.357 | that README, not XERJ |+| Cohere rerank-v3.5 | 0.745 | 0.340 | that README, not XERJ |+| XERJ hybrid RRF, local MiniLM | 0.6993 | 0.3448 | XERJ, `--embed-mode neural` |++XERJ did not run those three systems. They rerank that project's own first-stage shortlist, which is not XERJ's. The dataset and the metric match, and nothing else is controlled.++What the table supports is narrow. On NFCorpus the local hybrid figure sits inside the range of the published reranker figures. On SciFact it is lower than all three. A controlled comparison needs a provider key and XERJ's `rerank` stage, and XERJ has not run one.++## When to choose a hosted reranker instead++Choose a hosted reranker when ranking quality on hard queries matters most. You pay for it in latency, in cost, and in text that leaves the machine. The published SciFact figures for all three hosted systems are higher than XERJ's local hybrid figure. A reranker also returns a calibrated probability, which a fused score is not.++Choose local hybrid when text must stay on the machine, or when a paid call per search is not acceptable. XERJ supports both. `hybrid` runs on the node with the built-in lexical or neural embedder. With proxy embeddings, the query text goes to that embeddings API. The `rerank` stage calls a judge when an operator has set a key.++## What this run does not show++Two datasets, one model and one run each. Both datasets are scientific and medical text, so the gap may differ on code, logs or chat. The node was a single shard on a shared host. These figures show the ordering of the four arms on this data, not what your corpus will score.++## FAQ++### How much better is hybrid search than BM25?++On two public BEIR datasets XERJ's hybrid RRF scored nDCG@10 0.6993 against 0.6572 for BM25 on SciFact, and 0.3448 against 0.3016 on NFCorpus. Both runs used `--embed-mode neural`.++### Do these numbers apply to a default XERJ node?++Only the BM25 row does. The vector and hybrid rows need `--embed-mode neural`. The default embedder is lexical feature hashing, which has no model in it.++### Is reranking a BM25 shortlist with the same embedding model a good idea?++Not in this run. Reordering the BM25 top 30 by MiniLM scored 0.6855 on SciFact, below 0.6993 for fusing with the same model. Fusion beat reordering on both datasets.++### How does XERJ hybrid compare to hosted rerankers?++XERJ did not run them. Figures published elsewhere put Jev at 0.768 on SciFact and XERJ hybrid at 0.6993, but they use a different first stage, so it is not a controlled comparison.++### What does hybrid search cost in latency?++In this run hybrid queries cost about 220 to 370 ms at p50 against about 18 ms for BM25, on a 5,000-document index on CPU. The machine was also compiling, so read those as upper bounds.++### How fast is neural indexing?++About 3 to 7 documents per second on CPU in this run. SciFact's 5,183 short abstracts took about 25 minutes to index.++## Evidence++- On BEIR SciFact (300 test queries, 5,183 documents) nDCG@10 was 0.6572 for BM25, 0.6764 for MiniLM vectors only, 0.6855 for a BM25 top-30 shortlist reordered by MiniLM, and 0.6993 for hybrid RRF. — `benchmarks/beir-hybrid/results/scifact.txt`+- On BEIR NFCorpus (323 test queries, 3,633 documents) nDCG@10 was 0.3016 for BM25, 0.3291 for MiniLM vectors only, 0.3323 for the reorder arm, and 0.3448 for hybrid RRF. BM25 returned zero hits for 25 of the 323 queries. — `benchmarks/beir-hybrid/results/nfcorpus.txt`+- Exactly 25 of the 323 NFCorpus test queries share no token with any document title or text, which accounts for every empty BM25 result list; the same check reports 0 for SciFact. — `benchmarks/beir-hybrid/results/nfcorpus-lexical-gap.txt`+- The run was made on 2026-09-18 with xerj v1.0.0-rc.74 and --embed-mode neural, using the built-in all-MiniLM-L6-v2 model at 384 dimensions on CPU, one node and one shard with default settings. Neural indexing ran at about 3 to 7 documents per second, SciFact took about 25 minutes, and hybrid queries cost about 220 to 370 ms at p50 against about 18 ms for BM25. — `benchmarks/beir-hybrid/README.md`+- The nDCG@10 figures for Jev (0.768 SciFact, 0.358 NFCorpus), Voyage rerank-3 (0.755, 0.357) and Cohere rerank-v3.5 (0.745, 0.340) are published in the hev/jev-rerank README. XERJ did not run them. — [https://github.com/hev/jev-rerank](https://github.com/hev/jev-rerank)++## Related++- [What is Reciprocal Rank Fusion?](/answers/reciprocal-rank-fusion-when-to-use)+- [When should I turn on neural embeddings, and when is the lexical default enough?](/answers/do-search-embeddings-help)+- [How do I rerank search results with a calibrated relevance judge?](/answers/rerank-search-results-calibrated-judge)+- [I want local search that works offline, no cloud embeddings. What are my options?](/answers/local-embeddings-without-openai-api)diff --git a/landing/answers/improve-basic-faiss-rag-pipeline.html b/landing/answers/improve-basic-faiss-rag-pipeline.htmlindex 46d559e25..476f64997 100644--- a/landing/answers/improve-basic-faiss-rag-pipeline.html+++ b/landing/answers/improve-basic-faiss-rag-pipeline.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class="active"><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class="active"><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/index-csv-with-many-columns.html b/landing/answers/index-csv-with-many-columns.htmlindex 09cc01686..56c49c565 100644--- a/landing/answers/index-csv-with-many-columns.html+++ b/landing/answers/index-csv-with-many-columns.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class="active"><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class="active"><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/index-markdown-into-elasticsearch-api.html b/landing/answers/index-markdown-into-elasticsearch-api.htmlindex 0a28008e7..ae4259dec 100644--- a/landing/answers/index-markdown-into-elasticsearch-api.html+++ b/landing/answers/index-markdown-into-elasticsearch-api.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class="active"><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class="active"><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/index-monorepo-for-agent.html b/landing/answers/index-monorepo-for-agent.htmlindex ae205b109..473d2e6ca 100644--- a/landing/answers/index-monorepo-for-agent.html+++ b/landing/answers/index-monorepo-for-agent.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class="active"><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class="active"><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/index-multiple-csv-files.html b/landing/answers/index-multiple-csv-files.htmlindex fd8668d9b..2a0a4cfc5 100644--- a/landing/answers/index-multiple-csv-files.html+++ b/landing/answers/index-multiple-csv-files.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class="active"><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class="active"><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/index-reference-library-for-coding-agent.html b/landing/answers/index-reference-library-for-coding-agent.htmlindex f627ce567..765570ba0 100644--- a/landing/answers/index-reference-library-for-coding-agent.html+++ b/landing/answers/index-reference-library-for-coding-agent.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class="active"><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/index.html b/landing/answers/index.htmlindex 723560fd5..f60362590 100644--- a/landing/answers/index.html+++ b/landing/answers/index.html@@ -135,58 +135,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>@@ -260,6 +263,10 @@<ul><li><a href="/answers/index-reference-library-for-coding-agent">How do I set up reference coding so the agent retrieves a mechanism before writing code?</a> — Clone the library you are about to write against, index it with xerj autoindex, and retrieve the passage first. The published study, with the tracks where it loses.</li></ul>+ <h2>Decisions as retrieval</h2>+ <ul>+ <li><a href="/answers/classify-text-from-labelled-history-without-llm">Can I route tickets or filter spam from labelled history, without an LLM call?</a> — Index labelled examples and classify new text by its nearest neighbours. Measured: 0.933 accuracy on 77-way intent routing, 0.983 on SMS spam, no LLM or API call.</li>+ </ul><h2>Embeddings: cost</h2><ul><li><a href="/answers/reduce-embedding-api-cost">How do I reduce embedding API cost?</a> — XERJ embeds documents inside the node by default, so a captured lexical run made 0 outbound calls. Proxy mode sent exactly 2 calls for the same 100 documents.</li>@@ -312,6 +319,14 @@<ul><li><a href="/answers/rag-without-vector-database">How do I build RAG without a vector database?</a> — Build a RAG retrieval layer on one XERJ index. One hybrid request fuses BM25 and kNN with Reciprocal Rank Fusion, and no second service runs.</li></ul>+ <h2>Hybrid retrieval: quality</h2>+ <ul>+ <li><a href="/answers/hybrid-search-quality-measured-ndcg">How much better is hybrid search than BM25? Measured nDCG numbers</a> — XERJ measured BM25, vectors and hybrid RRF on two public BEIR datasets. Hybrid won both: nDCG@10 0.6993 against 0.6572 on SciFact, with a neural embedder.</li>+ </ul>+ <h2>Hybrid retrieval: reranking</h2>+ <ul>+ <li><a href="/answers/rerank-search-results-calibrated-judge">How do I rerank search results with a calibrated relevance judge?</a> — Add a rerank block to a XERJ search: the top 30 hits go to an external judge, a 0 to 1 probability replaces the score, and min_score becomes a real cut-off.</li>+ </ul><h2>Hybrid retrieval: RRF</h2><ul><li><a href="/answers/reciprocal-rank-fusion-when-to-use">What is Reciprocal Rank Fusion?</a> — Reciprocal Rank Fusion scores a document by 1 divided by k plus its rank in each list. XERJ uses k 60, and this page checks the arithmetic by hand.</li>diff --git a/landing/answers/index.json b/landing/answers/index.jsonindex 1ea28d0a5..657460e68 100644--- a/landing/answers/index.json+++ b/landing/answers/index.json@@ -119,6 +119,15 @@"updated": "2026-08-23","summary": "Clone the library you are about to write against, index it with xerj autoindex, and retrieve the passage first. The published study, with the tracks where it loses."},+ {+ "title": "Can I route or spam-filter text without an LLM call?",+ "question": "Can I classify text from labelled history without calling an LLM?",+ "canonical_url": "https://xerj.org/answers/classify-text-from-labelled-history-without-llm",+ "markdown_url": "https://xerj.org/answers/classify-text-from-labelled-history-without-llm.md",+ "cluster": "Decisions as retrieval",+ "updated": "2026-09-18",+ "summary": "Index labelled examples and classify new text by its nearest neighbours. Measured: 0.933 accuracy on 77-way intent routing, 0.983 on SMS spam, no LLM or API call."+ },{"title": "How do I reduce embedding API cost?","question": "My embedding bill is too high",@@ -425,6 +434,24 @@"updated": "2026-08-23","summary": "Build a RAG retrieval layer on one XERJ index. One hybrid request fuses BM25 and kNN with Reciprocal Rank Fusion, and no second service runs."},+ {+ "title": "How much better is hybrid search than BM25? Measured.",+ "question": "How much better is hybrid search than BM25 alone?",+ "canonical_url": "https://xerj.org/answers/hybrid-search-quality-measured-ndcg",+ "markdown_url": "https://xerj.org/answers/hybrid-search-quality-measured-ndcg.md",+ "cluster": "Hybrid retrieval: quality",+ "updated": "2026-09-19",+ "summary": "XERJ measured BM25, vectors and hybrid RRF on two public BEIR datasets. Hybrid won both: nDCG@10 0.6993 against 0.6572 on SciFact, with a neural embedder."+ },+ {+ "title": "How do I rerank search results with a relevance judge?",+ "question": "How do I rerank search results with a calibrated relevance judge?",+ "canonical_url": "https://xerj.org/answers/rerank-search-results-calibrated-judge",+ "markdown_url": "https://xerj.org/answers/rerank-search-results-calibrated-judge.md",+ "cluster": "Hybrid retrieval: reranking",+ "updated": "2026-09-19",+ "summary": "Add a rerank block to a XERJ search: the top 30 hits go to an external judge, a 0 to 1 probability replaces the score, and min_score becomes a real cut-off."+ },{"title": "What is Reciprocal Rank Fusion?","question": "What is reciprocal rank fusion and when should I use it?",diff --git a/landing/answers/is-pdf-indexing-efficient.html b/landing/answers/is-pdf-indexing-efficient.htmlindex 2bbb68f2d..666612151 100644--- a/landing/answers/is-pdf-indexing-efficient.html+++ b/landing/answers/is-pdf-indexing-efficient.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class="active"><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class="active"><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/keep-index-current-as-files-change.html b/landing/answers/keep-index-current-as-files-change.htmlindex 3ce2063e8..f00c6dc4f 100644--- a/landing/answers/keep-index-current-as-files-change.html+++ b/landing/answers/keep-index-current-as-files-change.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class="active"><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class="active"><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/knowledge-graphs-for-agent-memory.html b/landing/answers/knowledge-graphs-for-agent-memory.htmlindex 017e5a000..a87e1b27c 100644--- a/landing/answers/knowledge-graphs-for-agent-memory.html+++ b/landing/answers/knowledge-graphs-for-agent-memory.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/local-embeddings-without-openai-api.html b/landing/answers/local-embeddings-without-openai-api.htmlindex e27e28b79..8dd0f7a0e 100644--- a/landing/answers/local-embeddings-without-openai-api.html+++ b/landing/answers/local-embeddings-without-openai-api.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class="active"><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class="active"><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/mcp-api-search-or-vector-search.html b/landing/answers/mcp-api-search-or-vector-search.htmlindex 5c5ad4c6b..a8f8a643d 100644--- a/landing/answers/mcp-api-search-or-vector-search.html+++ b/landing/answers/mcp-api-search-or-vector-search.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/notion-export-duplicate-search-results.html b/landing/answers/notion-export-duplicate-search-results.htmlindex 737e77a95..c5bfad099 100644--- a/landing/answers/notion-export-duplicate-search-results.html+++ b/landing/answers/notion-export-duplicate-search-results.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class="active"><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class="active"><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/private-agent-memory-namespaces.html b/landing/answers/private-agent-memory-namespaces.htmlindex 6a7f62aa1..5d0db9e1c 100644--- a/landing/answers/private-agent-memory-namespaces.html+++ b/landing/answers/private-agent-memory-namespaces.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/rag-without-vector-database.html b/landing/answers/rag-without-vector-database.htmlindex de57a7569..b78502ef0 100644--- a/landing/answers/rag-without-vector-database.html+++ b/landing/answers/rag-without-vector-database.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class="active"><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class="active"><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/read-autoindex-progress.html b/landing/answers/read-autoindex-progress.htmlindex 6f9547db4..8ce98e06d 100644--- a/landing/answers/read-autoindex-progress.html+++ b/landing/answers/read-autoindex-progress.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class="active"><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class="active"><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/reciprocal-rank-fusion-when-to-use.html b/landing/answers/reciprocal-rank-fusion-when-to-use.htmlindex 02fb1dfc6..b068366be 100644--- a/landing/answers/reciprocal-rank-fusion-when-to-use.html+++ b/landing/answers/reciprocal-rank-fusion-when-to-use.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class="active"><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class="active"><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/reduce-embedding-api-cost.html b/landing/answers/reduce-embedding-api-cost.htmlindex eb97df31d..e82ca68bd 100644--- a/landing/answers/reduce-embedding-api-cost.html+++ b/landing/answers/reduce-embedding-api-cost.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class="active"><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class="active"><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/reindex-taking-forever.html b/landing/answers/reindex-taking-forever.htmlindex e3aeb4148..dd3fb01e9 100644--- a/landing/answers/reindex-taking-forever.html+++ b/landing/answers/reindex-taking-forever.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class="active"><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class="active"><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/rerank-search-results-calibrated-judge.html b/landing/answers/rerank-search-results-calibrated-judge.htmlnew file mode 100644index 000000000..3cb457523--- /dev/null+++ b/landing/answers/rerank-search-results-calibrated-judge.html@@ -0,0 +1,415 @@+<!-- generated by build_articles.py — edit content/answers/rerank-search-results-calibrated-judge.md instead -->+<!DOCTYPE html>+<html lang="en" data-theme="day">+<head>+<meta charset="UTF-8">+<meta name="viewport" content="width=device-width, initial-scale=1.0">+<title>How do I rerank search results with a relevance judge?</title>+<meta name="description" content="Add a rerank block to a XERJ search: the top 30 hits go to an external judge, a 0 to 1 probability replaces the score, and min_score becomes a real cut-off.">+<link rel="preconnect" href="https://fonts.googleapis.com">+<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>+<link href="https://fonts.googleapis.com/css2?family=Big+Shoulders+Display:wght@400;700;800;900&family=Inter:wght@400;500;600;700&family=IBM+Plex+Sans:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500;700&display=swap" rel="stylesheet">+<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'><rect width='100' height='100' fill='%230b0b0d'/><text x='50' y='68' font-size='60' font-family='Big Shoulders Display,sans-serif' font-weight='900' fill='%23ffc400' text-anchor='middle'>X</text></svg>">+<link rel="stylesheet" href="/style.css?v=90f643e">+<link rel="alternate" type="text/markdown" href="/answers/rerank-search-results-calibrated-judge.md">+<!-- seo:begin — generated by scripts/seo/fix_heads.py · do not hand-edit -->+<link rel="canonical" href="https://xerj.org/answers/rerank-search-results-calibrated-judge">+<meta property="og:site_name" content="XERJ.ai">+<meta property="og:type" content="article">+<meta property="og:title" content="How do I rerank search results with a relevance judge?">+<meta property="og:description" content="Add a rerank block to a XERJ search: the top 30 hits go to an external judge, a 0 to 1 probability replaces the score, and min_score becomes a real cut-off.">+<meta property="og:url" content="https://xerj.org/answers/rerank-search-results-calibrated-judge">+<meta property="og:image" content="https://xerj.org/og/xerj-card.png">+<meta property="og:image:width" content="1200">+<meta property="og:image:height" content="630">+<meta property="og:image:alt" content="XERJ.ai — the search engine built for AI agents">+<meta property="og:locale" content="en_US">+<meta name="twitter:card" content="summary_large_image">+<meta name="twitter:title" content="How do I rerank search results with a relevance judge?">+<meta name="twitter:description" content="Add a rerank block to a XERJ search: the top 30 hits go to an external judge, a 0 to 1 probability replaces the score, and min_score becomes a real cut-off.">+<meta name="twitter:image" content="https://xerj.org/og/xerj-card.png">+<script type="application/ld+json">+{+ "@context": "https://schema.org",+ "@type": "Organization",+ "@id": "https://xerj.org/#organization",+ "name": "XERJ",+ "alternateName": "XERJ.ai",+ "url": "https://xerj.org/",+ "logo": "https://xerj.org/brandbook/xerj-mark.svg",+ "sameAs": [+ "https://github.com/xerj-org/xerj"+ ]+}+</script>+<script type="application/ld+json">+{+ "@context": "https://schema.org",+ "@type": "WebSite",+ "@id": "https://xerj.org/#website",+ "name": "XERJ.ai",+ "url": "https://xerj.org/",+ "inLanguage": "en",+ "publisher": {+ "@id": "https://xerj.org/#organization"+ }+}+</script>+<script type="application/ld+json">+{+ "@context": "https://schema.org",+ "@type": "TechArticle",+ "headline": "How do I rerank search results with a relevance judge?",+ "description": "Add a rerank block to a XERJ search: the top 30 hits go to an external judge, a 0 to 1 probability replaces the score, and min_score becomes a real cut-off.",+ "url": "https://xerj.org/answers/rerank-search-results-calibrated-judge",+ "image": "https://xerj.org/og/xerj-card.png",+ "inLanguage": "en",+ "datePublished": "2026-09-18",+ "dateModified": "2026-09-19",+ "author": {+ "@type": "Person",+ "name": "XERJ documentation team"+ },+ "publisher": {+ "@id": "https://xerj.org/#organization"+ },+ "isPartOf": {+ "@id": "https://xerj.org/#website"+ },+ "mainEntityOfPage": {+ "@type": "WebPage",+ "@id": "https://xerj.org/answers/rerank-search-results-calibrated-judge"+ },+ "reviewedBy": {+ "@type": "Person",+ "name": "XERJ engineering team"+ }+}+</script>+<script type="application/ld+json">+{+ "@context": "https://schema.org",+ "@type": "BreadcrumbList",+ "itemListElement": [+ {+ "@type": "ListItem",+ "position": 1,+ "name": "Home",+ "item": "https://xerj.org/"+ },+ {+ "@type": "ListItem",+ "position": 2,+ "name": "Answers",+ "item": "https://xerj.org/answers/"+ },+ {+ "@type": "ListItem",+ "position": 3,+ "name": "How do I rerank search results with a calibrated relevance judge?"+ }+ ]+}+</script>+<!-- seo:end -->+</head>+<body>+<nav class="nav" aria-label="Primary">+ <a href="/" class="brand">XERJ.AI</a>+ <a href="/">HOME</a>+ <a href="/product">PRODUCT</a>+ <span class="nav-drop">+ <a href="/use-cases">USE CASES</a>+ </span>+ <a href="/case-studies">CASE STUDIES</a>+ <a href="/benchmarks/">BENCHMARKS</a>+ <span class="spacer"></span>+ <a href="/for-agents">FOR AGENTS</a>+ <a href="/docs/" class="active">DOCS</a>+ <a href="/docs/recipes/">RECIPES</a>+ <a href="/playground">DASHBOARDS</a>+ <a href="https://github.com/xerj-org/xerj" rel="external">GITHUB</a>+ <span class="spacer"></span>+ <span class="theme">+ <button type="button" data-theme="day" class="active">DAY</button>+ <span class="dash">·</span>+ <button type="button" data-theme="night">NIGHT</button>+ </span>+</nav>+<div class="docs-shell">+<nav class="docs-nav" aria-label="Article sections">+ <div class="group">+ <span class="label">ANSWERS</span>+ <a href="/answers/"><span class="eyebrow">00</span>All Answers</a>+ <a href="/answers/give-chatgpt-claude-local-file-access" class=""><span class="eyebrow">01</span>Can ChatGPT search a folder on my laptop, or do I need something else?</a>+ <a href="/answers/code-search-mcp-for-claude-code" class=""><span class="eyebrow">02</span>I need my agent to find a passage and cite the file, not dump the whole tree. How?</a>+ <a href="/answers/mcp-api-search-or-vector-search" class=""><span class="eyebrow">03</span>Should an agent use API search or vector search?</a>+ <a href="/answers/knowledge-graphs-for-agent-memory" class=""><span class="eyebrow">04</span>Should agent memory use a knowledge graph?</a>+ <a href="/answers/store-agent-memory-without-llm-call" class=""><span class="eyebrow">05</span>How do I stop an agent memory store from filling with near-duplicate notes?</a>+ <a href="/answers/evidence-for-agent-memory-links" class=""><span class="eyebrow">06</span>How can agent-memory links retain evidence?</a>+ <a href="/answers/private-agent-memory-namespaces" class=""><span class="eyebrow">07</span>How do I give two agents private memory on one laptop?</a>+ <a href="/answers/coding-agent-memory-across-sessions" class=""><span class="eyebrow">08</span>How do people store user preferences so an agent can recall them next week?</a>+ <a href="/answers/set-mcp-memory-storage-path" class=""><span class="eyebrow">09</span>How do I set the MCP memory storage path?</a>+ <a href="/answers/agent-memory-without-postgres-qdrant" class=""><span class="eyebrow">10</span>What's a simple way to give an agent long-term memory on my laptop without Qdrant?</a>+ <a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a>+ <a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a>+ <a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class="active"><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ </div>+ <div class="group">+ <span class="label">COMPARISONS</span>+ <a href="/compare/"><span class="eyebrow">00</span>All Comparisons</a>+ <a href="/compare/xerj-vs-web-agent-search" class=""><span class="eyebrow">01</span>How do I let an agent search a folder on my laptop instead of crawling the web?</a>+ <a href="/compare/xerj-vs-ast-grep" class=""><span class="eyebrow">02</span>ast-grep vs a code index for an agent?</a>+ <a href="/compare/xerj-vs-ripgrep-for-code-agents" class=""><span class="eyebrow">03</span>Is ripgrep enough for a coding agent, or do I need a code index?</a>+ <a href="/compare/xerj-vs-ripgrep-all" class=""><span class="eyebrow">04</span>How do I grep PDFs and Word docs from the command line?</a>+ <a href="/compare/xerj-vs-docfetcher" class=""><span class="eyebrow">05</span>What's a free DocFetcher alternative that an agent can call?</a>+ <a href="/compare/xerj-vs-recoll" class=""><span class="eyebrow">06</span>What's the best local desktop search for a folder of PDFs and docs?</a>+ <a href="/compare/xerj-vs-houdahspot" class=""><span class="eyebrow">07</span>XERJ vs HoudahSpot — Spotlight GUI vs an agent index</a>+ <a href="/compare/xerj-vs-spotlight" class=""><span class="eyebrow">08</span>XERJ vs macOS Spotlight for folder content an agent can query</a>+ <a href="/compare/xerj-vs-localsynapse" class=""><span class="eyebrow">09</span>What's a local MCP search engine for Claude?</a>+ <a href="/compare/xerj-vs-elasticsearch-local" class=""><span class="eyebrow">10</span>Is there an Elasticsearch-compatible engine that isn't Elasticsearch?</a>+ <a href="/compare/xerj-vs-obsidian-omnisearch" class=""><span class="eyebrow">11</span>How do I search an Obsidian vault from outside Obsidian?</a>+ <a href="/compare/xerj-vs-khoj" class=""><span class="eyebrow">12</span>What's a simpler alternative to Khoj if I only need search and memory?</a>+ <a href="/compare/xerj-vs-typesense" class=""><span class="eyebrow">13</span>Do I need Typesense for an agent to search my docs?</a>+ <a href="/compare/xerj-vs-meilisearch" class=""><span class="eyebrow">14</span>What's a local alternative to Meilisearch that indexes a folder by itself?</a>+ <a href="/compare/xerj-vs-pagefind" class=""><span class="eyebrow">15</span>Pagefind vs indexing a local HTML dump for an agent?</a>+ <a href="/compare/xerj-vs-vector-database" class=""><span class="eyebrow">16</span>Should my agent call a search API or stand up Qdrant for persistent memory?</a>+ </div>+</nav>+<main class="docs-content">+ <div class="breadcrumb"><a href="/">HOME</a><span class="sep">·</span><a href="/answers/">ANSWERS</a><span class="sep">·</span>HOW DO I RERANK SEARCH RESULTS WITH A CALIBRATED RELEVANCE JUDGE?</div>+ <div class="kicker"><span>ANSWERS · HYBRID RETRIEVAL: RERANKING</span></div>+ <h1>How do I rerank search results with a calibrated relevance judge?</h1>+ <div class="source-note">AUTHOR · XERJ documentation team · REVIEWED BY · XERJ engineering team · PUBLISHED · 2026-09-18 · UPDATED · 2026-09-19</div>+ <p><strong>TL;DR</strong> — Add <code class="inline">"rerank": {}</code> to a XERJ <code class="inline">_search</code> body. The top 30 hits go to an external judge, which returns a 0 to 1 relevance probability per document. That probability replaces <code class="inline">_score</code>, so <code class="inline">min_score</code> becomes a real cut-off. It sends document text off the machine. XERJ has not verified ranking quality with the real model.</p>+ <div class="prompt-card">+ <span class="prompt-label">PASTE THIS TO YOUR AI AGENT — IT DOES THE REST</span>+ <code>Act as a coding agent. Read https://xerj.org/llms.txt, ask the operator whether a rerank provider key is set on the XERJ node and whether document text may leave the machine, then run one search without the rerank block and one with it, and report the _rerank block of the second response, including whether applied is true.</code>+ <button type="button" class="prompt-copy" data-copy="Act as a coding agent. Read https://xerj.org/llms.txt, ask the operator whether a rerank provider key is set on the XERJ node and whether document text may leave the machine, then run one search without the rerank block and one with it, and report the _rerank block of the second response, including whether applied is true.">COPY THE PROMPT</button>+ </div>+ <div class="prompt-card">+ <span class="prompt-label">RUN THIS XERJ COMMAND</span>+ <code>curl -s 'http://127.0.0.1:9200/_xerj/rerank'</code>+ <button type="button" class="prompt-copy" data-copy="curl -s 'http://127.0.0.1:9200/_xerj/rerank'">COPY COMMAND</button>+ <span class="hero-proof">Ask the node whether a rerank provider is configured. The answer names where the key came from and never prints the key. A node started with --insecure needs no credentials. On any other node the route is superuser-only, so send the admin key from the admin.key file in the data directory.</span>+ </div>+ <div class="prompt-card">+ <span class="prompt-label">RUN THIS XERJ COMMAND</span>+ <code>curl -s -XPOST 'http://127.0.0.1:9200/kb/_search' -H 'content-type: application/json' -d '{"query":{"match":{"body":"vitamin d bone density"}},"size":5,"rerank":{"min_score":0.5}}'</code>+ <button type="button" class="prompt-copy" data-copy="curl -s -XPOST 'http://127.0.0.1:9200/kb/_search' -H 'content-type: application/json' -d '{"query":{"match":{"body":"vitamin d bone density"}},"size":5,"rerank":{"min_score":0.5}}'">COPY COMMAND</button>+ <span class="hero-proof">Rerank the top 30 hits and keep the ones the judge scores at 0.5 or higher. This request sends the text of those hits to the provider.</span>+ </div>+ <div class="prompt-card">+ <span class="prompt-label">RUN THIS XERJ COMMAND</span>+ <code>curl -s -XPOST 'http://127.0.0.1:9200/kb/_search' -H 'content-type: application/json' -d '{"query":{"match":{"body":"vitamin d bone density"}},"_source":["title"],"rerank":{}}'</code>+ <button type="button" class="prompt-copy" data-copy="curl -s -XPOST 'http://127.0.0.1:9200/kb/_search' -H 'content-type: application/json' -d '{"query":{"match":{"body":"vitamin d bone density"}},"_source":["title"],"rerank":{}}'">COPY COMMAND</button>+ <span class="hero-proof">Send titles only. The judge sees exactly what the response returns, so source filtering is also the control on what leaves the machine.</span>+ </div>+ <h2>What the rerank stage does</h2>+ <p>The engine retrieves and ranks as usual. The stage then sends the question and the top <code class="inline">window</code> hits to a relevance judge. The judge answers one yes-or-no question per document and returns a probability that the document is relevant.</p>+ <p>XERJ reorders the hits by that probability. The probability replaces <code class="inline">_score</code>, and <code class="inline">hits.max_score</code> becomes the top hit's probability.</p>+ <pre class="code"><code class="language-json">{+ "query": { "match": { "body": "vitamin d bone density" } },+ "size": 5,+ "rerank": { "min_score": 0.5 }+}+</code></pre>+ <p>The provider is TypeSafe AI's Jev model, reached through its System One API. XERJ asks one question per document rather than one multiple-choice question across all of them, because a real corpus has more than one relevant document. That request shape is the approach used by the <code class="inline">hev/jev-rerank</code> project, which is Apache-2.0.</p>+ <h2>Why a probability is worth more than a better order</h2>+ <p>A BM25 score orders results but has no absolute meaning. A score of 7.2 on one query is not comparable to 7.2 on another query, so you cannot use it as a threshold.</p>+ <p>A calibrated probability means the same thing on every query. <code class="inline">rerank.min_score: 0.5</code> drops every document the judge scored below 0.5, whatever the query was. The top-level <code class="inline">min_score</code> stays the engine's threshold on engine scores, and the two compose.</p>+ <h2>This sends your data off the machine</h2>+ <p>Reranking is the only search-time feature that sends document text to a third party. Two other features send text off the node, and both are off by default. Proxy embeddings (<code class="inline">[embedding] default_endpoint</code>) send document text at indexing time and query text at search time to an external embeddings API. The WAL tap replays every write on tapped indices to an external <code class="inline">_bulk</code> endpoint. An operator has to turn either one on.</p>+ <p>The node has two more outbound connections, and neither carries document or query text. Neural mode downloads its model from the HuggingFace Hub once. Cluster mode, which a default single-node deployment never starts, sends Raft messages, such as index names and mappings, to your own peers. The reference, <code class="inline">docs/RERANK.md</code>, lists every outbound connection a node can open.</p>+ <p>A rerank request POSTs the question and the text of up to <code class="inline">window</code> hits to the provider.</p>+ <p>Three controls exist. Reranking does nothing until an operator sets a provider key. A search only triggers it by carrying a <code class="inline">rerank</code> block. An operator can forbid it with <code class="inline">enabled = false</code> under <code class="inline">[rerank]</code>, which refuses every rerank request with HTTP 403.</p>+ <p>The judge sees exactly what the response returns, and nothing else. Source filtering is therefore also the control on what leaves the machine: <code class="inline">"_source": ["title"]</code> sends titles only. <code class="inline">rerank.fields</code> is exhaustive, so <code class="inline">["body"]</code> sends the body and not even the title. Document ids, index names, numbers and vectors are never sent.</p>+ <h2>Set the provider key</h2>+ <p>Put the key in the config file or in the environment. A value in the config file wins over the environment. A stray variable therefore cannot silently change where document text is sent.</p>+ <pre class="code"><code class="language-toml">[rerank]+enabled = true+api_key = "ts-..."+endpoint = "https://api.typesafe.ai/v1/systemone"+</code></pre>+ <p>The environment fallbacks are <code class="inline">TYPESAFE_API_KEY</code> and <code class="inline">TYPESAFE_ENDPOINT</code>. There is no per-request key, because a key in a search body would land in logs. <code class="inline">GET /_xerj/rerank</code> reports whether a key is set and where it came from. It never returns the key.</p>+ <h2>Read the _rerank block</h2>+ <p>Every reranked response carries a <code class="inline">_rerank</code> block ahead of <code class="inline">hits</code>. Read <code class="inline">applied</code> before you trust the order.</p>+ <table>+ <thead><tr><th>Field</th><th>Meaning</th></tr></thead>+ <tbody>+ <tr><td><code class="inline">applied</code></td><td><code class="inline">true</code> means the order is the judge's. <code class="inline">false</code> means the engine's order stands, and <code class="inline">reason</code> says why.</td></tr>+ <tr><td><code class="inline">score_kind</code></td><td><code class="inline">probability</code> or <code class="inline">engine</code>.</td></tr>+ <tr><td><code class="inline">judged</code></td><td>Documents the provider returned a verdict for.</td></tr>+ <tr><td><code class="inline">pruned_below_min_score</code></td><td>Judged hits removed by <code class="inline">rerank.min_score</code>.</td></tr>+ <tr><td><code class="inline">fields_without_text</code></td><td>Named fields that no hit in the window returned text for.</td></tr>+ <tr><td><code class="inline">usage</code></td><td>Tokens the provider reported for this search.</td></tr>+ </tbody>+ </table>+ <p><code class="inline">hits.total</code> and aggregations stay the engine's. They describe the full match set, not the window. When four documents match and two are judged irrelevant, <code class="inline">total</code> is 4 and <code class="inline">pruned_below_min_score</code> is 2.</p>+ <h2>What happens when the provider fails</h2>+ <p>The policy is split: degrade on deadline, surface on contract.</p>+ <table>+ <thead><tr><th>What happened</th><th>Result</th></tr></thead>+ <tbody>+ <tr><td>The provider missed <code class="inline">timeout_ms</code></td><td>HTTP 200, engine order, <code class="inline">applied: false</code>.</td></tr>+ <tr><td>The <code class="inline">rerank</code> block is malformed or the combination is refused</td><td>HTTP 400, nothing sent.</td></tr>+ <tr><td>The operator disabled reranking</td><td>HTTP 403, nothing sent.</td></tr>+ <tr><td>No provider key on the node</td><td>HTTP 503, nothing sent.</td></tr>+ <tr><td>The provider answered 401 or a malformed body</td><td>HTTP 502, no hits.</td></tr>+ </tbody>+ </table>+ <p>A slow third party is not a reason to deny you results you already have. A wrong key does not fix itself, and hiding it would mislead you about which ranking you hold. This split follows the approach Meilisearch uses in its personalization module (MIT). The approach and the retry back-off constants were adapted from it, and the code cites the lines.</p>+ <h2>Cost facts</h2>+ <p>Every document in the window is a paid judgement. The window defaults to 30 and the server caps it at 300. A provider call carries at most 30 documents, so <code class="inline">window: 35</code> is two calls. Scores from different calls stay comparable, because each one is an absolute probability rather than a rank inside its batch.</p>+ <p>XERJ sends 8 calls at once by default and 16 at most. The <code class="inline">usage</code> field is the provider's own token count for the search. XERJ does not price it.</p>+ <p>Every request is judged from scratch. There is no verdict cache, so three page requests over one 30-document window are three provider calls and 90 paid judgements. To keep the cost down, fetch the window once and cut the pages out of it in your own code. Page two continues page one only if the provider returns the same probabilities on a repeat call, which XERJ has not verified for the real model.</p>+ <p>The strings have ceilings too. <code class="inline">instructions</code> is limited to 2,000 characters, because the provider's wire format repeats it once per judged document. The question is limited to 4,000 characters and <code class="inline">model</code> to 128. A longer value is an HTTP 400 that names the field and the limit.</p>+ <h2>What is not verified</h2>+ <p>XERJ had no provider key. Ranking quality with the real Jev model is not verified by this project. Every test of the stage runs against a test double that scores by a table the test supplies.</p>+ <p>The <code class="inline">hev/jev-rerank</code> README publishes nDCG@10 of 0.768 on SciFact and 0.358 on NFCorpus for Jev. XERJ did not run those systems, and they rerank that project's own first-stage shortlist, not XERJ's. XERJ's own hybrid search scored 0.6993 and 0.3448 on the same datasets. Those two figures were measured with <code class="inline">--embed-mode neural</code> and the all-MiniLM-L6-v2 model. The default embedder is lexical feature hashing, which has no model in it. This is not a controlled comparison.</p>+ <h2>When reranking is not worth it</h2>+ <p>Do not rerank an exact identifier lookup, a filter, or an aggregation-only request. The first stage already answers those, and the stage adds a paid network call per search. Rerank when the right document is probably in the top 30 but not at the top, or when you need an absolute relevance cut-off.</p>+ <h2>FAQ</h2>+ <h3>How do I rerank search results in XERJ?</h3>+ <p>Add a <code class="inline">rerank</code> object to the body of <code class="inline">POST /{index}/_search</code>. An empty object uses every default: the top 30 hits are judged and reordered by relevance probability.</p>+ <h3>Does reranking send my documents to a third party?</h3>+ <p>Yes. It is the only search-time feature that sends document text off the node. Proxy embeddings and the WAL tap also send text off the node when an operator configures them, and all three are off by default. The question and the text of up to <code class="inline">window</code> hits go to the provider. Only fields the response returns are sent.</p>+ <h3>What happens when the rerank provider is slow?</h3>+ <p>The search still answers with HTTP 200. You get the engine's order and scores, and the <code class="inline">_rerank</code> block says <code class="inline">applied: false</code> with the reason. A slow provider degrades; it does not fail the search.</p>+ <h3>What happens when the provider key is wrong or missing?</h3>+ <p>The search fails loudly. A missing key is HTTP 503, a provider 401 is HTTP 502, both with type <code class="inline">rerank_exception</code> and no hits. XERJ does not hand back lexical order as if it had been reranked.</p>+ <h3>Is the ranking quality of the rerank model verified?</h3>+ <p>No. XERJ had no provider key, so ranking quality with the real model is not verified by this project. The tests use a test double. Only the mechanism is verified.</p>+ <h3>Can I combine rerank with sort or aggregations?</h3>+ <p>Aggregations work and still describe the full match set. <code class="inline">sort</code>, <code class="inline">search_after</code>, <code class="inline">collapse</code>, <code class="inline">scroll</code> and <code class="inline">size: 0</code> are refused with an HTTP 400 that names the conflict.</p>+ <h3>Can an AI agent ask for reranking through MCP?</h3>+ <p>Yes. The <code class="inline">xerj_search</code> and <code class="inline">xerj_hybrid_search</code> tools take an optional <code class="inline">rerank</code> argument. It only works when the operator has set a provider key on the node.</p>+ <h2>Evidence</h2>+ <ul>+ <li>Reranking is the only search-time feature that sends document text off the node. Proxy embeddings and the WAL tap also send text off the node when configured; the neural model download and cluster Raft messages carry no document or query text. A test checks that list against every outbound network client in the engine source. — <code>engine/crates/xerj-rerank/tests/egress_inventory.rs</code></li>+ <li>The rerank stage defaults to a window of 30 hits, allows at most 300, sends at most 30 documents per provider call, runs 8 calls in flight by default and 16 at most, and defaults to a 10000 ms stage budget capped at 60000 ms. — <code>engine/crates/xerj-rerank/src/lib.rs</code></li>+ <li>The failure policy, every refused combination, the 400, 403, 502 and 503 statuses, paging inside the window, and the rule that only returned text is sent are each pinned by an HTTP test against an in-process stub of the provider wire format. — <code>engine/crates/xerj-api/tests/rerank_stage_http.rs</code></li>+ <li>The nDCG@10 figures for Jev (0.768 SciFact, 0.358 NFCorpus), Voyage rerank-3 (0.755, 0.357) and Cohere rerank-v3.5 (0.745, 0.340) are published in the hev/jev-rerank README. XERJ did not run them, and they rerank that project's own first-stage shortlist. — <a href="https://github.com/hev/jev-rerank">https://github.com/hev/jev-rerank</a></li>+ <li>XERJ hybrid RRF scored nDCG@10 0.6993 on SciFact and 0.3448 on NFCorpus with --embed-mode neural and all-MiniLM-L6-v2, not with the default lexical embedder. — <code>benchmarks/beir-hybrid/README.md</code></li>+ </ul>+ <h2>Related</h2>+ <ul>+ <li><a href="/answers/reciprocal-rank-fusion-when-to-use">What is Reciprocal Rank Fusion?</a></li>+ <li><a href="/answers/hybrid-search-quality-measured-ndcg">How much better is hybrid search than BM25? Measured nDCG numbers</a></li>+ <li><a href="/answers/how-xerj-combines-search">How does XERJ combine BM25 and kNN?</a></li>+ <li><a href="/docs/config">Config TOML</a></li>+ </ul>+</main>+</div>+<!-- seo:footer-begin — generated by scripts/seo/fix_links.py · do not hand-edit -->+<nav class="footer-map" aria-label="Site map">+ <div class="fm-col">+ <span class="fm-h">PRODUCT</span>+ <a href="/product">PRODUCT</a>+ <a href="/architecture/">ARCHITECTURE</a>+ <a href="/benchmarks/">BENCHMARKS</a>+ <a href="/pricing/">PRICING</a>+ <a href="/security/">SECURITY</a>+ <a href="/playground">DASHBOARDS</a>+ <a href="/demo/">REAL-DATA DEMO</a>+ <a href="/aise-demo">PRODUCT TOUR</a>+ </div>+ <div class="fm-col">+ <span class="fm-h">SOLUTIONS</span>+ <a href="/solutions/">SOLUTIONS</a>+ <a href="/use-cases">USE CASES</a>+ <a href="/case-studies">CASE STUDIES</a>+ <a href="/industries/">INDUSTRIES</a>+ <a href="/industries/finserv">FINANCIAL SERVICES</a>+ <a href="/industries/public-sector">PUBLIC SECTOR</a>+ <a href="/industries/healthcare">HEALTHCARE</a>+ <a href="/industries/retail">RETAIL</a>+ <a href="/use-cases/code-security-audit">AI CODE-SECURITY AUDIT</a>+ <a href="/use-cases/semantic-analytics">SEMANTIC ANALYTICS</a>+ </div>+ <div class="fm-col">+ <span class="fm-h">DOCS</span>+ <a href="/docs/">DOCUMENTATION</a>+ <a href="/docs/quickstart">QUICKSTART</a>+ <a href="/docs/install">INSTALL</a>+ <a href="/docs/migration-from-es">MIGRATE FROM ELASTICSEARCH</a>+ <a href="/docs/recipes/">RECIPES</a>+ <a href="/docs/agents/">DOCS FOR AI</a>+ <a href="/for-agents">FOR AI AGENTS</a>+ <a href="/agent-search/">AGENTIC SEARCH DEMO</a>+ <a href="/answers/">ANSWERS</a>+ <a href="/compare/">COMPARISONS</a>+ </div>+ <div class="fm-col">+ <span class="fm-h">RESOURCES</span>+ <a href="/resources/">SALES KIT</a>+ <a href="/brand">BRAND</a>+ <a href="/brandbook/">BRAND BOOK</a>+ <a href="/llms.txt">LLMS.TXT</a>+ <a href="https://github.com/xerj-org/xerj" rel="external">GITHUB</a>+ </div>+</nav>+<!-- seo:footer-end -->+<footer class="footer">+ <span>XERJ.AI · ANSWERS</span>+ <span>READ THE SOURCE</span>+ <span><a href="/">HOME</a> · <a href="/product">PRODUCT</a> · <a href="/brand">BRAND</a> · <a href="/playground">DASHBOARDS</a> · <a href="/llms.txt">LLMS.TXT</a></span>+</footer>+<script src="/interact.js"></script>+</body>+</html>diff --git a/landing/answers/rerank-search-results-calibrated-judge.md b/landing/answers/rerank-search-results-calibrated-judge.mdnew file mode 100644index 000000000..5db44c824--- /dev/null+++ b/landing/answers/rerank-search-results-calibrated-judge.md@@ -0,0 +1,184 @@+<!-- generated by build_articles.py — edit content/answers/rerank-search-results-calibrated-judge.md instead -->+---+title: "How do I rerank search results with a relevance judge?"+canonical: "https://xerj.org/answers/rerank-search-results-calibrated-judge"+updated: "2026-09-19"+source: "content/answers/rerank-search-results-calibrated-judge.md"+---++# How do I rerank search results with a calibrated relevance judge?++**TL;DR** — Add `"rerank": {}` to a XERJ `_search` body. The top 30 hits go to an external judge, which returns a 0 to 1 relevance probability per document. That probability replaces `_score`, so `min_score` becomes a real cut-off. It sends document text off the machine. XERJ has not verified ranking quality with the real model.++## Agent prompt++```text+Act as a coding agent. Read https://xerj.org/llms.txt, ask the operator whether a rerank provider key is set on the XERJ node and whether document text may leave the machine, then run one search without the rerank block and one with it, and report the _rerank block of the second response, including whether applied is true.+```++## Commands++### Command 1++Note: Ask the node whether a rerank provider is configured. The answer names where the key came from and never prints the key. A node started with --insecure needs no credentials. On any other node the route is superuser-only, so send the admin key from the admin.key file in the data directory.++```sh+curl -s 'http://127.0.0.1:9200/_xerj/rerank'+```++### Command 2++Note: Rerank the top 30 hits and keep the ones the judge scores at 0.5 or higher. This request sends the text of those hits to the provider.++```sh+curl -s -XPOST 'http://127.0.0.1:9200/kb/_search' -H 'content-type: application/json' -d '{"query":{"match":{"body":"vitamin d bone density"}},"size":5,"rerank":{"min_score":0.5}}'+```++### Command 3++Note: Send titles only. The judge sees exactly what the response returns, so source filtering is also the control on what leaves the machine.++```sh+curl -s -XPOST 'http://127.0.0.1:9200/kb/_search' -H 'content-type: application/json' -d '{"query":{"match":{"body":"vitamin d bone density"}},"_source":["title"],"rerank":{}}'+```++## What the rerank stage does++The engine retrieves and ranks as usual. The stage then sends the question and the top `window` hits to a relevance judge. The judge answers one yes-or-no question per document and returns a probability that the document is relevant.++XERJ reorders the hits by that probability. The probability replaces `_score`, and `hits.max_score` becomes the top hit's probability.++```json+{+ "query": { "match": { "body": "vitamin d bone density" } },+ "size": 5,+ "rerank": { "min_score": 0.5 }+}+```++The provider is TypeSafe AI's Jev model, reached through its System One API. XERJ asks one question per document rather than one multiple-choice question across all of them, because a real corpus has more than one relevant document. That request shape is the approach used by the `hev/jev-rerank` project, which is Apache-2.0.++## Why a probability is worth more than a better order++A BM25 score orders results but has no absolute meaning. A score of 7.2 on one query is not comparable to 7.2 on another query, so you cannot use it as a threshold.++A calibrated probability means the same thing on every query. `rerank.min_score: 0.5` drops every document the judge scored below 0.5, whatever the query was. The top-level `min_score` stays the engine's threshold on engine scores, and the two compose.++## This sends your data off the machine++Reranking is the only search-time feature that sends document text to a third party. Two other features send text off the node, and both are off by default. Proxy embeddings (`[embedding] default_endpoint`) send document text at indexing time and query text at search time to an external embeddings API. The WAL tap replays every write on tapped indices to an external `_bulk` endpoint. An operator has to turn either one on.++The node has two more outbound connections, and neither carries document or query text. Neural mode downloads its model from the HuggingFace Hub once. Cluster mode, which a default single-node deployment never starts, sends Raft messages, such as index names and mappings, to your own peers. The reference, `docs/RERANK.md`, lists every outbound connection a node can open.++A rerank request POSTs the question and the text of up to `window` hits to the provider.++Three controls exist. Reranking does nothing until an operator sets a provider key. A search only triggers it by carrying a `rerank` block. An operator can forbid it with `enabled = false` under `[rerank]`, which refuses every rerank request with HTTP 403.++The judge sees exactly what the response returns, and nothing else. Source filtering is therefore also the control on what leaves the machine: `"_source": ["title"]` sends titles only. `rerank.fields` is exhaustive, so `["body"]` sends the body and not even the title. Document ids, index names, numbers and vectors are never sent.++## Set the provider key++Put the key in the config file or in the environment. A value in the config file wins over the environment. A stray variable therefore cannot silently change where document text is sent.++```toml+[rerank]+enabled = true+api_key = "ts-..."+endpoint = "https://api.typesafe.ai/v1/systemone"+```++The environment fallbacks are `TYPESAFE_API_KEY` and `TYPESAFE_ENDPOINT`. There is no per-request key, because a key in a search body would land in logs. `GET /_xerj/rerank` reports whether a key is set and where it came from. It never returns the key.++## Read the _rerank block++Every reranked response carries a `_rerank` block ahead of `hits`. Read `applied` before you trust the order.++| Field | Meaning |+| --- | --- |+| `applied` | `true` means the order is the judge's. `false` means the engine's order stands, and `reason` says why. |+| `score_kind` | `probability` or `engine`. |+| `judged` | Documents the provider returned a verdict for. |+| `pruned_below_min_score` | Judged hits removed by `rerank.min_score`. |+| `fields_without_text` | Named fields that no hit in the window returned text for. |+| `usage` | Tokens the provider reported for this search. |++`hits.total` and aggregations stay the engine's. They describe the full match set, not the window. When four documents match and two are judged irrelevant, `total` is 4 and `pruned_below_min_score` is 2.++## What happens when the provider fails++The policy is split: degrade on deadline, surface on contract.++| What happened | Result |+| --- | --- |+| The provider missed `timeout_ms` | HTTP 200, engine order, `applied: false`. |+| The `rerank` block is malformed or the combination is refused | HTTP 400, nothing sent. |+| The operator disabled reranking | HTTP 403, nothing sent. |+| No provider key on the node | HTTP 503, nothing sent. |+| The provider answered 401 or a malformed body | HTTP 502, no hits. |++A slow third party is not a reason to deny you results you already have. A wrong key does not fix itself, and hiding it would mislead you about which ranking you hold. This split follows the approach Meilisearch uses in its personalization module (MIT). The approach and the retry back-off constants were adapted from it, and the code cites the lines.++## Cost facts++Every document in the window is a paid judgement. The window defaults to 30 and the server caps it at 300. A provider call carries at most 30 documents, so `window: 35` is two calls. Scores from different calls stay comparable, because each one is an absolute probability rather than a rank inside its batch.++XERJ sends 8 calls at once by default and 16 at most. The `usage` field is the provider's own token count for the search. XERJ does not price it.++Every request is judged from scratch. There is no verdict cache, so three page requests over one 30-document window are three provider calls and 90 paid judgements. To keep the cost down, fetch the window once and cut the pages out of it in your own code. Page two continues page one only if the provider returns the same probabilities on a repeat call, which XERJ has not verified for the real model.++The strings have ceilings too. `instructions` is limited to 2,000 characters, because the provider's wire format repeats it once per judged document. The question is limited to 4,000 characters and `model` to 128. A longer value is an HTTP 400 that names the field and the limit.++## What is not verified++XERJ had no provider key. Ranking quality with the real Jev model is not verified by this project. Every test of the stage runs against a test double that scores by a table the test supplies.++The `hev/jev-rerank` README publishes nDCG@10 of 0.768 on SciFact and 0.358 on NFCorpus for Jev. XERJ did not run those systems, and they rerank that project's own first-stage shortlist, not XERJ's. XERJ's own hybrid search scored 0.6993 and 0.3448 on the same datasets. Those two figures were measured with `--embed-mode neural` and the all-MiniLM-L6-v2 model. The default embedder is lexical feature hashing, which has no model in it. This is not a controlled comparison.++## When reranking is not worth it++Do not rerank an exact identifier lookup, a filter, or an aggregation-only request. The first stage already answers those, and the stage adds a paid network call per search. Rerank when the right document is probably in the top 30 but not at the top, or when you need an absolute relevance cut-off.++## FAQ++### How do I rerank search results in XERJ?++Add a `rerank` object to the body of `POST /{index}/_search`. An empty object uses every default: the top 30 hits are judged and reordered by relevance probability.++### Does reranking send my documents to a third party?++Yes. It is the only search-time feature that sends document text off the node. Proxy embeddings and the WAL tap also send text off the node when an operator configures them, and all three are off by default. The question and the text of up to `window` hits go to the provider. Only fields the response returns are sent.++### What happens when the rerank provider is slow?++The search still answers with HTTP 200. You get the engine's order and scores, and the `_rerank` block says `applied: false` with the reason. A slow provider degrades; it does not fail the search.++### What happens when the provider key is wrong or missing?++The search fails loudly. A missing key is HTTP 503, a provider 401 is HTTP 502, both with type `rerank_exception` and no hits. XERJ does not hand back lexical order as if it had been reranked.++### Is the ranking quality of the rerank model verified?++No. XERJ had no provider key, so ranking quality with the real model is not verified by this project. The tests use a test double. Only the mechanism is verified.++### Can I combine rerank with sort or aggregations?++Aggregations work and still describe the full match set. `sort`, `search_after`, `collapse`, `scroll` and `size: 0` are refused with an HTTP 400 that names the conflict.++### Can an AI agent ask for reranking through MCP?++Yes. The `xerj_search` and `xerj_hybrid_search` tools take an optional `rerank` argument. It only works when the operator has set a provider key on the node.++## Evidence++- Reranking is the only search-time feature that sends document text off the node. Proxy embeddings and the WAL tap also send text off the node when configured; the neural model download and cluster Raft messages carry no document or query text. A test checks that list against every outbound network client in the engine source. — `engine/crates/xerj-rerank/tests/egress_inventory.rs`+- The rerank stage defaults to a window of 30 hits, allows at most 300, sends at most 30 documents per provider call, runs 8 calls in flight by default and 16 at most, and defaults to a 10000 ms stage budget capped at 60000 ms. — `engine/crates/xerj-rerank/src/lib.rs`+- The failure policy, every refused combination, the 400, 403, 502 and 503 statuses, paging inside the window, and the rule that only returned text is sent are each pinned by an HTTP test against an in-process stub of the provider wire format. — `engine/crates/xerj-api/tests/rerank_stage_http.rs`+- The nDCG@10 figures for Jev (0.768 SciFact, 0.358 NFCorpus), Voyage rerank-3 (0.755, 0.357) and Cohere rerank-v3.5 (0.745, 0.340) are published in the hev/jev-rerank README. XERJ did not run them, and they rerank that project's own first-stage shortlist. — [https://github.com/hev/jev-rerank](https://github.com/hev/jev-rerank)+- XERJ hybrid RRF scored nDCG@10 0.6993 on SciFact and 0.3448 on NFCorpus with --embed-mode neural and all-MiniLM-L6-v2, not with the default lexical embedder. — `benchmarks/beir-hybrid/README.md`++## Related++- [What is Reciprocal Rank Fusion?](/answers/reciprocal-rank-fusion-when-to-use)+- [How much better is hybrid search than BM25? Measured nDCG numbers](/answers/hybrid-search-quality-measured-ndcg)+- [How does XERJ combine BM25 and kNN?](/answers/how-xerj-combines-search)+- [Config TOML](/docs/config)diff --git a/landing/answers/resume-interrupted-autoindex-run.html b/landing/answers/resume-interrupted-autoindex-run.htmlindex 9bb4715e7..214b0076c 100644--- a/landing/answers/resume-interrupted-autoindex-run.html+++ b/landing/answers/resume-interrupted-autoindex-run.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class="active"><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class="active"><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-all-pdfs-in-a-folder.html b/landing/answers/search-all-pdfs-in-a-folder.htmlindex 8b88861d3..9a15da0c2 100644--- a/landing/answers/search-all-pdfs-in-a-folder.html+++ b/landing/answers/search-all-pdfs-in-a-folder.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class="active"><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class="active"><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-bvh-motion-capture-files.html b/landing/answers/search-bvh-motion-capture-files.htmlindex 57384af07..0527819d6 100644--- a/landing/answers/search-bvh-motion-capture-files.html+++ b/landing/answers/search-bvh-motion-capture-files.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class="active"><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class="active"><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-confluence-html-export.html b/landing/answers/search-confluence-html-export.htmlindex ac7464745..f2305745a 100644--- a/landing/answers/search-confluence-html-export.html+++ b/landing/answers/search-confluence-html-export.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class="active"><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class="active"><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-engine-without-docker.html b/landing/answers/search-engine-without-docker.htmlindex 46f8be64a..0dd8ce0c6 100644--- a/landing/answers/search-engine-without-docker.html+++ b/landing/answers/search-engine-without-docker.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class="active"><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class="active"><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-file-contents-in-a-folder.html b/landing/answers/search-file-contents-in-a-folder.htmlindex 84f3dee5f..077c7b180 100644--- a/landing/answers/search-file-contents-in-a-folder.html+++ b/landing/answers/search-file-contents-in-a-folder.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class="active"><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class="active"><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-files-in-an-s3-bucket.html b/landing/answers/search-files-in-an-s3-bucket.htmlindex 3859b2f61..49e5441cd 100644--- a/landing/answers/search-files-in-an-s3-bucket.html+++ b/landing/answers/search-files-in-an-s3-bucket.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class="active"><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class="active"><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-gmail-takeout-mbox-locally.html b/landing/answers/search-gmail-takeout-mbox-locally.htmlindex 6325f8bad..724b633a3 100644--- a/landing/answers/search-gmail-takeout-mbox-locally.html+++ b/landing/answers/search-gmail-takeout-mbox-locally.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class="active"><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class="active"><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-gzip-logs-without-zgrep.html b/landing/answers/search-gzip-logs-without-zgrep.htmlindex ffc24683e..a8f1a0b3a 100644--- a/landing/answers/search-gzip-logs-without-zgrep.html+++ b/landing/answers/search-gzip-logs-without-zgrep.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class="active"><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class="active"><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-gzipped-csv-or-json.html b/landing/answers/search-gzipped-csv-or-json.htmlindex 92d792380..b7a465190 100644--- a/landing/answers/search-gzipped-csv-or-json.html+++ b/landing/answers/search-gzipped-csv-or-json.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class="active"><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class="active"><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-html-export.html b/landing/answers/search-html-export.htmlindex 252f3c809..829c25cd1 100644--- a/landing/answers/search-html-export.html+++ b/landing/answers/search-html-export.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class="active"><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class="active"><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-json-and-jsonl-logs.html b/landing/answers/search-json-and-jsonl-logs.htmlindex 0bd8b9f1f..ae1aa6245 100644--- a/landing/answers/search-json-and-jsonl-logs.html+++ b/landing/answers/search-json-and-jsonl-logs.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class="active"><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class="active"><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-jupyter-notebook-cells.html b/landing/answers/search-jupyter-notebook-cells.htmlindex fba44eef2..b604d121b 100644--- a/landing/answers/search-jupyter-notebook-cells.html+++ b/landing/answers/search-jupyter-notebook-cells.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class="active"><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class="active"><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-kubernetes-pod-logs-on-disk.html b/landing/answers/search-kubernetes-pod-logs-on-disk.htmlindex 4439722e6..6102c5723 100644--- a/landing/answers/search-kubernetes-pod-logs-on-disk.html+++ b/landing/answers/search-kubernetes-pod-logs-on-disk.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class="active"><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class="active"><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-obsidian-pdf-docx-attachments.html b/landing/answers/search-obsidian-pdf-docx-attachments.htmlindex ff4279001..2b4e4a677 100644--- a/landing/answers/search-obsidian-pdf-docx-attachments.html+++ b/landing/answers/search-obsidian-pdf-docx-attachments.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class="active"><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class="active"><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-openapi-spec-for-agent.html b/landing/answers/search-openapi-spec-for-agent.htmlindex 35aab97c2..7ab0fcfc9 100644--- a/landing/answers/search-openapi-spec-for-agent.html+++ b/landing/answers/search-openapi-spec-for-agent.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class="active"><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class="active"><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-slack-json-export.html b/landing/answers/search-slack-json-export.htmlindex cad5e83af..79ae5eebf 100644--- a/landing/answers/search-slack-json-export.html+++ b/landing/answers/search-slack-json-export.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class="active"><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class="active"><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-sql-dump-file.html b/landing/answers/search-sql-dump-file.htmlindex 6b9397699..c8677acce 100644--- a/landing/answers/search-sql-dump-file.html+++ b/landing/answers/search-sql-dump-file.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class="active"><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class="active"><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-unity-project-assets.html b/landing/answers/search-unity-project-assets.htmlindex 69e56fc0c..e566aa329 100644--- a/landing/answers/search-unity-project-assets.html+++ b/landing/answers/search-unity-project-assets.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class="active"><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class="active"><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-word-documents-in-a-folder.html b/landing/answers/search-word-documents-in-a-folder.htmlindex e2af38aef..82180688e 100644--- a/landing/answers/search-word-documents-in-a-folder.html+++ b/landing/answers/search-word-documents-in-a-folder.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class="active"><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class="active"><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-yaml-xml-config-repository.html b/landing/answers/search-yaml-xml-config-repository.htmlindex df0b29d60..df100e844 100644--- a/landing/answers/search-yaml-xml-config-repository.html+++ b/landing/answers/search-yaml-xml-config-repository.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class="active"><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class="active"><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/search-zendesk-ticket-json-export.html b/landing/answers/search-zendesk-ticket-json-export.htmlindex f540c86ef..9b7d874de 100644--- a/landing/answers/search-zendesk-ticket-json-export.html+++ b/landing/answers/search-zendesk-ticket-json-export.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class="active"><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class="active"><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/set-mcp-memory-storage-path.html b/landing/answers/set-mcp-memory-storage-path.htmlindex 410a97e02..5c3ea91e1 100644--- a/landing/answers/set-mcp-memory-storage-path.html+++ b/landing/answers/set-mcp-memory-storage-path.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/store-agent-memory-without-llm-call.html b/landing/answers/store-agent-memory-without-llm-call.htmlindex 0efefdae6..9039ee77d 100644--- a/landing/answers/store-agent-memory-without-llm-call.html+++ b/landing/answers/store-agent-memory-without-llm-call.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/syntax-aware-code-search-refactoring.html b/landing/answers/syntax-aware-code-search-refactoring.htmlindex a8bfc59b7..f7ca03b45 100644--- a/landing/answers/syntax-aware-code-search-refactoring.html+++ b/landing/answers/syntax-aware-code-search-refactoring.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class="active"><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class="active"><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/vector-database-vs-full-text-search.html b/landing/answers/vector-database-vs-full-text-search.htmlindex 6deb3d7dc..04259b2a8 100644--- a/landing/answers/vector-database-vs-full-text-search.html+++ b/landing/answers/vector-database-vs-full-text-search.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class="active"><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class="active"><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/what-is-xerj.html b/landing/answers/what-is-xerj.htmlindex 4673fcc54..da61e073b 100644--- a/landing/answers/what-is-xerj.html+++ b/landing/answers/what-is-xerj.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class="active"><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class="active"><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/answers/why-autoindex-skipped-files.html b/landing/answers/why-autoindex-skipped-files.htmlindex 951efe00d..2ca70dbed 100644--- a/landing/answers/why-autoindex-skipped-files.html+++ b/landing/answers/why-autoindex-skipped-files.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class="active"><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class="active"><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/benchmarks/index.html b/landing/benchmarks/index.htmlindex 3ebfedc47..70088a7e0 100644--- a/landing/benchmarks/index.html+++ b/landing/benchmarks/index.html@@ -209,9 +209,9 @@<div class="uc-hero"><div class="uc-eyebrow">BENCHMARKS · WINS AND LOSSES, WITH RECEIPTS</div><h1>EVERY NUMBER HERE<br><span class="accent">TRACES TO A RUN.</span></h1>- <p class="uc-lead">Four things a search engine for AI agents has to prove, each measured, each linked to the+ <p class="uc-lead">Five things a search engine for AI agents has to prove, each measured, each linked to theharness or the case study that produced it: <a href="#performance">engine performance</a>,- <a href="#retrieval">code retrieval</a>, <a href="#tokens">token savings</a>, and the design that ties them+ <a href="#retrieval">code retrieval</a>, <a href="#ranking">ranking quality</a>, <a href="#tokens">token savings</a>, and the design that ties themtogether — the <a href="#zta">zero-token architecture</a>. The red cells are published too, root-caused with<code class="inline">file:line</code>, because a benchmark page that hides its losses is marketing.</p></div>@@ -219,6 +219,7 @@<nav class="uc-subnav" aria-label="Sections"><a href="#performance">PERFORMANCE</a><a href="#retrieval">RETRIEVAL</a>+ <a href="#ranking">RANKING QUALITY</a><a href="#tokens">TOKEN SAVINGS</a><a href="#zta">ZERO-TOKEN ARCHITECTURE</a></nav>@@ -386,9 +387,102 @@<a href="https://github.com/xerj-org/xerj/tree/main/docs/case-studies/reference-coding" rel="external">docs/case-studies/reference-coding</a></p></section>+<!-- ============================ RANKING QUALITY ============================ -->+<section class="scene-section" id="ranking">+ <div class="kicker"><span class="accent">03</span><span class="dash">·</span><span>RANKING QUALITY · PUBLIC DATASETS, NEURAL MODE</span></div>+ <h2 class="scene" style="font-size:var(--fs-56);">FUSE FIRST.<br><span class="accent">JUDGE WHAT'S LEFT.</span></h2>+ <p class="wide">Two questions, both answered on public datasets anyone can download. How much does hybrid+ retrieval add over BM25? And how many routing and spam decisions can labelled history settle with no model+ call at all? <strong>Every vector and hybrid figure in this section was measured with+ <code class="inline">--embed-mode neural</code> and the built-in all-MiniLM-L6-v2 model — not with the default+ embedder, which is lexical feature hashing and has no model in it.</strong> The BM25 rows use no embedder and+ hold on any node.</p>++ <div class="vt-wrap">+ <table class="vt">+ <thead><tr><th>nDCG@10, BEIR <code class="inline">test</code> split</th><th>SciFact · 300 q · 5,183 docs</th><th>NFCorpus · 323 q · 3,633 docs</th><th>Run by</th></tr></thead>+ <tbody>+ <tr><td>XERJ BM25 (<code class="inline">multi_match</code>, no embedder)</td>+ <td class="n">0.6572</td><td class="n">0.3016</td><td>us · any node</td></tr>+ <tr><td>XERJ MiniLM vectors only (<code class="inline">semantic</code>)</td>+ <td class="n">0.6764</td><td class="n">0.3291</td><td>us · <code class="inline">--embed-mode neural</code></td></tr>+ <tr><td>XERJ BM25 top-30, reordered by MiniLM</td>+ <td class="n">0.6855</td><td class="n">0.3323</td><td>us · <code class="inline">--embed-mode neural</code></td></tr>+ <tr><td><strong>XERJ hybrid RRF</strong> (<code class="inline">hybrid</code>, server-side) — the best arm XERJ has, on both</td>+ <td class="n w">0.6993</td><td class="n w">0.3448</td><td>us · <code class="inline">--embed-mode neural</code></td></tr>+ <tr><td>Jev (TypeSafe AI), hosted reranker</td>+ <td class="n">0.768</td><td class="n">0.358</td><td><strong>not run by us</strong> — published in the hev/jev-rerank README</td></tr>+ <tr><td>Voyage rerank-3, hosted reranker</td>+ <td class="n">0.755</td><td class="n">0.357</td><td><strong>not run by us</strong> — same README</td></tr>+ <tr><td>Cohere rerank-v3.5, hosted reranker</td>+ <td class="n">0.745</td><td class="n">0.340</td><td><strong>not run by us</strong> — same README</td></tr>+ </tbody>+ </table>+ </div>++ <ul class="deep">+ <li><strong>The three hosted rows are not a controlled comparison, and carry no verdict.</strong> XERJ did not+ run those systems. They rerank that project's own first-stage shortlist, which is not XERJ's. The dataset+ and the metric match; nothing else is controlled. What the table supports is narrow: on NFCorpus the local+ hybrid figure sits inside the range of the published reranker figures, and <strong>on SciFact it is lower+ than all three</strong>.</li>+ <li><strong>Fusing with a model scored above reranking with it.</strong> Reordering a BM25 shortlist can only+ move documents BM25 already returned; fusion also admits the ones only the vector list found. On NFCorpus+ BM25 returned zero hits for 25 of 323 queries, and no reorder repairs an empty list. That is why XERJ ships+ no local "MiniLM rerank" provider — it would score below the hybrid search that already exists.</li>+ <li><strong>What neural mode costs.</strong> In the same run: about 3–7 documents per second to index on CPU+ (SciFact took about 25 minutes), and about 220–370 ms p50 per hybrid query against about 18 ms for BM25.+ The machine was also compiling, so read the latencies as upper bounds.</li>+ <li><strong>The <code class="inline">rerank</code> search stage exists, and its quality is not verified.</strong>+ XERJ can hand the top hits to an external relevance judge and reorder by the calibrated probability that+ comes back. No provider key was available to this project, so ranking quality with the real model has+ <strong>not</strong> been measured by us; the stage is tested against a test double. It is also the one+ XERJ feature that sends document text off the machine —+ <a href="/answers/rerank-search-results-calibrated-judge">read that before turning it on</a>.</li>+ </ul>++ <div class="vt-wrap">+ <table class="vt">+ <thead><tr><th>Decisions from labelled history · k = 10 · 1,000 held-out items</th><th>Accuracy</th><th>Calibration error</th><th>Decided at confidence ≥ 0.8</th><th>ms / item</th></tr></thead>+ <tbody>+ <tr><td>Banking77, 77-way intent routing — BM25, no model of any kind</td>+ <td class="n">0.819</td><td class="n">0.089</td><td class="n">50.6% at 0.996</td><td class="n">2.2</td></tr>+ <tr><td>Banking77 — MiniLM neighbours (<code class="inline">--embed-mode neural</code>)</td>+ <td class="n w">0.933</td><td class="n w">0.012</td><td class="n w">86.9% at 0.979</td><td class="n">8.6</td></tr>+ <tr><td>Banking77 — hybrid RRF (<code class="inline">--embed-mode neural</code>)</td>+ <td class="n w">0.937</td><td class="n">0.052</td><td class="n">79.2% at 0.989</td><td class="n">12.4</td></tr>+ <tr><td>SMS spam, yes / no — BM25, no model of any kind</td>+ <td class="n w">0.983</td><td class="n">0.017</td><td class="n">95.4% at 0.992</td><td class="n w">0.9</td></tr>+ <tr><td>SMS spam — MiniLM neighbours (<code class="inline">--embed-mode neural</code>)</td>+ <td class="n">0.982</td><td class="n">0.009</td><td class="n">95.8% at 0.989</td><td class="n">74.5</td></tr>+ <tr><td>SMS spam — hybrid RRF (<code class="inline">--embed-mode neural</code>)</td>+ <td class="n w">0.989</td><td class="n">0.015</td><td class="n">95.7% at 0.997</td><td class="n">67.2</td></tr>+ </tbody>+ </table>+ </div>++ <ul class="deep">+ <li><strong>What it means for you.</strong> Where you already have labelled examples, index them and classify+ new text by a rank-weighted vote over its ten nearest neighbours. On Banking77 the neural arm settled 86.9%+ of traffic at confidence 0.8 or higher and was right on 0.979 of it, with no tokens spent; only the rest+ needs a judge model or a person. Spam detection reached 0.983 with plain BM25, under a millisecond per+ message.</li>+ <li><strong>What it is not.</strong> It is not zero-shot: a new category has no history to vote from, and that+ case needs a judge model. XERJ did not run any judge model on these datasets and claims nothing about how+ one would score. Two datasets, one run each.</li>+ </ul>+ <p class="src">Scripts, raw output and reproduce commands:+ <a href="https://github.com/xerj-org/xerj/tree/main/benchmarks/beir-hybrid" rel="external">benchmarks/beir-hybrid</a> ·+ <a href="https://github.com/xerj-org/xerj/tree/main/benchmarks/decisions-as-retrieval" rel="external">benchmarks/decisions-as-retrieval</a> ·+ run 2026-09-18 on xerj v1.0.0-rc.74 · written up in+ <a href="/answers/hybrid-search-quality-measured-ndcg">hybrid search quality</a> and+ <a href="/answers/classify-text-from-labelled-history-without-llm">decisions without a model call</a>.+ Hosted-reranker figures: <a href="https://github.com/hev/jev-rerank" rel="external">hev/jev-rerank</a> (Apache-2.0).</p>+</section>+<!-- ============================ TOKEN SAVINGS ============================ --><section class="scene-section" id="tokens">- <div class="kicker"><span class="accent">03</span><span class="dash">·</span><span>TOKEN SAVINGS · REAL ACCOUNTING, NO API KEY REQUIRED</span></div>+ <div class="kicker"><span class="accent">04</span><span class="dash">·</span><span>TOKEN SAVINGS · REAL ACCOUNTING, NO API KEY REQUIRED</span></div><h2 class="scene" style="font-size:var(--fs-56);">THE TOKENS YOU<br><span class="accent">DON'T SPEND.</span></h2><p class="wide">Output tokens are the expensive kind — priced roughly 5× above input on Claude models — andretry loops on unknown APIs burn exactly those. Three arms of the same agent on the same tasks: memory only,@@ -457,7 +551,7 @@<!-- ============================ ZTA ============================ --><section class="scene-section" id="zta">- <div class="kicker"><span class="accent">04</span><span class="dash">·</span><span>ZTA · THE DESIGN BEHIND THE NUMBERS</span></div>+ <div class="kicker"><span class="accent">05</span><span class="dash">·</span><span>ZTA · THE DESIGN BEHIND THE NUMBERS</span></div><h2 class="scene" style="font-size:var(--fs-56);">ZERO-TOKEN<br><span class="accent">ARCHITECTURE.</span></h2><p class="wide">The cheapest token is the one your model never generates. <strong>ZTA is the design rule thatproduced every number above:</strong> spend compute once, at index time, so agents stop spending inferencediff --git a/landing/compare/index.html b/landing/compare/index.htmlindex 5cdc3f33f..8dff45edf 100644--- a/landing/compare/index.html+++ b/landing/compare/index.html@@ -135,58 +135,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/compare/xerj-vs-ast-grep.html b/landing/compare/xerj-vs-ast-grep.htmlindex 0b16fbb53..68bea507f 100644--- a/landing/compare/xerj-vs-ast-grep.html+++ b/landing/compare/xerj-vs-ast-grep.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/compare/xerj-vs-docfetcher.html b/landing/compare/xerj-vs-docfetcher.htmlindex 595d0d947..312dd08b2 100644--- a/landing/compare/xerj-vs-docfetcher.html+++ b/landing/compare/xerj-vs-docfetcher.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/compare/xerj-vs-elasticsearch-local.html b/landing/compare/xerj-vs-elasticsearch-local.htmlindex ce49b0374..50b505b08 100644--- a/landing/compare/xerj-vs-elasticsearch-local.html+++ b/landing/compare/xerj-vs-elasticsearch-local.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/compare/xerj-vs-houdahspot.html b/landing/compare/xerj-vs-houdahspot.htmlindex d75c6eb8e..6907cb41c 100644--- a/landing/compare/xerj-vs-houdahspot.html+++ b/landing/compare/xerj-vs-houdahspot.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/compare/xerj-vs-khoj.html b/landing/compare/xerj-vs-khoj.htmlindex 5830db27d..8f54dfef7 100644--- a/landing/compare/xerj-vs-khoj.html+++ b/landing/compare/xerj-vs-khoj.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/compare/xerj-vs-localsynapse.html b/landing/compare/xerj-vs-localsynapse.htmlindex 5ade4ffaf..3d7d99d8b 100644--- a/landing/compare/xerj-vs-localsynapse.html+++ b/landing/compare/xerj-vs-localsynapse.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/compare/xerj-vs-meilisearch.html b/landing/compare/xerj-vs-meilisearch.htmlindex 44f59001d..be9b0ab3a 100644--- a/landing/compare/xerj-vs-meilisearch.html+++ b/landing/compare/xerj-vs-meilisearch.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/compare/xerj-vs-obsidian-omnisearch.html b/landing/compare/xerj-vs-obsidian-omnisearch.htmlindex f5290e929..298102ec0 100644--- a/landing/compare/xerj-vs-obsidian-omnisearch.html+++ b/landing/compare/xerj-vs-obsidian-omnisearch.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/compare/xerj-vs-pagefind.html b/landing/compare/xerj-vs-pagefind.htmlindex d5d924eba..5d8e848e1 100644--- a/landing/compare/xerj-vs-pagefind.html+++ b/landing/compare/xerj-vs-pagefind.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/compare/xerj-vs-recoll.html b/landing/compare/xerj-vs-recoll.htmlindex c0e4a8255..70605694c 100644--- a/landing/compare/xerj-vs-recoll.html+++ b/landing/compare/xerj-vs-recoll.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/compare/xerj-vs-ripgrep-all.html b/landing/compare/xerj-vs-ripgrep-all.htmlindex 85514bf5b..04f396b71 100644--- a/landing/compare/xerj-vs-ripgrep-all.html+++ b/landing/compare/xerj-vs-ripgrep-all.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/compare/xerj-vs-ripgrep-for-code-agents.html b/landing/compare/xerj-vs-ripgrep-for-code-agents.htmlindex c6673f993..37280f5be 100644--- a/landing/compare/xerj-vs-ripgrep-for-code-agents.html+++ b/landing/compare/xerj-vs-ripgrep-for-code-agents.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/compare/xerj-vs-spotlight.html b/landing/compare/xerj-vs-spotlight.htmlindex 8045d8faa..33554ef43 100644--- a/landing/compare/xerj-vs-spotlight.html+++ b/landing/compare/xerj-vs-spotlight.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/compare/xerj-vs-typesense.html b/landing/compare/xerj-vs-typesense.htmlindex 4cfd2921a..26f71291d 100644--- a/landing/compare/xerj-vs-typesense.html+++ b/landing/compare/xerj-vs-typesense.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/compare/xerj-vs-vector-database.html b/landing/compare/xerj-vs-vector-database.htmlindex 685d756f8..332980bcc 100644--- a/landing/compare/xerj-vs-vector-database.html+++ b/landing/compare/xerj-vs-vector-database.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/compare/xerj-vs-web-agent-search.html b/landing/compare/xerj-vs-web-agent-search.htmlindex c66f817f3..5378bdb4e 100644--- a/landing/compare/xerj-vs-web-agent-search.html+++ b/landing/compare/xerj-vs-web-agent-search.html@@ -154,58 +154,61 @@<a href="/answers/does-xerj-support-s3-alerting-plugins" class=""><span class="eyebrow">11</span>Does XERJ support S3 storage, alerting or custom plugins yet?</a><a href="/answers/audit-php-codebase-with-an-index" class=""><span class="eyebrow">12</span>How do I audit a PHP codebase with an index instead of grepping 1,400 files?</a><a href="/answers/index-reference-library-for-coding-agent" class=""><span class="eyebrow">13</span>How do I set up reference coding so the agent retrieves a mechanism before writing code?</a>- <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">14</span>How do I reduce embedding API cost?</a>- <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">15</span>I want local search that works offline, no cloud embeddings. What are my options?</a>- <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">16</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>- <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">17</span>How do I find every place a config key is set across YAML and XML in a repo?</a>- <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">18</span>How do I full-text search a SQLite database I just copied onto disk?</a>- <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">19</span>How do I index a CSV with many columns?</a>- <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">20</span>How do I index Markdown into the Elasticsearch API?</a>- <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">21</span>How do I run syntax-aware code search?</a>- <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">22</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>- <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">23</span>How do I search a Confluence export?</a>- <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">24</span>How do I search a Slack JSON export?</a>- <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">25</span>How do I search BVH motion-capture files?</a>- <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">26</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>- <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">27</span>How do I search Kubernetes pod logs?</a>- <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">28</span>How do I search through a folder of contracts in .docx?</a>- <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">29</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>- <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">30</span>How do I stop my agent from reading the whole repo into context?</a>- <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">31</span>How should an agent figure out what's in a messy data folder before searching?</a>- <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">32</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>- <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">33</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>- <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">34</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>- <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">35</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>- <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">36</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>- <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">37</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>- <a href="/answers/search-html-export" class=""><span class="eyebrow">38</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>- <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">39</span>Is it efficient to index PDF files?</a>- <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">40</span>Is there a way to full-text search my browser history file itself?</a>- <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">41</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>- <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">42</span>What's the best way to search markdown notes and PDFs in the same vault?</a>- <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">43</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>- <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">44</span>Why does my Notion export duplicate titles?</a>- <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">45</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>- <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">46</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>- <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">47</span>How do I build RAG without a vector database?</a>- <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">48</span>What is Reciprocal Rank Fusion?</a>- <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">49</span>How does XERJ combine BM25 and kNN?</a>- <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">50</span>How do I search files that live in an S3 bucket?</a>- <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">51</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>- <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">52</span>How do I estimate folder-indexing time?</a>- <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">53</span>Why would a folder search miss files that I can see on disk?</a>- <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">54</span>The indexer exited 3 (or 4). Did it fail?</a>- <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">55</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>- <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">56</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>- <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">57</span>Is there a single-binary local search engine I can just run?</a>- <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">58</span>What can I use for low-volume log search?</a>- <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">59</span>How do I read autoindex progress?</a>- <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">60</span>The indexer died overnight. Do I have to start over?</a>- <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">61</span>Reindexing my files is taking forever. What usually causes that?</a>- <a href="/answers/what-is-xerj" class=""><span class="eyebrow">62</span>What is XERJ?</a>- <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">63</span>How do I improve a basic FAISS RAG pipeline?</a>- <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">64</span>Why did my filtered vector search get slower when I added a filter?</a>- <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">65</span>What's the actual difference between a vector database and full-text search when I have a folder?</a>+ <a href="/answers/classify-text-from-labelled-history-without-llm" class=""><span class="eyebrow">14</span>Can I route tickets or filter spam from labelled history, without an LLM call?</a>+ <a href="/answers/reduce-embedding-api-cost" class=""><span class="eyebrow">15</span>How do I reduce embedding API cost?</a>+ <a href="/answers/local-embeddings-without-openai-api" class=""><span class="eyebrow">16</span>I want local search that works offline, no cloud embeddings. What are my options?</a>+ <a href="/answers/do-search-embeddings-help" class=""><span class="eyebrow">17</span>When should I turn on neural embeddings, and when is the lexical default enough?</a>+ <a href="/answers/search-yaml-xml-config-repository" class=""><span class="eyebrow">18</span>How do I find every place a config key is set across YAML and XML in a repo?</a>+ <a href="/answers/full-text-search-sqlite-database" class=""><span class="eyebrow">19</span>How do I full-text search a SQLite database I just copied onto disk?</a>+ <a href="/answers/index-csv-with-many-columns" class=""><span class="eyebrow">20</span>How do I index a CSV with many columns?</a>+ <a href="/answers/index-markdown-into-elasticsearch-api" class=""><span class="eyebrow">21</span>How do I index Markdown into the Elasticsearch API?</a>+ <a href="/answers/syntax-aware-code-search-refactoring" class=""><span class="eyebrow">22</span>How do I run syntax-aware code search?</a>+ <a href="/answers/search-gzipped-csv-or-json" class=""><span class="eyebrow">23</span>How do I search a .csv.gz or .json.gz without unzipping it first?</a>+ <a href="/answers/search-confluence-html-export" class=""><span class="eyebrow">24</span>How do I search a Confluence export?</a>+ <a href="/answers/search-slack-json-export" class=""><span class="eyebrow">25</span>How do I search a Slack JSON export?</a>+ <a href="/answers/search-bvh-motion-capture-files" class=""><span class="eyebrow">26</span>How do I search BVH motion-capture files?</a>+ <a href="/answers/search-gzip-logs-without-zgrep" class=""><span class="eyebrow">27</span>How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them?</a>+ <a href="/answers/search-kubernetes-pod-logs-on-disk" class=""><span class="eyebrow">28</span>How do I search Kubernetes pod logs?</a>+ <a href="/answers/search-word-documents-in-a-folder" class=""><span class="eyebrow">29</span>How do I search through a folder of contracts in .docx?</a>+ <a href="/answers/search-unity-project-assets" class=""><span class="eyebrow">30</span>How do I search Unity YAML scenes and the C# that references them as one project?</a>+ <a href="/answers/index-monorepo-for-agent" class=""><span class="eyebrow">31</span>How do I stop my agent from reading the whole repo into context?</a>+ <a href="/answers/catalog-files-with-autoindex-map" class=""><span class="eyebrow">32</span>How should an agent figure out what's in a messy data folder before searching?</a>+ <a href="/answers/search-all-pdfs-in-a-folder" class=""><span class="eyebrow">33</span>I downloaded a bunch of papers as PDFs. How do I search across them?</a>+ <a href="/answers/search-sql-dump-file" class=""><span class="eyebrow">34</span>I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres?</a>+ <a href="/answers/index-multiple-csv-files" class=""><span class="eyebrow">35</span>I have a directory of CSV exports. How do I query them without opening each one in Excel?</a>+ <a href="/answers/search-file-contents-in-a-folder" class=""><span class="eyebrow">36</span>I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once?</a>+ <a href="/answers/search-zendesk-ticket-json-export" class=""><span class="eyebrow">37</span>I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search?</a>+ <a href="/answers/search-jupyter-notebook-cells" class=""><span class="eyebrow">38</span>I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them?</a>+ <a href="/answers/search-html-export" class=""><span class="eyebrow">39</span>I saved a help-center as HTML. How do I search it like the real help-center search?</a>+ <a href="/answers/is-pdf-indexing-efficient" class=""><span class="eyebrow">40</span>Is it efficient to index PDF files?</a>+ <a href="/answers/full-text-search-browser-history" class=""><span class="eyebrow">41</span>Is there a way to full-text search my browser history file itself?</a>+ <a href="/answers/search-openapi-spec-for-agent" class=""><span class="eyebrow">42</span>My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way?</a>+ <a href="/answers/search-obsidian-pdf-docx-attachments" class=""><span class="eyebrow">43</span>What's the best way to search markdown notes and PDFs in the same vault?</a>+ <a href="/answers/search-json-and-jsonl-logs" class=""><span class="eyebrow">44</span>What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?</a>+ <a href="/answers/notion-export-duplicate-search-results" class=""><span class="eyebrow">45</span>Why does my Notion export duplicate titles?</a>+ <a href="/answers/search-gmail-takeout-mbox-locally" class=""><span class="eyebrow">46</span>I exported my Gmail with Google Takeout. How do I search that mbox file locally?</a>+ <a href="/answers/how-xerj-autoindexes-a-folder" class=""><span class="eyebrow">47</span>I want to point something at a folder and then ask questions about what's in it. What should I use?</a>+ <a href="/answers/rag-without-vector-database" class=""><span class="eyebrow">48</span>How do I build RAG without a vector database?</a>+ <a href="/answers/hybrid-search-quality-measured-ndcg" class=""><span class="eyebrow">49</span>How much better is hybrid search than BM25? Measured nDCG numbers</a>+ <a href="/answers/rerank-search-results-calibrated-judge" class=""><span class="eyebrow">50</span>How do I rerank search results with a calibrated relevance judge?</a>+ <a href="/answers/reciprocal-rank-fusion-when-to-use" class=""><span class="eyebrow">51</span>What is Reciprocal Rank Fusion?</a>+ <a href="/answers/how-xerj-combines-search" class=""><span class="eyebrow">52</span>How does XERJ combine BM25 and kNN?</a>+ <a href="/answers/search-files-in-an-s3-bucket" class=""><span class="eyebrow">53</span>How do I search files that live in an S3 bucket?</a>+ <a href="/answers/check-codebase-index-is-complete" class=""><span class="eyebrow">54</span>My codebase indexer says indexed but I don't see my code. How do I check it actually finished?</a>+ <a href="/answers/estimate-autoindex-time-before-running" class=""><span class="eyebrow">55</span>How do I estimate folder-indexing time?</a>+ <a href="/answers/why-autoindex-skipped-files" class=""><span class="eyebrow">56</span>Why would a folder search miss files that I can see on disk?</a>+ <a href="/answers/autoindex-exit-codes" class=""><span class="eyebrow">57</span>The indexer exited 3 (or 4). Did it fail?</a>+ <a href="/answers/autoindex-says-extract-the-archive-first" class=""><span class="eyebrow">58</span>xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why?</a>+ <a href="/answers/keep-index-current-as-files-change" class=""><span class="eyebrow">59</span>How do I keep a search index up to date as files change, without re-indexing everything?</a>+ <a href="/answers/search-engine-without-docker" class=""><span class="eyebrow">60</span>Is there a single-binary local search engine I can just run?</a>+ <a href="/answers/cheap-low-volume-log-search" class=""><span class="eyebrow">61</span>What can I use for low-volume log search?</a>+ <a href="/answers/read-autoindex-progress" class=""><span class="eyebrow">62</span>How do I read autoindex progress?</a>+ <a href="/answers/resume-interrupted-autoindex-run" class=""><span class="eyebrow">63</span>The indexer died overnight. Do I have to start over?</a>+ <a href="/answers/reindex-taking-forever" class=""><span class="eyebrow">64</span>Reindexing my files is taking forever. What usually causes that?</a>+ <a href="/answers/what-is-xerj" class=""><span class="eyebrow">65</span>What is XERJ?</a>+ <a href="/answers/improve-basic-faiss-rag-pipeline" class=""><span class="eyebrow">66</span>How do I improve a basic FAISS RAG pipeline?</a>+ <a href="/answers/filter-knn-exact-scan-caveat" class=""><span class="eyebrow">67</span>Why did my filtered vector search get slower when I added a filter?</a>+ <a href="/answers/vector-database-vs-full-text-search" class=""><span class="eyebrow">68</span>What's the actual difference between a vector database and full-text search when I have a folder?</a></div><div class="group"><span class="label">COMPARISONS</span>diff --git a/landing/docs/agents/schemas/anthropic-tools.json b/landing/docs/agents/schemas/anthropic-tools.jsonindex 61b8b29fe..d61446004 100644--- a/landing/docs/agents/schemas/anthropic-tools.json+++ b/landing/docs/agents/schemas/anthropic-tools.json@@ -42,6 +42,11 @@"track_total_hits": {"description": "Whether to compute the exact total hit count. Pass true for an exact count, false for a fast approximate/capped count, or an integer to cap the tracked total. Maps to the ES `track_total_hits` field.","type": ["boolean", "integer"]+ },+ "rerank": {+ "type": "object",+ "description": "Optional second-stage reranking. Maps to the request body's `rerank` field. XERJ sends the text of the top `window` hits plus the question to an external relevance model, which returns a calibrated probability (0..1) per document; hits are reordered by it, that probability REPLACES `_score`, and `min_score` prunes by it. WORTH IT WHEN: a natural-language question over prose where the right document is probably in the top 30 but not at the top, or you need an absolute relevance cut-off rather than a relative rank. NOT WORTH IT FOR: exact identifier lookups, filters, sorted or aggregation-only requests - it adds a paid network round-trip per search. NEEDS A CONFIGURED PROVIDER: the node's operator must have set a provider key (`[rerank] api_key` or TYPESAFE_API_KEY); you cannot supply one in the call. Without it the search fails with HTTP 503 `rerank_exception` - do not retry, repeat the search without `rerank`. HTTP 403 means the operator disabled it. PRIVACY: this is the only search-time feature that sends document text off the node. Proxy embeddings (`[embedding] default_endpoint`) and the WAL tap also send text off the node when an operator configures them, so never tell a user that nothing else leaves the machine. Only fields the response returns are sent, so `source` filtering also limits what leaves. Pass {} for defaults, or: `query` (string: the natural-language question to judge against; REQUIRED when the search `query` is a `bool`, a `knn` or anything else with no single question in it, otherwise read from `match`, `match_phrase`, `multi_match`, `semantic` or `simple_query_string`); `window` (int, default 30, max 300 - every judged document is paid, and `from`+`size` must fit inside it); `min_score` (0..1); `fields` (string[], exhaustive list of returned fields to send); `instructions` (string); `timeout_ms` (int, default 10000); `model` (string). Cannot be combined with `sort`, `search_after`, `collapse`, scroll or `size: 0`. READ THE RESPONSE'S `_rerank` BLOCK: `applied: true` means the order is the judge's and every `_score` is a probability or null (a hit with no verdict, counted in `unjudged`, sorts last); `applied: false` means the provider missed the deadline or answered nothing usable, and the engine's own order and scores were returned, with the cause in `reason`. Ranking quality with the provider's model has not been verified by the XERJ project.",+ "additionalProperties": true}},"required": ["index", "query"]@@ -179,6 +184,11 @@"description": "Maximum number of fused hits to return. Defaults to 10.","minimum": 0,"default": 10+ },+ "rerank": {+ "type": "object",+ "description": "Optional second-stage reranking. Maps to the request body's `rerank` field. XERJ sends the text of the top `window` hits plus the question to an external relevance model, which returns a calibrated probability (0..1) per document; hits are reordered by it, that probability REPLACES `_score`, and `min_score` prunes by it. WORTH IT WHEN: a natural-language question over prose where the right document is probably in the top 30 but not at the top, or you need an absolute relevance cut-off rather than a relative rank. NOT WORTH IT FOR: exact identifier lookups, filters, sorted or aggregation-only requests - it adds a paid network round-trip per search. NEEDS A CONFIGURED PROVIDER: the node's operator must have set a provider key (`[rerank] api_key` or TYPESAFE_API_KEY); you cannot supply one in the call. Without it the search fails with HTTP 503 `rerank_exception` - do not retry, repeat the search without `rerank`. HTTP 403 means the operator disabled it. PRIVACY: this is the only search-time feature that sends document text off the node. Proxy embeddings (`[embedding] default_endpoint`) and the WAL tap also send text off the node when an operator configures them, so never tell a user that nothing else leaves the machine. Only fields the response returns are sent, so `source` filtering also limits what leaves. Pass {} for defaults, or: `query` (string, REQUIRED for this tool: a hybrid search has several sub-queries and no single question); `window` (int, default 30, max 300 - every judged document is paid, and `from`+`size` must fit inside it); `min_score` (0..1); `fields` (string[], exhaustive list of returned fields to send); `instructions` (string); `timeout_ms` (int, default 10000); `model` (string). Cannot be combined with `sort`, `search_after`, `collapse`, scroll or `size: 0`. READ THE RESPONSE'S `_rerank` BLOCK: `applied: true` means the order is the judge's and every `_score` is a probability or null (a hit with no verdict, counted in `unjudged`, sorts last); `applied: false` means the provider missed the deadline or answered nothing usable, and the engine's own order and scores were returned, with the cause in `reason`. Ranking quality with the provider's model has not been verified by the XERJ project.",+ "additionalProperties": true}},"required": ["index", "queries"]diff --git a/landing/docs/agents/schemas/mcp-tools.json b/landing/docs/agents/schemas/mcp-tools.jsonindex 7d810cfdf..9ffbdf745 100644--- a/landing/docs/agents/schemas/mcp-tools.json+++ b/landing/docs/agents/schemas/mcp-tools.json@@ -2,7 +2,7 @@"tools": [{"name": "xerj_search",- "description": "Full-text / keyword / structured search over a XERJ index. Proxies POST /{index}/_search. Pass `query` as a PLAIN STRING for the definition-first code search (ranks the file that defines a symbol above files that merely mention it — the right default for code), or as an ES query-DSL object (e.g. {\"term\":{\"status\":\"open\"}}, or a bool clause) for structured queries. Omit `query` for match_all.",+ "description": "Full-text / keyword / structured search over a XERJ index. Proxies POST /{index}/_search. Pass `query` as a PLAIN STRING for the definition-first code search (ranks the file that defines a symbol above files that merely mention it — the right default for code), or as an ES query-DSL object (e.g. {\"term\":{\"status\":\"open\"}}, or a bool clause) for structured queries. Omit `query` for match_all. Optional `rerank` adds a second stage that re-judges the top hits with an external relevance model — read that argument's description before using it: it needs a provider the operator configured, and it sends document text off the machine.","inputSchema": {"type": "object","properties": {@@ -30,6 +30,13 @@},"_source": {"description": "Source filtering (bool, field, or {includes,excludes})."+ },+ "rerank": {+ "type": [+ "object",+ "boolean"+ ],+ "description": "Optional second-stage reranking. XERJ sends the text of the top `window` hits plus the question to an external relevance model, which returns a calibrated probability (0..1) per document. Hits are reordered by it, that probability REPLACES `_score`, and `min_score` prunes by it — an absolute relevance cut, which a BM25 score cannot give you. WORTH IT WHEN: a natural-language question over prose where the right document is probably in the top 30 but not at the top; or you need a yes/no relevance threshold rather than a relative rank. NOT WORTH IT FOR: exact identifier or symbol lookups (the first stage already ranks the definition first), filters, sorted or aggregation-only requests — it adds a paid network round-trip per search. NEEDS A CONFIGURED PROVIDER: the node's operator must have set a provider key (`[rerank] api_key` or TYPESAFE_API_KEY). You cannot supply one in this call. Without it the search fails with HTTP 503 `rerank_exception`; do not retry, repeat the search without `rerank`. HTTP 403 means the operator disabled it. PRIVACY: this is the only search-time feature that sends document text off the node. Proxy embeddings (`[embedding] default_endpoint`) and the WAL tap also send text off the node when an operator configures them, so never tell a user that nothing else leaves the machine — do not use rerank on data that must stay local unless the user has agreed. Only fields the response returns are sent, so `_source` filtering also limits what leaves. Pass true or {} for defaults, or an object with: `query` (string): the natural-language question to judge documents against. Filled in for you when the tool's `query` is a plain string (that mode also returns the matching passage, which leads the text the judge reads); REQUIRED when the tool's `query` is a `bool`, a `knn`, or anything else with no single question in it (the node refuses rather than guess). `window` (int, default 30, max 300): how many top hits are judged — every one is a paid judgement; `from`+`size` must fit inside it. `min_score` (0..1): drop hits judged below this probability. `fields` (string[]): which returned fields to send; default is every returned string field. `instructions` (string): override what \"relevant\" means for this corpus. `timeout_ms` (int, default 10000). `model` (string). Cannot be combined with `sort`. READ THE RESPONSE'S `_rerank` BLOCK: `applied: true` means the order is the judge's and every `_score` is a probability or null (a hit with no verdict, counted in `unjudged`, sorts last); `applied: false` means the provider missed the deadline or answered nothing usable, and you are looking at the engine's own order with engine scores, stated in `reason`."}},"required": [@@ -112,7 +119,7 @@},{"name": "xerj_hybrid_search",- "description": "Hybrid search: fuse several sub-queries (e.g. a lexical `match` plus a vector `knn`) into one ranked list. Fusion is `rrf` (reciprocal-rank) or `linear` (weighted). Proxies POST /{index}/_search with {\"query\":{\"hybrid\":{\"queries\":[...],\"fusion\":...}}}.",+ "description": "Hybrid search: fuse several sub-queries (e.g. a lexical `match` plus a vector `knn`) into one ranked list. Fusion is `rrf` (reciprocal-rank) or `linear` (weighted). Proxies POST /{index}/_search with {\"query\":{\"hybrid\":{\"queries\":[...],\"fusion\":...}}}. Optional `rerank` re-judges the fused top hits with an external relevance model; here `rerank.query` is REQUIRED, because a hybrid search has several sub-queries and no single question to read.","inputSchema": {"type": "object","properties": {@@ -148,6 +155,13 @@"size": {"type": "integer","description": "Max fused hits to return."+ },+ "rerank": {+ "type": [+ "object",+ "boolean"+ ],+ "description": "Optional second-stage reranking. XERJ sends the text of the top `window` hits plus the question to an external relevance model, which returns a calibrated probability (0..1) per document. Hits are reordered by it, that probability REPLACES `_score`, and `min_score` prunes by it — an absolute relevance cut, which a BM25 score cannot give you. WORTH IT WHEN: a natural-language question over prose where the right document is probably in the top 30 but not at the top; or you need a yes/no relevance threshold rather than a relative rank. NOT WORTH IT FOR: exact identifier or symbol lookups (the first stage already ranks the definition first), filters, sorted or aggregation-only requests — it adds a paid network round-trip per search. NEEDS A CONFIGURED PROVIDER: the node's operator must have set a provider key (`[rerank] api_key` or TYPESAFE_API_KEY). You cannot supply one in this call. Without it the search fails with HTTP 503 `rerank_exception`; do not retry, repeat the search without `rerank`. HTTP 403 means the operator disabled it. PRIVACY: this is the only search-time feature that sends document text off the node. Proxy embeddings (`[embedding] default_endpoint`) and the WAL tap also send text off the node when an operator configures them, so never tell a user that nothing else leaves the machine — do not use rerank on data that must stay local unless the user has agreed. Only fields the response returns are sent, so `_source` filtering also limits what leaves. Pass true or {} for defaults, or an object with: `query` (string, REQUIRED for this tool): the natural-language question to judge documents against. `window` (int, default 30, max 300): how many top hits are judged — every one is a paid judgement; `from`+`size` must fit inside it. `min_score` (0..1): drop hits judged below this probability. `fields` (string[]): which returned fields to send; default is every returned string field. `instructions` (string): override what \"relevant\" means for this corpus. `timeout_ms` (int, default 10000). `model` (string). Cannot be combined with `sort`. READ THE RESPONSE'S `_rerank` BLOCK: `applied: true` means the order is the judge's and every `_score` is a probability or null (a hit with no verdict, counted in `unjudged`, sorts last); `applied: false` means the provider missed the deadline or answered nothing usable, and you are looking at the engine's own order with engine scores, stated in `reason`."}},"required": [diff --git a/landing/docs/agents/schemas/openai-tools.json b/landing/docs/agents/schemas/openai-tools.jsonindex ad147f859..9ba8be932 100644--- a/landing/docs/agents/schemas/openai-tools.json+++ b/landing/docs/agents/schemas/openai-tools.json@@ -42,6 +42,11 @@"track_total_hits": {"type": ["boolean", "integer"],"description": "true for an exact total hit count, false for a fast approximate count, or an integer cap."+ },+ "rerank": {+ "type": "object",+ "description": "Optional second-stage reranking. Maps to the request body's `rerank` field. XERJ sends the text of the top `window` hits plus the question to an external relevance model, which returns a calibrated probability (0..1) per document; hits are reordered by it, that probability REPLACES `_score`, and `min_score` prunes by it. WORTH IT WHEN: a natural-language question over prose where the right document is probably in the top 30 but not at the top, or you need an absolute relevance cut-off rather than a relative rank. NOT WORTH IT FOR: exact identifier lookups, filters, sorted or aggregation-only requests - it adds a paid network round-trip per search. NEEDS A CONFIGURED PROVIDER: the node's operator must have set a provider key (`[rerank] api_key` or TYPESAFE_API_KEY); you cannot supply one in the call. Without it the search fails with HTTP 503 `rerank_exception` - do not retry, repeat the search without `rerank`. HTTP 403 means the operator disabled it. PRIVACY: this is the only search-time feature that sends document text off the node. Proxy embeddings (`[embedding] default_endpoint`) and the WAL tap also send text off the node when an operator configures them, so never tell a user that nothing else leaves the machine. Only fields the response returns are sent, so `source` filtering also limits what leaves. Pass {} for defaults, or: `query` (string: the natural-language question to judge against; REQUIRED when the search `query` is a `bool`, a `knn` or anything else with no single question in it, otherwise read from `match`, `match_phrase`, `multi_match`, `semantic` or `simple_query_string`); `window` (int, default 30, max 300 - every judged document is paid, and `from`+`size` must fit inside it); `min_score` (0..1); `fields` (string[], exhaustive list of returned fields to send); `instructions` (string); `timeout_ms` (int, default 10000); `model` (string). Cannot be combined with `sort`, `search_after`, `collapse`, scroll or `size: 0`. READ THE RESPONSE'S `_rerank` BLOCK: `applied: true` means the order is the judge's and every `_score` is a probability or null (a hit with no verdict, counted in `unjudged`, sorts last); `applied: false` means the provider missed the deadline or answered nothing usable, and the engine's own order and scores were returned, with the cause in `reason`. Ranking quality with the provider's model has not been verified by the XERJ project.",+ "additionalProperties": true}},"required": ["index", "query"]@@ -180,6 +185,11 @@"type": "integer","description": "Maximum number of fused hits to return. Defaults to 10.","minimum": 0+ },+ "rerank": {+ "type": "object",+ "description": "Optional second-stage reranking. Maps to the request body's `rerank` field. XERJ sends the text of the top `window` hits plus the question to an external relevance model, which returns a calibrated probability (0..1) per document; hits are reordered by it, that probability REPLACES `_score`, and `min_score` prunes by it. WORTH IT WHEN: a natural-language question over prose where the right document is probably in the top 30 but not at the top, or you need an absolute relevance cut-off rather than a relative rank. NOT WORTH IT FOR: exact identifier lookups, filters, sorted or aggregation-only requests - it adds a paid network round-trip per search. NEEDS A CONFIGURED PROVIDER: the node's operator must have set a provider key (`[rerank] api_key` or TYPESAFE_API_KEY); you cannot supply one in the call. Without it the search fails with HTTP 503 `rerank_exception` - do not retry, repeat the search without `rerank`. HTTP 403 means the operator disabled it. PRIVACY: this is the only search-time feature that sends document text off the node. Proxy embeddings (`[embedding] default_endpoint`) and the WAL tap also send text off the node when an operator configures them, so never tell a user that nothing else leaves the machine. Only fields the response returns are sent, so `source` filtering also limits what leaves. Pass {} for defaults, or: `query` (string, REQUIRED for this tool: a hybrid search has several sub-queries and no single question); `window` (int, default 30, max 300 - every judged document is paid, and `from`+`size` must fit inside it); `min_score` (0..1); `fields` (string[], exhaustive list of returned fields to send); `instructions` (string); `timeout_ms` (int, default 10000); `model` (string). Cannot be combined with `sort`, `search_after`, `collapse`, scroll or `size: 0`. READ THE RESPONSE'S `_rerank` BLOCK: `applied: true` means the order is the judge's and every `_score` is a probability or null (a hit with no verdict, counted in `unjudged`, sorts last); `applied: false` means the provider missed the deadline or answered nothing usable, and the engine's own order and scores were returned, with the cause in `reason`. Ranking quality with the provider's model has not been verified by the XERJ project.",+ "additionalProperties": true}},"required": ["index", "queries"]diff --git a/landing/docs/api-es-compat.html b/landing/docs/api-es-compat.htmlindex 5751ec72d..fd9ace505 100644--- a/landing/docs/api-es-compat.html+++ b/landing/docs/api-es-compat.html@@ -63,7 +63,7 @@"image": "https://xerj.org/og/xerj-card.png","inLanguage": "en","datePublished": "2026-06-29",- "dateModified": "2026-09-08",+ "dateModified": "2026-09-19","author": {"@id": "https://xerj.org/#organization"},@@ -252,7 +252,7 @@<div class="cell m">POST</div><div class="cell p" id="search">/:index/_search</div>- <div class="cell dsc">ES query DSL. See the <a href="/docs/queries">query types</a> reference.</div>+ <div class="cell dsc">ES query DSL. See the <a href="/docs/queries">query types</a> reference. Accepts one top-level key that is <strong>not</strong> Elasticsearch's: <a href="#rerank"><code class="inline">rerank</code></a>.</div><div class="cell m">POST</div><div class="cell p" id="bulk">/:index/_bulk</div>@@ -321,6 +321,16 @@ $ curl -sX POST http://localhost:9200/logs/_search \-d '{"size":1000,"sort":[{"_id":"asc"}],"search_after":["10897"],"query":{"match_all":{}}}'# → 12 pages, all 11,450 documents, no ceiling</pre>+ <h2 id="rerank">The <code class="inline">rerank</code> stage — a XERJ extension</h2>+ <p><strong>This feature sends your data off the machine.</strong> <code class="inline">rerank</code> is a top-level <code class="inline">_search</code> key that Elasticsearch does not have, and the only search-time feature that sends document text off the node (proxy embeddings and the WAL tap also send text off the node when an operator configures them; both are off by default, and <a href="https://github.com/xerj-org/xerj/blob/main/docs/RERANK.md#every-way-data-leaves-a-xerj-node" rel="external">docs/RERANK.md</a> lists every outbound connection a node can open). It POSTs the question and the text of the top <code class="inline">window</code> hits (default 30, maximum 300) to a third-party relevance judge, and reorders the hits by the 0–1 probability that comes back. That probability replaces <code class="inline">_score</code>, so <code class="inline">rerank.min_score</code> is an absolute cut-off. It does nothing until the node's operator sets a provider key under <a href="/docs/config#rerank-section"><code class="inline">[rerank]</code></a>, a request only triggers it by carrying the block, and only fields the response returns are sent.</p>+<pre class="code">$ curl -sX POST http://localhost:9200/kb/_search \+ -H 'Content-Type: application/json' \+ -d '{"query":{"match":{"body":"vitamin d bone density"}},"size":5,"rerank":{"min_score":0.5}}'+# → "_rerank":{"applied":true,"score_kind":"probability","judged":30,...} ahead of "hits"+# → a provider that misses its deadline is still a 200: "applied":false, engine order+# → no provider key on the node: 503 rerank_exception, no hits, nothing sent</pre>+ <p><code class="inline">hits.total</code> and <code class="inline">aggregations</code> stay the engine's and describe the full match set, not the window. <code class="inline">sort</code>, <code class="inline">search_after</code>, <code class="inline">collapse</code>, scroll and <code class="inline">size: 0</code> are refused with a 400, and <code class="inline">_msearch</code>, search templates, <code class="inline">_async_search</code> and <code class="inline">_rank_eval</code> (per request, under <code class="inline">failures</code>) refuse the block by name rather than dropping it. The strings have server-side ceilings like the window does: <code class="inline">rerank.instructions</code> is limited to 2,000 characters because the provider's wire format repeats it once per judged document, the question to 4,000 and <code class="inline">model</code> to 128; a longer value is a 400 naming the field. Every page request judges the window again, so fetch the window once and page client-side when cost matters. <code class="inline">GET /_xerj/rerank</code> (superuser only) reports whether a provider is configured, never the key. Ranking quality with the provider's model has <strong>not been verified</strong> by this project. Full reference: <a href="https://github.com/xerj-org/xerj/blob/main/docs/RERANK.md" rel="external">docs/RERANK.md</a> · <a href="/answers/rerank-search-results-calibrated-judge">the walkthrough</a>.</p>+<h2>Bulk indexing example</h2><pre class="code">$ curl -sX POST http://localhost:9200/logs/_bulk \-H 'Content-Type: application/x-ndjson' \@@ -331,7 +341,7 @@ $ curl -sX POST http://localhost:9200/logs/_search \{"@timestamp":"2026-04-14T12:00:01Z","service":"auth","level":"warn","message":"rate limited"}JSON</pre>- <p class="source-note">Source · engine/crates/xerj-api/src/es_compat.rs</p>+ <p class="source-note">Source · engine/crates/xerj-api/src/es_compat.rs · engine/crates/xerj-api/src/rerank_stage.rs</p><div class="next">diff --git a/landing/docs/config.html b/landing/docs/config.htmlindex 488eb7724..b6c470998 100644--- a/landing/docs/config.html+++ b/landing/docs/config.html@@ -63,7 +63,7 @@"image": "https://xerj.org/og/xerj-card.png","inLanguage": "en","datePublished": "2026-06-29",- "dateModified": "2026-09-08",+ "dateModified": "2026-09-19","author": {"@id": "https://xerj.org/#organization"},@@ -220,7 +220,7 @@<p>Paths for the config are, in precedence order: <code class="inline">--config /path/to/xerj.toml</code> on the command line, then <code class="inline">/etc/xerj/xerj.toml</code>, then <code class="inline">./xerj.toml</code> in the working directory. If none exist, the full default is used.</p><h2>Full key table</h2>- <div class="cfg"><div class="head">KEY</div><div class="head">TYPE</div><div class="head">DEFAULT</div><div class="head">DESCRIPTION</div><div class="group" id="server">[server]</div><div class="cell k">rest_port</div><div class="cell t">u16</div><div class="cell d">8080</div><div class="cell dsc">Native REST API listener port. All native endpoints (/v1/*) live here.</div><div class="cell k">grpc_port</div><div class="cell t">u16</div><div class="cell d">8081</div><div class="cell dsc">Reserved for a future gRPC API. Not wired in v0.1 — leave at default.</div><div class="cell k">es_compat_port</div><div class="cell t">u16</div><div class="cell d">9200</div><div class="cell dsc">Elasticsearch-compatible wire port. Point Kibana, Logstash, or the ES client here. Set to 0 to disable.</div><div class="cell k">bind_address</div><div class="cell t">string</div><div class="cell d">"127.0.0.1"</div><div class="cell dsc">Interface to bind every listener to. Loopback by default, so an unconfigured node is unreachable from the network. Set "0.0.0.0" or a private address to expose it (also --bind / XERJ_BIND_ADDRESS). Must be an IPv4 or IPv6 literal — host names are not resolved, and a node given one refuses to start.</div><div class="cell k">allow_insecure_network_bind</div><div class="cell t">bool</div><div class="cell d">false</div><div class="cell dsc">Permit a non-loopback bind_address while tls.enabled = false. Startup refuses that combination otherwise: every listener would serve plain HTTP, so the API key in every Authorization header crosses the network in cleartext. Set true only when a proxy, sidecar, mesh or container boundary terminates TLS in front of the node. Env: XERJ_ALLOW_INSECURE_NETWORK_BIND.</div><div class="cell k">data_dir</div><div class="cell t">path</div><div class="cell d">"./data"</div><div class="cell dsc">Root for indices, WAL files, and segments. Needs fast I/O and enough free space.</div><div class="group" id="auth">[auth]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">true</div><div class="cell dsc">Require an API key on every request. Set to false only on a trusted network.</div><div class="cell k">admin_api_key</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Static admin key. Left empty, a 256-bit key is generated on first run and written to <data_dir>/admin.key.</div><div class="group" id="tls">[tls]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">false</div><div class="cell dsc">Terminate TLS at the server. Requires cert_path + key_path. If you terminate at a proxy, leave false.</div><div class="cell k">cert_path</div><div class="cell t">path</div><div class="cell d">""</div><div class="cell dsc">PEM X.509 certificate. Use a CA-signed cert in production.</div><div class="cell k">key_path</div><div class="cell t">path</div><div class="cell d">""</div><div class="cell dsc">PEM private key. Permissions should be 0600.</div><div class="group" id="storage">[storage]</div><div class="cell k">wal_sync</div><div class="cell t">enum</div><div class="cell d">"batched"</div><div class="cell dsc">"sync" · "batched" · "async". Durability vs throughput. Batched is the recommended default.</div><div class="cell k">wal_batch_ms</div><div class="cell t">u32</div><div class="cell d">100</div><div class="cell dsc">Fsync cadence when wal_sync="batched" (ms). Range 1–10000. Lower = smaller loss window.</div><div class="cell k">wal_max_size_mb</div><div class="cell t">u32</div><div class="cell d">1024</div><div class="cell dsc">WAL rollover threshold (MiB). Larger means fewer rollovers, longer crash recovery.</div><div class="cell k">flush_size_mb</div><div class="cell t">u32</div><div class="cell d">512</div><div class="cell dsc">Memtable buffer size that triggers a segment flush (MiB).</div><div class="cell k">flush_interval_secs</div><div class="cell t">u32</div><div class="cell d">30</div><div class="cell dsc">Maximum wall-clock interval between flushes even if the buffer is not full.</div><div class="group" id="merge">[merge]</div><div class="cell k">strategy</div><div class="cell t">enum</div><div class="cell d">"size_tiered"</div><div class="cell dsc">Only "size_tiered" is implemented; "log_structured" is refused at startup rather than silently running the other policy.</div><div class="cell k">min_segments</div><div class="cell t">u32</div><div class="cell d">10</div><div class="cell dsc">Accepted and validated (must be ≥ 2) but <strong>not wired</strong>: the merge trigger is per-tier and comes from min_merge_count. Setting it away from the default logs a warning at startup.</div><div class="cell k">max_segment_mb</div><div class="cell t">u32</div><div class="cell d">8192</div><div class="cell dsc">Upper bound on a mergeable segment (MiB). Segments at or above this size are never merged again.</div><div class="cell k">io_rate_mb_per_sec</div><div class="cell t">u32</div><div class="cell d">100</div><div class="cell dsc">Accepted but <strong>not wired</strong>: merge I/O is not throttled in this build — the rate limiter that honours it sits in an unused storage-crate merge executor. Setting it away from the default logs a warning at startup.</div><div class="cell k">max_concurrent</div><div class="cell t">u8</div><div class="cell d">1</div><div class="cell dsc">Accepted but <strong>not wired</strong>: merge parallelism comes from the XERJ_MERGE_PARALLELISM environment variable, which also defaults to 1. Setting it away from the default logs a warning at startup.</div><div class="group" id="compression">[compression]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">true</div><div class="cell dsc">Accepted but <strong>not wired</strong>: every durable artifact is a compressed envelope with no uncompressed write path, so false changes nothing on disk. Setting it away from the default logs a warning at startup.</div><div class="cell k">level</div><div class="cell t">enum</div><div class="cell d">"balanced"</div><div class="cell dsc">"fast" (Zstd L1), "balanced" (Zstd L3, default), "best" (Zstd L6, cold storage). Applied when segments are re-encoded <strong>at merge</strong>; flush always writes L3, because raising the flush level collapses sustained ingest. Force-merge to apply a change to data already on disk.</div><div class="cell k">block_size_docs</div><div class="cell t">u32</div><div class="cell d">128</div><div class="cell dsc">Accepted but <strong>not wired</strong>: the stored codec is columnar over the whole segment section, not blocked by document count. Range 16–4096 is enforced at startup, and a non-default value logs a warning.</div><div class="group" id="fts">[fts]</div><div class="cell k">default_analyzer</div><div class="cell t">enum</div><div class="cell d">"standard"</div><div class="cell dsc">"standard" · "whitespace" · "simple" · "english". Override per-field in the mapping.</div><div class="group" id="vector">[vector]</div><div class="cell k">default_metric</div><div class="cell t">enum</div><div class="cell d">"cosine"</div><div class="cell dsc">"cosine" · "dot_product" · "euclidean". Cosine for text embeddings is the usual choice.</div><div class="cell k">hnsw_m</div><div class="cell t">u32</div><div class="cell d">16</div><div class="cell dsc">Accepted and validated; the built-in HNSW graph currently builds with fixed M=16 — this key is not yet wired to the build.</div><div class="cell k">hnsw_ef_construction</div><div class="cell t">u32</div><div class="cell d">200</div><div class="cell dsc">Accepted; must be ≥ hnsw_m. The graph currently builds with fixed ef_construction=200 — this key is not yet wired to the build.</div><div class="cell k">hnsw_ef_search</div><div class="cell t">u32</div><div class="cell d">100</div><div class="cell dsc">Accepted but not read by the serving path — the ANN beam width comes from the request's num_candidates (floored at 800).</div><div class="cell k">default_quantization</div><div class="cell t">enum</div><div class="cell d">"none"</div><div class="cell dsc">"none" (the default) or "scalar8" = kNN scored from 1-byte-per-dimension codes, 1–2% recall loss. scalar8 buys precision, <strong>not</strong> RAM: the scan reads the full-precision vector from <code class="inline">_source</code> and quantizes it per query, so resident memory is unchanged and the field also gives up HNSW-served kNN (<a href="https://github.com/xerj-org/xerj/issues/392">#392</a>). "binary" has no quantizer behind it and is refused at startup rather than silently stored at full precision; "scalar4" is not a value this key accepts at all (the 4-bit quantizer in the vector crate is unreachable from config or mapping).</div><div class="cell k">max_dimensions</div><div class="cell t">u32</div><div class="cell d">16384</div><div class="cell dsc">Upper bound on vector dimensionality. 4× the Elasticsearch limit of 4096.</div><div class="group" id="logs">[logs]</div><div class="cell k">retention_days</div><div class="cell t">u32</div><div class="cell d">90</div><div class="cell dsc">Auto-delete log docs older than N days. 0 disables.</div><div class="cell k">time_partition</div><div class="cell t">enum</div><div class="cell d">"1h"</div><div class="cell dsc">"1m" · "5m" · "15m" · "1h" · "6h" · "1d". Time-slice granularity for retention pruning.</div><div class="group" id="embedding">[embedding]</div><div class="cell k">default_endpoint</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">OpenAI-compatible embeddings URL. Empty disables auto-embedding on ingest. Example: "https://api.openai.com/v1/embeddings".</div><div class="cell k">default_model</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Model name passed to the endpoint. Example: "text-embedding-3-small" or "nomic-embed-text".</div><div class="cell k">batch_size</div><div class="cell t">u32</div><div class="cell d">64</div><div class="cell dsc">Docs per embedding API call. Range 1–2048. Bigger batches amortise round-trip cost.</div><div class="cell k">timeout_ms</div><div class="cell t">u32</div><div class="cell d">5000</div><div class="cell dsc">HTTP timeout for embedding calls (ms). Ingest fails with a timeout error if exceeded.</div><div class="group" id="limits">[limits]</div><div class="cell k">max_query_memory_mb</div><div class="cell t">u32</div><div class="cell d">512</div><div class="cell dsc">Per-query memory cap (MiB). Queries that exceed it are cancelled.</div><div class="cell k">max_concurrent_searches</div><div class="cell t">u32</div><div class="cell d">64</div><div class="cell dsc">Global in-flight search ceiling. Extras are queued.</div><div class="cell k">max_fields_per_index</div><div class="cell t">u32</div><div class="cell d">500</div><div class="cell dsc">Field-explosion protection. ES default is 1000; 500 is intentionally stricter.</div><div class="group" id="indexing">[indexing]</div><div class="cell k">turbo_batch_size</div><div class="cell t">u32</div><div class="cell d">1000</div><div class="cell dsc">Docs per batch in turbo mode. Range 500–5000. Larger = higher throughput, slightly higher latency.</div><div class="cell k">turbo_parallel</div><div class="cell t">bool</div><div class="cell d">true</div><div class="cell dsc">Parallel tokenisation on Rayon threads. Disable only for debugging.</div><div class="cell k">turbo_fast_analyzer</div><div class="cell t">bool</div><div class="cell d">false</div><div class="cell dsc">Skip stemming and stop-word removal in turbo mode. Trades recall for speed.</div><div class="group" id="cluster">[cluster]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">false</div><div class="cell dsc">Enable multi-node cluster mode. When true, the Raft state machine and cluster transport are started on port.</div><div class="cell k">port</div><div class="cell t">u16</div><div class="cell d">9300</div><div class="cell dsc">TCP port for intra-cluster Raft and search messages. Must be reachable from every peer.</div><div class="cell k">peers</div><div class="cell t">array</div><div class="cell d">[]</div><div class="cell dsc">Peer list in "node_id=host:port" format. The local node identifies itself from the entry matching bind_address:port. Example: ["a=10.0.0.1:9300","b=10.0.0.2:9300","c=10.0.0.3:9300"].</div><div class="cell k">tick_ms</div><div class="cell t">u64</div><div class="cell d">50</div><div class="cell dsc">Raft tick interval (ms). Lower = faster leader election at the cost of CPU.</div><div class="cell k">auth_secret</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Cluster-wide shared secret authenticating every control frame on port (HMAC-SHA256 over a per-connection challenge). Mandatory when enabled = true — with cluster mode on and no secret here or in XERJ_CLUSTER_AUTH_SECRET the node refuses to start. Same value on every node; minimum 16 chars; generate with `openssl rand -hex 32`. Authenticates, does not encrypt.</div><div class="group" id="wal_tap">[wal_tap]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">false</div><div class="cell dsc">Push a filtered subset of indices to an external ES-compatible target (Elasticsearch, OpenSearch, or another XERJ node) in near-real-time. One-directional and single-node — this is not cross-cluster replication. Off by default.</div><div class="cell k">target_url</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Base URL of the target cluster; the tap POSTs to {target_url}/_bulk. Example: "https://central:9200". Empty disables the tap even with enabled = true. Must not carry credentials in the URL (user:pass@host is refused at startup): target_url is echoed by GET /_xerj/wal_tap, by _stats and in the log. Put the credential in target_auth.</div><div class="cell k">target_auth</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Verbatim Authorization header for the target, e.g. "ApiKey abc123". Write-only: GET /_xerj/wal_tap reports whether one is set, never its value.</div><div class="cell k">indices</div><div class="cell t">array</div><div class="cell d">[]</div><div class="cell dsc">Index allowlist, glob patterns. Empty ships nothing; ["edge-*","metrics"] ships those. A wildcard never expands to a hidden index, and XERJ's own .xerj* system indices are never shipped whatever this says.</div><div class="cell k">poll_interval_ms</div><div class="cell t">u64</div><div class="cell d">500</div><div class="cell dsc">How often each allowlisted index's WAL is polled (ms). Range 50–60000. This is the floor on end-to-end latency; the tap adds nothing to the write path.</div><div class="cell k">max_batch_docs</div><div class="cell t">usize</div><div class="cell d">1000</div><div class="cell dsc">Maximum WAL entries read and shipped per poll, per index. Must be at least 1.</div><div class="cell k">max_batch_bytes</div><div class="cell t">usize</div><div class="cell d">5242880</div><div class="cell dsc">Maximum _bulk body size (bytes). Bounds one HTTP request against the target's http.max_content_length — a larger batch is split across requests, never truncated.</div><div class="cell k">request_timeout_secs</div><div class="cell t">u64</div><div class="cell d">30</div><div class="cell dsc">Per-request timeout against the target. A timed-out batch is retried; the cursor does not advance, so nothing is dropped.</div><div class="cell k">max_retry_backoff_secs</div><div class="cell t">u64</div><div class="cell d">60</div><div class="cell dsc">Ceiling on the exponential, jittered retry backoff when the target is down. Range 1–86400 (one day), enforced at startup as well as by PUT /_xerj/wal_tap.</div><div class="cell k">min_retained_generations</div><div class="cell t">u64</div><div class="cell d">0</div><div class="cell dsc">Rotated WAL generations kept per shard after every entry in them is durable in a segment, so a tap whose target is briefly unreachable still finds them. At the default 0, WAL retention never waits for the target: an outage longer than storage.flush_interval_secs loses entries and reports them as gaps in GET /_xerj/wal_tap/_stats. This is a bounded floor, not a retention lease — the extra cost is at most n × storage.wal_max_size_mb per WAL shard per index, whether or not a tap is running, so a stalled target can never fill this node's disk. Capped at 64 and range-checked at startup, not only by PUT /_xerj/wal_tap — the config file must not be a way around the bound.</div></div>+ <div class="cfg"><div class="head">KEY</div><div class="head">TYPE</div><div class="head">DEFAULT</div><div class="head">DESCRIPTION</div><div class="group" id="server">[server]</div><div class="cell k">rest_port</div><div class="cell t">u16</div><div class="cell d">8080</div><div class="cell dsc">Native REST API listener port. All native endpoints (/v1/*) live here.</div><div class="cell k">grpc_port</div><div class="cell t">u16</div><div class="cell d">8081</div><div class="cell dsc">Reserved for a future gRPC API. Not wired in v0.1 — leave at default.</div><div class="cell k">es_compat_port</div><div class="cell t">u16</div><div class="cell d">9200</div><div class="cell dsc">Elasticsearch-compatible wire port. Point Kibana, Logstash, or the ES client here. Set to 0 to disable.</div><div class="cell k">bind_address</div><div class="cell t">string</div><div class="cell d">"127.0.0.1"</div><div class="cell dsc">Interface to bind every listener to. Loopback by default, so an unconfigured node is unreachable from the network. Set "0.0.0.0" or a private address to expose it (also --bind / XERJ_BIND_ADDRESS). Must be an IPv4 or IPv6 literal — host names are not resolved, and a node given one refuses to start.</div><div class="cell k">allow_insecure_network_bind</div><div class="cell t">bool</div><div class="cell d">false</div><div class="cell dsc">Permit a non-loopback bind_address while tls.enabled = false. Startup refuses that combination otherwise: every listener would serve plain HTTP, so the API key in every Authorization header crosses the network in cleartext. Set true only when a proxy, sidecar, mesh or container boundary terminates TLS in front of the node. Env: XERJ_ALLOW_INSECURE_NETWORK_BIND.</div><div class="cell k">data_dir</div><div class="cell t">path</div><div class="cell d">"./data"</div><div class="cell dsc">Root for indices, WAL files, and segments. Needs fast I/O and enough free space.</div><div class="group" id="auth">[auth]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">true</div><div class="cell dsc">Require an API key on every request. Set to false only on a trusted network.</div><div class="cell k">admin_api_key</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Static admin key. Left empty, a 256-bit key is generated on first run and written to <data_dir>/admin.key.</div><div class="group" id="tls">[tls]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">false</div><div class="cell dsc">Terminate TLS at the server. Requires cert_path + key_path. If you terminate at a proxy, leave false.</div><div class="cell k">cert_path</div><div class="cell t">path</div><div class="cell d">""</div><div class="cell dsc">PEM X.509 certificate. Use a CA-signed cert in production.</div><div class="cell k">key_path</div><div class="cell t">path</div><div class="cell d">""</div><div class="cell dsc">PEM private key. Permissions should be 0600.</div><div class="group" id="storage">[storage]</div><div class="cell k">wal_sync</div><div class="cell t">enum</div><div class="cell d">"batched"</div><div class="cell dsc">"sync" · "batched" · "async". Durability vs throughput. Batched is the recommended default.</div><div class="cell k">wal_batch_ms</div><div class="cell t">u32</div><div class="cell d">100</div><div class="cell dsc">Fsync cadence when wal_sync="batched" (ms). Range 1–10000. Lower = smaller loss window.</div><div class="cell k">wal_max_size_mb</div><div class="cell t">u32</div><div class="cell d">1024</div><div class="cell dsc">WAL rollover threshold (MiB). Larger means fewer rollovers, longer crash recovery.</div><div class="cell k">flush_size_mb</div><div class="cell t">u32</div><div class="cell d">512</div><div class="cell dsc">Memtable buffer size that triggers a segment flush (MiB).</div><div class="cell k">flush_interval_secs</div><div class="cell t">u32</div><div class="cell d">30</div><div class="cell dsc">Maximum wall-clock interval between flushes even if the buffer is not full.</div><div class="group" id="merge">[merge]</div><div class="cell k">strategy</div><div class="cell t">enum</div><div class="cell d">"size_tiered"</div><div class="cell dsc">Only "size_tiered" is implemented; "log_structured" is refused at startup rather than silently running the other policy.</div><div class="cell k">min_segments</div><div class="cell t">u32</div><div class="cell d">10</div><div class="cell dsc">Accepted and validated (must be ≥ 2) but <strong>not wired</strong>: the merge trigger is per-tier and comes from min_merge_count. Setting it away from the default logs a warning at startup.</div><div class="cell k">max_segment_mb</div><div class="cell t">u32</div><div class="cell d">8192</div><div class="cell dsc">Upper bound on a mergeable segment (MiB). Segments at or above this size are never merged again.</div><div class="cell k">io_rate_mb_per_sec</div><div class="cell t">u32</div><div class="cell d">100</div><div class="cell dsc">Accepted but <strong>not wired</strong>: merge I/O is not throttled in this build — the rate limiter that honours it sits in an unused storage-crate merge executor. Setting it away from the default logs a warning at startup.</div><div class="cell k">max_concurrent</div><div class="cell t">u8</div><div class="cell d">1</div><div class="cell dsc">Accepted but <strong>not wired</strong>: merge parallelism comes from the XERJ_MERGE_PARALLELISM environment variable, which also defaults to 1. Setting it away from the default logs a warning at startup.</div><div class="group" id="compression">[compression]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">true</div><div class="cell dsc">Accepted but <strong>not wired</strong>: every durable artifact is a compressed envelope with no uncompressed write path, so false changes nothing on disk. Setting it away from the default logs a warning at startup.</div><div class="cell k">level</div><div class="cell t">enum</div><div class="cell d">"balanced"</div><div class="cell dsc">"fast" (Zstd L1), "balanced" (Zstd L3, default), "best" (Zstd L6, cold storage). Applied when segments are re-encoded <strong>at merge</strong>; flush always writes L3, because raising the flush level collapses sustained ingest. Force-merge to apply a change to data already on disk.</div><div class="cell k">block_size_docs</div><div class="cell t">u32</div><div class="cell d">128</div><div class="cell dsc">Accepted but <strong>not wired</strong>: the stored codec is columnar over the whole segment section, not blocked by document count. Range 16–4096 is enforced at startup, and a non-default value logs a warning.</div><div class="group" id="fts">[fts]</div><div class="cell k">default_analyzer</div><div class="cell t">enum</div><div class="cell d">"standard"</div><div class="cell dsc">"standard" · "whitespace" · "simple" · "english". Override per-field in the mapping.</div><div class="group" id="vector">[vector]</div><div class="cell k">default_metric</div><div class="cell t">enum</div><div class="cell d">"cosine"</div><div class="cell dsc">"cosine" · "dot_product" · "euclidean". Cosine for text embeddings is the usual choice.</div><div class="cell k">hnsw_m</div><div class="cell t">u32</div><div class="cell d">16</div><div class="cell dsc">Accepted and validated; the built-in HNSW graph currently builds with fixed M=16 — this key is not yet wired to the build.</div><div class="cell k">hnsw_ef_construction</div><div class="cell t">u32</div><div class="cell d">200</div><div class="cell dsc">Accepted; must be ≥ hnsw_m. The graph currently builds with fixed ef_construction=200 — this key is not yet wired to the build.</div><div class="cell k">hnsw_ef_search</div><div class="cell t">u32</div><div class="cell d">100</div><div class="cell dsc">Accepted but not read by the serving path — the ANN beam width comes from the request's num_candidates (floored at 800).</div><div class="cell k">default_quantization</div><div class="cell t">enum</div><div class="cell d">"none"</div><div class="cell dsc">"none" (the default) or "scalar8" = kNN scored from 1-byte-per-dimension codes, 1–2% recall loss. scalar8 buys precision, <strong>not</strong> RAM: the scan reads the full-precision vector from <code class="inline">_source</code> and quantizes it per query, so resident memory is unchanged and the field also gives up HNSW-served kNN (<a href="https://github.com/xerj-org/xerj/issues/392">#392</a>). "binary" has no quantizer behind it and is refused at startup rather than silently stored at full precision; "scalar4" is not a value this key accepts at all (the 4-bit quantizer in the vector crate is unreachable from config or mapping).</div><div class="cell k">max_dimensions</div><div class="cell t">u32</div><div class="cell d">16384</div><div class="cell dsc">Upper bound on vector dimensionality. 4× the Elasticsearch limit of 4096.</div><div class="group" id="logs">[logs]</div><div class="cell k">retention_days</div><div class="cell t">u32</div><div class="cell d">90</div><div class="cell dsc">Auto-delete log docs older than N days. 0 disables.</div><div class="cell k">time_partition</div><div class="cell t">enum</div><div class="cell d">"1h"</div><div class="cell dsc">"1m" · "5m" · "15m" · "1h" · "6h" · "1d". Time-slice granularity for retention pruning.</div><div class="group" id="embedding">[embedding]</div><div class="cell k">default_endpoint</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">OpenAI-compatible embeddings URL. Empty disables auto-embedding on ingest. Example: "https://api.openai.com/v1/embeddings".</div><div class="cell k">default_model</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Model name passed to the endpoint. Example: "text-embedding-3-small" or "nomic-embed-text".</div><div class="cell k">batch_size</div><div class="cell t">u32</div><div class="cell d">64</div><div class="cell dsc">Docs per embedding API call. Range 1–2048. Bigger batches amortise round-trip cost.</div><div class="cell k">timeout_ms</div><div class="cell t">u32</div><div class="cell d">5000</div><div class="cell dsc">HTTP timeout for embedding calls (ms). Ingest fails with a timeout error if exceeded.</div><div class="group" id="limits">[limits]</div><div class="cell k">max_query_memory_mb</div><div class="cell t">u32</div><div class="cell d">512</div><div class="cell dsc">Per-query memory cap (MiB). Queries that exceed it are cancelled.</div><div class="cell k">max_concurrent_searches</div><div class="cell t">u32</div><div class="cell d">64</div><div class="cell dsc">Global in-flight search ceiling. Extras are queued.</div><div class="cell k">max_fields_per_index</div><div class="cell t">u32</div><div class="cell d">500</div><div class="cell dsc">Field-explosion protection. ES default is 1000; 500 is intentionally stricter.</div><div class="group" id="indexing">[indexing]</div><div class="cell k">turbo_batch_size</div><div class="cell t">u32</div><div class="cell d">1000</div><div class="cell dsc">Docs per batch in turbo mode. Range 500–5000. Larger = higher throughput, slightly higher latency.</div><div class="cell k">turbo_parallel</div><div class="cell t">bool</div><div class="cell d">true</div><div class="cell dsc">Parallel tokenisation on Rayon threads. Disable only for debugging.</div><div class="cell k">turbo_fast_analyzer</div><div class="cell t">bool</div><div class="cell d">false</div><div class="cell dsc">Skip stemming and stop-word removal in turbo mode. Trades recall for speed.</div><div class="group" id="cluster">[cluster]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">false</div><div class="cell dsc">Enable multi-node cluster mode. When true, the Raft state machine and cluster transport are started on port.</div><div class="cell k">port</div><div class="cell t">u16</div><div class="cell d">9300</div><div class="cell dsc">TCP port for intra-cluster Raft and search messages. Must be reachable from every peer.</div><div class="cell k">peers</div><div class="cell t">array</div><div class="cell d">[]</div><div class="cell dsc">Peer list in "node_id=host:port" format. The local node identifies itself from the entry matching bind_address:port. Example: ["a=10.0.0.1:9300","b=10.0.0.2:9300","c=10.0.0.3:9300"].</div><div class="cell k">tick_ms</div><div class="cell t">u64</div><div class="cell d">50</div><div class="cell dsc">Raft tick interval (ms). Lower = faster leader election at the cost of CPU.</div><div class="cell k">auth_secret</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Cluster-wide shared secret authenticating every control frame on port (HMAC-SHA256 over a per-connection challenge). Mandatory when enabled = true — with cluster mode on and no secret here or in XERJ_CLUSTER_AUTH_SECRET the node refuses to start. Same value on every node; minimum 16 chars; generate with `openssl rand -hex 32`. Authenticates, does not encrypt.</div><div class="group" id="wal_tap">[wal_tap]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">false</div><div class="cell dsc">Push a filtered subset of indices to an external ES-compatible target (Elasticsearch, OpenSearch, or another XERJ node) in near-real-time. One-directional and single-node — this is not cross-cluster replication. Off by default.</div><div class="cell k">target_url</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Base URL of the target cluster; the tap POSTs to {target_url}/_bulk. Example: "https://central:9200". Empty disables the tap even with enabled = true. Must not carry credentials in the URL (user:pass@host is refused at startup): target_url is echoed by GET /_xerj/wal_tap, by _stats and in the log. Put the credential in target_auth.</div><div class="cell k">target_auth</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Verbatim Authorization header for the target, e.g. "ApiKey abc123". Write-only: GET /_xerj/wal_tap reports whether one is set, never its value.</div><div class="cell k">indices</div><div class="cell t">array</div><div class="cell d">[]</div><div class="cell dsc">Index allowlist, glob patterns. Empty ships nothing; ["edge-*","metrics"] ships those. A wildcard never expands to a hidden index, and XERJ's own .xerj* system indices are never shipped whatever this says.</div><div class="cell k">poll_interval_ms</div><div class="cell t">u64</div><div class="cell d">500</div><div class="cell dsc">How often each allowlisted index's WAL is polled (ms). Range 50–60000. This is the floor on end-to-end latency; the tap adds nothing to the write path.</div><div class="cell k">max_batch_docs</div><div class="cell t">usize</div><div class="cell d">1000</div><div class="cell dsc">Maximum WAL entries read and shipped per poll, per index. Must be at least 1.</div><div class="cell k">max_batch_bytes</div><div class="cell t">usize</div><div class="cell d">5242880</div><div class="cell dsc">Maximum _bulk body size (bytes). Bounds one HTTP request against the target's http.max_content_length — a larger batch is split across requests, never truncated.</div><div class="cell k">request_timeout_secs</div><div class="cell t">u64</div><div class="cell d">30</div><div class="cell dsc">Per-request timeout against the target. A timed-out batch is retried; the cursor does not advance, so nothing is dropped.</div><div class="cell k">max_retry_backoff_secs</div><div class="cell t">u64</div><div class="cell d">60</div><div class="cell dsc">Ceiling on the exponential, jittered retry backoff when the target is down. Range 1–86400 (one day), enforced at startup as well as by PUT /_xerj/wal_tap.</div><div class="cell k">min_retained_generations</div><div class="cell t">u64</div><div class="cell d">0</div><div class="cell dsc">Rotated WAL generations kept per shard after every entry in them is durable in a segment, so a tap whose target is briefly unreachable still finds them. At the default 0, WAL retention never waits for the target: an outage longer than storage.flush_interval_secs loses entries and reports them as gaps in GET /_xerj/wal_tap/_stats. This is a bounded floor, not a retention lease — the extra cost is at most n × storage.wal_max_size_mb per WAL shard per index, whether or not a tap is running, so a stalled target can never fill this node's disk. Capped at 64 and range-checked at startup, not only by PUT /_xerj/wal_tap — the config file must not be a way around the bound.</div><div class="group" id="rerank">[rerank]</div><div class="cell k">enabled</div><div class="cell t">bool</div><div class="cell d">true</div><div class="cell dsc">Allow the <code class="inline">rerank</code> search stage. It is the only search-time feature that sends document text off this node, and it does nothing until a key is set. <strong>false</strong> refuses every rerank request with HTTP 403, whatever the environment holds, and nothing is sent.</div><div class="cell k">api_key</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Rerank provider key. Empty falls back to the TYPESAFE_API_KEY environment variable; a value here wins over the environment. Empty in both places means reranking is not configured and a rerank request gets HTTP 503. Write-only: GET /_xerj/rerank reports whether a key is set and where it came from, never its value. There is no per-request key.</div><div class="cell k">endpoint</div><div class="cell t">string</div><div class="cell d">""</div><div class="cell dsc">Where judged document text is POSTed. Empty falls back to TYPESAFE_ENDPOINT, then to https://api.typesafe.ai/v1/systemone. Must be an absolute http:// or https:// URL with no credentials in it (user:pass@host is refused at startup, whether it came from this file or from TYPESAFE_ENDPOINT): the endpoint is echoed by GET /_xerj/rerank.</div></div><h2 id="server-section">[server]</h2>@@ -340,6 +340,14 @@ peers = [ # not a default ([]) — your node roster]tick_ms = 50</pre>+ <h2 id="rerank-section">[rerank]</h2>+ <p><strong>This feature sends your data off the machine.</strong> A search that carries a <code class="inline">rerank</code> block POSTs the question and the text of up to <code class="inline">window</code> hits (default 30, maximum 300) to a third-party relevance judge, and reorders the hits by the probability that comes back. It is the only search-time feature that sends document text off the node. Two other features send text off the node, both operator configuration and off by default: <code class="inline">[embedding] default_endpoint</code> (proxy embeddings) and the WAL tap. The neural model download and cluster Raft traffic carry no document text. Nothing is sent until an operator sets a key, a request only triggers it by asking for it, and <code class="inline">enabled = false</code> forbids it outright. Settings are resolved once at startup; a value in this file wins over the environment. Ranking quality with the provider's model has <strong>not been verified</strong> by this project. Full reference: <a href="https://github.com/xerj-org/xerj/blob/main/docs/RERANK.md" rel="external">docs/RERANK.md</a>.</p>+<pre class="code">[rerank]+enabled = true+api_key = "ts-..." # not a default ("") — or set TYPESAFE_API_KEY+endpoint = "https://api.typesafe.ai/v1/systemone" # not a default ("") — this is where "" resolves to</pre>+ <p>Check it without printing the key: <code class="inline">GET /_xerj/rerank</code> (superuser only) returns <code class="inline">configured</code>, whether a key is set and whether it came from <code class="inline">config</code> or <code class="inline">env</code>, the endpoint with any query string removed, and the server-side limits.</p>+<p class="source-note">Source · engine/xerj.default.toml · engine/crates/xerj-common/src/config.rs</p>diff --git a/landing/docs/env.html b/landing/docs/env.htmlindex 01198231b..09ca9b8e9 100644--- a/landing/docs/env.html+++ b/landing/docs/env.html@@ -63,7 +63,7 @@"image": "https://xerj.org/og/xerj-card.png","inLanguage": "en","datePublished": "2026-06-29",- "dateModified": "2026-09-08",+ "dateModified": "2026-09-18","author": {"@id": "https://xerj.org/#organization"},@@ -215,7 +215,7 @@<h1>Environment variables</h1>- <p>XERJ deliberately does not expose feature flags via environment variables. Behaviour lives in the TOML. The only env vars the engine actually reads are the two needed to find the config file and filter logs.</p>+ <p>XERJ deliberately does not expose feature flags via environment variables. Behaviour lives in the TOML. The variables below find the config file and filter logs; the <a href="#rerank-env">rerank provider pair</a> further down carries a credential, not behaviour; and a few config keys name their own environment override on the <a href="/docs/config">Config TOML</a> page.</p><div class="cfg" style="grid-template-columns: 1fr 1fr 2.5fr;"><div class="head">NAME</div>@@ -237,7 +237,24 @@<p style="margin-top: var(--sp-6);">There are no feature gates via env. If you want to change XERJ's behaviour, change the config.</p>- <p class="source-note">Source · engine/crates/xerj-server/src/main.rs</p>+ <h2 id="rerank-env">Rerank provider</h2>+ <p>The <code class="inline">rerank</code> search stage sends document text to a third-party judge, so it needs a provider key. The key and endpoint may come from the environment as a <strong>fallback</strong>: a non-empty value under <code class="inline">[rerank]</code> in the config file wins, so a stray variable in a service's environment cannot silently re-point where document text is sent. Both are read <strong>once, at startup</strong> — changing them on a running node does nothing. Setting the key does not switch reranking on for any search; a request still has to carry a <code class="inline">rerank</code> block, and <code class="inline">[rerank] enabled = false</code> overrides both variables.</p>++ <div class="cfg" style="grid-template-columns: 1fr 1fr 2.5fr;">+ <div class="head">NAME</div>+ <div class="head">DEFAULT</div>+ <div class="head">DESCRIPTION</div>++ <div class="cell k">TYPESAFE_API_KEY</div>+ <div class="cell d">—</div>+ <div class="cell dsc">Rerank provider key, used when <code class="inline">rerank.api_key</code> is empty. Unset in both places means a rerank request gets HTTP 503. Never echoed: <code class="inline">GET /_xerj/rerank</code> reports only that a key is set and that it came from <code class="inline">env</code>.</div>++ <div class="cell k">TYPESAFE_ENDPOINT</div>+ <div class="cell d">https://api.typesafe.ai/v1/systemone</div>+ <div class="cell dsc">Rerank provider endpoint, used when <code class="inline">rerank.endpoint</code> is empty. Point it at a proxy or a test double; it is where judged document text is POSTed. Validated at boot like the config value: it must be an absolute http:// or https:// URL with no <code class="inline">user:password@</code> in it, or the node refuses to start.</div>+ </div>++ <p class="source-note">Source · engine/crates/xerj-server/src/main.rs · engine/crates/xerj-rerank/src/lib.rs</p><div class="next">diff --git a/landing/docs/metrics.html b/landing/docs/metrics.htmlindex 7959b6171..8bb2014fc 100644--- a/landing/docs/metrics.html+++ b/landing/docs/metrics.html@@ -63,7 +63,7 @@"image": "https://xerj.org/og/xerj-card.png","inLanguage": "en","datePublished": "2026-06-29",- "dateModified": "2026-09-08",+ "dateModified": "2026-09-18","author": {"@id": "https://xerj.org/#organization"},@@ -223,6 +223,9 @@<div class="item">queries_executed<span class="dsc">Total queries served.</span></div><div class="item">bytes_written<span class="dsc">Bytes to segment + WAL.</span></div><div class="item">bytes_read<span class="dsc">Bytes read back from segments.</span></div>+ <div class="item">rerank_requests<span class="dsc">Searches that carried a <code class="inline">rerank</code> block, by <code class="inline">outcome</code>: applied, degraded, refused (nothing sent), failed.</span></div>+ <div class="item">rerank_documents_judged<span class="dsc">Documents the rerank provider returned a verdict for — the billing unit.</span></div>+ <div class="item">rerank_provider_tokens<span class="dsc">Tokens the rerank provider reported, by <code class="inline">kind</code>: input, output.</span></div></div><h2>Histograms</h2>diff --git a/landing/docs/recipes/air-gapped-deployment.html b/landing/docs/recipes/air-gapped-deployment.htmlindex c2a58e3a7..67479ae17 100644--- a/landing/docs/recipes/air-gapped-deployment.html+++ b/landing/docs/recipes/air-gapped-deployment.html@@ -62,7 +62,7 @@"image": "https://xerj.org/og/xerj-card.png","inLanguage": "en","datePublished": "2026-08-16",- "dateModified": "2026-08-30",+ "dateModified": "2026-09-19","author": {"@id": "https://xerj.org/#organization"},@@ -174,6 +174,7 @@<table><thead><tr><th>Component</th><th>Default</th><th>Offline meaning</th></tr></thead><tbody><tr><td>Embedding</td><td><code class="inline">auto</code></td><td>Lexical unless an endpoint is configured; no model/network for lexical mode.</td></tr><tr><td>WAL tap</td><td>Disabled</td><td>A persisted runtime overlay can override TOML on a reused data directory.</td></tr>+ <tr><td>Rerank provider</td><td>Inert: no key is configured</td><td>The only search-time feature that POSTs document text to a third party. The external proxy and WAL tap rows above also send text off the node when enabled; the neural model download and the cluster transport carry no document text; all of them are off by default. It stays inert with no <code class="inline">rerank.api_key</code> and no <code class="inline">TYPESAFE_API_KEY</code> in the service environment, and a search must carry a <code class="inline">rerank</code> block to trigger it. Set <code class="inline">[rerank] enabled = false</code> to refuse it outright, whatever the environment holds; <code class="inline">GET /_xerj/rerank</code> confirms <code class="inline">"enabled": false</code>.</td></tr><tr><td>Cluster</td><td>Disabled</td><td>Single-node startup does not initialize the Raft transport.</td></tr><tr><td>REST / ES-compatible listeners</td><td><code class="inline">127.0.0.1</code></td><td>Loopback-only by default; keep it there unless the production auth/TLS procedure is followed.</td></tr><tr><td>autoindex / MCP</td><td>Localhost defaults</td><td>Local clients; they do not create a public listener or silently upload corpus data.</td></tr>diff --git a/landing/for-agents.html b/landing/for-agents.htmlindex 5a78f0a4c..41eb44bb6 100644--- a/landing/for-agents.html+++ b/landing/for-agents.html@@ -418,7 +418,7 @@ $ curl -sXPOST localhost:9200/docfolder/_search \<ul class="pitch" style="max-width:760px;"><li><a href="/llms.txt"><code class="inline">/llms.txt</code></a> — a plain-text manifest of the base URL, the seven agent operations (autoindex included), and the honesty caveats, written for an LLM to read directly.</li><li><a href="/llms-full.txt"><code class="inline">/llms-full.txt</code></a> — the expanded reference: the <code class="inline">autoindex</code> CLI contract, field types, request and response shapes, and the full endpoint catalogue.</li>- <li><a href="/openapi.json"><code class="inline">/openapi.json</code></a> — an OpenAPI description of the six <em>HTTP</em> agent operations (7 routes), for tool/codegen pipelines — <code class="inline">xerj_autoindex</code>, the seventh, is CLI-only and has no route to describe. It is a deliberately partial spec, not the whole ~210-route REST surface: an endpoint missing from it is "not written up yet", never "not supported".</li>+ <li><a href="/openapi.json"><code class="inline">/openapi.json</code></a> — an OpenAPI description of the six <em>HTTP</em> agent operations (7 routes), for tool/codegen pipelines — <code class="inline">xerj_autoindex</code>, the seventh, is CLI-only and has no route to describe. It also describes one operator route that is not an agent operation: <code class="inline">GET /_xerj/rerank</code>, which says whether the optional <code class="inline">rerank</code> search stage has a provider configured, without printing the key. It is a deliberately partial spec, not the whole ~210-route REST surface: an endpoint missing from it is "not written up yet", never "not supported".</li><li><strong>Tool schemas — paste straight into a tool definition.</strong> The same six HTTP operations, one file per calling convention:<a href="/docs/agents/schemas/anthropic-tools.json"><code class="inline">/docs/agents/schemas/anthropic-tools.json</code></a> (Anthropic <code class="inline">tools</code> array),<a href="/docs/agents/schemas/openai-tools.json"><code class="inline">/docs/agents/schemas/openai-tools.json</code></a> (OpenAI function-calling),diff --git a/landing/llms-full.txt b/landing/llms-full.txtindex 2ac91fee1..57ae4159e 100644--- a/landing/llms-full.txt+++ b/landing/llms-full.txt@@ -56,6 +56,8 @@ CONTENTS — search for a section number (e.g. "3.4", "10.") to jumpfor `_passage`2.h what the fully-exercised AI-core surface is2.i performance is supporting evidence, not the headline+ 2.j `rerank` is the only search-time feature that sends document text off+ the node, and it needs an operator-configured provider3. THE SEVEN CANONICAL AGENT OPERATIONS — full contract for each3.0 xerj_autoindex ....... make a folder searchable (CLI-only, zero config)3.1 xerj_search .......... lexical / structured search, and the code-document@@ -64,6 +66,8 @@ CONTENTS — search for a section number (e.g. "3.4", "10.") to jump3.2 xerj_semantic_search . meaning over a `semantic_text` field3.3 xerj_vector_search ... kNN over `dense_vector`3.4 xerj_hybrid_search ... fuse lexical + vector/semantic+ 3.4b `rerank` ............. optional second stage on 3.1 / 3.4 (an argument,+ NOT an eighth operation)3.5 xerj_memory_store .... write a namespaced memory3.6 xerj_memory_recall ... recall by meaning4. USE CASES & RECIPES ...... reference coding · zero-config folder indexing ·@@ -308,6 +312,25 @@ For a complete disconnected deployment procedure, including current GitHub relearead-under-write); they are published, not hidden. Acked deletes surviveSIGTERM/SIGKILL restarts (11/11 adversarial crash cells). Measure on your workload.+- `rerank` is the only SEARCH-TIME feature that sends document text off the node.+ Two other features send text off the node, both operator configuration and off by+ default: `[embedding] default_endpoint` / `--embed-mode proxy` (document text at+ ingest and query text at search time to an external embeddings API) and the WAL tap+ (`PUT /_xerj/wal_tap`, every write on tapped indices to an external `_bulk`+ endpoint). The node's other outbound connections carry no document or query text:+ the one-time HuggingFace model download for `--embed-mode neural`, and cluster-mode+ Raft messages (index names, mappings) to your own peers. A `_search` body carrying `"rerank": {...}` POSTs the question and the text+ of up to `window` hits (default 30, max 300) to a third-party relevance judge+ (TypeSafe AI's Jev, `POST /v1/systemone`). Three controls: it is inert until the+ node's OPERATOR sets a provider key (`[rerank] api_key`, or `TYPESAFE_API_KEY`; a+ config value wins over the environment; you cannot pass a key in a request); a+ request only triggers it by asking; and `[rerank] enabled = false` refuses every+ rerank request with 403. Do NOT add `rerank` on data that must stay local unless the+ user has agreed. NOT VERIFIED: ranking quality with the real model — no provider key+ was available to this project, and every test of the stage runs against a test+ double that scores by a table the test supplies. Only the mechanism is verified.+ See §3.4b.+=====================================================================3. THE SEVEN CANONICAL AGENT OPERATIONS=====================================================================@@ -567,6 +590,107 @@ POST /{index}/_search- `fusion`: "rrf" (default, k=60) or "linear". Object form: {"type":"rrf","k":60}.- `fusion:"learned"` ERRORS (see §2). `queries` must be non-empty; per-query `weight` defaults 1.0.+--- 3.4b `rerank` — optional second stage (NOT an eighth operation) ---+An argument on xerj_search (3.1) and xerj_hybrid_search (3.4). READ §2.j FIRST: it+sends document text to a third party and needs an operator-configured provider key.+POST /{index}/_search+ { "query": { "match": { "body": "vitamin d bone density" } },+ "size": 5,+ "rerank": { "min_score": 0.5 } } // {} = every default+What it does: the engine ranks as usual; the top `window` hits go to a relevance judge+that answers one yes/no question PER DOCUMENT and returns a calibrated probability+(0..1). Hits are reordered by it, the probability REPLACES `_score` (and+`hits.max_score`), and `rerank.min_score` prunes by it — an absolute relevance cut,+which a BM25/fusion score cannot give you.+ WORTH IT WHEN: a natural-language question over prose where the right document is+ probably in the top 30 but not at the top; or you need a yes/no relevance threshold.+ NOT WORTH IT FOR: exact identifier/symbol lookups, filters, sorted or+ aggregation-only requests — it adds a paid network round-trip per search.+`rerank` fields (unknown keys are REFUSED BY NAME, never ignored):+ provider "jev" (default) | "typesafe" (alias) | "none"/"disabled" (skip, applied:false)+ model default "jev-latest", sent verbatim+ window default 30, 1..300. EVERY judged document is paid. from+size <= window.+ min_score 0..1 probability threshold. NOT the top-level `min_score`, which is the+ engine's, applied to engine scores first; the two compose.+ query the question. Read for you from `match`, `match_phrase`, `multi_match`,+ `semantic`, `simple_query_string`. REQUIRED for `bool`, `hybrid`, a+ top-level `knn` with no text query, or no query — XERJ refuses to guess.+ fields EXHAUSTIVE allow-list of returned fields to send. ["body"] sends the body+ and nothing else, not even the title. Default: title + every other+ returned top-level string field.+ instructions override what "relevant" means for this corpus+ timeout_ms default 10000, max 60000 — budget for the WHOLE stage+ max_doc_chars default 1200, max 8000 (title and text each)+ batch default 30; values above 30 are lowered to 30 (provider ceiling)+ max_concurrency default 8, max 16+Response: a top-level `_rerank` block, emitted BEFORE `hits`. ALWAYS READ `applied`.+ applied:true → { provider, model, score_kind:"probability", window, judged,+ skipped_no_text, pruned_below_min_score, dropped_unjudged,+ partial_failures, usage:{input_tokens,output_tokens}, query, took_ms,+ fields_without_text? } // last one only when a named field+ // returned no text anywhere in the window+ applied:false → { reason, provider, score_kind:"engine", query, took_ms } — the+ order AND scores are the engine's.+ Unjudged hits keep the engine score and sort AFTER every judged hit.+What stays the engine's (describes the FULL match set, not the window):+ `hits.total` / `track_total_hits`, `aggregations`, `profile`, `suggest`. Four docs+ match and two are judged irrelevant → total is 4, pruned_below_min_score is 2.+ `highlight`, `fields`, `inner_hits` travel with their hit. `explain` is kept under a+ new top node whose value equals the new `_score`. Multi-index/wildcard searches work+ (the window is cut from the merged result). Top-level `knn` works; vectors are never+ sent. `rescore` runs first.+FAILURE POLICY — degrade on deadline, surface on contract:+ provider missed timeout_ms ............ 200, engine order, applied:false+ malformed block / refused combination . 400 illegal_argument_exception, nothing sent+ operator disabled it .................. 403 rerank_exception, nothing sent+ no provider key on the node ........... 503 rerank_exception, nothing sent+ provider 401/4xx, malformed, unreachable 502 rerank_exception, NO HITS+ provider 429/529/5xx .................. retried (max 3 attempts/call) inside the budget+ On 403/502/503 do not retry — repeat the search without `rerank`.+REFUSED with 400 (never silently dropped): `sort` (body or ?sort=), `search_after`,+ `collapse`, `?scroll=`, `size:0`, `_source:false` with no `fields` clause; and on+ `_msearch` (per item — the rest of the batch runs), `_search/template`,+ `_msearch/template`, `_async_search`, `_search_scroll`, the native+ `POST /v1/indices/{name}/search` API, and the gRPC Search RPC.+WHAT IS SENT: the question, `instructions`, the model name, and per window hit its title+ + text as the RESPONSE returns them, each cut to max_doc_chars. If you asked for+ `"fields":["_passage"]`, the matching passage LEADS the text (so the clip never cuts+ the part that matched); `"rerank":{"fields":["_passage"]}` sends the passage ONLY —+ the smallest egress for long documents. The `_xerj.hints` corrected request keeps+ your `rerank` block. So `_source`+ filtering is also the egress control: `"_source":["title"]` sends titles only. NEVER+ sent: `_id`s, index names (documents are keyed d0, d1, … by position), numbers,+ booleans, vectors, nested objects, any field the response does not return. The key+ travels only as a bearer token.+COST: 30 documents per provider call (window 35 = 2 calls, window 300 = 10), 8 calls+ in flight by default. Scores from different calls are comparable — each is an+ absolute probability, not a rank within its batch.+Is it configured? GET /_xerj/rerank (superuser only) → { enabled, configured,+ api_key:{set,source:"config"|"env"|null}, endpoint:{url,source}, defaults, limits,+ data_egress }. Never the key. Operator meter: /v1/metrics →+ xerj_rerank_requests_total{outcome}, xerj_rerank_documents_judged_total,+ xerj_rerank_provider_tokens_total{kind}.+Via MCP: optional `rerank` argument on xerj_search and xerj_hybrid_search (`true` or+ {} = defaults). A PLAIN-STRING `query` is passed as `rerank.query` for you;+ xerj_hybrid_search REQUIRES `rerank.query`.+QUALITY — what is and is not measured. NOT VERIFIED: ranking quality with the real Jev+ model (no key was available). Published elsewhere and NOT RUN BY US, reranking a+ different first stage: hev/jev-rerank's README gives nDCG@10 SciFact/NFCorpus of+ 0.768/0.358 (Jev), 0.755/0.357 (Voyage rerank-3), 0.745/0.340 (Cohere rerank-v3.5).+ Measured by us, with `--embed-mode neural` + all-MiniLM-L6-v2 — NOT the default+ lexical embedder: BM25 0.6572/0.3016 · MiniLM vectors 0.6764/0.3291 · BM25 top-30+ reordered by MiniLM 0.6855/0.3323 · hybrid RRF 0.6993/0.3448. Fusing with the model+ scored above reordering with it on both sets, which is why there is no local rerank+ provider. Not a controlled comparison. Raw output + scripts:+ https://github.com/xerj-org/xerj/tree/main/benchmarks/beir-hybrid+ Related — decisions from labelled history with no model call (Banking77 accuracy+ 0.819 BM25 / 0.933 MiniLM / 0.937 hybrid; SMS spam 0.983 / 0.982 / 0.989; the MiniLM+ and hybrid figures need `--embed-mode neural`, the BM25 ones use no embedder):+ https://github.com/xerj-org/xerj/tree/main/benchmarks/decisions-as-retrieval+Credits: one-noul-per-document request shape — hev/jev-rerank (Apache-2.0). Failure+ policy — Meilisearch's personalization module (approach adapted, no code copied).+Full reference: https://github.com/xerj-org/xerj/blob/main/docs/RERANK.md+--- 3.5 xerj_memory_store — write a namespaced memory --------------POST /_memory/{namespace}{ "text": "user prefers metric units",diff --git a/landing/llms.txt b/landing/llms.txtindex 43395c8a9..0980f1e3f 100644--- a/landing/llms.txt+++ b/landing/llms.txt@@ -40,6 +40,7 @@ Then, from an unknown folder to an answered question: `xerj autoindex <folder>`## Caveats (read before you rely on a capability)> - **kNN is HNSW-served (approximate) when unfiltered, exact brute-force otherwise.** An unfiltered `knn` on a full-precision cosine field with ≥1,024 docs runs a beam search over a persisted HNSW graph and exact-rescores its candidates, so `_score`s match the exact path bit-for-bit. `num_candidates` is the beam width (floored at 800; ES's 1.5×k default applies when omitted). On the 50k × 128-d bench corpus: recall@10 1.00 on the official bench query, 100-probe mean 0.976 / min 0.90 (ES 8.13.4, same protocol: 0.937 / 0.70). Recall on the ANN path is **measured, not guaranteed**, and it can return fewer than k hits. Filtered or nested kNN, non-cosine similarity, SQ8-quantized fields, indexes under 1,024 docs, a stale graph, and `semantic` / `_memory` recall all use the exact scan (recall 1.00 by construction; latency scales with vectors scanned).+> - **`rerank` is the only SEARCH-TIME feature that sends document text off the node — and it needs an operator-configured provider.** Adding `"rerank": {…}` to a `_search` body POSTs the question and the text of up to `window` hits (default 30, max 300) to a third-party relevance judge (TypeSafe AI's Jev), which returns a calibrated 0–1 probability per document; that probability replaces `_score` and `rerank.min_score` prunes by it. Two other features send text off the node, both operator configuration and off by default: `[embedding] default_endpoint` / `--embed-mode proxy` (document text at ingest and query text at search time to an external embeddings API) and the WAL tap (`PUT /_xerj/wal_tap`, every write on tapped indices to an external `_bulk` endpoint); a default node contacts neither, and its other outbound connections (the one-time neural model download, cluster Raft to your own peers) carry no document text. `rerank` is opt-in per request, inert until the node's operator sets a key (`[rerank] api_key` or `TYPESAFE_API_KEY` — you cannot pass one in a request), and `[rerank] enabled = false` forbids it. **Do not add `rerank` on data that must stay local unless the user has agreed.** Only fields the response returns are sent, so `_source` filtering also limits what leaves; `rerank.fields` is an exhaustive allow-list. Failure policy is *degrade on deadline, surface on contract*: a slow provider → HTTP 200 with the engine's order and `_rerank.applied:false` (**always read `_rerank.applied`**); no key → `503`, disabled → `403`, rejected key / malformed body / unreachable → `502`, all `rerank_exception` with **no hits** — do not retry, repeat the search without `rerank`. Refused (400), never silently ignored: with `sort`, `search_after`, `collapse`, scroll (open and continuation), `size:0`, and on `_msearch`, search templates, `_async_search`, the native `/v1` search API and gRPC. When applied, every `_score` is a probability or `null` (hits with no verdict, counted in `_rerank.unjudged`, sort last) — never a BM25 value beside probabilities. **Ranking quality with the real model is NOT verified by this project** — no provider key was available; the stage is tested against a test double. Reference: [docs/RERANK.md](https://github.com/xerj-org/xerj/blob/main/docs/RERANK.md).> - **Hybrid fusion is `rrf` or `linear` ONLY.** `fusion:"learned"` is NOT implemented and fails loudly — `400`, `hybrid fusion learned is not yet supported; use rrf or linear`. Do not depend on it.> - **The DEFAULT embedding mode is lexical, not neural.** With no extra flags, semantic search and memory recall use a deterministic feature-hashing embedder (384-d, L2-normalised → cosine): hybrid lexical+vector retrieval — word / sub-word overlap, not deep semantic understanding or synonym-level recall. For real neural semantics start the server with `--embed-mode neural` — a built-in in-process BERT encoder (all-MiniLM-L6-v2) that ships in the binary and auto-downloads its model (~90 MB) on first use — or `--embed-mode proxy` (any external OpenAI-compatible `/v1/embeddings`). Only describe output as neural when one of those is actually running.> * **`--embed-mode` is a server-start flag that applies AT INGEST.** Text is embedded as it is indexed, so the mode in force when a document was written is the mode that document keeps. Switching lexical → neural does **not** re-embed anything already indexed: you have to re-index every corpus you care about. **Decide before you index your first corpus.**@@ -67,7 +68,8 @@ Then, from an unknown folder to an answered question: `xerj autoindex <folder>`- `PUT /{index}/_mapping` · `GET /{index}/_mapping`.- `POST /{index}/_doc` (auto ID) · `PUT`/`GET`/`DELETE`/`HEAD /{index}/_doc/{id}` · `POST /{index}/_update/{id}`.- `POST /_bulk` · `POST /{index}/_bulk` (NDJSON: action line + doc line).-- `POST|GET /{index}/_search` — full query DSL, `semantic`, `knn`, `hybrid`, top-level `knn`, `aggs`, `sort`, `_source`, `highlight` · `POST /_search` (all indices) · `POST /_msearch`.+- `POST|GET /{index}/_search` — full query DSL, `semantic`, `knn`, `hybrid`, top-level `knn`, `aggs`, `sort`, `_source`, `highlight`, and the optional `rerank` second stage (see Caveats: sends text off the machine, needs a provider key) · `POST /_search` (all indices) · `POST /_msearch`.+- `GET /_xerj/rerank` — is a rerank provider configured? Reports that a key is set and whether it came from `config` or `env`, never the key. Superuser only; without superuser rights a `rerank` search tells you the same thing (`503` no key, `403` disabled).- `POST|GET /{index}/_count` · `POST /{index}/_delete_by_query` · `POST /{index}/_update_by_query` · `POST /_mget` · `POST /{index}/_mget`.- `POST /_memory/{namespace}` (also `GET` list, `DELETE` namespace) · `POST /_memory/{namespace}/_recall` · `DELETE /_memory/{namespace}/{id}`.- `POST /autoindex-catalog/_search` — the self-describing catalog written by `xerj autoindex`. CLI equivalents: `xerj autoindex map [--json]`, `xerj autoindex status`.@@ -106,6 +108,8 @@ Direct answers for you, the AI agent: the link, then the question that page answ- [Audit a PHP codebase with an index, not grep](https://xerj.org/answers/audit-php-codebase-with-an-index.md): How do I search a large PHP repo for dangerous sinks without reading every file?- **Coding agents: reference retrieval**- [Index a reference library for a coding agent](https://xerj.org/answers/index-reference-library-for-coding-agent.md): How do I set up reference coding so the agent retrieves a mechanism before writing code?+- **Decisions as retrieval**+ - [Can I route or spam-filter text without an LLM call?](https://xerj.org/answers/classify-text-from-labelled-history-without-llm.md): Can I classify text from labelled history without calling an LLM?- **Embeddings: cost**- [How do I reduce embedding API cost?](https://xerj.org/answers/reduce-embedding-api-cost.md): My embedding bill is too high- **Embeddings: local execution**@@ -146,6 +150,10 @@ Direct answers for you, the AI agent: the link, then the question that page answ- [Search a Gmail Takeout mbox on your own machine](https://xerj.org/answers/search-gmail-takeout-mbox-locally.md): How do I search a Google Takeout mbox file locally?- **Hybrid retrieval: architecture**- [RAG without a vector database](https://xerj.org/answers/rag-without-vector-database.md): How do I build RAG without a vector database?+- **Hybrid retrieval: quality**+ - [How much better is hybrid search than BM25? Measured.](https://xerj.org/answers/hybrid-search-quality-measured-ndcg.md): How much better is hybrid search than BM25 alone?+- **Hybrid retrieval: reranking**+ - [How do I rerank search results with a relevance judge?](https://xerj.org/answers/rerank-search-results-calibrated-judge.md): How do I rerank search results with a calibrated relevance judge?- **Hybrid retrieval: RRF**- [What is Reciprocal Rank Fusion?](https://xerj.org/answers/reciprocal-rank-fusion-when-to-use.md): What is reciprocal rank fusion and when should I use it?- **Hybrid search**diff --git a/landing/openapi.json b/landing/openapi.jsonindex 4fcf1d9a9..35f67cbe2 100644--- a/landing/openapi.json+++ b/landing/openapi.json@@ -193,6 +193,15 @@},"size": 10}+ },+ "rerank": {+ "summary": "Rerank the top hits with an external judge (sends document text off the machine; needs an operator-configured provider key)",+ "value": {+ "query": { "match": { "body": "vpn outage" } },+ "size": 5,+ "_source": ["title", "body"],+ "rerank": { "min_score": 0.5 }+ }}}}@@ -200,14 +209,34 @@},"responses": {"200": {- "description": "Search results.",+ "description": "Search results. With `rerank`, read `_rerank.applied`: a provider that missed its deadline still yields a 200, with the engine's order.","content": {"application/json": {"schema": { "$ref": "#/components/schemas/SearchResponse" }}}},- "400": { "$ref": "#/components/responses/Error" }+ "400": { "$ref": "#/components/responses/Error" },+ "403": { "description": "`rerank` only: the operator disabled reranking (`[rerank] enabled = false`). Nothing was sent. Error type `rerank_exception`.", "content": { "application/json": { "schema": { "type": "object", "additionalProperties": true } } } },+ "502": { "description": "`rerank` only: the provider was called and the contract broke (rejected key, malformed body, unreachable). No hits are returned. Error type `rerank_exception`.", "content": { "application/json": { "schema": { "type": "object", "additionalProperties": true } } } },+ "503": { "description": "`rerank` only: no provider key is configured on this node. Nothing was sent. Repeat the search without `rerank`. Error type `rerank_exception`.", "content": { "application/json": { "schema": { "type": "object", "additionalProperties": true } } } }+ }+ }+ },+ "/_xerj/rerank": {+ "get": {+ "operationId": "rerankStatus",+ "summary": "Is a rerank provider configured?",+ "description": "Reports whether the `rerank` search stage can run on this node: whether a provider key is set and where it came from (`config` or `env`), the endpoint with userinfo/query/fragment removed, and the server-side limits. NEVER returns the key. Superuser only, like the rest of the `/_xerj/*` operator namespace; a caller without superuser rights learns the same thing from a `rerank` search: HTTP 503 means no key, 403 means disabled.",+ "responses": {+ "200": {+ "description": "Rerank provider status.",+ "content": {+ "application/json": {+ "schema": { "$ref": "#/components/schemas/RerankStatus" }+ }+ }+ }}}},@@ -433,10 +462,65 @@"_source": { "description": "Which source fields to return: true/false, or an array of field names.", "type": ["boolean", "array", "string"] },"aggs": { "type": "object", "description": "Aggregations. Not supported together with a hybrid query.", "additionalProperties": true },"highlight": { "type": "object", "description": "Highlight configuration.", "additionalProperties": true },- "track_total_hits": { "description": "true/false, or an integer cap, controlling exact total-hit counting.", "type": ["boolean", "integer"] }+ "track_total_hits": { "description": "true/false, or an integer cap, controlling exact total-hit counting.", "type": ["boolean", "integer"] },+ "rerank": { "$ref": "#/components/schemas/RerankBlock" }},"additionalProperties": true},+ "RerankBlock": {+ "type": "object",+ "description": "XERJ extension: optional second-stage reranking. NOT an Elasticsearch key. PRIVACY: this is the only search-time feature that sends document text off the node - the question and the text of up to `window` hits are POSTed to a third-party relevance judge (proxy embeddings via `[embedding] default_endpoint` and the WAL tap also send text off the node when an operator configures them; docs/RERANK.md lists every outbound connection a node can open). Only fields the response returns are sent, so `_source` filtering also limits what leaves. NEEDS A CONFIGURED PROVIDER: the node's operator must have set a key (`[rerank] api_key` or TYPESAFE_API_KEY); there is no per-request key. The judge returns a calibrated probability (0..1) per document, which REPLACES `_score`; `hits.total` and `aggregations` still describe the full match set. Cannot be combined with `sort`, `search_after`, `collapse`, scroll (open or continuation) or `size: 0`, and is refused (not ignored) on _msearch, search templates, _async_search, the native search API and gRPC; `null` is the same as absent. When applied, every hit's `_score` is a probability or null (no verdict), never an engine score beside probabilities. `{}` uses every default. Unknown keys are refused by name. Ranking quality with the provider's model has not been verified by the XERJ project.",+ "properties": {+ "provider": { "type": "string", "enum": ["jev", "typesafe", "none", "disabled"], "default": "jev", "description": "`none`/`disabled` skips the call and reports `applied: false`." },+ "model": { "type": "string", "maxLength": 128, "default": "jev-latest", "description": "Sent to the provider verbatim." },+ "window": { "type": "integer", "minimum": 1, "maximum": 300, "default": 30, "description": "How many of the engine's top hits are judged. Every one is a paid judgement. `from` + `size` must be <= `window`." },+ "min_score": { "type": "number", "minimum": 0, "maximum": 1, "description": "Drop hits whose relevance probability is below this. Distinct from the top-level `min_score`, which is applied to engine scores first." },+ "query": { "type": "string", "minLength": 1, "maxLength": 4000, "description": "The question documents are judged against (at most 4,000 characters, also when it is inferred). Inferred from `match`, `match_phrase`, `multi_match`, `semantic` and `simple_query_string`; REQUIRED for anything else (`bool`, `hybrid`, a top-level `knn` with no text query)." },+ "fields": { "type": "array", "items": { "type": "string", "maxLength": 256 }, "minItems": 1, "maxItems": 64, "description": "EXHAUSTIVE list of returned fields to send. `[\"body\"]` sends the body and nothing else, not even the title. Default: the title, the matching passage if returned, every other returned top-level string field of `_source`, then string values returned under the hit's `fields` that `_source` did not already return." },+ "instructions": { "type": "string", "maxLength": 2000, "description": "Overrides what 'relevant' means for this corpus. At most 2,000 characters: the provider's wire format repeats it inside every per-document question, so it is sent once per judged document." },+ "timeout_ms": { "type": "integer", "minimum": 1, "maximum": 60000, "default": 10000, "description": "Wall-clock budget for the whole stage. Exceeding it DEGRADES: HTTP 200 with the engine's order and `_rerank.applied: false`." },+ "max_doc_chars": { "type": "integer", "minimum": 1, "maximum": 8000, "default": 1200, "description": "Title and text are each cut to this many characters." },+ "batch": { "type": "integer", "minimum": 1, "default": 30, "description": "Documents per provider call. Values above 30 are lowered to 30." },+ "max_concurrency": { "type": "integer", "minimum": 1, "maximum": 16, "default": 8, "description": "Provider calls in flight at once." }+ },+ "additionalProperties": false+ },+ "RerankInfo": {+ "type": "object",+ "description": "Present only when the request carried `rerank`. Emitted BEFORE `hits` on the wire. Always read `applied`.",+ "properties": {+ "applied": { "type": "boolean", "description": "true: the order is the judge's. false: the provider missed the deadline (or `provider` was `none`) and the order and scores are the engine's; see `reason`." },+ "reason": { "type": "string", "description": "Why reranking was not applied. Only when `applied` is false." },+ "provider": { "type": "string" },+ "model": { "type": "string" },+ "score_kind": { "type": "string", "enum": ["probability", "engine"], "description": "What `_score` means in this response." },+ "window": { "type": "integer", "description": "Hits considered: min(rerank.window, hits the engine returned)." },+ "judged": { "type": "integer", "description": "Documents the provider returned a verdict for - one per document; answers for documents not sent in that call are not counted." },+ "unjudged": { "type": "integer", "description": "Hits in the response with no verdict: `_score` is null and they sort after every judged hit." },+ "skipped_no_text": { "type": "integer", "description": "Window hits with no text in the response. Not sent." },+ "pruned_below_min_score": { "type": "integer" },+ "dropped_unjudged": { "type": "integer", "description": "With `min_score` set, unjudged hits cannot be shown to clear the bar and are dropped." },+ "partial_failures": { "type": "integer", "description": "Provider calls that did not deliver a verdict while others did: failed calls, and calls the deadline stopped from being sent." },+ "fields_without_text": { "type": "array", "items": { "type": "string" }, "description": "Only when non-empty: names in `rerank.fields` that no hit in the window returned text for." },+ "usage": { "type": "object", "properties": { "input_tokens": { "type": "integer" }, "output_tokens": { "type": "integer" } }, "description": "Tokens the provider reported for this search. Advisory: a count the provider reports as anything but a non-negative number meters as 0." },+ "query": { "type": "string", "description": "The question that was judged, inferred or given." },+ "took_ms": { "type": "integer", "description": "Time spent in the stage; the response's `took` includes it." }+ }+ },+ "RerankStatus": {+ "type": "object",+ "description": "Whether a rerank provider is configured. Never contains the key.",+ "properties": {+ "enabled": { "type": "boolean", "description": "false when the operator set `[rerank] enabled = false`." },+ "configured": { "type": "boolean", "description": "enabled AND a key is set." },+ "providers": { "type": "array", "items": { "type": "string" } },+ "api_key": { "type": "object", "properties": { "set": { "type": "boolean" }, "source": { "type": ["string", "null"], "enum": ["config", "env", null] } } },+ "endpoint": { "type": "object", "properties": { "url": { "type": "string", "description": "Userinfo, query string and fragment removed." }, "source": { "type": "string", "enum": ["config", "env", "default"] } } },+ "defaults": { "type": "object", "additionalProperties": true },+ "limits": { "type": "object", "additionalProperties": true, "description": "Server-side ceilings on the caller-chosen cost knobs: `max_docs_per_call`, `max_window`, `max_concurrency`, `max_doc_chars`, `max_timeout_ms`, and the string ceilings `max_instructions_chars`, `max_query_chars`, `max_model_chars`, `max_fields`, `max_field_name_chars`. A value over a ceiling is a 400 naming the field." },+ "data_egress": { "type": "string" }+ }+ },"KnnClause": {"type": "object","description": "ES 8.x top-level kNN clause. Unfiltered kNN is HNSW-served with exact rescoring; filtered kNN runs the exact brute-force scan.",@@ -480,7 +564,8 @@}}},- "aggregations": { "type": "object", "additionalProperties": true }+ "aggregations": { "type": "object", "additionalProperties": true },+ "_rerank": { "$ref": "#/components/schemas/RerankInfo" }}},"BulkResponse": {diff --git a/landing/sitemap.xml b/landing/sitemap.xmlindex 0f053e0a1..8e978ba19 100644--- a/landing/sitemap.xml+++ b/landing/sitemap.xml@@ -44,6 +44,10 @@<loc>https://xerj.org/answers/check-codebase-index-is-complete</loc><lastmod>2026-09-20</lastmod></url>+ <url>+ <loc>https://xerj.org/answers/classify-text-from-labelled-history-without-llm</loc>+ <lastmod>2026-09-20</lastmod>+ </url><url><loc>https://xerj.org/answers/code-search-mcp-for-claude-code</loc><lastmod>2026-09-20</lastmod>@@ -92,6 +96,10 @@<loc>https://xerj.org/answers/how-xerj-combines-search</loc><lastmod>2026-09-20</lastmod></url>+ <url>+ <loc>https://xerj.org/answers/hybrid-search-quality-measured-ndcg</loc>+ <lastmod>2026-09-20</lastmod>+ </url><url><loc>https://xerj.org/answers/improve-basic-faiss-rag-pipeline</loc><lastmod>2026-09-20</lastmod>@@ -164,6 +172,10 @@<loc>https://xerj.org/answers/reindex-taking-forever</loc><lastmod>2026-09-20</lastmod></url>+ <url>+ <loc>https://xerj.org/answers/rerank-search-results-calibrated-judge</loc>+ <lastmod>2026-09-20</lastmod>+ </url><url><loc>https://xerj.org/answers/resume-interrupted-autoindex-run</loc><lastmod>2026-09-20</lastmod>@@ -282,7 +294,7 @@</url><url><loc>https://xerj.org/benchmarks/</loc>- <lastmod>2026-08-30</lastmod>+ <lastmod>2026-09-18</lastmod></url><url><loc>https://xerj.org/benchmarks/elasticsearch</loc>@@ -414,7 +426,7 @@</url><url><loc>https://xerj.org/docs/api-es-compat</loc>- <lastmod>2026-09-08</lastmod>+ <lastmod>2026-09-19</lastmod></url><url><loc>https://xerj.org/docs/api-native</loc>@@ -438,11 +450,11 @@</url><url><loc>https://xerj.org/docs/config</loc>- <lastmod>2026-09-08</lastmod>+ <lastmod>2026-09-19</lastmod></url><url><loc>https://xerj.org/docs/env</loc>- <lastmod>2026-09-08</lastmod>+ <lastmod>2026-09-18</lastmod></url><url><loc>https://xerj.org/docs/ingest</loc>@@ -454,7 +466,7 @@</url><url><loc>https://xerj.org/docs/metrics</loc>- <lastmod>2026-09-08</lastmod>+ <lastmod>2026-09-18</lastmod></url><url><loc>https://xerj.org/docs/migration-from-es</loc>@@ -502,7 +514,7 @@</url><url><loc>https://xerj.org/docs/recipes/air-gapped-deployment</loc>- <lastmod>2026-08-30</lastmod>+ <lastmod>2026-09-19</lastmod></url><url><loc>https://xerj.org/docs/recipes/anomaly-detection</loc>@@ -570,7 +582,7 @@</url><url><loc>https://xerj.org/for-agents</loc>- <lastmod>2026-09-08</lastmod>+ <lastmod>2026-09-18</lastmod></url><url><loc>https://xerj.org/industries/</loc>