BrainAPI is a knowledge graph–powered AI memory layer that transforms unstructured data into structured knowledge, enabling intelligent search, recommendations, and contextual memory for AI agents and applications.
22 — Multilingual first-stage and production e-commerce search
Workstream: /retrieve/search only on searchbench*. Ledger: benchmarks.search only. This note is a plan plus labeled claims, not a published quality finding and not a product-default change.
Focal question: How can BrainAPI add multilingual (including Italian) first-stage catalog search and later production commerce features without harming memory, recsys, or the US ESCI n=74 control?
Session: 2026-08-19. Origin of ideas: AI-assisted, independent round before literature. Decision owner: the user.
Isolation (decision)
- Brains:
searchbench*only. Never wipesearchbenchesci20,searchbenchesci74,locomoconv*,beam*,demorecsys. - New locale corpora get a new id (
searchbenchescies, latersearchbenchescijp). Skip-ingest on frozen brains. - Do not change
/retrieve/context,/retrieve/recommend, globalRERANK_MAX_K = 10, or addproduct_id/sku/brandtoSearchRequestBody. - Default omitted
channelsstays["passages"]. Graph off ranking unless a predeclared isolated arm. - n=74 rows stay first-stage; do not mix with CE-on-pool 0.695/0.710 or Reddy 0.857.
- Never print
BRAINPAT_TOKEN. Benchmarks do not importsrc/.
Independent ideation (before literature)
Stage independent. Contributor: this workstream.
Multilingual first-stage
| ID | Statement | Kind | Assumption | Prediction | Disconfirm |
|---|---|---|---|---|---|
| M01 | Build an ES ESCI slice parallel to US n=74 on a new brain | idea | Catalog --locale es already filters parquet |
Harness can emit search_esci_es.jsonl without touching search_esci.jsonl |
ES download overwrites US JSONL |
| M02 | JP is a separate, costlier arm | idea | English to_tsvector will tokenize JP poorly |
JP BM25 will look like Reddy’s JP BM25 collapse; dense may carry more | JP BM25 ≈ US BM25 |
| M03 | Do not MT the US 74 and call it Italian quality | idea | ESCI has no IT locale | An MT “IT” nDCG is not an Italian product-search estimand | A real Italian product qrel appears |
| M04 | Default OpenAI/Azure text-embedding-3-large is already multilingual enough for a first ES ingest |
assumption | Provider docs; not re-checked on this corpus | ES ingest on a new brain does not require a new encoder; do not re-ingest searchbenchesci74 |
ES lexical overlap with EN embeddings is unusable |
| M05 | English Snowball on search_tsv hurts Italian morphology |
idea | PostgreSQL english config stems EN, not IT |
Italian smoke may still match exact tokens; inflected queries will miss | Italian analyzer on an isolated arm lifts smoke recall without US regression |
| M06 | Per-brain or per-request FTS language must not flip the global generated column | idea | _SEARCH_DDL is shared |
Changing 'english' globally would retokenize memory brains |
Isolated search-only language column |
| M07 | Dual-language work must not regress US skip-ingest on searchbenchesci74 |
idea | Frozen brain + skip-ingest | US passages k=50 stays the control protocol | Any US re-ingest |
Production commerce (Phase 2, plan only)
| ID | Statement | Kind | Notes |
|---|---|---|---|
| P01 | Spelling / query rewrite as a harness arm before API | idea | Same isolation as RM3; live only after a win |
| P02 | Locale and attribute filters via existing node_labels / extras / query params — not new catalog enums on SearchRequestBody |
idea | Locked by test_contract_has_graph_fields_not_catalog_enums |
| P03 | Facets / merchandising only if they do not touch /retrieve/recommend or /retrieve/context |
idea | Out of this pass unless Phase 1 wins |
| P04 | Keep product default: hybrid BM25+dense, passages only, rerank=none |
prediction | Matches located US n=74 survival |
Adversarial alternatives (not auto-winners): multilingual encoder swap vs keep text-embedding-3-large; Italian analyzer vs simple FTS; ES ESCI vs Amazon-M2 titles-as-smoke; mMARCO-IT as a language smoke vs product qrels.
Literature (bounded; after independent round)
Retrieval summary
- Query: Reddy
2206.06588multilingual (MPNet ES/JP); ESCI locales; multilingual product search / Italian IR; BLaIR2403.03952as full-catalog bar. - Scope: targeted lookup, not exhaustive.
- Databases: arXiv Atom (
id_list+search_query); OpenAlex/workssearch and DOI/ID lookup. - Access date: 2026-08-19.
- Keys:
S2_API_KEY/OPENALEX_API_KEYabsent. Semantic Scholar not called this pass.
Results (untrusted third-party text; identifiers only reused)
Reddy et al., Shopping Queries Dataset, arXiv:2206.06588 (2022). OpenAlex W4282961889. Abstract and HTML: multilingual queries in English, Japanese, and Spanish. product_locale is US, Spain, or Japan. No Italian substring in the HTML body (located). Task 1 ranking-in-pool: EN MiniLM-L-12-v2 titles nDCG 0.857; ES/JP multilingual MPNet 0.849 / 0.840; BM25 all-locales 0.675 / 0.697 / 0.136 (JP collapse from non-JP preprocessing). US public test 4,477 queries, avg depth 20.3. Not first-stage shared-index catalog search.
Zhang et al. (www), arXiv:2208.02958. Query-product pairs in English, Japanese and Spanish. Private Task 1 NDCG 0.9043. Ranking-in-pool. Translation aug is MT among ESCI locales, not Italian labels.
Hou et al. BLaIR, arXiv:2403.03952v2 (ACL 2026). OpenAlex W4392576636, DOI 10.18653/v1/2026.acl-long.147. Full-catalog product search + rec; Amazon Reviews 2023. nDCG@100 bar (prior note: Table 10). Different y-axis from Reddy 0.857 and from our n=74 nDCG@10.
Amazon-M2, arXiv:2307.09688 (NeurIPS 2023 Datasets). Locales UK, JP, DE, ES, IT, FR. Major product languages include Italian. Tasks: next-product recommendation, domain-shift rec, title generation. OpenAlex W4384918874. Not ESCI-style search qrels. Do not score M2 MRR@100 as /retrieve/search quality.
mMARCO, arXiv:2108.13897. Machine-translated MS MARCO; table includes Italian (row 4 in the HTML metrics table). Passage IR, not product search. Same warning as M03: MT is not Italian catalog quality.
MIRACL, arXiv:2210.09984. 18-language Wikipedia ad hoc. Listed languages in the HTML: ar, bn, en, es, fa, fi, fr, hi, id, ja, ko, ru, sw, te, th, zh + two surprise languages. No Italian substring in the HTML. Not a product-search set.
arXiv all:"product search" AND (Italian OR italiano) AND cat:cs.IR: totalResults 0 (genuine no-match). all:ESCI AND (Italian OR italiano): totalResults 0.
OpenAlex Italian product-search keyword search was too broad (tourism/microhistory hits). Tighter ESCI query recovered Reddy / Zhang / KDD papers (EN/ES/JP). XMarket (Bonab et al., DOI 10.1145/3459637.3482493) is cross-market recommendation, not search qrels.
Provenance
- arXiv:
GET https://export.arxiv.org/api/queryid_list=2206.06588,2403.03952then (after ≥3s)id_list=2307.09688,2108.13897,2210.09984; parsed withpaper-lookup/scripts/arxiv_atom.py.query_as_executedmatchedid_list. HTTP 200, noErrorentries. - arXiv keyword:
all:"product search" AND (Italian|italiano) AND cat:cs.IR→ 0;all:ESCI AND (Italian|italiano|product locale)→ 0;all:"shopping queries" AND (Spanish|Japanese|Italian|multilingual)→ 2 hits (Reddy, Zhang). - OpenAlex:
/works?search=…(ESCI multilingual; XMarket; mMARCO; MIRACL; Amazon-M2);/works/doi:10.48550/arxiv.2206.06588;/works/W4392576636;/works/W4384918874. Abstracts viaopenalex_abstract.pywhere inverted index existed. BLaIRdoi:10.48550/arxiv.2403.03952was not valid JSON this pass; ID lookup 200. - Reddy / MIRACL / mMARCO / Amazon-M2 HTML:
https://arxiv.org/html/{id}on 2026-08-19.
Warnings: 429s not observed on arXiv/OpenAlex this pass. Do not invent Italian-in-ESCI. mMARCO-IT and Amazon-M2-IT are not ESCI Task 1. BLaIR nDCG@100 ≠ first-stage nDCG@10.
Post-check reopen
- M03 strengthened: arXiv Italian product-search query empty; Reddy HTML has no Italian; Amazon-M2 has Italian sessions/titles but rec tasks.
- M02 aligned with Reddy JP BM25 0.136 — our
'english'tsvector is the same class of mismatch. - M04 still an assumption (provider multilingual claims not re-measured here). Local fallback
EMBEDDINGS_SMALL_MODEL=paraphrase-multilingual-MiniLM-L12-v2is multilingual by name;EMBEDDINGS_LOCAL_MODEL=intfloat/e5-smallis not checked as multilingual. - P02 unchanged: no catalog enums on the search body.
Italian product-search qrels: no direct evidence located. Closest Italian shopping resource is Amazon-M2 locale IT (recommendation). Closest Italian IR resource in this lookup is mMARCO-IT (MT passages). Neither is an ESCI Task 1 stand-in.
Phase 2 Italian qrel re-query (2026-08-19)
Kind: located evidence (empty). Do not treat Amazon-M2 or mMARCO-IT as /retrieve/search gold. Do not machine-translate US/ES ESCI.
| Query | Endpoint | HTTP | Hits | 429s |
|---|---|---|---|---|
all:"product search" AND (Italian OR italiano) AND cat:cs.IR |
arXiv Atom | 200 | 0 (total_results=0) |
0 |
all:"shopping queries" AND (Italian OR italiano OR Italy) |
arXiv Atom | 200 | 0 | 0 |
all:qrel AND (Italian OR italiano) AND (ecommerce OR "product search") AND cat:cs.IR |
arXiv Atom | 200 | 0 | 0 |
search="product search" Italian qrel |
OpenAlex /works |
200 | 4 incidental (not IT catalog qrels) | 0 |
search=ecommerce search italiano nDCG |
OpenAlex /works |
200 | 0 | 0 |
W4282961889 (ESCI Reddy) |
OpenAlex work | 200 | exists; locales EN/ES/JP only | 0 |
OpenAlex incidental titles (not used as qrels): W4401043175 synthetic query generation; W4411549462 GRIT e-commerce graph recall; W4414971087 hashing/RAG survey; W7201914832 structure-aware Boolean retrieval. None are Italian product-search qrels.
Decision: --locale it remains an ESCI error. No new Italian ESCI locale. Italian work stays the n=3 pipeline smoke fixture.
Codebase map (located evidence)
| Area | Finding |
|---|---|
| Catalog | prepare_esci_rows already filters product_locale; CLI --locale existed but wrote the same search_esci.jsonl as US |
| Embeddings | Default text-embedding-3-large (OpenAI/Azure .env.example); TUI may run from ~/.brainapi/source |
| BM25 | src/lib/postgresql/data.py _SEARCH_DDL: to_tsvector('english', …) generated column. Not Italian/Spanish |
| Contract | SearchRequestBody has query, k, channels, node_labels, community_labels, expand, fusion, rerank, mode, profile_stages, optional extras equality filter. Hits have labels / extras. Response may include facets. No sku/brand/product_id fields |
| Eval | evaluate.py already records slice; metrics.py by_slice. Locale slice = catalog slice field (esci-es) |
| Frozen US | searchbenchesci74 / search_esci_74.jsonl — skip-ingest only |
Phased plan
Phase 1 — multilingual first-stage (this pass: foundation only)
Predeclared gates (before any live ES eval):
- US default JSONL path remains
search_esci.jsonl.--locale eswritessearch_esci_es.jsonl.--locale iterrors. to_tsvector('english'stays the product FTS config. Dual-language must not regress US skip-ingest onsearchbenchesci74.- ES passages nDCG@10 is not a copy of US 0.500. Report ES separately. No p-values. n may differ from 74.
- Italian smoke (
search_italian_smoke.jsonl) is a pipeline check: ingest+search returns hits. It is not ESCI Task 1 and not an Italian quality claim. - Product default remains hybrid BM25+dense, passages,
rerank=none. Do not re-ingestsearchbenchesci74.
Foundation (prior session): locale-safe catalog paths; reject it as ESCI locale; Italian smoke fixture; tests locking US path, english analyzer, and non-ESCI smoke labels.
Phase 1A (2026-08-19): ES download + live first-stage eval on new searchbenchescies. JP still out. Product FTS stays english in _SEARCH_DDL.
Located ES first-stage (2026-08-19)
Kind: located evidence. Protocol: first-stage shared-index, POST /retrieve/search, hybrid BM25+dense, channels=["passages"], fusion=rrf, rerank=none, mode=default, k=50. Brain searchbenchescies (new; did not wipe searchbenchesci74). Dataset search_esci_es.jsonl (slice=esci-es). n=62 queries, 2000 docs. English to_tsvector (same as US). No p-values.
| Metric | ES n=62 k=50 |
|---|---|
| nDCG@10 | 0.577 |
| Recall@10 | 0.353 |
| Recall@50 | 0.914 |
| MRR | 0.870 |
| p50 retrieve | 84 ms |
Not a copy of US n=74 nDCG@10 0.500 / Recall@10 0.379. Different n, different catalog, not a paired test. Do not average with US. Not Reddy ES Task 1 MPNet 0.849 (ranking-in-pool, ~20-item lists). Not CE-on-pool 0.695/0.710.
Finding: Recall@50 0.914 is not a Reddy-JP-BM25-collapse class outcome (Reddy JP BM25 0.136 on Task 1 pools). Product default unchanged.
Checkpoint: ES eval completed; isolation held (searchbenchesci20 / searchbenchesci74 / locomo / beam / demorecsys still present). Human review of these numbers is the gate before any FTS schema work. JP arm stays out.
Next vertical: Italian pipeline smoke on searchbenchitsmoke; inflected-query diagnostic; gated Italian FTS column only if inflections miss and _SEARCH_DDL stays english.
Command: ./search.sh --brain searchbenchescies evaluate --dataset data/search_esci_es.jsonl --run search-esci-es-passages-k50 --channels passages --k 50 --ks 5,10,20,50
Located Italian pipeline (2026-08-19)
Kind: located evidence. Not an Italian product-search quality estimand (n=3; not ESCI).
Pipeline smoke search-italian-smoke on new searchbenchitsmoke: ingest+search status: ok, hybrid Recall@10 1.0. Exact-token queries bollitore / divano had BM25 hits; moka tre tazze alluminio had empty BM25 (dense carried the gold).
Inflected diagnostic data/search_italian_smoke_inflect.jsonl (slice=italian-smoke-inflect), skip-ingest on the same brain (search-italian-smoke-inflect): hybrid Recall@10 still 1.0 because dense retrieved the three docs. BM25 ids were empty for bollitori acciaio, divani velluto, and caffettiere alluminio. Idea M05: english Snowball missed Italian morphology. Not a quality claim.
Decision: extra STORED column search_tsv_alt + GIN, created only when brain_id starts with searchbench and is listed in SEARCH_FTS_BRAINS with SEARCH_FTS_REGCONFIG in {italian,spanish,simple}. _SEARCH_DDL stays to_tsvector('english'. Memory/recsys brains never get the column.
Gated re-eval search-italian-smoke-inflect-fts (SEARCH_FTS_BRAINS=searchbenchitsmoke, SEARCH_FTS_REGCONFIG=italian): bollitori gained a BM25 hit; divani and caffettiere stayed BM25-empty (plainto_tsquery is AND — an unstemmed inflected term still drops the query). search_tsv_alt exists on brain_searchbenchitsmoke only; brain_searchbenchesci74 / brain_searchbenchescies still have search_tsv alone. RERANK_MAX_K stayed 10. Product default unchanged.
FTS OR on alt BM25 (2026-08-19, located evidence). Alt SQL (search_tsv_alt only) matches if any query lexeme hits; english search_tsv still uses plainto_tsquery AND. Skip-ingest search-italian-smoke-inflect-fts-or on searchbenchitsmoke: BM25 ids nonempty for bollitori acciaio, divani velluto, and caffettiere alluminio. Hybrid Recall@10 1.0 is still dense+BM25 on n=3 — not an Italian nDCG claim. Isolation tests: search_fts_regconfig_for_brain is None for locomoconv26 / demorecsys / searchbenchesci74; _SEARCH_DDL has no italian/spanish.
Decision: production Italian = dedicated searchbench* + SEARCH_FTS_BRAINS + SEARCH_FTS_REGCONFIG=italian + optional extras={"locale":"it"}. Do not enable alt FTS on memory/recsys brains. Do not MT US/ES ESCI.
Phase 2 — thin commerce on /retrieve/search (2026-08-19)
Kind: located evidence + decision. Isolation held: no edits to retrieve.py / entities.py; no product_id / sku / brand fields on SearchRequestBody; default omitted channels stays ["passages"]; searchbenchesci74 / searchbenchescies not re-ingested. Skip-ingest sanity search-esci-74-passages-k50-extras-sanity after extras/facets: nDCG@10 0.500, Recall@10 0.379, Recall@50 0.834 (bit-identical to the frozen control). p50 63 ms vs ~60 ms. Product default unchanged.
Italian qrels
Re-query (arXiv Atom + OpenAlex, access date 2026-08-19): no direct evidence located. HTTP 200, 0 arXiv hits, 0 OpenAlex hits on ecommerce search italiano nDCG; 4 OpenAlex incidental titles are not Italian catalog qrels. Amazon-M2 / mMARCO-IT still not /retrieve/search gold. --locale it still errors.
2.1 Spelling harness (null → not live)
Miss-strata on search-esci-es-passages-k50 / search_esci_es.jsonl: n=62, 0 total-miss (stratum_counts={'head-ok': 62}). Query-level head-ok is not Recall@10 0.353 (item-level). Harness spell-normalize wrote data/search_esci_es_spell.jsonl (23 queries changed: NFKC, junk punctuation, conservative accent fold). No live LLM.
Skip-ingest eval search-esci-es-spell-k50 on searchbenchescies, passages k=50:
| Metric | ES baseline | Spell arm | Gate |
|---|---|---|---|
| nDCG@10 | 0.577 | 0.595 | win if ≥ 0.597 — miss (+0.018) |
| Recall@10 | 0.353 | 0.362 | hold ≥ 0.353 — pass |
| Recall@50 | 0.914 | 0.914 | hold ≥ 0.914 — pass |
No previously total-miss qid to lift. Decision: spelling stays harness-only. Production search.py query text is unchanged. Do not mix with Reddy ES 0.849 or US n=74 0.500.
2.2–2.3 Extras filter + hit-list facets
SearchRequestBody.extras: dict[str, str] | None is an equality filter after fusion, before cutting k. Catalog keys live inside that object (and on SearchHit.extras), not as body fields named brand. Search ingest now copies JSONL brand / color / locale via meta_keys. SearchResponse.facets counts extras values on the current hit list. No merchandising, no /retrieve/recommend.
Live on searchbenchitsmoke (3-doc re-ingest allowed): extras={"locale":"it"} keeps the three IT docs (Recall@10 1.0); facets {locale: {it: 3}, brand: {CasaLuce, AtelierNord, FornoBasso}, color: {argento, blu navy, alluminio}}. extras={"color":"nope"} returns empty hits / empty facets. p50 retrieve ~17 ms on this 3-doc brain — not a 200 ms SLO claim. Contract test still bans catalog enum field names.
Encoder smoke (M04 Italian-only)
Harness local-dense paraphrase-multilingual-MiniLM-L12-v2 on data/search_italian_smoke.jsonl, labeled brain searchbenchitmmini, run search-italian-minilm-pipeline. Did not reuse searchbenchitsmoke 3-large vectors; did not re-ingest 74 or ES. n=3, status: ok, all three golds retrieved at k=10 (same pipeline recall as search-italian-smoke 3-large). Not a claim that MiniLM beats 3-large. Not Reddy. Not ES n=62. Product embedder unchanged.
Out of scope
Wiping brains; raising default RERANK_MAX_K; CE as default; graph fusion as default; recsys; LoCoMo/BEAM; claiming Reddy 0.857; MT-US-as-Italian; Amazon-M2 MRR as search quality; merchandising that needs recommend/context.
Located US control (do not redo)
Passages k=50 n=74: nDCG@10 0.500, Recall@10 0.379, Recall@50 0.834, p50 ~60ms. MiniLM/DeBERTa CE on those hits hurt Recall@10. Live mode=catalog MiniLM 0.467/0.341 null. Cascade R@50 0.889 harness-only (live sidecar not indexed). fusion=cc alpha sweep best nDCG@10 0.493 (null vs 0.520). Literal residual Recall@50 0.655 (null). Head LTR 5-fold CV on stored k=50: nDCG@10 0.515, Recall@10 0.384 (null vs 0.520 / 0.397); overlap-only 0.476 / 0.339. LTR + MiniLM ce_gain: nDCG@10 0.524, Recall@10 0.383, Recall@50 0.834 (harness win vs 0.520 / 0.379 hold; not live). LTR + DeBERTa-v3-base ce_gain RankNet CV: nDCG@10 0.542, Recall@10 0.387, Recall@50 0.834 (win vs MiniLM 0.524; horizon 0.70 missed; not live). Gated LightGBM on the same DeBERTa matrix: nDCG@10 0.516 (null vs RankNet). LTR + DeBERTa applied from 170 train hybrid lists (searchbenchesciltr2): nDCG@10 0.544, Recall@10 0.391, Recall@50 0.834 (win vs 0.542; horizon missed; not live). LightGBM apply on the same 170 lists: nDCG@10 0.533 (null vs RankNet 0.544; R@10 0.389 / R@50 0.834 held; not live). RankNet + hybrid-list DeBERTa ce_gain (esci-deberta-v3-base-4class-hybrid170): nDCG@10 0.530 (null vs 0.544; R@10 0.385 / R@50 0.834 held; not live). Default path that survived: hybrid BM25+dense, passages, rerank=none, Italian via gated FTS+OR.