Skip to content

BrainAPI is a knowledge graph–powered AI memory layer that transforms unstructured data into structured knowledge, enabling intelligent search, recommendations, and contextual memory for AI agents and applications.

Support dense-only search on heavy deployments

Christianc0bae139d559
diff --git a/README.md b/README.md
index 52a07b6..cf418d0 100644
--- a/README.md
+++ b/README.md
@@ -210,7 +210,7 @@ BrainAPI follows an **append-only philosophy**: nothing is deleted from the grap
- **`accurate`** (default) — the full swarm runs: Scout extracts every entity, the Observations agent writes notes, and the Janitor validates and deduplicates the Architect's relationships.
- **`lightweight`** — a faster, cheaper path that extracts only the most important entities and skips the heavier validation steps. Switch with `--pipeline lightweight` or `PIPELINE_MODE` in `.env`.
-Search (`GET|POST /retrieve/search`) is **off by default**. Set `SEARCH_ENABLED=true` (requires `DATA_DB=postgresql`) to register the hit-list API, write BM25 `tsvector` indexes, and use halfvec HNSW when embeddings are >2000-d. Default retrieval is both channels fused (`SEARCH_FUSION=rrf`). The search p50 SLO is **< 200 ms excluding `embed.query`** — ask for `profile_stages` to split embed RTT from retrieve. `/retrieve/context` stays dense ∪ ILIKE unless search is on and `CONTEXT_PASSAGE_MODE` is `hybrid` / `bm25` / `dense`.
+Search (`GET|POST /retrieve/search`) is **off by default**. Set `SEARCH_ENABLED=true` to register the hit-list API. BM25 requires `DATA_DB=postgresql`; dense-only search (`SEARCH_USE_BM25=false`) is supported by either production profile. PostgreSQL writes BM25 `tsvector` indexes and uses halfvec HNSW when embeddings are >2000-d. Default light-profile retrieval is both channels fused (`SEARCH_FUSION=rrf`). The search p50 SLO is **< 200 ms excluding `embed.query`** — ask for `profile_stages` to split embed RTT from retrieve. `/retrieve/context` stays dense ∪ ILIKE unless search is on and `CONTEXT_PASSAGE_MODE` is `hybrid` / `bm25` / `dense`.
---
diff --git a/deploy/env.heavy.example b/deploy/env.heavy.example
index f4170ba..052731f 100644
--- a/deploy/env.heavy.example
+++ b/deploy/env.heavy.example
@@ -176,7 +176,8 @@ INGEST_ARCHITECT_DENSE_ENTITY_THRESHOLD="12"
INGEST_ARCHITECT_DENSE_MAX_CHARS="1200"
INGEST_ARCHITECT_PRIOR_CONTEXT="auto"
INGEST_ARCHITECT_SCRATCHPAD_TOKEN_CAP="500"
-# Search (off by default). SEARCH_ENABLED=true requires DATA_DB=postgresql.
+# Search (off by default). BM25 requires DATA_DB=postgresql; heavy can enable
+# dense-only search with SEARCH_USE_BM25=false.
# p50 SLO < 200 ms excluding embed.query RTT.
SEARCH_ENABLED="false"
SEARCH_USE_DENSE="true"
diff --git a/src/config.py b/src/config.py
index 95d134d..00eca61 100644
--- a/src/config.py
+++ b/src/config.py
@@ -109,10 +109,10 @@ def validate_search_config(
)
if not enabled:
return
- if (data_db or "").strip().lower() != "postgresql":
+ if use_bm25 and (data_db or "").strip().lower() != "postgresql":
raise ValueError(
- "SEARCH_ENABLED=true requires DATA_DB=postgresql "
- "(v1 BM25 indexes are Postgres-only)."
+ "SEARCH_USE_BM25=true requires DATA_DB=postgresql "
+ "(BM25 indexes are Postgres-only)."
)
if not use_dense and not use_bm25:
raise ValueError(
diff --git a/tests/test_search_config.py b/tests/test_search_config.py
index 0b9db26..872372f 100644
--- a/tests/test_search_config.py
+++ b/tests/test_search_config.py
@@ -51,7 +51,7 @@ class SearchConfigValidationTests(unittest.TestCase):
bm25_b=0.75,
)
- def test_enabled_requires_postgres_data_db(self):
+ def test_bm25_requires_postgres_data_db(self):
with self.assertRaises(ValueError) as ctx:
validate_search_config(
enabled=True,
@@ -63,6 +63,16 @@ class SearchConfigValidationTests(unittest.TestCase):
)
self.assertIn("DATA_DB=postgresql", str(ctx.exception))
+ def test_dense_only_search_supports_non_postgres_data_db(self):
+ validate_search_config(
+ enabled=True,
+ use_dense=True,
+ use_bm25=False,
+ data_db="mongo",
+ bm25_k1=1.2,
+ bm25_b=0.75,
+ )
+
def test_enabled_requires_at_least_one_channel(self):
with self.assertRaises(ValueError) as ctx:
validate_search_config(