Build and run agents you can see, understand and trust.
Python agentchatbotlarge-language-modelsllmllm-agentmcpmulti-agentmulti-modalreact-agentrealtime-agentvoice-agentvoice-assistant
feat(rag): add MongoDB vector store backend (#2008)
xu-supingb2d92f31e0d7
diff --git a/examples/rag/README.md b/examples/rag/README.mdindex 1e7e1abde..dc4589889 100644--- a/examples/rag/README.md+++ b/examples/rag/README.md@@ -7,7 +7,7 @@ Two library-mode walk-throughs of `agentscope.rag` — no FastAPI service, no ma| [`index_and_search.py`](./index_and_search.py) | The minimal pipeline: parse → chunk → embed → insert, then `KnowledgeBase.search`. Start here. || [`integrate_with_agent.py`](./integrate_with_agent.py) | Attaches the same `KnowledgeBase` to an `Agent` via `RAGMiddleware`, in both `static` (auto-inject) and `agentic` (tool-driven) modes. |-Both examples use an in-memory Qdrant store (`location=":memory:"`) and the DashScope `text-embedding-v4` model, so no external services are required.+Both examples use an in-memory Qdrant store (`location=":memory:"`) and the DashScope `text-embedding-v4` model, so no external services are required. The sections below show how to swap in Milvus Lite or MongoDB instead; those backends need additional setup.## Install@@ -19,6 +19,8 @@ uv pip install "agentscope[rag]"uv pip install -e ".[rag]"```+### Milvus Lite (local persistence)+To use a local persistent Milvus Lite vector store instead of thein-memory Qdrant store, install the optional extra:@@ -28,7 +30,8 @@ uv pip install "agentscope[milvuslite]"uv pip install -e ".[milvuslite]"```-Then replace the vector store construction:+Then replace the vector store construction in `index_and_search.py`+and/or `integrate_with_agent.py`:```pythonfrom agentscope.rag import MilvusLiteStore@@ -36,6 +39,88 @@ from agentscope.rag import MilvusLiteStorestore = MilvusLiteStore(uri="./rag_demo.db")```+### MongoDB Vector Search++To use MongoDB as the vector backend instead of the in-memory Qdrant+store — useful when your team already runs MongoDB as the primary data+store and wants to avoid maintaining a separate vector database — install+the optional extra:++```bash+uv pip install "agentscope[mongodb]"+# Or from source (repo root)+uv pip install -e ".[mongodb]"+```++**Prerequisites**++- A MongoDB deployment with [Vector Search](https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-overview/) enabled:+ - **MongoDB Atlas** — create a cluster and enable Vector Search on the target database; or+ - **Self-hosted** — MongoDB 7.0+ replica set with Vector Search enabled.+- A connection URI available in the environment (do not hard-code credentials):++```bash+export MONGODB_URI="mongodb+srv://user:[email protected]/?retryWrites=true&w=majority"+# Self-hosted example:+# export MONGODB_URI="mongodb://localhost:27017"+```++Then replace the vector store construction in `index_and_search.py`+and/or `integrate_with_agent.py`:++```python+import os++from agentscope.rag import MongoDBStore++store = MongoDBStore(+ uri=os.environ["MONGODB_URI"],+ database="agentscope_rag",+ # Declare every field you plan to filter on in search().+ # Required for metadata_filter; defaults to ["document_id"] only.+ filter_fields=[+ "document_id",+ # "chunk.metadata.tenant_id", # uncomment if you use metadata_filter+ ],+)++# MongoDBStore is also an async context manager — same as QdrantStore.+async with store:+ knowledge = KnowledgeBase(+ name="demo-kb",+ description="A toy corpus on cats and AgentScope.",+ embedding_model=embedding_model,+ vector_store=store,+ collection=COLLECTION,+ )+ ...+```++**Notes**++- The examples use DashScope `text-embedding-v4` with `dimensions=1024`.+ `MongoDBStore.create_collection` is called automatically on the first+ index operation with that dimension — keep the embedding model and+ index dimensions aligned.+- Unlike Qdrant `:memory:` or Milvus Lite (local `.db` file), MongoDB is+ an external service; you must have a reachable cluster before running+ the scripts.+- If `search(..., metadata_filter={...})` returns no results or errors,+ ensure each metadata key is listed in `filter_fields` as+ `chunk.metadata.<key>` when constructing `MongoDBStore`.+- For the full FastAPI RAG service, pass the same `MongoDBStore` instance+ to `create_app(vector_store=...)` in `examples/agent_service/main.py`+ (the default there uses in-memory Qdrant for zero-setup demos).++### Choosing a vector backend++| | Qdrant (default) | Milvus Lite | MongoDB |+| --- | --- | --- | --- |+| Install extra | `agentscope[rag]` | `agentscope[milvuslite]` | `agentscope[mongodb]` |+| External service | No | No | Yes |+| Persistence | No (`:memory:`) | Yes (local `.db`) | Yes (server) |+| Best for | Quick start / tests | Local dev with persistence | Teams already on MongoDB |+`integrate_with_agent.py` additionally uses `DashScopeChatModel`, which is already in the base `agentscope` dependencies.## Run@@ -47,7 +132,8 @@ python examples/rag/index_and_search.pypython examples/rag/integrate_with_agent.py```+When using MongoDB, also export `MONGODB_URI` before running.+## Service modeThe two scripts above are library-mode — you drive the pipeline yourself in a single process. For the full service-mode experience (FastAPI endpoints for knowledge base CRUD, document upload, indexing workers, and search), see [`examples/agent_service`](../agent_service) for the backend and [`examples/web_ui`](../web_ui) for the chat-style UI.-diff --git a/pyproject.toml b/pyproject.tomlindex 81158f37b..ffd601aec 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -88,6 +88,10 @@ milvuslite = ["pymilvus[milvus-lite]>=2.4.0","milvus-lite>=3.0; sys_platform == 'win32'",]+# ------------ MongoDB vector store ------------+mongodb = [+ "pymongo>=4.7",+]# ------------ S3-compatible blob store ------------s3 = [@@ -114,6 +118,7 @@ full = ["agentscope[tools]","agentscope[rag]","agentscope[milvuslite]",+ "agentscope[mongodb]","agentscope[s3]","agentscope[mem0]","agentscope[reme]",@@ -153,7 +158,7 @@ include-package-data = true# ``"*"`` applies to every discovered package. Patterns:# - ``py.typed`` marks the typed package per PEP 561.# - ``_models/*.yaml`` reaches into the data-only ``_models/``-# subdirs that ship with each model provider (they have no+# subdirs that ship with each model provider (they have no# ``__init__.py`` and are not packages, but the files still# live alongside the parent package's source on disk)."*" = [@@ -162,7 +167,6 @@ include-package-data = true"_cosyvoice_models/*.yaml",]"agentscope.workspace._docker" = ["Dockerfile*.template"]-[build-system]requires = ["setuptools>=45", "wheel"]build-backend = "setuptools.build_meta"diff --git a/src/agentscope/rag/__init__.py b/src/agentscope/rag/__init__.pyindex 78697cb40..5d2c3a12e 100644--- a/src/agentscope/rag/__init__.py+++ b/src/agentscope/rag/__init__.py@@ -14,6 +14,7 @@ from ._vdb import (VectorRecord,VectorSearchResult,QdrantStore,+ MongoDBStore,)from ._knowledge import KnowledgeBase@@ -34,4 +35,5 @@ __all__ = ["VectorSearchResult","QdrantStore","KnowledgeBase",+ "MongoDBStore",]diff --git a/src/agentscope/rag/_vdb/__init__.py b/src/agentscope/rag/_vdb/__init__.pyindex 58774cfd9..68c712032 100644--- a/src/agentscope/rag/_vdb/__init__.py+++ b/src/agentscope/rag/_vdb/__init__.py@@ -7,8 +7,9 @@ from ._vector_store import (VectorSearchResult,VectorStoreBase,)-from ._milvus_lite import MilvusLiteStorefrom ._qdrant import QdrantStore+from ._mongodb import MongoDBStore+from ._milvus_lite import MilvusLiteStore__all__ = ["DocumentSummary",@@ -17,4 +18,5 @@ __all__ = ["VectorRecord","VectorSearchResult","QdrantStore",+ "MongoDBStore",]diff --git a/src/agentscope/rag/_vdb/_mongodb.py b/src/agentscope/rag/_vdb/_mongodb.pynew file mode 100644index 000000000..358c8a808--- /dev/null+++ b/src/agentscope/rag/_vdb/_mongodb.py@@ -0,0 +1,464 @@+# -*- coding: utf-8 -*-+"""MongoDB implementation of the vector store backend.++Built on ``pymongo.AsyncMongoClient`` and MongoDB Vector Search+(``$vectorSearch``), so all operations are non-blocking and safe to+call from the application's event loop.++The same class supports MongoDB Atlas and self-hosted deployments+through the constructor arguments:++- ``uri="mongodb+srv://..."`` — MongoDB Atlas cluster+- ``uri="mongodb://localhost:27017"`` — self-hosted replica set+ (MongoDB 7.0+ with Vector Search enabled)+"""+import asyncio+import time+from typing import TYPE_CHECKING, Any, Literal++from ._vector_store import (+ DocumentSummary,+ VectorRecord,+ VectorSearchResult,+ VectorStoreBase,+)+from .._document import Chunk++if TYPE_CHECKING:+ from pymongo import AsyncMongoClient+++class MongoDBStore(VectorStoreBase):+ """Vector store backend backed by `MongoDB Vector Search \+<https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-overview/>`_.++ Each knowledge base maps to one MongoDB collection inside a single+ database. Every document stores the owning ``document_id`` plus+ the serialized :class:`~agentscope.rag.Chunk`, which is+ reconstructed on retrieval.++ .. note:: Requires ``pymongo`` with ``AsyncMongoClient`` support+ (``pymongo>=4.7``). Install with+ ``pip install agentscope[mongodb]``.++ .. note:: Fields used in :meth:`search` ``metadata_filter`` must be+ declared in ``filter_fields`` at construction time so they are+ included in the vector search index definition.++ .. code-block:: python++ # MongoDB Atlas+ store = MongoDBStore(+ uri="mongodb+srv://user:[email protected]",+ database="agentscope_rag",+ filter_fields=[+ "document_id",+ "chunk.metadata.tenant_id",+ ],+ )++ async with store:+ await store.create_collection("kb-1", dimensions=768)++ """++ def __init__(+ self,+ uri: str,+ database: str,+ distance: Literal["cosine", "euclidean", "dotProduct"] = "cosine",+ index_name: str = "vector_index",+ filter_fields: list[str] | None = None,+ client_kwargs: dict[str, Any] | None = None,+ ) -> None:+ """Initialize the MongoDB vector store.++ Args:+ uri (`str`):+ The MongoDB connection URI, e.g.+ ``"mongodb+srv://..."`` for Atlas or+ ``"mongodb://localhost:27017"`` for a self-hosted+ replica set.+ database (`str`):+ The database name. Each knowledge base maps to one+ collection inside this database.+ distance (`Literal["cosine", "euclidean", "dotProduct"]`, \+ defaults to ``"cosine"``):+ The similarity metric used when creating vector search+ indexes.+ index_name (`str`, defaults to ``"vector_index"``):+ The name of the vector search index created on each+ collection. All collections share this index name.+ filter_fields (`list[str] | None`, optional):+ Field paths declared as filter fields in the vector+ search index. Required for ``metadata_filter`` in+ :meth:`search`. Defaults to ``["document_id"]``.+ client_kwargs (`dict[str, Any] | None`, optional):+ Extra keyword arguments forwarded to+ :class:`~pymongo.AsyncMongoClient`.+ """+ self._uri = uri+ self._database_name = database+ self._distance = distance+ self._index_name = index_name+ self._filter_fields = filter_fields or ["document_id"]+ self._client_kwargs = client_kwargs or {}+ self._client: "AsyncMongoClient | None" = None++ def get_client(self) -> "AsyncMongoClient":+ """Lazily create and cache the async MongoDB client.++ Returns:+ `AsyncMongoClient`:+ The shared async client instance.+ """+ if self._client is None:+ from pymongo import AsyncMongoClient++ self._client = AsyncMongoClient(+ self._uri,+ **self._client_kwargs,+ )+ return self._client++ async def __aexit__(+ self,+ exc_type: type[BaseException] | None,+ exc_val: BaseException | None,+ exc_tb: Any,+ ) -> None:+ """Exit the async context — close the underlying client."""+ if self._client is not None:+ await self._client.close()+ self._client = None++ # ------------------------------------------------------------------+ # Collection management+ # ------------------------------------------------------------------++ def _db(self) -> Any:+ """Return the configured database handle."""+ return self.get_client()[self._database_name]++ def _col(self, collection: str) -> Any:+ """Return a collection handle inside the configured database."""+ return self._db()[collection]++ async def create_collection(self, name: str, dimensions: int) -> None:+ """Create a new MongoDB collection with a vector search index.++ No-op if the collection already exists.++ Args:+ name (`str`):+ The collection name. Typically, the knowledge base ID.+ dimensions (`int`):+ The fixed vector dimensionality for this collection.+ """+ if await self.has_collection(name):+ return++ from pymongo.operations import SearchIndexModel++ await self._db().create_collection(name)+ col = self._col(name)++ fields: list[dict[str, Any]] = [+ {+ "type": "vector",+ "path": "vector",+ "numDimensions": dimensions,+ "similarity": self._distance,+ },+ ]+ fields.extend(+ {"type": "filter", "path": field_path}+ for field_path in self._filter_fields+ )++ model = SearchIndexModel(+ definition={"fields": fields},+ name=self._index_name,+ type="vectorSearch",+ )+ await col.create_search_index(model)+ await self._wait_for_index_ready(col, name)++ async def delete_collection(self, name: str) -> None:+ """Delete a collection and all its data.++ Args:+ name (`str`):+ The collection name to delete.+ """+ await self._col(name).drop()++ async def has_collection(self, name: str) -> bool:+ """Check whether a collection exists.++ Args:+ name (`str`):+ The collection name to check.++ Returns:+ `bool`: ``True`` if the collection exists.+ """+ return name in await self._db().list_collection_names()++ async def _wait_for_index_ready(+ self,+ collection: Any,+ collection_name: str,+ timeout: float = 30.0,+ ) -> None:+ """Poll until the vector search index becomes queryable.++ Args:+ collection:+ The MongoDB collection handle.+ collection_name (`str`):+ The collection name (used in timeout error messages).+ timeout (`float`, defaults to ``30.0``):+ Maximum seconds to wait before raising+ :class:`TimeoutError`.++ Raises:+ `TimeoutError`:+ If the index is not queryable within ``timeout`` seconds.+ """+ deadline = time.monotonic() + timeout+ while time.monotonic() < deadline:+ async for index in await collection.list_search_indexes(+ self._index_name,+ ):+ if index.get("queryable"):+ return+ break+ await asyncio.sleep(0.5)++ raise TimeoutError(+ f"Vector search index {self._index_name!r} on collection "+ f"{collection_name!r} was not queryable within {timeout}s",+ )++ async def _ensure_index_ready(self, collection: str) -> None:+ """Ensure the vector search index is queryable before reads."""+ await self._wait_for_index_ready(self._col(collection), collection)++ # ------------------------------------------------------------------+ # Data operations+ # ------------------------------------------------------------------++ async def insert(+ self,+ collection: str,+ records: list[VectorRecord],+ ) -> None:+ """Insert records into a collection.++ Each document stores :attr:`VectorRecord.document_id` under the+ ``document_id`` key and the serialized :class:`Chunk` under the+ ``chunk`` key, so that :meth:`delete` can remove all records of+ one document.++ Args:+ collection (`str`):+ The target collection name.+ records (`list[VectorRecord]`):+ The records to insert (each carrying a+ :class:`Chunk` and its embedding vector).+ """+ if not records:+ return++ from pymongo import ReplaceOne++ col = self._col(collection)+ operations = [+ ReplaceOne(+ {"_id": f"{record.document_id}_{record.chunk.chunk_index}"},+ {+ "_id": f"{record.document_id}_{record.chunk.chunk_index}",+ "document_id": record.document_id,+ "vector": record.vector,+ "chunk": record.chunk.model_dump(mode="json"),+ },+ upsert=True,+ )+ for record in records+ ]+ await col.bulk_write(operations, ordered=False)++ async def delete(+ self,+ collection: str,+ document_id: str,+ ) -> None:+ """Delete all records belonging to one source document.++ Matches the ``document_id`` field written by :meth:`insert` from+ :attr:`VectorRecord.document_id`.++ Args:+ collection (`str`):+ The target collection name.+ document_id (`str`):+ The source document ID whose records should be+ removed.+ """+ await self._col(collection).delete_many({"document_id": document_id})++ # ------------------------------------------------------------------+ # Search+ # ------------------------------------------------------------------++ async def search(+ self,+ collection: str,+ query_vector: list[float],+ top_k: int = 5,+ metadata_filter: dict[str, Any] | None = None,+ ) -> list[VectorSearchResult]:+ """Find the most similar records to a query vector.++ Args:+ collection (`str`):+ The collection to search.+ query_vector (`list[float]`):+ The query embedding vector.+ top_k (`int`, defaults to ``5``):+ Maximum number of results to return.+ metadata_filter (`dict[str, Any] | None`, optional):+ If provided, restrict the search to records whose+ ``chunk.metadata`` matches every ``key == value`` pair+ in this dict (translated into a MongoDB filter against+ ``chunk.metadata.<key>``). Each key must correspond to+ a path declared in ``filter_fields``.++ Returns:+ `list[VectorSearchResult]`:+ Results ordered by descending similarity score.+ """+ await self._ensure_index_ready(collection)+ col = self._col(collection)++ vector_search: dict[str, Any] = {+ "index": self._index_name,+ "path": "vector",+ "queryVector": query_vector,+ "numCandidates": max(100, top_k * 20),+ "limit": top_k,+ }+ filter_expr = self._build_metadata_filter(metadata_filter)+ if filter_expr is not None:+ vector_search["filter"] = filter_expr++ pipeline = [+ {"$vectorSearch": vector_search},+ {+ "$project": {+ "document_id": 1,+ "chunk": 1,+ "score": {"$meta": "vectorSearchScore"},+ },+ },+ ]++ results: list[VectorSearchResult] = []+ async for doc in await col.aggregate(pipeline):+ results.append(+ VectorSearchResult(+ score=float(doc["score"]),+ document_id=doc["document_id"],+ chunk=Chunk.model_validate(doc["chunk"]),+ ),+ )+ return results++ @staticmethod+ def _build_metadata_filter(+ metadata_filter: dict[str, Any] | None,+ ) -> dict[str, Any] | None:+ """Translate a flat ``{key: value}`` filter into a MongoDB filter.++ Each ``key`` is matched against the corresponding nested path+ ``chunk.metadata.<key>`` written by :meth:`insert`. Returns+ ``None`` when ``metadata_filter`` is empty so that callers+ skip the filter argument entirely.++ Args:+ metadata_filter (`dict[str, Any] | None`):+ The flat filter, or ``None`` for no filter.++ Returns:+ `dict[str, Any] | None`:+ A MongoDB filter document, or ``None``.+ """+ if not metadata_filter:+ return None++ return {+ "$and": [+ {f"chunk.metadata.{key}": {"$eq": value}}+ for key, value in metadata_filter.items()+ ],+ }++ # ------------------------------------------------------------------+ # Document listing+ # ------------------------------------------------------------------++ async def list_documents(+ self,+ collection: str,+ metadata_filter: dict[str, Any] | None = None,+ ) -> list[DocumentSummary]:+ """List all distinct source documents indexed in a collection.++ Aggregates records by ``document_id``. The first chunk encountered+ for each document supplies the ``source`` filename and the+ document-level ``metadata``.++ Args:+ collection (`str`):+ The target collection name.+ metadata_filter (`dict[str, Any] | None`, optional):+ If provided, restrict aggregation to records whose+ ``chunk.metadata`` matches every ``key == value`` pair.++ Returns:+ `list[DocumentSummary]`:+ One summary per distinct ``document_id``.+ """+ pipeline: list[dict[str, Any]] = []+ if metadata_filter:+ pipeline.append(+ {+ "$match": {+ f"chunk.metadata.{key}": value+ for key, value in metadata_filter.items()+ },+ },+ )+ pipeline.append(+ {+ "$group": {+ "_id": "$document_id",+ "source": {"$first": "$chunk.source"},+ "metadata": {"$first": "$chunk.metadata"},+ "chunk_count": {"$sum": 1},+ },+ },+ )++ summaries: list[DocumentSummary] = []+ async for row in await self._col(collection).aggregate(pipeline):+ summaries.append(+ DocumentSummary(+ document_id=row["_id"],+ source=row["source"],+ chunk_count=row["chunk_count"],+ metadata=row["metadata"] or {},+ ),+ )+ return summariesdiff --git a/tests/rag_vdb_mongodb_test.py b/tests/rag_vdb_mongodb_test.pynew file mode 100644index 000000000..b4c3ed5a7--- /dev/null+++ b/tests/rag_vdb_mongodb_test.py@@ -0,0 +1,647 @@+# -*- coding: utf-8 -*-+# pylint: disable=protected-access,missing-function-docstring+"""Unit tests for the MongoDBStore class (mocked pymongo backend)."""+from __future__ import annotations++import math+from contextlib import AsyncExitStack+from typing import Any+from unittest.async_case import IsolatedAsyncioTestCase+from unittest.mock import patch++from utils import AnyString++from agentscope.message import TextBlock+from agentscope.rag import (+ Chunk,+ MongoDBStore,+ VectorRecord,+ VectorSearchResult,+)+++def _dump_results(results: list[VectorSearchResult]) -> list[dict]:+ """Convert search results into plain dicts for whole-structure+ comparison.++ Args:+ results (`list[VectorSearchResult]`):+ The search results to convert.++ Returns:+ `list[dict]`:+ The results as plain dicts.+ """+ return [result.model_dump() for result in results]+++def _make_record(+ text: str,+ vector: list[float],+ document_id: str,+ chunk_index: int = 0,+ total_chunks: int = 1,+) -> VectorRecord:+ """Build a VectorRecord for testing.++ Args:+ text (`str`):+ The chunk text content.+ vector (`list[float]`):+ The embedding vector.+ document_id (`str`):+ The ID of the source document the record belongs to.+ chunk_index (`int`, defaults to ``0``):+ The chunk index within the document.+ total_chunks (`int`, defaults to ``1``):+ The total number of chunks in the document.++ Returns:+ `VectorRecord`:+ The constructed record.+ """+ return VectorRecord(+ vector=vector,+ document_id=document_id,+ chunk=Chunk(+ content=TextBlock(text=text),+ source=f"{document_id}.txt",+ chunk_index=chunk_index,+ total_chunks=total_chunks,+ ),+ )+++def _cosine_similarity(a: list[float], b: list[float]) -> float:+ """Cosine similarity aligned with Qdrant cosine scoring in tests."""+ dot = sum(x * y for x, y in zip(a, b, strict=True))+ norm_a = math.sqrt(sum(x * x for x in a))+ norm_b = math.sqrt(sum(x * x for x in b))+ if norm_a == 0.0 or norm_b == 0.0:+ return 0.0+ return dot / (norm_a * norm_b)+++class _FakeAsyncIterator:+ """Minimal async iterator for mocked MongoDB cursors."""++ def __init__(self, items: list[Any]) -> None:+ self._items = items+ self._index = 0++ def __aiter__(self) -> "_FakeAsyncIterator":+ return self++ async def __anext__(self) -> Any:+ if self._index >= len(self._items):+ raise StopAsyncIteration+ item = self._items[self._index]+ self._index += 1+ return item+++class _FakeMongoCollection:+ """In-memory collection that implements the async API MongoDBStore uses."""++ def __init__(self, database: "_FakeMongoDatabase", name: str) -> None:+ self._database = database+ self._name = name+ self._docs: dict[str, dict[str, Any]] = {}+ self._index_queryable = False++ async def bulk_write(+ self,+ operations: list[Any],+ ordered: bool = False,+ ) -> None:+ del ordered+ for operation in operations:+ self._docs[operation._doc["_id"]] = dict(operation._doc)++ async def delete_many(self, filter_doc: dict[str, Any]) -> None:+ document_id = filter_doc["document_id"]+ to_remove = [+ doc_id+ for doc_id, doc in self._docs.items()+ if doc.get("document_id") == document_id+ ]+ for doc_id in to_remove:+ del self._docs[doc_id]++ async def create_search_index(self, model: Any) -> None:+ del model+ self._index_queryable = True++ async def list_search_indexes(self, name: str) -> _FakeAsyncIterator:+ del name+ return _FakeAsyncIterator([{"queryable": self._index_queryable}])++ async def aggregate(+ self,+ pipeline: list[dict[str, Any]],+ ) -> _FakeAsyncIterator:+ return _FakeAsyncIterator(self._run_aggregate(pipeline))++ async def drop(self) -> None:+ self._database._collections.pop(self._name, None)++ def _run_aggregate(+ self,+ pipeline: list[dict[str, Any]],+ ) -> list[dict[str, Any]]:+ if pipeline and "$vectorSearch" in pipeline[0]:+ return self._vector_search(pipeline[0]["$vectorSearch"])+ return self._list_documents(pipeline)++ def _vector_search(self, stage: dict[str, Any]) -> list[dict[str, Any]]:+ query_vector = stage["queryVector"]+ top_k = stage["limit"]+ metadata_filter = stage.get("filter")++ scored: list[dict[str, Any]] = []+ for doc in self._docs.values():+ if not self._matches_metadata_filter(doc, metadata_filter):+ continue+ scored.append(+ {+ "document_id": doc["document_id"],+ "chunk": doc["chunk"],+ "score": _cosine_similarity(query_vector, doc["vector"]),+ },+ )++ scored.sort(key=lambda item: item["score"], reverse=True)+ return scored[:top_k]++ def _list_documents(+ self,+ pipeline: list[dict[str, Any]],+ ) -> list[dict[str, Any]]:+ docs = list(self._docs.values())+ for stage in pipeline:+ if "$match" in stage:+ docs = [+ doc+ for doc in docs+ if self._matches_match_stage(doc, stage["$match"])+ ]+ elif "$group" in stage:+ return self._group_documents(docs, stage["$group"])+ return []++ @staticmethod+ def _matches_match_stage(+ doc: dict[str, Any],+ match: dict[str, Any],+ ) -> bool:+ for key, expected in match.items():+ value = doc+ for part in key.split("."):+ value = value[part]+ if value != expected:+ return False+ return True++ @staticmethod+ def _matches_metadata_filter(+ doc: dict[str, Any],+ metadata_filter: dict[str, Any] | None,+ ) -> bool:+ if not metadata_filter:+ return True++ for clause in metadata_filter.get("$and", []):+ for key, condition in clause.items():+ expected = condition["$eq"]+ value = doc+ for part in key.split("."):+ value = value[part]+ if value != expected:+ return False+ return True++ @staticmethod+ def _group_documents(+ docs: list[dict[str, Any]],+ group_stage: dict[str, Any],+ ) -> list[dict[str, Any]]:+ del group_stage+ grouped: dict[str, dict[str, Any]] = {}+ for doc in docs:+ document_id = doc["document_id"]+ if document_id not in grouped:+ grouped[document_id] = {+ "_id": document_id,+ "source": doc["chunk"]["source"],+ "metadata": doc["chunk"].get("metadata") or {},+ "chunk_count": 0,+ }+ grouped[document_id]["chunk_count"] += 1+ return list(grouped.values())+++class _FakeMongoDatabase:+ """In-memory database handle."""++ def __init__(self) -> None:+ self._collections: dict[str, _FakeMongoCollection] = {}++ async def list_collection_names(self) -> list[str]:+ return list(self._collections.keys())++ async def create_collection(self, name: str) -> _FakeMongoCollection:+ collection = _FakeMongoCollection(self, name)+ self._collections[name] = collection+ return collection++ def __getitem__(self, name: str) -> _FakeMongoCollection:+ if name not in self._collections:+ self._collections[name] = _FakeMongoCollection(self, name)+ return self._collections[name]+++class _FakeMongoClient:+ """In-memory async MongoDB client."""++ def __init__(self, database_name: str) -> None:+ self._database_name = database_name+ self._databases: dict[str, _FakeMongoDatabase] = {}++ def __getitem__(self, database_name: str) -> _FakeMongoDatabase:+ if database_name not in self._databases:+ self._databases[database_name] = _FakeMongoDatabase()+ return self._databases[database_name]++ async def close(self) -> None:+ self._databases.clear()+++class MongoDBStoreTest(IsolatedAsyncioTestCase):+ """The test cases for the MongoDBStore class."""++ async def asyncSetUp(self) -> None:+ """Create a MongoDB store with a mocked pymongo client."""+ self._fake_client = _FakeMongoClient("test-db")+ self._client_patcher = patch.object(+ MongoDBStore,+ "get_client",+ return_value=self._fake_client,+ )+ self._client_patcher.start()++ self._exit_stack = AsyncExitStack()+ self.store = MongoDBStore(uri="mongodb://mock", database="test-db")+ await self._exit_stack.enter_async_context(self.store)++ async def asyncTearDown(self) -> None:+ """Close the store and stop patches after each test."""+ await self._exit_stack.aclose()+ self._client_patcher.stop()++ async def test_collection_lifecycle(self) -> None:+ """Collections can be created, checked, and deleted."""+ self.assertEqual(await self.store.has_collection("kb-1"), False)++ await self.store.create_collection("kb-1", dimensions=3)+ self.assertEqual(await self.store.has_collection("kb-1"), True)++ # Creating an existing collection is a no-op+ await self.store.create_collection("kb-1", dimensions=3)+ self.assertEqual(await self.store.has_collection("kb-1"), True)++ await self.store.delete_collection("kb-1")+ self.assertEqual(await self.store.has_collection("kb-1"), False)++ async def test_insert_and_search(self) -> None:+ """Inserted records are searchable, ordered by similarity."""+ await self.store.create_collection("kb-1", dimensions=3)+ await self.store.insert(+ "kb-1",+ [+ _make_record(+ "Hello world!",+ [1.0, 0.0, 0.0],+ document_id="doc-1",+ chunk_index=0,+ total_chunks=2,+ ),+ _make_record(+ "Goodbye world!",+ [0.0, 1.0, 0.0],+ document_id="doc-1",+ chunk_index=1,+ total_chunks=2,+ ),+ ],+ )++ results = await self.store.search(+ "kb-1",+ query_vector=[1.0, 0.0, 0.0],+ top_k=2,+ )++ self.assertEqual(+ _dump_results(results),+ [+ {+ "score": 1.0,+ "document_id": "doc-1",+ "chunk": {+ "content": {+ "type": "text",+ "text": "Hello world!",+ "id": AnyString(),+ },+ "source": "doc-1.txt",+ "chunk_index": 0,+ "total_chunks": 2,+ "metadata": {},+ },+ },+ {+ "score": 0.0,+ "document_id": "doc-1",+ "chunk": {+ "content": {+ "type": "text",+ "text": "Goodbye world!",+ "id": AnyString(),+ },+ "source": "doc-1.txt",+ "chunk_index": 1,+ "total_chunks": 2,+ "metadata": {},+ },+ },+ ],+ )++ async def test_search_top_k(self) -> None:+ """top_k limits the number of returned results."""+ await self.store.create_collection("kb-1", dimensions=3)+ await self.store.insert(+ "kb-1",+ [+ _make_record("A", [1.0, 0.0, 0.0], document_id="doc-1"),+ _make_record("B", [0.9, 0.1, 0.0], document_id="doc-2"),+ _make_record("C", [0.0, 0.0, 1.0], document_id="doc-3"),+ ],+ )++ results = await self.store.search(+ "kb-1",+ query_vector=[1.0, 0.0, 0.0],+ top_k=1,+ )++ self.assertEqual(+ _dump_results(results),+ [+ {+ "score": 1.0,+ "document_id": "doc-1",+ "chunk": {+ "content": {+ "type": "text",+ "text": "A",+ "id": AnyString(),+ },+ "source": "doc-1.txt",+ "chunk_index": 0,+ "total_chunks": 1,+ "metadata": {},+ },+ },+ ],+ )++ async def test_delete_by_document_id(self) -> None:+ """delete removes all records of one document only."""+ await self.store.create_collection("kb-1", dimensions=3)+ await self.store.insert(+ "kb-1",+ [+ _make_record(+ "doc1-chunk0",+ [1.0, 0.0, 0.0],+ document_id="doc-1",+ chunk_index=0,+ total_chunks=2,+ ),+ _make_record(+ "doc1-chunk1",+ [0.9, 0.1, 0.0],+ document_id="doc-1",+ chunk_index=1,+ total_chunks=2,+ ),+ _make_record(+ "doc2-chunk0",+ [0.0, 1.0, 0.0],+ document_id="doc-2",+ ),+ ],+ )++ await self.store.delete("kb-1", document_id="doc-1")++ results = await self.store.search(+ "kb-1",+ query_vector=[1.0, 0.0, 0.0],+ top_k=5,+ )++ self.assertEqual(+ _dump_results(results),+ [+ {+ "score": 0.0,+ "document_id": "doc-2",+ "chunk": {+ "content": {+ "type": "text",+ "text": "doc2-chunk0",+ "id": AnyString(),+ },+ "source": "doc-2.txt",+ "chunk_index": 0,+ "total_chunks": 1,+ "metadata": {},+ },+ },+ ],+ )++ async def test_insert_empty_records(self) -> None:+ """Inserting an empty record list is a no-op."""+ await self.store.create_collection("kb-1", dimensions=3)+ await self.store.insert("kb-1", [])++ results = await self.store.search(+ "kb-1",+ query_vector=[1.0, 0.0, 0.0],+ )++ self.assertEqual(_dump_results(results), [])++ async def test_list_documents_aggregates_by_document_id(self) -> None:+ """list_documents groups chunks by document_id."""+ await self.store.create_collection("kb-1", dimensions=3)++ def _record_with_metadata(+ text: str,+ document_id: str,+ metadata: dict,+ chunk_index: int = 0,+ total_chunks: int = 1,+ ) -> VectorRecord:+ return VectorRecord(+ vector=[1.0, 0.0, 0.0],+ document_id=document_id,+ chunk=Chunk(+ content=TextBlock(text=text),+ source=metadata.get("filename", f"{document_id}.txt"),+ chunk_index=chunk_index,+ total_chunks=total_chunks,+ metadata=metadata,+ ),+ )++ await self.store.insert(+ "kb-1",+ [+ _record_with_metadata(+ "A",+ "doc-1",+ {"filename": "alpha.txt", "media_type": "text/plain"},+ 0,+ 2,+ ),+ _record_with_metadata(+ "B",+ "doc-1",+ {"filename": "alpha.txt", "media_type": "text/plain"},+ 1,+ 2,+ ),+ _record_with_metadata(+ "C",+ "doc-2",+ {"filename": "beta.md", "media_type": "text/markdown"},+ 0,+ 1,+ ),+ ],+ )++ summaries = sorted(+ await self.store.list_documents("kb-1"),+ key=lambda summary: summary.document_id,+ )+ self.assertEqual(+ [summary.model_dump() for summary in summaries],+ [+ {+ "document_id": "doc-1",+ "source": "alpha.txt",+ "chunk_count": 2,+ "metadata": {+ "filename": "alpha.txt",+ "media_type": "text/plain",+ },+ },+ {+ "document_id": "doc-2",+ "source": "beta.md",+ "chunk_count": 1,+ "metadata": {+ "filename": "beta.md",+ "media_type": "text/markdown",+ },+ },+ ],+ )++ async def test_search_metadata_filter(self) -> None:+ """search applies the metadata_filter as a payload predicate."""+ await self.store.create_collection("kb-1", dimensions=3)++ def _record(+ text: str,+ document_id: str,+ kb_scope: str,+ ) -> VectorRecord:+ return VectorRecord(+ vector=[1.0, 0.0, 0.0],+ document_id=document_id,+ chunk=Chunk(+ content=TextBlock(text=text),+ source=f"{document_id}.txt",+ chunk_index=0,+ total_chunks=1,+ metadata={"kb_scope": kb_scope},+ ),+ )++ await self.store.insert(+ "kb-1",+ [+ _record("A", "doc-1", "kb-a"),+ _record("B", "doc-2", "kb-b"),+ ],+ )++ results = await self.store.search(+ "kb-1",+ query_vector=[1.0, 0.0, 0.0],+ top_k=5,+ metadata_filter={"kb_scope": "kb-a"},+ )+ self.assertEqual(+ _dump_results(results),+ [+ {+ "score": 1.0,+ "document_id": "doc-1",+ "chunk": {+ "content": {+ "type": "text",+ "text": "A",+ "id": AnyString(),+ },+ "source": "doc-1.txt",+ "chunk_index": 0,+ "total_chunks": 1,+ "metadata": {"kb_scope": "kb-a"},+ },+ },+ ],+ )++ results = await self.store.search(+ "kb-1",+ query_vector=[1.0, 0.0, 0.0],+ top_k=5,+ metadata_filter={"kb_scope": "kb-b"},+ )+ self.assertEqual(+ _dump_results(results),+ [+ {+ "score": 1.0,+ "document_id": "doc-2",+ "chunk": {+ "content": {+ "type": "text",+ "text": "B",+ "id": AnyString(),+ },+ "source": "doc-2.txt",+ "chunk_index": 0,+ "total_chunks": 1,+ "metadata": {"kb_scope": "kb-b"},+ },+ },+ ],+ )