Skip to content

BrainAPI is a knowledge graph–powered AI memory layer that transforms unstructured data into structured knowledge, enabling intelligent search, recommendations, and contextual memory for AI agents and applications.

feat(benchmarks): added locomo + doc files and misc fixes

Christian06e3cb1af044
diff --git a/AGENTS.md b/AGENTS.md
new file mode 100644
index 0000000..e69de29
diff --git a/CLAUDE.md b/CLAUDE.md
new file mode 100644
index 0000000..43c994c
--- /dev/null
+++ b/CLAUDE.md
@@ -0,0 +1 @@
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
new file mode 100644
index 0000000..e69de29
diff --git a/benchmarks/.env.example b/benchmarks/.env.example
new file mode 100644
index 0000000..24405da
--- /dev/null
+++ b/benchmarks/.env.example
@@ -0,0 +1,14 @@
+BRAINAPI_URL=http://localhost:8000
+BRAINPAT_TOKEN=
+
+# DeepSeek (OpenAI-compatible). Get a key at https://platform.deepseek.com/
+DEEPSEEK_API_KEY=
+BENCH_LLM_BASE_URL=https://api.deepseek.com
+BENCH_ANSWER_MODEL=deepseek-v4-flash
+BENCH_JUDGE_MODEL=deepseek-v4-flash
+
+# Optional aliases if you prefer another OpenAI-compatible provider:
+# OPENAI_API_KEY=
+# BENCH_LLM_BASE_URL=https://api.openai.com/v1
+# BENCH_ANSWER_MODEL=gpt-4o-mini
+# BENCH_JUDGE_MODEL=gpt-4o
diff --git a/benchmarks/.gitignore b/benchmarks/.gitignore
new file mode 100644
index 0000000..ee5ec2e
--- /dev/null
+++ b/benchmarks/.gitignore
@@ -0,0 +1,10 @@
+data/*
+!data/.gitkeep
+runs/*
+!runs/.gitkeep
+.venv/
+venv/
+__pycache__/
+*.pyc
+.env
+.DS_Store
diff --git a/benchmarks/AGENTS.md b/benchmarks/AGENTS.md
new file mode 100644
index 0000000..e69de29
diff --git a/benchmarks/CLAUDE.md b/benchmarks/CLAUDE.md
new file mode 100644
index 0000000..43c994c
--- /dev/null
+++ b/benchmarks/CLAUDE.md
@@ -0,0 +1 @@
diff --git a/benchmarks/README.md b/benchmarks/README.md
new file mode 100644
index 0000000..a4c06e2
--- /dev/null
+++ b/benchmarks/README.md
@@ -0,0 +1,147 @@
+# LoCoMo benchmark for BrainAPI
+
+Standalone harness that:
+
+1. Downloads the [LoCoMo](https://github.com/snap-research/LoCoMo) dataset
+2. Ingests conversations into BrainAPI over HTTP (`POST /ingest/`)
+3. Answers annotated QA using `POST /retrieve/context` + an LLM answerer
+4. Scores with an LLM judge plus F1 / BLEU-1
+
+This package does **not** import anything from `src/`. It only talks to a running BrainAPI instance.
+
+## Prerequisites
+
+- BrainAPI running locally (`make start-all` or `brainapi start`)
+- `BRAIN_CREATION_ALLOWED="true"` in the server env (needed so brains like `locomoconv26` are auto-created)
+- A PAT token (`BRAINPAT_TOKEN`)
+- A DeepSeek API key for answering and judging (or any OpenAI-compatible provider)
+
+Brain IDs must be alphanumeric. This harness maps `conv-26` → `locomoconv26`.
+
+## Setup
+
+```bash
+cd benchmarks
+python3 -m venv .venv
+source .venv/bin/activate
+# If `python` is aliased to a system binary (common on macOS), prefer the wrapper:
+# ./locomo.sh ...
+# or call the venv interpreter directly:
+# .venv/bin/python -m locomo ...
+pip install -r requirements.txt
+cp .env.example .env
+# edit .env: BRAINPAT_TOKEN, DEEPSEEK_API_KEY
+```
+
+## Quickstart
+
+```bash
+# 1. Download dataset
+./locomo.sh download
+./locomo.sh dataset-stats
+
+# 2. Smoke-test ingest + retrieve against your API
+./locomo.sh smoke --brain locomosmoke
+
+# 3. Ingest one conversation (session-level; recommended start)
+./locomo.sh ingest --sample conv-26 --run locomo-conv26
+
+# 4. Evaluate a small slice first
+./locomo.sh evaluate --sample conv-26 --run locomo-conv26 --limit 20
+
+# 5. Full non-adversarial QA for that conversation
+./locomo.sh evaluate --sample conv-26 --run locomo-conv26
+
+# 6. Rebuild the report anytime (no API calls)
+./locomo.sh report --run locomo-conv26
+```
+
+Equivalent: `.venv/bin/python -m locomo ...` (avoid bare `python` if you have a shell alias).
+
+Reports land in `runs/<run_id>/`:
+
+- `ingest.jsonl` — per-session ingest outcomes
+- `answers.jsonl` — per-QA predictions and scores
+- `manifest.json` — run config
+- `report.md` / `report.json` — aggregated metrics
+
+## Commands
+
+| Command | Purpose |
+|---|---|
+| `download` | Fetch `data/locomo10.json` |
+| `dataset-stats` | Print sessions / turns / QA per sample |
+| `smoke` | One-shot ingest + retrieve health check |
+| `ingest` | Push sessions (or turns) into BrainAPI |
+| `answer-once` | Retrieve + answer a single question |
+| `selftest-metrics` | Local F1 / BLEU-1 sanity checks |
+| `evaluate` | Retrieve → answer → judge for QA items |
+| `report` | Aggregate `answers.jsonl` into markdown/JSON |
+
+### Useful flags
+
+**ingest**
+
+- `--sample conv-26` (repeatable; default = all 10)
+- `--granularity session|turn` (default `session`)
+- `--concurrency 2`
+- `--limit-sessions N`
+- `--dry-run`
+- `--no-resume`
+- `--run <id>` (resume / continue a named run)
+
+**evaluate**
+
+- `--sample conv-26`
+- `--limit 20`
+- `--categories 1,2,4`
+- `--include-adversarial` (category 5 is skipped by default)
+- `--concurrency 2`
+- `--run <id>`
+
+## Scoring
+
+- **Headline number**: LLM-judge accuracy excluding category 5 (adversarial), matching common LoCoMo memory-system reporting.
+- Also reported: overall accuracy, per-category breakdown, mean F1, mean BLEU-1, retrieval latency p50/p95, total LLM tokens.
+- Categories: `1` multi-hop, `2` temporal, `3` open-domain, `4` single-hop, `5` adversarial.
+
+Defaults use **DeepSeek** (override in `.env`):
+
+- Base URL: `https://api.deepseek.com` (`BENCH_LLM_BASE_URL`)
+- Answerer + judge: `deepseek-v4-flash` (`BENCH_ANSWER_MODEL` / `BENCH_JUDGE_MODEL`)
+- API key: `DEEPSEEK_API_KEY` (or `OPENAI_API_KEY` for other providers)
+
+Other OpenAI-compatible providers work the same way — set `BENCH_LLM_BASE_URL`, model ids, and the key.
+
+## Cost and time expectations
+
+LoCoMo conversations are long (roughly 19–32 sessions and 370–690 turns each). With `PIPELINE_MODE=accurate`, ingesting even one conversation can take a long time and burn substantial LLM tokens inside BrainAPI.
+
+Recommendations:
+
+1. Start with `conv-26` only.
+2. Prefer `--granularity session` (default).
+3. Use `--limit-sessions` / `--limit` while iterating.
+4. Optionally compare a `PIPELINE_MODE=lightweight` ingest run for cost/quality tradeoffs.
+5. Do **not** ingest the `qa` annotations — that leaks the test set.
+
+Ingest and evaluate are resumable: re-running the same `--run` skips completed units / QA rows.
+
+## Adding another benchmark
+
+Create a sibling package under `benchmarks/`:
+
+```text
+benchmarks/
+ locomo/ # this package
+ yourbench/ # new package with its own cli / client / metrics
+ requirements.txt # shared deps, or a per-bench requirements file
+```
+
+Reuse patterns from `locomo/`:
+
+- HTTP-only `BrainAPIClient`
+- JSONL resume files under `runs/`
+- Separate ingest → evaluate → report stages
+
+Keep each benchmark self-contained and free of imports from BrainAPI's `src/`.
diff --git a/benchmarks/data/.gitkeep b/benchmarks/data/.gitkeep
new file mode 100644
index 0000000..e69de29
diff --git a/benchmarks/locomo.sh b/benchmarks/locomo.sh
new file mode 100755
index 0000000..1963159
--- /dev/null
+++ b/benchmarks/locomo.sh
@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+set -euo pipefail
+ROOT="$(cd "$(dirname "$0")" && pwd)"
+PYTHON="${ROOT}/.venv/bin/python"
+if [[ ! -x "$PYTHON" ]]; then
+ echo "Missing ${PYTHON}. Create it with:"
+ echo " cd benchmarks && python3 -m venv .venv && .venv/bin/pip install -r requirements.txt"
+ exit 1
+fi
+exec "$PYTHON" -m locomo "$@"
diff --git a/benchmarks/locomo/__init__.py b/benchmarks/locomo/__init__.py
new file mode 100644
index 0000000..5d04af0
--- /dev/null
+++ b/benchmarks/locomo/__init__.py
@@ -0,0 +1,3 @@
+"""LoCoMo benchmark harness for BrainAPI."""
+
+__version__ = "0.1.0"
diff --git a/benchmarks/locomo/__main__.py b/benchmarks/locomo/__main__.py
new file mode 100644
index 0000000..881fb03
--- /dev/null
+++ b/benchmarks/locomo/__main__.py
@@ -0,0 +1,4 @@
+from locomo.cli import main
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/benchmarks/locomo/answer.py b/benchmarks/locomo/answer.py
new file mode 100644
index 0000000..36d6764
--- /dev/null
+++ b/benchmarks/locomo/answer.py
@@ -0,0 +1,57 @@
+from __future__ import annotations
+
+import time
+from dataclasses import dataclass
+from typing import Any
+
+from openai import OpenAI
+
+from locomo.config import Settings
+from locomo.prompts import build_answer_messages
+
+
+@dataclass
+class AnswerResult:
+ answer: str
+ model: str
+ latency_ms: float
+ prompt_tokens: int | None
+ completion_tokens: int | None
+ total_tokens: int | None
+
+
+def _make_openai_client(settings: Settings) -> OpenAI:
+ settings.require_llm()
+ kwargs: dict[str, Any] = {"api_key": settings.openai_api_key}
+ if settings.llm_base_url:
+ kwargs["base_url"] = settings.llm_base_url
+ return OpenAI(**kwargs)
+
+
+def answer_question(
+ settings: Settings,
+ question: str,
+ context: dict[str, Any],
+ *,
+ model: str | None = None,
+) -> AnswerResult:
+ client = _make_openai_client(settings)
+ model_name = model or settings.answer_model
+ messages = build_answer_messages(question, context)
+ started = time.perf_counter()
+ response = client.chat.completions.create(
+ model=model_name,
+ messages=messages,
+ temperature=0,
+ )
+ latency_ms = (time.perf_counter() - started) * 1000
+ content = (response.choices[0].message.content or "").strip()
+ usage = response.usage
+ return AnswerResult(
+ answer=content,
+ model=model_name,
+ latency_ms=latency_ms,
+ prompt_tokens=getattr(usage, "prompt_tokens", None) if usage else None,
+ completion_tokens=getattr(usage, "completion_tokens", None) if usage else None,
+ total_tokens=getattr(usage, "total_tokens", None) if usage else None,
+ )
diff --git a/benchmarks/locomo/cli.py b/benchmarks/locomo/cli.py
new file mode 100644
index 0000000..9f3fcb8
--- /dev/null
+++ b/benchmarks/locomo/cli.py
@@ -0,0 +1,279 @@
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+
+from rich.console import Console
+from rich.table import Table
+
+from locomo.answer import answer_question
+from locomo.client import BrainAPIClient
+from locomo.config import Settings
+from locomo.dataset import (
+ brain_id_for,
+ dataset_stats,
+ download_dataset,
+ get_sample,
+ load_dataset,
+ resolve_samples,
+)
+from locomo.evaluate import evaluate_samples
+from locomo.ingest import ensure_run_dir, ingest_samples, write_manifest
+from locomo.metrics import selftest_metrics
+from locomo.report import print_report_table, write_report
+
+console = Console()
+
+
+def _parse_categories(raw: str | None) -> set[int] | None:
+ if not raw:
+ return None
+ return {int(part.strip()) for part in raw.split(",") if part.strip()}
+
+
+def cmd_download(args: argparse.Namespace, settings: Settings) -> int:
+ path = download_dataset(settings.dataset_path, settings.dataset_url)
+ console.print(f"[green]Dataset ready:[/green] {path}")
+ return 0
+
+
+def cmd_dataset_stats(args: argparse.Namespace, settings: Settings) -> int:
+ if not settings.dataset_path.exists():
+ download_dataset(settings.dataset_path, settings.dataset_url)
+ rows = dataset_stats(load_dataset(settings.dataset_path))
+ table = Table(title="LoCoMo dataset")
+ for col in ("sample_id", "brain_id", "speakers", "sessions", "turns", "qa"):
+ table.add_column(col)
+ for row in rows:
+ table.add_row(
+ str(row["sample_id"]),
+ str(row["brain_id"]),
+ f"{row['speaker_a']} & {row['speaker_b']}",
+ str(row["sessions"]),
+ str(row["turns"]),
+ str(row["qa"]),
+ )
+ console.print(table)
+ assert brain_id_for("conv-26") == "locomoconv26"
+ console.print("[dim]brain_id_for('conv-26') == 'locomoconv26' OK[/dim]")
+ return 0
+
+
+def cmd_smoke(args: argparse.Namespace, settings: Settings) -> int:
+ brain_id = args.brain
+ text = (
+ "Emily organized the AI Ethics Meetup in London on March 8, 2024. "
+ "Caroline attended an LGBTQ support group on 7 May 2023."
+ )
+ with BrainAPIClient(settings) as client:
+ submitted = client.ingest_text(text, brain_id)
+ task_id = (submitted.data or {}).get("task_id")
+ console.print(f"Queued ingest task_id={task_id}")
+ waited = client.wait_for_task(task_id, brain_id, timeout_s=args.timeout)
+ console.print(f"Task status: {(waited.data or {}).get('status')}")
+ ctx = client.retrieve_context(
+ "When did Caroline go to the LGBTQ support group?", brain_id
+ )
+ text_context = (ctx.data or {}).get("text_context") or ""
+ console.print(f"text_context ({len(text_context)} chars): {text_context[:400]}")
+ if not text_context.strip():
+ console.print("[yellow]Warning: empty text_context[/yellow]")
+ return 1
+ console.print("[green]Smoke test passed[/green]")
+ return 0
+
+
+def cmd_ingest(args: argparse.Namespace, settings: Settings) -> int:
+ sample_ids = args.sample or None
+ samples = resolve_samples(settings, sample_ids)
+ run_id, run_dir = ensure_run_dir(settings, args.run)
+ write_manifest(
+ run_dir,
+ {
+ "run_id": run_id,
+ "command": "ingest",
+ "samples": [s["sample_id"] for s in samples],
+ "granularity": args.granularity,
+ "concurrency": args.concurrency,
+ "limit_sessions": args.limit_sessions,
+ "dry_run": args.dry_run,
+ "brainapi_url": settings.brainapi_url,
+ "answer_model": settings.answer_model,
+ "judge_model": settings.judge_model,
+ },
+ )
+ ingest_samples(
+ settings,
+ samples,
+ run_dir=run_dir,
+ granularity=args.granularity,
+ concurrency=args.concurrency,
+ limit_sessions=args.limit_sessions,
+ dry_run=args.dry_run,
+ resume=not args.no_resume,
+ task_timeout_s=args.timeout,
+ )
+ console.print(f"[green]Run directory:[/green] {run_dir}")
+ return 0
+
+
+def cmd_answer_once(args: argparse.Namespace, settings: Settings) -> int:
+ sample = get_sample(load_dataset(settings.dataset_path), args.sample)
+ brain_id = brain_id_for(str(sample["sample_id"]))
+ question = args.question
+ if not question:
+ qa = (sample.get("qa") or [None])[0]
+ if not qa:
+ raise SystemExit("No question provided and sample has no QA")
+ question = qa["question"]
+ console.print(f"[dim]Using first QA: {question}[/dim]")
+
+ with BrainAPIClient(settings) as client:
+ retrieved = client.retrieve_context(question, brain_id)
+ result = answer_question(settings, question, retrieved.data or {})
+ console.print(result.answer)
+ console.print(
+ f"[dim]model={result.model} latency_ms={result.latency_ms:.0f} "
+ f"tokens={result.total_tokens}[/dim]"
+ )
+ return 0
+
+
+def cmd_selftest_metrics(args: argparse.Namespace, settings: Settings) -> int:
+ errors = selftest_metrics()
+ if errors:
+ for err in errors:
+ console.print(f"[red]FAIL[/red] {err}")
+ return 1
+ console.print("[green]selftest-metrics passed[/green]")
+ return 0
+
+
+def cmd_evaluate(args: argparse.Namespace, settings: Settings) -> int:
+ sample_ids = args.sample or None
+ samples = resolve_samples(settings, sample_ids)
+ run_id, run_dir = ensure_run_dir(settings, args.run)
+ categories = _parse_categories(args.categories)
+ write_manifest(
+ run_dir,
+ {
+ "run_id": run_id,
+ "command": "evaluate",
+ "samples": [s["sample_id"] for s in samples],
+ "concurrency": args.concurrency,
+ "skip_adversarial": not args.include_adversarial,
+ "categories": sorted(categories) if categories else None,
+ "limit": args.limit,
+ "brainapi_url": settings.brainapi_url,
+ "answer_model": settings.answer_model,
+ "judge_model": settings.judge_model,
+ "historical_limit": args.historical_limit,
+ },
+ )
+ evaluate_samples(
+ settings,
+ samples,
+ run_dir=run_dir,
+ concurrency=args.concurrency,
+ categories=categories,
+ skip_adversarial=not args.include_adversarial,
+ limit=args.limit,
+ resume=not args.no_resume,
+ historical_limit=args.historical_limit,
+ )
+ report = write_report(run_dir)
+ print_report_table(report)
+ console.print(f"[green]Report:[/green] {run_dir / 'report.md'}")
+ return 0
+
+
+def cmd_report(args: argparse.Namespace, settings: Settings) -> int:
+ run_dir = settings.runs_dir / args.run
+ if not run_dir.exists():
+ raise SystemExit(f"Run directory not found: {run_dir}")
+ report = write_report(run_dir)
+ print_report_table(report)
+ console.print(report["run_dir"] + "/report.md")
+ if args.json:
+ console.print_json(json.dumps(report))
+ return 0
+
+
+def build_parser() -> argparse.ArgumentParser:
+ parser = argparse.ArgumentParser(
+ prog="python -m locomo",
+ description="LoCoMo benchmark harness for BrainAPI",
+ )
+ parser.add_argument(
+ "--env-file",
+ type=Path,
+ default=None,
+ help="Optional path to .env (defaults to benchmarks/.env)",
+ )
+ sub = parser.add_subparsers(dest="command", required=True)
+
+ p_download = sub.add_parser("download", help="Download locomo10.json")
+ p_download.set_defaults(func=cmd_download)
+
+ p_stats = sub.add_parser("dataset-stats", help="Print dataset statistics")
+ p_stats.set_defaults(func=cmd_dataset_stats)
+
+ p_smoke = sub.add_parser("smoke", help="Smoke-test ingest + retrieve")
+ p_smoke.add_argument("--brain", default="locomosmoke")
+ p_smoke.add_argument("--timeout", type=float, default=600.0)
+ p_smoke.set_defaults(func=cmd_smoke)
+
+ p_ingest = sub.add_parser("ingest", help="Ingest LoCoMo conversations")
+ p_ingest.add_argument(
+ "--sample",
+ action="append",
+ help="Sample id (repeatable). Default: all samples.",
+ )
+ p_ingest.add_argument("--run", default=None, help="Existing or new run id")
+ p_ingest.add_argument(
+ "--granularity", choices=("session", "turn"), default="session"
+ )
+ p_ingest.add_argument("--concurrency", type=int, default=2)
+ p_ingest.add_argument("--limit-sessions", type=int, default=None)
+ p_ingest.add_argument("--dry-run", action="store_true")
+ p_ingest.add_argument("--no-resume", action="store_true")
+ p_ingest.add_argument("--timeout", type=float, default=900.0)
+ p_ingest.set_defaults(func=cmd_ingest)
+
+ p_answer = sub.add_parser("answer-once", help="Retrieve + answer one question")
+ p_answer.add_argument("--sample", default="conv-26")
+ p_answer.add_argument("--question", default=None)
+ p_answer.set_defaults(func=cmd_answer_once)
+
+ p_self = sub.add_parser("selftest-metrics", help="Run local metrics self-test")
+ p_self.set_defaults(func=cmd_selftest_metrics)
+
+ p_eval = sub.add_parser("evaluate", help="Answer and score LoCoMo QA")
+ p_eval.add_argument("--sample", action="append")
+ p_eval.add_argument("--run", default=None)
+ p_eval.add_argument("--concurrency", type=int, default=2)
+ p_eval.add_argument("--categories", default=None, help="e.g. 1,2,4")
+ p_eval.add_argument("--limit", type=int, default=None)
+ p_eval.add_argument(
+ "--include-adversarial",
+ action="store_true",
+ help="Include category 5 questions",
+ )
+ p_eval.add_argument("--no-resume", action="store_true")
+ p_eval.add_argument("--historical-limit", type=int, default=10)
+ p_eval.set_defaults(func=cmd_evaluate)
+
+ p_report = sub.add_parser("report", help="Rebuild report from a run directory")
+ p_report.add_argument("--run", required=True)
+ p_report.add_argument("--json", action="store_true")
+ p_report.set_defaults(func=cmd_report)
+
+ return parser
+
+
+def main(argv: list[str] | None = None) -> int:
+ parser = build_parser()
+ args = parser.parse_args(argv)
+ settings = Settings.load(args.env_file)
+ return args.func(args, settings)
diff --git a/benchmarks/locomo/client.py b/benchmarks/locomo/client.py
new file mode 100644
index 0000000..1715bb7
--- /dev/null
+++ b/benchmarks/locomo/client.py
@@ -0,0 +1,157 @@
+from __future__ import annotations
+
+import time
+from dataclasses import dataclass, field
+from typing import Any
+
+import httpx
+
+from locomo.config import TERMINAL_TASK_STATUSES, Settings
+
+
+@dataclass
+class TimedResult:
+ data: Any
+ latency_ms: float
+
+
+@dataclass
+class BrainAPIClient:
+ settings: Settings
+ timeout: float = 120.0
+ max_retries: int = 5
+ _client: httpx.Client = field(init=False, repr=False)
+
+ def __post_init__(self) -> None:
+ self.settings.require_brainapi()
+ self._client = httpx.Client(
+ base_url=self.settings.brainapi_url,
+ headers={
+ "Authorization": f"Bearer {self.settings.brainpat_token}",
+ "Content-Type": "application/json",
+ },
+ timeout=self.timeout,
+ )
+
+ def close(self) -> None:
+ self._client.close()
+
+ def __enter__(self) -> "BrainAPIClient":
+ return self
+
+ def __exit__(self, *args: object) -> None:
+ self.close()
+
+ def _request(
+ self,
+ method: str,
+ path: str,
+ *,
+ json: dict | None = None,
+ params: dict | None = None,
+ ) -> TimedResult:
+ last_error: Exception | None = None
+ for attempt in range(self.max_retries):
+ started = time.perf_counter()
+ try:
+ response = self._client.request(
+ method, path, json=json, params=params
+ )
+ latency_ms = (time.perf_counter() - started) * 1000
+ if response.status_code in (429, 500, 502, 503, 504):
+ wait = min(2**attempt * 0.5, 30)
+ time.sleep(wait)
+ last_error = httpx.HTTPStatusError(
+ f"HTTP {response.status_code}",
+ request=response.request,
+ response=response,
+ )
+ continue
+ response.raise_for_status()
+ if response.status_code == 204 or not response.content:
+ return TimedResult(data=None, latency_ms=latency_ms)
+ return TimedResult(data=response.json(), latency_ms=latency_ms)
+ except (httpx.TransportError, httpx.TimeoutException) as exc:
+ last_error = exc
+ wait = min(2**attempt * 0.5, 30)
+ time.sleep(wait)
+ raise RuntimeError(f"Request failed after retries: {method} {path}: {last_error}")
+
+ def ingest_text(
+ self,
+ text: str,
+ brain_id: str,
+ *,
+ observate_for: list[str] | None = None,
+ preferred_extraction_entities: list[str] | None = None,
+ ) -> TimedResult:
+ body = {
+ "data": {"data_type": "text", "text_data": text},
+ "brain_id": brain_id,
+ "observate_for": observate_for
+ or [
+ "facts about each speaker",
+ "events and when they happened",
+ "relationships between people",
+ ],
+ "preferred_extraction_entities": preferred_extraction_entities
+ or ["Person", "Event", "Location", "Date"],
+ }
+ return self._request("POST", "/ingest/", json=body)
+
+ def get_task(self, task_id: str, brain_id: str) -> TimedResult:
+ return self._request(
+ "GET", f"/tasks/{task_id}", params={"brain_id": brain_id}
+ )
+
+ def wait_for_task(
+ self,
+ task_id: str,
+ brain_id: str,
+ *,
+ timeout_s: float = 600.0,
+ poll_interval_s: float = 2.0,
+ ) -> TimedResult:
+ started = time.perf_counter()
+ while True:
+ result = self.get_task(task_id, brain_id)
+ status = (result.data or {}).get("status", "unknown")
+ if status in TERMINAL_TASK_STATUSES:
+ return TimedResult(
+ data=result.data,
+ latency_ms=(time.perf_counter() - started) * 1000,
+ )
+ if time.perf_counter() - started > timeout_s:
+ raise TimeoutError(
+ f"Task {task_id} did not finish within {timeout_s}s "
+ f"(last status={status})"
+ )
+ time.sleep(poll_interval_s)
+
+ def retrieve_context(
+ self,
+ text: str,
+ brain_id: str,
+ *,
+ historical_limit: int = 10,
+ ) -> TimedResult:
+ body = {
+ "text": text,
+ "brain_id": brain_id,
+ "historical_limit": historical_limit,
+ }
+ return self._request("POST", "/retrieve/context", json=body)
+
+ def entity_status(self, target: str, brain_id: str) -> TimedResult:
+ return self._request(
+ "GET",
+ "/retrieve/entity/status",
+ params={"target": target, "brain_id": brain_id},
+ )
+
+ def list_entities(self, brain_id: str, *, limit: int = 5) -> TimedResult:
+ return self._request(
+ "GET",
+ "/retrieve/entities",
+ params={"brain_id": brain_id, "limit": limit},
+ )
diff --git a/benchmarks/locomo/config.py b/benchmarks/locomo/config.py
new file mode 100644
index 0000000..d2f39cf
--- /dev/null
+++ b/benchmarks/locomo/config.py
@@ -0,0 +1,82 @@
+from __future__ import annotations
+
+import os
+from dataclasses import dataclass
+from pathlib import Path
+
+from dotenv import load_dotenv
+
+BENCHMARKS_ROOT = Path(__file__).resolve().parent.parent
+DATA_DIR = BENCHMARKS_ROOT / "data"
+RUNS_DIR = BENCHMARKS_ROOT / "runs"
+DEFAULT_DATASET_URL = (
+ "https://raw.githubusercontent.com/snap-research/locomo/main/data/locomo10.json"
+)
+DEFAULT_DATASET_PATH = DATA_DIR / "locomo10.json"
+DEFAULT_LLM_BASE_URL = "https://api.deepseek.com"
+DEFAULT_ANSWER_MODEL = "deepseek-v4-flash"
+DEFAULT_JUDGE_MODEL = "deepseek-v4-flash"
+
+TERMINAL_TASK_STATUSES = frozenset({"completed", "failed", "partial_failed"})
+CATEGORY_NAMES = {
+ 1: "multi-hop",
+ 2: "temporal",
+ 3: "open-domain",
+ 4: "single-hop",
+ 5: "adversarial",
+}
+
+
+@dataclass
+class Settings:
+ brainapi_url: str
+ brainpat_token: str
+ openai_api_key: str
+ llm_base_url: str | None
+ answer_model: str
+ judge_model: str
+ dataset_path: Path
+ dataset_url: str
+ runs_dir: Path
+
+ @classmethod
+ def load(cls, env_file: Path | None = None) -> "Settings":
+ if env_file is None:
+ env_file = BENCHMARKS_ROOT / ".env"
+ if env_file.exists():
+ load_dotenv(env_file, override=False)
+ else:
+ load_dotenv(override=False)
+
+ api_key = (
+ os.getenv("DEEPSEEK_API_KEY")
+ or os.getenv("OPENAI_API_KEY")
+ or ""
+ )
+
+ return cls(
+ brainapi_url=os.getenv("BRAINAPI_URL", "http://localhost:8000").rstrip("/"),
+ brainpat_token=os.getenv("BRAINPAT_TOKEN", ""),
+ openai_api_key=api_key,
+ llm_base_url=os.getenv("BENCH_LLM_BASE_URL", DEFAULT_LLM_BASE_URL) or None,
+ answer_model=os.getenv("BENCH_ANSWER_MODEL", DEFAULT_ANSWER_MODEL),
+ judge_model=os.getenv("BENCH_JUDGE_MODEL", DEFAULT_JUDGE_MODEL),
+ dataset_path=Path(
+ os.getenv("BENCH_DATASET_PATH", str(DEFAULT_DATASET_PATH))
+ ),
+ dataset_url=os.getenv("BENCH_DATASET_URL", DEFAULT_DATASET_URL),
+ runs_dir=Path(os.getenv("BENCH_RUNS_DIR", str(RUNS_DIR))),
+ )
+
+ def require_brainapi(self) -> None:
+ if not self.brainpat_token:
+ raise SystemExit(
+ "BRAINPAT_TOKEN is required. Copy .env.example to .env and set it."
+ )
+
+ def require_llm(self) -> None:
+ if not self.openai_api_key:
+ raise SystemExit(
+ "DEEPSEEK_API_KEY (or OPENAI_API_KEY) is required for answer/judge "
+ "LLM calls. Copy .env.example to .env and set it."
+ )
diff --git a/benchmarks/locomo/dataset.py b/benchmarks/locomo/dataset.py
new file mode 100644
index 0000000..2360899
--- /dev/null
+++ b/benchmarks/locomo/dataset.py
@@ -0,0 +1,181 @@
+from __future__ import annotations
+
+import json
+import re
+from pathlib import Path
+from typing import Any
+from urllib.request import urlretrieve
+
+from locomo.config import DEFAULT_DATASET_PATH, DEFAULT_DATASET_URL, Settings
+
+
+def brain_id_for(sample_id: str) -> str:
+ slug = re.sub(r"[^a-zA-Z0-9]", "", f"locomo{sample_id}")
+ if not slug.isalnum():
+ raise ValueError(f"Could not derive alphanumeric brain_id from {sample_id!r}")
+ return slug.lower()
+
+
+def download_dataset(
+ dest: Path | None = None,
+ url: str = DEFAULT_DATASET_URL,
+) -> Path:
+ dest = dest or DEFAULT_DATASET_PATH
+ dest.parent.mkdir(parents=True, exist_ok=True)
+ if dest.exists() and dest.stat().st_size > 0:
+ return dest
+ urlretrieve(url, dest)
+ return dest
+
+
+def load_dataset(path: Path | None = None) -> list[dict[str, Any]]:
+ path = path or DEFAULT_DATASET_PATH
+ if not path.exists():
+ raise FileNotFoundError(
+ f"Dataset not found at {path}. Run: python -m locomo download"
+ )
+ with path.open(encoding="utf-8") as f:
+ data = json.load(f)
+ if not isinstance(data, list):
+ raise ValueError("Expected locomo10.json to be a JSON array")
+ return data
+
+
+def get_sample(dataset: list[dict[str, Any]], sample_id: str) -> dict[str, Any]:
+ for sample in dataset:
+ if sample.get("sample_id") == sample_id:
+ return sample
+ raise KeyError(f"sample_id {sample_id!r} not found")
+
+
+def session_keys(conversation: dict[str, Any]) -> list[str]:
+ keys = [
+ k
+ for k, v in conversation.items()
+ if k.startswith("session_")
+ and not k.endswith("date_time")
+ and isinstance(v, list)
+ ]
+
+ def sort_key(name: str) -> int:
+ try:
+ return int(name.split("_", 1)[1])
+ except (IndexError, ValueError):
+ return 0
+
+ return sorted(keys, key=sort_key)
+
+
+def format_turn(turn: dict[str, Any]) -> str:
+ text = (turn.get("text") or "").strip()
+ caption = turn.get("blip_caption")
+ if caption:
+ text = f"{text} [image: {caption}]".strip()
+ speaker = turn.get("speaker") or "Unknown"
+ dia_id = turn.get("dia_id") or "?"
+ return f"{speaker} ({dia_id}): {text}"
+
+
+def format_session(conversation: dict[str, Any], session_key: str) -> str:
+ when = conversation.get(f"{session_key}_date_time", "unknown time")
+ speaker_a = conversation.get("speaker_a", "Speaker A")
+ speaker_b = conversation.get("speaker_b", "Speaker B")
+ turns = conversation.get(session_key) or []
+ lines = [
+ f"Conversation between {speaker_a} and {speaker_b}.",
+ f"Session time: {when}.",
+ f"Session id: {session_key}.",
+ "",
+ ]
+ for turn in turns:
+ if isinstance(turn, dict):
+ lines.append(format_turn(turn))
+ return "\n".join(lines)
+
+
+def format_turn_unit(
+ conversation: dict[str, Any], session_key: str, turn: dict[str, Any]
+) -> str:
+ when = conversation.get(f"{session_key}_date_time", "unknown time")
+ speaker_a = conversation.get("speaker_a", "Speaker A")
+ speaker_b = conversation.get("speaker_b", "Speaker B")
+ return "\n".join(
+ [
+ f"Conversation between {speaker_a} and {speaker_b}.",
+ f"Session time: {when}.",
+ f"Session id: {session_key}.",
+ "",
+ format_turn(turn),
+ ]
+ )
+
+
+def iter_ingest_units(
+ sample: dict[str, Any],
+ granularity: str = "session",
+ limit_sessions: int | None = None,
+) -> list[dict[str, Any]]:
+ conversation = sample["conversation"]
+ keys = session_keys(conversation)
+ if limit_sessions is not None:
+ keys = keys[:limit_sessions]
+
+ units: list[dict[str, Any]] = []
+ if granularity == "session":
+ for key in keys:
+ units.append(
+ {
+ "unit_id": key,
+ "session_key": key,
+ "text": format_session(conversation, key),
+ }
+ )
+ return units
+
+ if granularity == "turn":
+ for key in keys:
+ for idx, turn in enumerate(conversation.get(key) or []):
+ if not isinstance(turn, dict):
+ continue
+ dia_id = turn.get("dia_id") or f"{key}:{idx}"
+ units.append(
+ {
+ "unit_id": f"{key}:{dia_id}",
+ "session_key": key,
+ "dia_id": dia_id,
+ "text": format_turn_unit(conversation, key, turn),
+ }
+ )
+ return units
+
+ raise ValueError("granularity must be 'session' or 'turn'")
+
+
+def dataset_stats(dataset: list[dict[str, Any]]) -> list[dict[str, Any]]:
+ rows = []
+ for sample in dataset:
+ conversation = sample.get("conversation") or {}
+ keys = session_keys(conversation)
+ turns = sum(len(conversation.get(k) or []) for k in keys)
+ rows.append(
+ {
+ "sample_id": sample.get("sample_id"),
+ "brain_id": brain_id_for(str(sample.get("sample_id"))),
+ "speaker_a": conversation.get("speaker_a"),
+ "speaker_b": conversation.get("speaker_b"),
+ "sessions": len(keys),
+ "turns": turns,
+ "qa": len(sample.get("qa") or []),
+ }
+ )
+ return rows
+
+
+def resolve_samples(
+ settings: Settings,
+ sample_ids: list[str] | None = None,
+) -> list[dict[str, Any]]:
+ dataset = load_dataset(settings.dataset_path)
+ if not sample_ids:
+ return dataset
+ return [get_sample(dataset, sid) for sid in sample_ids]
diff --git a/benchmarks/locomo/evaluate.py b/benchmarks/locomo/evaluate.py
new file mode 100644
index 0000000..6bbfb1e
--- /dev/null
+++ b/benchmarks/locomo/evaluate.py
@@ -0,0 +1,226 @@
+from __future__ import annotations
+
+import threading
+from concurrent.futures import ThreadPoolExecutor, as_completed
+from dataclasses import asdict, dataclass
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any
+
+from rich.console import Console
+from rich.progress import Progress
+
+from locomo.answer import answer_question
+from locomo.client import BrainAPIClient
+from locomo.config import Settings
+from locomo.dataset import brain_id_for
+from locomo.ingest import append_jsonl, load_jsonl
+from locomo.judge import judge_answer
+from locomo.metrics import bleu1_score, f1_score
+
+console = Console()
+_write_lock = threading.Lock()
+
+
+@dataclass
+class AnswerRecord:
+ sample_id: str
+ brain_id: str
+ qa_index: int
+ question: str
+ gold: str
+ prediction: str
+ category: int
+ evidence: list[str]
+ judge_correct: bool
+ judge_reason: str
+ judge_raw: str
+ f1: float
+ bleu1: float
+ retrieve_latency_ms: float | None
+ answer_latency_ms: float | None
+ judge_latency_ms: float | None
+ answer_model: str | None
+ judge_model: str | None
+ answer_total_tokens: int | None
+ judge_total_tokens: int | None
+ text_context: str
+ error: str | None
+ timestamp: str
+
+
+def _utc_now() -> str:
+ return datetime.now(timezone.utc).isoformat()
+
+
+def qa_key(sample_id: str, qa_index: int) -> str:
+ return f"{sample_id}::{qa_index}"
+
+
+def completed_qa_keys(answers_path: Path) -> set[str]:
+ done = set()
+ for row in load_jsonl(answers_path):
+ if row.get("error"):
+ continue
+ if row.get("prediction") is None:
+ continue
+ done.add(qa_key(str(row.get("sample_id")), int(row.get("qa_index"))))
+ return done
+
+
+def iter_qa_jobs(
+ samples: list[dict[str, Any]],
+ *,
+ categories: set[int] | None = None,
+ skip_adversarial: bool = True,
+ limit: int | None = None,
+) -> list[dict[str, Any]]:
+ jobs: list[dict[str, Any]] = []
+ for sample in samples:
+ sample_id = str(sample["sample_id"])
+ brain_id = brain_id_for(sample_id)
+ for idx, qa in enumerate(sample.get("qa") or []):
+ cat = int(qa.get("category") or 0)
+ if skip_adversarial and cat == 5:
+ continue
+ if categories is not None and cat not in categories:
+ continue
+ jobs.append(
+ {
+ "sample_id": sample_id,
+ "brain_id": brain_id,
+ "qa_index": idx,
+ "qa": qa,
+ }
+ )
+ if limit is not None and len(jobs) >= limit:
+ return jobs
+ return jobs
+
+
+def evaluate_samples(
+ settings: Settings,
+ samples: list[dict[str, Any]],
+ *,
+ run_dir: Path,
+ concurrency: int = 2,
+ categories: set[int] | None = None,
+ skip_adversarial: bool = True,
+ limit: int | None = None,
+ resume: bool = True,
+ historical_limit: int = 10,
+) -> list[AnswerRecord]:
+ run_dir.mkdir(parents=True, exist_ok=True)
+ answers_path = run_dir / "answers.jsonl"
+ already_done = completed_qa_keys(answers_path) if resume else set()
+
+ jobs = iter_qa_jobs(
+ samples,
+ categories=categories,
+ skip_adversarial=skip_adversarial,
+ limit=limit,
+ )
+ jobs = [
+ j
+ for j in jobs
+ if qa_key(j["sample_id"], j["qa_index"]) not in already_done
+ ]
+
+ if not jobs:
+ console.print("[green]Nothing to evaluate (all QA already scored).[/green]")
+ return []
+
+ console.print(
+ f"Evaluating {len(jobs)} QA items "
+ f"(concurrency={concurrency}, skip_adversarial={skip_adversarial})"
+ )
+
+ records: list[AnswerRecord] = []
+
+ def _one(job: dict[str, Any]) -> AnswerRecord:
+ qa = job["qa"]
+ question = str(qa.get("question") or "")
+ gold = str(qa.get("answer") if qa.get("answer") is not None else "")
+ category = int(qa.get("category") or 0)
+ evidence = list(qa.get("evidence") or [])
+
+ try:
+ with BrainAPIClient(settings) as client:
+ retrieved = client.retrieve_context(
+ question,
+ job["brain_id"],
+ historical_limit=historical_limit,
+ )
+ context = retrieved.data or {}
+ answered = answer_question(settings, question, context)
+ judged = judge_answer(
+ settings, question, gold, answered.answer
+ )
+ record = AnswerRecord(
+ sample_id=job["sample_id"],
+ brain_id=job["brain_id"],
+ qa_index=job["qa_index"],
+ question=question,
+ gold=gold,
+ prediction=answered.answer,
+ category=category,
+ evidence=evidence,
+ judge_correct=judged.correct,
+ judge_reason=judged.reason,
+ judge_raw=judged.raw,
+ f1=f1_score(answered.answer, gold),
+ bleu1=bleu1_score(answered.answer, gold),
+ retrieve_latency_ms=retrieved.latency_ms,
+ answer_latency_ms=answered.latency_ms,
+ judge_latency_ms=judged.latency_ms,
+ answer_model=answered.model,
+ judge_model=judged.model,
+ answer_total_tokens=answered.total_tokens,
+ judge_total_tokens=judged.total_tokens,
+ text_context=str(context.get("text_context") or "")[:4000],
+ error=None,
+ timestamp=_utc_now(),
+ )
+ except Exception as exc:
+ record = AnswerRecord(
+ sample_id=job["sample_id"],
+ brain_id=job["brain_id"],
+ qa_index=job["qa_index"],
+ question=question,
+ gold=gold,
+ prediction="",
+ category=category,
+ evidence=evidence,
+ judge_correct=False,
+ judge_reason="",
+ judge_raw="",
+ f1=0.0,
+ bleu1=0.0,
+ retrieve_latency_ms=None,
+ answer_latency_ms=None,
+ judge_latency_ms=None,
+ answer_model=None,
+ judge_model=None,
+ answer_total_tokens=None,
+ judge_total_tokens=None,
+ text_context="",
+ error=str(exc),
+ timestamp=_utc_now(),
+ )
+
+ with _write_lock:
+ append_jsonl(answers_path, asdict(record))
+ return record
+
+ with Progress(console=console) as progress:
+ task = progress.add_task("evaluate", total=len(jobs))
+ with ThreadPoolExecutor(max_workers=max(1, concurrency)) as pool:
+ futures = [pool.submit(_one, job) for job in jobs]
+ for future in as_completed(futures):
+ records.append(future.result())
+ progress.advance(task)
+
+ ok = sum(1 for r in records if not r.error)
+ failed = sum(1 for r in records if r.error)
+ console.print(f"Evaluate done: ok={ok} failed={failed} -> {answers_path}")
+ return records
diff --git a/benchmarks/locomo/ingest.py b/benchmarks/locomo/ingest.py
new file mode 100644
index 0000000..0de2870
--- /dev/null
+++ b/benchmarks/locomo/ingest.py
@@ -0,0 +1,215 @@
+from __future__ import annotations
+
+import json
+import threading
+from concurrent.futures import ThreadPoolExecutor, as_completed
+from dataclasses import asdict, dataclass
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any
+
+from rich.console import Console
+from rich.progress import Progress
+
+from locomo.client import BrainAPIClient
+from locomo.config import Settings
+from locomo.dataset import brain_id_for, iter_ingest_units
+
+console = Console()
+_write_lock = threading.Lock()
+
+
+@dataclass
+class IngestRecord:
+ sample_id: str
+ brain_id: str
+ unit_id: str
+ session_key: str
+ task_id: str | None
+ status: str
+ error: str | None
+ submit_latency_ms: float | None
+ wait_latency_ms: float | None
+ granularity: str
+ dry_run: bool
+ timestamp: str
+
+
+def _utc_now() -> str:
+ return datetime.now(timezone.utc).isoformat()
+
+
+def load_jsonl(path: Path) -> list[dict[str, Any]]:
+ if not path.exists():
+ return []
+ rows = []
+ with path.open(encoding="utf-8") as f:
+ for line in f:
+ line = line.strip()
+ if line:
+ rows.append(json.loads(line))
+ return rows
+
+
+def append_jsonl(path: Path, row: dict[str, Any]) -> None:
+ path.parent.mkdir(parents=True, exist_ok=True)
+ with _write_lock:
+ with path.open("a", encoding="utf-8") as f:
+ f.write(json.dumps(row, ensure_ascii=False) + "\n")
+
+
+def completed_unit_ids(ingest_path: Path) -> set[str]:
+ done = set()
+ for row in load_jsonl(ingest_path):
+ status = row.get("status")
+ if status in {"completed", "partial_failed", "dry_run"}:
+ key = f"{row.get('sample_id')}::{row.get('unit_id')}"
+ done.add(key)
+ return done
+
+
+def ingest_samples(
+ settings: Settings,
+ samples: list[dict[str, Any]],
+ *,
+ run_dir: Path,
+ granularity: str = "session",
+ concurrency: int = 2,
+ limit_sessions: int | None = None,
+ dry_run: bool = False,
+ resume: bool = True,
+ task_timeout_s: float = 900.0,
+) -> list[IngestRecord]:
+ run_dir.mkdir(parents=True, exist_ok=True)
+ ingest_path = run_dir / "ingest.jsonl"
+ already_done = completed_unit_ids(ingest_path) if resume else set()
+
+ jobs: list[dict[str, Any]] = []
+ for sample in samples:
+ sample_id = str(sample["sample_id"])
+ brain_id = brain_id_for(sample_id)
+ for unit in iter_ingest_units(
+ sample, granularity=granularity, limit_sessions=limit_sessions
+ ):
+ key = f"{sample_id}::{unit['unit_id']}"
+ if key in already_done:
+ continue
+ jobs.append(
+ {
+ "sample_id": sample_id,
+ "brain_id": brain_id,
+ "unit": unit,
+ }
+ )
+
+ if not jobs:
+ console.print("[green]Nothing to ingest (all units already completed).[/green]")
+ return []
+
+ console.print(
+ f"Ingesting {len(jobs)} units "
+ f"(granularity={granularity}, concurrency={concurrency}, dry_run={dry_run})"
+ )
+
+ records: list[IngestRecord] = []
+
+ def _one(job: dict[str, Any]) -> IngestRecord:
+ unit = job["unit"]
+ if dry_run:
+ record = IngestRecord(
+ sample_id=job["sample_id"],
+ brain_id=job["brain_id"],
+ unit_id=unit["unit_id"],
+ session_key=unit["session_key"],
+ task_id=None,
+ status="dry_run",
+ error=None,
+ submit_latency_ms=None,
+ wait_latency_ms=None,
+ granularity=granularity,
+ dry_run=True,
+ timestamp=_utc_now(),
+ )
+ append_jsonl(ingest_path, asdict(record))
+ return record
+
+ with BrainAPIClient(settings) as client:
+ try:
+ submitted = client.ingest_text(unit["text"], job["brain_id"])
+ task_id = (submitted.data or {}).get("task_id")
+ if not task_id:
+ raise RuntimeError(f"No task_id in ingest response: {submitted.data}")
+ waited = client.wait_for_task(
+ task_id, job["brain_id"], timeout_s=task_timeout_s
+ )
+ status = (waited.data or {}).get("status", "unknown")
+ error = (waited.data or {}).get("error")
+ record = IngestRecord(
+ sample_id=job["sample_id"],
+ brain_id=job["brain_id"],
+ unit_id=unit["unit_id"],
+ session_key=unit["session_key"],
+ task_id=task_id,
+ status=status,
+ error=error,
+ submit_latency_ms=submitted.latency_ms,
+ wait_latency_ms=waited.latency_ms,
+ granularity=granularity,
+ dry_run=False,
+ timestamp=_utc_now(),
+ )
+ except Exception as exc:
+ record = IngestRecord(
+ sample_id=job["sample_id"],
+ brain_id=job["brain_id"],
+ unit_id=unit["unit_id"],
+ session_key=unit["session_key"],
+ task_id=None,
+ status="failed",
+ error=str(exc),
+ submit_latency_ms=None,
+ wait_latency_ms=None,
+ granularity=granularity,
+ dry_run=False,
+ timestamp=_utc_now(),
+ )
+ append_jsonl(ingest_path, asdict(record))
+ return record
+
+ with Progress(console=console) as progress:
+ task = progress.add_task("ingest", total=len(jobs))
+ with ThreadPoolExecutor(max_workers=max(1, concurrency)) as pool:
+ futures = [pool.submit(_one, job) for job in jobs]
+ for future in as_completed(futures):
+ records.append(future.result())
+ progress.advance(task)
+
+ ok = sum(1 for r in records if r.status in {"completed", "dry_run"})
+ partial = sum(1 for r in records if r.status == "partial_failed")
+ failed = sum(1 for r in records if r.status == "failed")
+ console.print(
+ f"Ingest done: ok={ok} partial_failed={partial} failed={failed} "
+ f"-> {ingest_path}"
+ )
+ return records
+
+
+def ensure_run_dir(settings: Settings, run_id: str | None = None) -> tuple[str, Path]:
+ settings.runs_dir.mkdir(parents=True, exist_ok=True)
+ if not run_id:
+ run_id = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
+ run_dir = settings.runs_dir / run_id
+ run_dir.mkdir(parents=True, exist_ok=True)
+ return run_id, run_dir
+
+
+def write_manifest(run_dir: Path, manifest: dict[str, Any]) -> None:
+ path = run_dir / "manifest.json"
+ existing: dict[str, Any] = {}
+ if path.exists():
+ existing = json.loads(path.read_text(encoding="utf-8"))
+ existing.update(manifest)
+ existing["updated_at"] = _utc_now()
+ if "created_at" not in existing:
+ existing["created_at"] = _utc_now()
+ path.write_text(json.dumps(existing, indent=2), encoding="utf-8")
diff --git a/benchmarks/locomo/judge.py b/benchmarks/locomo/judge.py
new file mode 100644
index 0000000..2fead6d
--- /dev/null
+++ b/benchmarks/locomo/judge.py
@@ -0,0 +1,87 @@
+from __future__ import annotations
+
+import json
+import re
+import time
+from dataclasses import dataclass
+from typing import Any
+
+from openai import OpenAI
+
+from locomo.config import Settings
+from locomo.prompts import build_judge_messages
+
+
+@dataclass
+class JudgeResult:
+ correct: bool
+ reason: str
+ raw: str
+ model: str
+ latency_ms: float
+ prompt_tokens: int | None
+ completion_tokens: int | None
+ total_tokens: int | None
+
+
+def _make_openai_client(settings: Settings) -> OpenAI:
+ settings.require_llm()
+ kwargs: dict[str, Any] = {"api_key": settings.openai_api_key}
+ if settings.llm_base_url:
+ kwargs["base_url"] = settings.llm_base_url
+ return OpenAI(**kwargs)
+
+
+def _parse_judge_payload(raw: str) -> tuple[bool, str]:
+ text = raw.strip()
+ try:
+ data = json.loads(text)
+ return bool(data.get("correct")), str(data.get("reason", ""))
+ except json.JSONDecodeError:
+ match = re.search(r"\{.*\}", text, re.DOTALL)
+ if match:
+ try:
+ data = json.loads(match.group(0))
+ return bool(data.get("correct")), str(data.get("reason", ""))
+ except json.JSONDecodeError:
+ pass
+ lowered = text.lower()
+ if '"correct": true' in lowered or "correct: true" in lowered:
+ return True, text
+ if '"correct": false' in lowered or "correct: false" in lowered:
+ return False, text
+ return False, f"unparseable judge output: {text[:200]}"
+
+
+def judge_answer(
+ settings: Settings,
+ question: str,
+ gold: str,
+ prediction: str,
+ *,
+ model: str | None = None,
+) -> JudgeResult:
+ client = _make_openai_client(settings)
+ model_name = model or settings.judge_model
+ messages = build_judge_messages(question, gold, prediction)
+ started = time.perf_counter()
+ response = client.chat.completions.create(
+ model=model_name,
+ messages=messages,
+ temperature=0,
+ response_format={"type": "json_object"},
+ )
+ latency_ms = (time.perf_counter() - started) * 1000
+ raw = (response.choices[0].message.content or "").strip()
+ correct, reason = _parse_judge_payload(raw)
+ usage = response.usage
+ return JudgeResult(
+ correct=correct,
+ reason=reason,
+ raw=raw,
+ model=model_name,
+ latency_ms=latency_ms,
+ prompt_tokens=getattr(usage, "prompt_tokens", None) if usage else None,
+ completion_tokens=getattr(usage, "completion_tokens", None) if usage else None,
+ total_tokens=getattr(usage, "total_tokens", None) if usage else None,
+ )
diff --git a/benchmarks/locomo/metrics.py b/benchmarks/locomo/metrics.py
new file mode 100644
index 0000000..0fbe71f
--- /dev/null
+++ b/benchmarks/locomo/metrics.py
@@ -0,0 +1,140 @@
+from __future__ import annotations
+
+import math
+import string
+from collections import Counter
+from typing import Any, Iterable
+
+
+_ARTICLES = {"a", "an", "the"}
+
+
+def normalize_answer(text: str) -> str:
+ text = (text or "").lower()
+ text = "".join(ch if ch not in string.punctuation else " " for ch in text)
+ tokens = [t for t in text.split() if t and t not in _ARTICLES]
+ return " ".join(tokens)
+
+
+def tokenize(text: str) -> list[str]:
+ return normalize_answer(text).split()
+
+
+def f1_score(prediction: str, gold: str) -> float:
+ pred_tokens = tokenize(prediction)
+ gold_tokens = tokenize(gold)
+ if not pred_tokens and not gold_tokens:
+ return 1.0
+ if not pred_tokens or not gold_tokens:
+ return 0.0
+ common = Counter(pred_tokens) & Counter(gold_tokens)
+ num_same = sum(common.values())
+ if num_same == 0:
+ return 0.0
+ precision = num_same / len(pred_tokens)
+ recall = num_same / len(gold_tokens)
+ return 2 * precision * recall / (precision + recall)
+
+
+def bleu1_score(prediction: str, gold: str) -> float:
+ pred_tokens = tokenize(prediction)
+ gold_tokens = tokenize(gold)
+ if not pred_tokens or not gold_tokens:
+ return 1.0 if not pred_tokens and not gold_tokens else 0.0
+ gold_counts = Counter(gold_tokens)
+ match = 0
+ for token in pred_tokens:
+ if gold_counts[token] > 0:
+ match += 1
+ gold_counts[token] -= 1
+ precision = match / len(pred_tokens)
+ brevity = min(1.0, math.exp(1 - len(gold_tokens) / len(pred_tokens)))
+ return precision * brevity
+
+
+def percentile(values: list[float], p: float) -> float | None:
+ if not values:
+ return None
+ ordered = sorted(values)
+ if len(ordered) == 1:
+ return ordered[0]
+ k = (len(ordered) - 1) * (p / 100.0)
+ f = math.floor(k)
+ c = math.ceil(k)
+ if f == c:
+ return ordered[int(k)]
+ return ordered[f] * (c - k) + ordered[c] * (k - f)
+
+
+def mean(values: Iterable[float]) -> float | None:
+ vals = list(values)
+ if not vals:
+ return None
+ return sum(vals) / len(vals)
+
+
+def aggregate_answers(rows: list[dict[str, Any]]) -> dict[str, Any]:
+ by_category: dict[int, list[dict[str, Any]]] = {}
+ for row in rows:
+ cat = int(row.get("category") or 0)
+ by_category.setdefault(cat, []).append(row)
+
+ def _bucket(items: list[dict[str, Any]]) -> dict[str, Any]:
+ if not items:
+ return {
+ "n": 0,
+ "judge_accuracy": None,
+ "mean_f1": None,
+ "mean_bleu1": None,
+ }
+ correct = [1.0 if r.get("judge_correct") else 0.0 for r in items]
+ return {
+ "n": len(items),
+ "judge_accuracy": mean(correct),
+ "mean_f1": mean(float(r.get("f1") or 0.0) for r in items),
+ "mean_bleu1": mean(float(r.get("bleu1") or 0.0) for r in items),
+ }
+
+ non_adv = [r for r in rows if int(r.get("category") or 0) != 5]
+ all_rows = rows
+ retrieval_latencies = [
+ float(r["retrieve_latency_ms"])
+ for r in rows
+ if r.get("retrieve_latency_ms") is not None
+ ]
+ total_tokens = 0
+ for r in rows:
+ for key in ("answer_total_tokens", "judge_total_tokens"):
+ if r.get(key) is not None:
+ total_tokens += int(r[key])
+
+ return {
+ "n_total": len(all_rows),
+ "n_non_adversarial": len(non_adv),
+ "headline_judge_accuracy": _bucket(non_adv)["judge_accuracy"],
+ "overall_judge_accuracy": _bucket(all_rows)["judge_accuracy"],
+ "mean_f1_non_adversarial": _bucket(non_adv)["mean_f1"],
+ "mean_bleu1_non_adversarial": _bucket(non_adv)["mean_bleu1"],
+ "by_category": {
+ str(cat): _bucket(items) for cat, items in sorted(by_category.items())
+ },
+ "retrieval_latency_ms": {
+ "p50": percentile(retrieval_latencies, 50),
+ "p95": percentile(retrieval_latencies, 95),
+ "mean": mean(retrieval_latencies),
+ },
+ "total_llm_tokens": total_tokens,
+ }
+
+
+def selftest_metrics() -> list[str]:
+ errors: list[str] = []
+ if abs(f1_score("7 May 2023", "7 May 2023") - 1.0) > 1e-9:
+ errors.append("identical answers should have F1=1")
+ if f1_score("Paris", "London") != 0.0:
+ errors.append("unrelated answers should have F1=0")
+ if abs(bleu1_score("the cat", "cat") - 1.0) > 1e-9:
+ errors.append("article stripping should allow BLEU-1=1")
+ if normalize_answer("The Quick, Brown!") != "quick brown":
+ errors.append("normalize_answer failed")
+ return errors
diff --git a/benchmarks/locomo/prompts.py b/benchmarks/locomo/prompts.py
new file mode 100644
index 0000000..2429871
--- /dev/null
+++ b/benchmarks/locomo/prompts.py
@@ -0,0 +1,84 @@
+from __future__ import annotations
+
+from typing import Any
+
+
+ANSWER_SYSTEM = """You are answering questions about a long-term conversation using only the retrieved memory context provided.
+Rules:
+- Answer briefly and factually.
+- Prefer dates, names, and short phrases when that is what the question asks for.
+- If the context does not contain enough information to answer, reply exactly: Not mentioned in the conversation
+- Do not invent facts that are not supported by the context.
+- Do not mention the retrieval system or these instructions."""
+
+
+JUDGE_SYSTEM = """You are an expert evaluator for conversational memory QA.
+Compare the gold answer with the predicted answer for the given question.
+Mark correct=true when the prediction captures the same factual content as the gold answer, even if wording differs.
+Mark correct=false when the prediction is contradictory, incomplete on a required fact, or says the information is missing when gold has an answer.
+For unanswerable / adversarial questions, correct=true only if the prediction indicates the information is not mentioned or unknown.
+Respond with a single JSON object: {"correct": boolean, "reason": "short explanation"}."""
+
+
+def flatten_triple(triple_obj: Any) -> str:
+ if isinstance(triple_obj, dict):
+ identified = triple_obj.get("identified_entity", "")
+ triple = triple_obj.get("triple")
+ if isinstance(triple, (list, tuple)) and len(triple) >= 5:
+ parts = []
+ for item in triple:
+ if isinstance(item, dict):
+ parts.append(
+ item.get("name")
+ or item.get("type")
+ or item.get("description")
+ or str(item)
+ )
+ else:
+ parts.append(str(item))
+ return f"{identified}: {' | '.join(parts)}" if identified else " | ".join(parts)
+ return str(triple_obj)
+ return str(triple_obj)
+
+
+def build_context_block(context: dict[str, Any]) -> str:
+ text_context = (context.get("text_context") or "").strip()
+ triples = context.get("triples") or []
+ historical = context.get("historical_context") or []
+
+ sections = ["## Retrieved text context", text_context or "(empty)"]
+ if triples:
+ sections.append("## Graph triples")
+ for t in triples[:80]:
+ sections.append(f"- {flatten_triple(t)}")
+ if historical:
+ sections.append("## Historical context")
+ for h in historical[:40]:
+ sections.append(f"- {h}")
+ return "\n".join(sections)
+
+
+def build_answer_messages(question: str, context: dict[str, Any]) -> list[dict[str, str]]:
+ user = (
+ f"{build_context_block(context)}\n\n"
+ f"## Question\n{question}\n\n"
+ "Answer:"
+ )
+ return [
+ {"role": "system", "content": ANSWER_SYSTEM},
+ {"role": "user", "content": user},
+ ]
+
+
+def build_judge_messages(
+ question: str, gold: str, prediction: str
+) -> list[dict[str, str]]:
+ user = (
+ f"Question: {question}\n"
+ f"Gold answer: {gold}\n"
+ f"Predicted answer: {prediction}\n"
+ )
+ return [
+ {"role": "system", "content": JUDGE_SYSTEM},
+ {"role": "user", "content": user},
+ ]
diff --git a/benchmarks/locomo/report.py b/benchmarks/locomo/report.py
new file mode 100644
index 0000000..f35bfbd
--- /dev/null
+++ b/benchmarks/locomo/report.py
@@ -0,0 +1,188 @@
+from __future__ import annotations
+
+import json
+from pathlib import Path
+from typing import Any
+
+from rich.console import Console
+from rich.table import Table
+
+from locomo.config import CATEGORY_NAMES
+from locomo.ingest import load_jsonl
+from locomo.metrics import aggregate_answers
+
+console = Console()
+
+
+def _latest_ingest_rows(ingest_rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
+ """Keep the last status per sample/unit (resume appends retries)."""
+ latest: dict[tuple[str, str], dict[str, Any]] = {}
+ for row in ingest_rows:
+ key = (
+ str(row.get("sample_id") or ""),
+ str(row.get("unit_id") or row.get("session_key") or ""),
+ )
+ latest[key] = row
+ return list(latest.values())
+
+
+def build_report(run_dir: Path) -> dict[str, Any]:
+ answers_path = run_dir / "answers.jsonl"
+ ingest_path = run_dir / "ingest.jsonl"
+ manifest_path = run_dir / "manifest.json"
+
+ answers = [r for r in load_jsonl(answers_path) if not r.get("error")]
+ ingest_rows = load_jsonl(ingest_path)
+ ingest_latest = _latest_ingest_rows(ingest_rows)
+ manifest = {}
+ if manifest_path.exists():
+ manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
+
+ metrics = aggregate_answers(answers)
+ ingest_summary = {
+ "n": len(ingest_latest),
+ "completed": sum(1 for r in ingest_latest if r.get("status") == "completed"),
+ "partial_failed": sum(
+ 1 for r in ingest_latest if r.get("status") == "partial_failed"
+ ),
+ "failed": sum(1 for r in ingest_latest if r.get("status") == "failed"),
+ "dry_run": sum(1 for r in ingest_latest if r.get("status") == "dry_run"),
+ "attempts": len(ingest_rows),
+ }
+
+ report = {
+ "run_dir": str(run_dir),
+ "manifest": manifest,
+ "ingest": ingest_summary,
+ "metrics": metrics,
+ "warnings": [],
+ }
+ if ingest_summary["partial_failed"] or ingest_summary["failed"]:
+ report["warnings"].append(
+ "Some ingest units failed or partially failed; scores may be skewed."
+ )
+ return report
+
+
+def render_markdown(report: dict[str, Any]) -> str:
+ metrics = report["metrics"]
+ manifest = report.get("manifest") or {}
+ lines = [
+ "# LoCoMo BrainAPI Benchmark Report",
+ "",
+ f"- Run dir: `{report['run_dir']}`",
+ f"- Created: {manifest.get('created_at', 'n/a')}",
+ f"- Granularity: {manifest.get('granularity', 'n/a')}",
+ f"- Answer model: {manifest.get('answer_model', 'n/a')}",
+ f"- Judge model: {manifest.get('judge_model', 'n/a')}",
+ f"- Samples: {', '.join(manifest.get('samples', []) or ['n/a'])}",
+ "",
+ "## Headline",
+ "",
+ f"- Judge accuracy (excl. adversarial / cat 5): "
+ f"**{_pct(metrics.get('headline_judge_accuracy'))}** "
+ f"(n={metrics.get('n_non_adversarial', 0)})",
+ f"- Judge accuracy (all categories): "
+ f"**{_pct(metrics.get('overall_judge_accuracy'))}** "
+ f"(n={metrics.get('n_total', 0)})",
+ f"- Mean F1 (excl. adversarial): "
+ f"**{_num(metrics.get('mean_f1_non_adversarial'))}**",
+ f"- Mean BLEU-1 (excl. adversarial): "
+ f"**{_num(metrics.get('mean_bleu1_non_adversarial'))}**",
+ "",
+ "## Per-category",
+ "",
+ "| Category | Name | N | Judge Acc | Mean F1 | Mean BLEU-1 |",
+ "|---|---|---:|---:|---:|---:|",
+ ]
+ for cat, bucket in (metrics.get("by_category") or {}).items():
+ name = CATEGORY_NAMES.get(int(cat), "unknown")
+ lines.append(
+ f"| {cat} | {name} | {bucket.get('n', 0)} | "
+ f"{_pct(bucket.get('judge_accuracy'))} | "
+ f"{_num(bucket.get('mean_f1'))} | "
+ f"{_num(bucket.get('mean_bleu1'))} |"
+ )
+
+ lat = metrics.get("retrieval_latency_ms") or {}
+ lines.extend(
+ [
+ "",
+ "## Latency & tokens",
+ "",
+ f"- Retrieval latency p50: {_num(lat.get('p50'))} ms",
+ f"- Retrieval latency p95: {_num(lat.get('p95'))} ms",
+ f"- Retrieval latency mean: {_num(lat.get('mean'))} ms",
+ f"- Total LLM tokens (answer+judge): {metrics.get('total_llm_tokens', 0)}",
+ "",
+ "## Ingest",
+ "",
+ f"- Units: {report['ingest'].get('n', 0)}",
+ f"- Completed: {report['ingest'].get('completed', 0)}",
+ f"- Partial failed: {report['ingest'].get('partial_failed', 0)}",
+ f"- Failed: {report['ingest'].get('failed', 0)}",
+ *(
+ [f"- Attempts (incl. retries): {report['ingest'].get('attempts')}"]
+ if report["ingest"].get("attempts")
+ and report["ingest"].get("attempts") != report["ingest"].get("n")
+ else []
+ ),
+ "",
+ ]
+ )
+ if report.get("warnings"):
+ lines.append("## Warnings")
+ lines.append("")
+ for warning in report["warnings"]:
+ lines.append(f"- {warning}")
+ lines.append("")
+ return "\n".join(lines)
+
+
+def write_report(run_dir: Path) -> dict[str, Any]:
+ report = build_report(run_dir)
+ md = render_markdown(report)
+ (run_dir / "report.json").write_text(
+ json.dumps(report, indent=2), encoding="utf-8"
+ )
+ (run_dir / "report.md").write_text(md, encoding="utf-8")
+ return report
+
+
+def print_report_table(report: dict[str, Any]) -> None:
+ metrics = report["metrics"]
+ table = Table(title="LoCoMo per-category results")
+ table.add_column("Cat")
+ table.add_column("Name")
+ table.add_column("N", justify="right")
+ table.add_column("Judge Acc", justify="right")
+ table.add_column("F1", justify="right")
+ table.add_column("BLEU-1", justify="right")
+ for cat, bucket in (metrics.get("by_category") or {}).items():
+ table.add_row(
+ str(cat),
+ CATEGORY_NAMES.get(int(cat), "unknown"),
+ str(bucket.get("n", 0)),
+ _pct(bucket.get("judge_accuracy")),
+ _num(bucket.get("mean_f1")),
+ _num(bucket.get("mean_bleu1")),
+ )
+ console.print(table)
+ console.print(
+ f"Headline judge accuracy (excl. cat 5): "
+ f"[bold]{_pct(metrics.get('headline_judge_accuracy'))}[/bold]"
+ )
+ for warning in report.get("warnings") or []:
+ console.print(f"[yellow]Warning: {warning}[/yellow]")
+
+
+def _pct(value: float | None) -> str:
+ if value is None:
+ return "n/a"
+ return f"{value * 100:.1f}%"
+
+
+def _num(value: float | None) -> str:
+ if value is None:
+ return "n/a"
+ return f"{value:.3f}"
diff --git a/benchmarks/requirements.txt b/benchmarks/requirements.txt
new file mode 100644
index 0000000..fccaa95
--- /dev/null
+++ b/benchmarks/requirements.txt
@@ -0,0 +1,4 @@
+httpx>=0.27.0,<1.0.0
+openai>=1.40.0,<2.0.0
+python-dotenv>=1.0.0,<2.0.0
+rich>=13.0.0,<15.0.0
diff --git a/benchmarks/runs/.gitkeep b/benchmarks/runs/.gitkeep
new file mode 100644
index 0000000..e69de29
diff --git a/console/AGENTS.md b/console/AGENTS.md
new file mode 100644
index 0000000..e69de29
diff --git a/console/CLAUDE.md b/console/CLAUDE.md
new file mode 100644
index 0000000..43c994c
--- /dev/null
+++ b/console/CLAUDE.md
@@ -0,0 +1 @@
diff --git a/console/src/pages/GraphPage.tsx b/console/src/pages/GraphPage.tsx
index 5002975..ddac1b7 100644
--- a/console/src/pages/GraphPage.tsx
+++ b/console/src/pages/GraphPage.tsx
@@ -1,6 +1,6 @@
import { useCallback, useEffect, useMemo, useState } from "react";
import GraphCanvas from "../components/GraphCanvas";
-import { apiFetch } from "../lib/api";
+import { apiFetch, getSession } from "../lib/api";
import { colorForLabel, primaryLabel } from "../lib/graphColors";
import {
filterRelationshipsByEntities,
@@ -87,6 +87,7 @@ export default function GraphPage() {
const [error, setError] = useState<string | null>(null);
const [physics, setPhysics] = useState(true);
const [activeFilters, setActiveFilters] = useState<GraphFilters>(DEFAULT_FILTERS);
+ const activeBrainId = getSession()?.brainId ?? "default";
const loadGraph = useCallback(async (filters: GraphFilters) => {
setLoading(true);
@@ -244,6 +245,7 @@ export default function GraphPage() {
<div className="mr-2">
<h1 className="text-2xl font-semibold">Graph Explorer</h1>
<p className="text-xs text-slate-500">
+ Brain <span className="text-cyan-600">{activeBrainId}</span> ·{" "}
{entities.length} nodes · {edges.length} edges · {relationships.length}{" "}
relationships loaded · double-click to expand
</p>
@@ -319,10 +321,19 @@ export default function GraphPage() {
Loading graph…
</div>
) : entities.length === 0 ? (
- <div className="flex h-full items-center justify-center bg-[#1a1d23] text-slate-500">
- {hasActiveFilters
- ? "No matches — try a different label or search term"
- : "No nodes — ingest data or change filters"}
+ <div className="flex h-full flex-col items-center justify-center gap-2 bg-[#1a1d23] px-6 text-center text-slate-500">
+ <p>
+ {hasActiveFilters
+ ? "No matches — try a different label or search term"
+ : "No nodes in this brain"}
+ </p>
+ <p className="max-w-md text-xs text-slate-600">
+ Active brain:{" "}
+ <span className="text-cyan-700">{activeBrainId}</span>
+ {activeBrainId === "default"
+ ? ". System PAT defaults to default — switch to locomoconv26 (or your ingest brain) in the sidebar."
+ : ". If you expected data here, confirm ingest used this same brain id."}
+ </p>
</div>
) : (
<GraphCanvas
diff --git a/llms.txt b/llms.txt
new file mode 100644
index 0000000..e69de29
diff --git a/src/adapters/graph.py b/src/adapters/graph.py
index 99ee4e8..e78110f 100644
--- a/src/adapters/graph.py
+++ b/src/adapters/graph.py
@@ -8,6 +8,7 @@ Modified By: Christian Nonis <[email protected]>
-----
"""
+import re
from abc import ABC, abstractmethod
from typing import Dict, List, Literal, Optional, Tuple
from src.adapters.interfaces.graph import GraphClient, PredicateWithFlowKey
@@ -126,8 +127,17 @@ class GraphAdapter:
result = self.graph.execute_operation(operation, brain_id)
return serialize_graph_operation_result(result)
except Exception as e: # pylint: disable=broad-exception-caught
+ hint = ""
+ backend = getattr(self.graph, "graphdb_type", "") or ""
+ if "postgresql" in backend and re.search(
+ r"\bMATCH\b|\bRETURN\b|\bFROM\s+nodes\b", operation or "", re.I
+ ):
+ hint = (
+ " Hint: this backend expects SQL on kg_nodes/kg_relationships, "
+ "not Cypher and not a table named nodes."
+ )
print(f"Error executing graph operation: {e} - {operation}")
- return f"Error executing graph operation: {e}"
+ return f"Error executing graph operation: {e}.{hint}"
def add_nodes(
self,
diff --git a/src/core/agents/architect_agent.py b/src/core/agents/architect_agent.py
index f4b02b4..a6a79a1 100644
--- a/src/core/agents/architect_agent.py
+++ b/src/core/agents/architect_agent.py
@@ -770,7 +770,6 @@ class ArchitectAgent:
if mode == "granular":
from src.core.agents.janitor_agent import JanitorAgent
- from src.lib.neo4j.client import _neo4j_client
from src.services.input.agents import (
embeddings_adapter,
graph_adapter,
@@ -785,7 +784,7 @@ class ArchitectAgent:
kg=graph_adapter,
vector_store=vector_store_adapter,
embeddings=embeddings_adapter,
- database_desc=_neo4j_client.graphdb_description,
+ database_desc=graph_adapter.graphdb_description,
)
self.janitor_agent = janitor_agent
self._janitor_agent_brain_id = brain_id
diff --git a/src/core/agents/tools/architect_agent/ArchitectAgentCreateRelationshipTool.py b/src/core/agents/tools/architect_agent/ArchitectAgentCreateRelationshipTool.py
index cfe4b19..4286db3 100644
--- a/src/core/agents/tools/architect_agent/ArchitectAgentCreateRelationshipTool.py
+++ b/src/core/agents/tools/architect_agent/ArchitectAgentCreateRelationshipTool.py
@@ -30,7 +30,6 @@ from src.services.input.agents import (
llm_small_adapter,
vector_store_adapter,
)
-from src.lib.neo4j.client import _neo4j_client
from src.utils.cleanup import strip_properties
from src.utils.nlp.names import levenshtein_similarity
from src.utils.serialization.data import is_uuid
@@ -394,7 +393,7 @@ class ArchitectAgentCreateRelationshipTool(BaseTool):
kg=graph_adapter,
vector_store=vector_store_adapter,
embeddings=embeddings_adapter,
- database_desc=_neo4j_client.graphdb_description,
+ database_desc=graph_adapter.graphdb_description,
)
self.architect_agent.janitor_agent = janitor_agent
self.architect_agent._janitor_agent_brain_id = self.brain_id
diff --git a/src/core/agents/tools/janitor_agent/JanitorAgentExecuteGraphReadOperationTool.py b/src/core/agents/tools/janitor_agent/JanitorAgentExecuteGraphReadOperationTool.py
index 5d41bc4..e8a3e4b 100644
--- a/src/core/agents/tools/janitor_agent/JanitorAgentExecuteGraphReadOperationTool.py
+++ b/src/core/agents/tools/janitor_agent/JanitorAgentExecuteGraphReadOperationTool.py
@@ -93,8 +93,7 @@ class JanitorAgentExecuteGraphReadOperationTool(BaseTool):
"Use this tool for ad-hoc graph reads when search_entities is not enough. "
"The query should be a valid graph read only operation depending on the graph database type. "
"{database_desc}. "
- "For UNION queries, every branch must RETURN the same column aliases "
- "(e.g. RETURN n.uuid AS uuid, n.name AS name). "
+ "For UNION queries, every branch must select the same column aliases. "
"If you get an error, try again after fixing your query and don't give up. "
"If you get a result, return the result as a JSON object."
)
diff --git a/src/core/instances.py b/src/core/instances.py
index f0a77c4..5030a13 100644
--- a/src/core/instances.py
+++ b/src/core/instances.py
@@ -126,12 +126,17 @@ elif config.data_db == "postgresql":
data_adapter = DataAdapter()
data_adapter.add_client(get_postgresql_data_client())
-from src.lib.embeddings.client_small import _embeddings_small_client
-
_embeddings_primary = EmbeddingsAdapter()
_embeddings_primary.add_client(_build_embeddings(config.embeddings_provider))
_embeddings_small = EmbeddingsAdapter()
-_embeddings_small.add_client(_embeddings_small_client)
+if config.models_mode == "remote":
+ # Keep remote mode free of local torch/sentence-transformers.
+ # Architect similarity checks still work as long as both sides share a space.
+ _embeddings_small.add_client(_build_embeddings(config.embeddings_provider))
+else:
+ from src.lib.embeddings.client_small import _embeddings_small_client
+
+ _embeddings_small.add_client(_embeddings_small_client)
embeddings_adapter = _embeddings_primary
embeddings_small_adapter = _embeddings_small
diff --git a/src/core/layers/graph_consolidation/graph_consolidation.py b/src/core/layers/graph_consolidation/graph_consolidation.py
index 6dbac3a..a842ded 100644
--- a/src/core/layers/graph_consolidation/graph_consolidation.py
+++ b/src/core/layers/graph_consolidation/graph_consolidation.py
@@ -24,7 +24,6 @@ from src.services.input.agents import (
vector_store_adapter,
)
from src.core.instances import llm_small_adapter as llm_adapter
-from src.lib.neo4j.client import _neo4j_client
RELATIONSHIP_BATCH_SIZE = 20
@@ -59,7 +58,7 @@ def consolidate_graph(
kg=graph_adapter,
vector_store=vector_store_adapter,
embeddings=embeddings_adapter,
- database_desc=_neo4j_client.graphdb_description,
+ database_desc=graph_adapter.graphdb_description,
)
tasks = janitor_agent.run_graph_consolidator(
@@ -80,7 +79,7 @@ def consolidate_graph(
kg=graph_adapter,
vector_store=vector_store_adapter,
embeddings=embeddings_adapter,
- database_desc=_neo4j_client.graphdb_description,
+ database_desc=graph_adapter.graphdb_description,
)
for task in tasks:
try:
diff --git a/src/core/search/entity_info.py b/src/core/search/entity_info.py
index 041833e..d8015a2 100644
--- a/src/core/search/entity_info.py
+++ b/src/core/search/entity_info.py
@@ -25,6 +25,9 @@ from src.utils.similarity.vectors import cosine_similarity
vector_search = VectorSearchFacade(vector_store_adapter)
+_BRANCH_FACTOR = 3
+_MAX_EXPLORATION_WORK = 50
+
# ================================================================
# NOTE Currently not fully supported by the Event-Centric v2 kg
@@ -51,17 +54,46 @@ class EventSynergyRetriever:
self.memory_id = memory_id
self.brain_id = memory_id
+ def _score_neighbors(
+ self,
+ current_node_id: str,
+ query_embedding: List[float],
+ ) -> List[Tuple[Tuple[Predicate, Node], float]]:
+ neighbors = graph_adapter.get_neighbors(
+ [current_node_id], brain_id=self.brain_id
+ )
+ if not neighbors or current_node_id not in neighbors:
+ return []
+
+ conn_rels: List[Tuple[Predicate, Node]] = neighbors[current_node_id]
+ scored: List[Tuple[Tuple[Predicate, Node], float]] = []
+ for rel_tuple in conn_rels:
+ cr = rel_tuple[0]
+ v_id = cr.properties.get("v_id") if cr.properties else None
+ if v_id is None:
+ continue
+ cr_vs = vector_store_adapter.get_by_ids(
+ [v_id],
+ store="relationships",
+ brain_id=self.brain_id,
+ )
+ if not cr_vs or not getattr(cr_vs[0], "embeddings", None):
+ continue
+ scored.append(
+ (rel_tuple, cosine_similarity(cr_vs[0].embeddings, query_embedding))
+ )
+ scored.sort(key=lambda item: item[1], reverse=True)
+ return scored
+
def _recursive_explorer(
self,
current_node_id: str,
query_embedding: List[float],
depth: int,
visited_ids: Set[str],
- most_similar_conn_rel_tuple: Optional[
- Tuple[Tuple[Predicate, Node], float]
- ] = None,
- ):
- # -> List[SynergyPath]:
+ work_counter: Optional[List[int]] = None,
+ branch_factor: int = _BRANCH_FACTOR,
+ ) -> List[Tuple[Tuple[Predicate, Node], float]]:
"""
Traverse graph neighbors from the given node to locate the connection path most similar to the provided query embedding, searching up to the specified depth.
@@ -69,86 +101,53 @@ class EventSynergyRetriever:
current_node_id (str): UUID of the node to start exploration from.
query_embedding (List[float]): Embedding vector representing the query used to score neighbor relations.
depth (int): Remaining recursion depth; exploration stops when depth <= 0.
- visited_ids (Set[str]): Set of visited node UUIDs; this set is mutated to include nodes visited during traversal to avoid cycles.
- most_similar_conn_rel_tuple (Optional[Tuple[Tuple[Predicate, Node], float]]): Optional running best match (relation tuple and its similarity) to carry forward during recursion.
+ visited_ids (Set[str]): Set of visited node UUIDs used to avoid cycles on the current path.
+ work_counter (Optional[List[int]]): Mutable single-element list tracking explored nodes; stops when cap is reached.
+ branch_factor (int): Number of top-scoring neighbors to expand at each hop.
Returns:
- results (List[Tuple[Tuple[Predicate, Node], float]]): A list containing the most similar (predicate, node) tuple paired with its similarity score found along the explored path, or an empty list if no suitable connections are found.
+ results (List[Tuple[Tuple[Predicate, Node], float]]): Best path as an ordered list of (relation tuple, similarity) hops from the start node, or an empty list if no suitable connections are found.
"""
- if depth <= 0:
+ if work_counter is None:
+ work_counter = [0]
+
+ if depth <= 0 or work_counter[0] >= _MAX_EXPLORATION_WORK:
return []
if current_node_id in visited_ids:
return []
visited_ids.add(current_node_id)
+ work_counter[0] += 1
- neighbors = graph_adapter.get_neighbors(
- [current_node_id], brain_id=self.brain_id
- )
-
- if not neighbors or current_node_id not in neighbors:
+ scored = self._score_neighbors(current_node_id, query_embedding)
+ if not scored:
return []
- conn_rels: List[Tuple[Predicate, Node]] = neighbors[current_node_id]
+ best_path: List[Tuple[Tuple[Predicate, Node], float]] = []
+ best_score = float("-inf")
- conn_rels_w_embeddings: List[Tuple[Tuple[Predicate, Node], List[float]]] = []
-
- for rel_tuple in conn_rels:
- cr = rel_tuple[0]
- v_id = cr.properties.get("v_id")
- if v_id is None:
+ for rel_tuple, similarity in scored[: max(1, branch_factor)]:
+ predicate, node = rel_tuple
+ if not node or not predicate or not node.uuid:
continue
- cr_vs = vector_store_adapter.get_by_ids(
- [v_id],
- store="relationships",
- brain_id=self.brain_id,
- )
- if cr_vs:
- conn_rels_w_embeddings.append((rel_tuple, cr_vs[0].embeddings))
- if not conn_rels_w_embeddings:
- return []
-
- conn_rels_similarities: List[Tuple[Tuple[Predicate, Node], float]] = [
- (rel_tuple, cosine_similarity(cr_embedding, query_embedding))
- for rel_tuple, cr_embedding in conn_rels_w_embeddings
- ]
-
- conn_rels_similarities_sorted = sorted(
- conn_rels_similarities, key=lambda x: x[1], reverse=True
- )
-
- if not conn_rels_similarities_sorted:
- return []
- current_best = conn_rels_similarities_sorted[0]
-
- predicate, node = current_best[0]
- if not node or not predicate:
- return []
-
- if (
- most_similar_conn_rel_tuple is None
- or current_best[1] > most_similar_conn_rel_tuple[1]
- ):
- most_similar_conn_rel_tuple = current_best
-
- next_node = current_best[0][1]
- if not next_node or not next_node.uuid:
- return [most_similar_conn_rel_tuple] if most_similar_conn_rel_tuple else []
-
- results = self._recursive_explorer(
- next_node.uuid,
- query_embedding,
- depth - 1,
- visited_ids,
- most_similar_conn_rel_tuple,
- )
+ hop = (rel_tuple, similarity)
+ child_path = self._recursive_explorer(
+ node.uuid,
+ query_embedding,
+ depth - 1,
+ visited_ids.copy(),
+ work_counter,
+ branch_factor=branch_factor,
+ )
+ candidate = [hop] + child_path
+ path_score = max(score for _, score in candidate)
+ if path_score > best_score:
+ best_score = path_score
+ best_path = candidate
- if results:
- return results
- else:
- return [most_similar_conn_rel_tuple] if most_similar_conn_rel_tuple else []
+ return best_path
def retrieve_matches(
self, target: str, query: str, max_depth: int = 3
@@ -206,6 +205,7 @@ class EventSynergyRetriever:
query_embedding.embeddings,
depth=max_depth,
visited_ids=set(),
+ work_counter=[0],
)
if not synergy_paths:
@@ -216,8 +216,6 @@ class EventSynergyRetriever:
children=[],
)
- synergy_paths = list(reversed(synergy_paths))
-
match_paths = []
for path_tuple, similarity in synergy_paths:
predicate, node = path_tuple
@@ -260,4 +258,4 @@ class EventSynergyRetriever:
current_path.children.append(match_paths[i])
current_path = match_paths[i]
- return root_path
\ No newline at end of file
+ return root_path
diff --git a/src/lib/postgresql/_provisioning.py b/src/lib/postgresql/_provisioning.py
index 1aa36da..bf4bfcb 100644
--- a/src/lib/postgresql/_provisioning.py
+++ b/src/lib/postgresql/_provisioning.py
@@ -43,7 +43,9 @@ logger = logging.getLogger(__name__)
_MAX_OPEN_POOLS = 16
_DEFAULT_MIN_CONN = 1
-_DEFAULT_MAX_CONN = 4
+# Celery uses a threaded pool; keep enough connections for concurrent
+# brain reads/writes without blocking behind the in-memory graph lock.
+_DEFAULT_MAX_CONN = max(16, int(os.getenv("CELERY_WORKER_CONCURRENCY") or 4) * 4)
_MAINTENANCE_FALLBACK_DB = "postgres"
@@ -191,10 +193,21 @@ def ensure_system_database() -> str:
def list_brain_database_names() -> list[str]:
+ # LIKE '_' is a single-char wildcard; escape so prefix "brain_" is literal.
+ # Otherwise "brainapi" (system DB) matches "brain_%".
+ escaped_prefix = (
+ BRAIN_DB_PREFIX.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
+ )
+ system_db = _system_dbname()
with borrow(_registry.get(_maintenance_dbname())) as conn:
with conn.cursor() as cur:
cur.execute(
- "SELECT datname FROM pg_database WHERE datname LIKE %s ORDER BY datname",
- (f"{BRAIN_DB_PREFIX}%",),
+ "SELECT datname FROM pg_database "
+ "WHERE datname LIKE %s ESCAPE %s ORDER BY datname",
+ (f"{escaped_prefix}%", "\\"),
)
- return [row[0] for row in cur.fetchall()]
+ return [
+ row[0]
+ for row in cur.fetchall()
+ if row[0] != system_db
+ ]
diff --git a/src/lib/postgresql/graph_store.py b/src/lib/postgresql/graph_store.py
index 2ed7d39..dd1bdf1 100644
--- a/src/lib/postgresql/graph_store.py
+++ b/src/lib/postgresql/graph_store.py
@@ -9,6 +9,7 @@ from __future__ import annotations
from contextlib import contextmanager
import json
+import re
import threading
from typing import Any, Dict, List, Optional
@@ -35,6 +36,17 @@ class GraphDatabaseError(Exception):
self.code = code
+# Agents often write data->'field' ILIKE ..., which fails because -> returns jsonb.
+_JSONB_ILIKE_RE = re.compile(
+ r"\bdata\s*->\s*'([^']+)'\s+(ILIKE|LIKE)\b",
+ re.IGNORECASE,
+)
+
+
+def normalize_read_sql(sql: str) -> str:
+ return _JSONB_ILIKE_RE.sub(r"data->>'\1' \2", sql)
+
+
class _NodeProxy(dict):
def get(self, key: str, default: Any = None) -> Any:
return super().get(key, default)
@@ -145,6 +157,7 @@ class PostgreSQLGraphStore:
self._brains: dict[str, _BrainGraph] = {}
self._schema_ready: set[str] = set()
self._schema_lock = threading.Lock()
+ self._brains_lock = threading.RLock()
def _ensure_brain_schema(self, brain_id: str) -> None:
if brain_id in self._schema_ready:
@@ -171,9 +184,48 @@ class PostgreSQLGraphStore:
def get_brain(self, brain_id: str) -> _BrainGraph:
return self._load_brain(brain_id)
+ def invalidate_brain(self, brain_id: str) -> None:
+ with self._brains_lock:
+ self._brains.pop(brain_id, None)
+
+ def _db_graph_counts(self, brain_id: str) -> tuple[int, int]:
+ self._ensure_brain_schema(brain_id)
+ with borrow(get_brain_pool(brain_id)) as conn:
+ with conn.cursor() as cur:
+ cur.execute("SELECT COUNT(*) FROM kg_nodes")
+ nodes = int(cur.fetchone()[0])
+ cur.execute("SELECT COUNT(*) FROM kg_relationships")
+ rels = int(cur.fetchone()[0])
+ return nodes, rels
+
def _load_brain(self, brain_id: str) -> _BrainGraph:
- if brain_id in self._brains:
- return self._brains[brain_id]
+ """Read path: refresh from Postgres when another process wrote data."""
+ with self._brains_lock:
+ cached = self._brains.get(brain_id)
+ cached_counts = (
+ (cached.graph.number_of_nodes(), cached.graph.number_of_edges())
+ if cached is not None
+ else None
+ )
+
+ if cached is not None and cached_counts is not None:
+ try:
+ if self._db_graph_counts(brain_id) == cached_counts:
+ return cached
+ except Exception:
+ pass
+
+ with self._brains_lock:
+ return self._reload_brain_locked(brain_id)
+
+ def _ensure_brain_in_memory_locked(self, brain_id: str) -> _BrainGraph:
+ """Write path: reuse the in-process graph; load from DB only if missing."""
+ cached = self._brains.get(brain_id)
+ if cached is not None:
+ return cached
+ return self._reload_brain_locked(brain_id)
+
+ def _reload_brain_locked(self, brain_id: str) -> _BrainGraph:
self._ensure_brain_schema(brain_id)
brain = _BrainGraph(brain_id)
with self._connection(brain_id) as conn:
@@ -342,7 +394,7 @@ class PostgreSQLGraphStore:
max_rows: int = MAX_READ_QUERY_ROWS,
) -> dict[str, Any]:
try:
- sql = validate_read_only_sql(query)
+ sql = validate_read_only_sql(normalize_read_sql(query))
except ReadQueryValidationError as exc:
raise GraphDatabaseError(str(exc)) from exc
self._ensure_brain_schema(brain_id)
@@ -366,12 +418,32 @@ class PostgreSQLGraphStore:
identification: dict[str, Any],
properties: dict[str, Any],
) -> str:
- brain = self._load_brain(brain_id)
- merged_props = {**identification, **properties}
- node_uuid = brain.upsert_node(labels, identification, merged_props)
- self._persist_node(brain_id, node_uuid, brain.node_data(node_uuid))
+ with self._brains_lock:
+ brain = self._ensure_brain_in_memory_locked(brain_id)
+ merged_props = {**identification, **properties}
+ node_uuid = brain.upsert_node(labels, identification, merged_props)
+ payload = dict(brain.node_data(node_uuid))
+ self._persist_node(brain_id, node_uuid, payload)
return node_uuid
+ def _ensure_relationship_endpoint(
+ self,
+ brain: _BrainGraph,
+ labels: list[str],
+ name: Any,
+ node_uuid: Optional[str],
+ ) -> tuple[Optional[str], Optional[dict]]:
+ if node_uuid and node_uuid in brain.graph:
+ return node_uuid, None
+ if node_uuid:
+ resolved = brain.upsert_node(
+ labels,
+ {"uuid": node_uuid, "name": name},
+ {"uuid": node_uuid, "name": name},
+ )
+ return resolved, dict(brain.node_data(resolved))
+ return self.resolve_node_by_name_labels(brain, labels, name), None
+
def merge_relationship(
self,
brain_id: str,
@@ -384,34 +456,53 @@ class PostgreSQLGraphStore:
subject_uuid: Optional[str] = None,
object_uuid: Optional[str] = None,
) -> Optional[tuple[dict, dict]]:
- brain = self._load_brain(brain_id)
- source_uuid = subject_uuid if subject_uuid and subject_uuid in brain.graph else None
- target_uuid = object_uuid if object_uuid and object_uuid in brain.graph else None
- if not source_uuid:
- source_uuid = self.resolve_node_by_name_labels(
- brain, subject_labels, subject_name
+ pending_nodes: list[tuple[str, dict]] = []
+ with self._brains_lock:
+ brain = self._ensure_brain_in_memory_locked(brain_id)
+ source_uuid, source_payload = self._ensure_relationship_endpoint(
+ brain, subject_labels, subject_name, subject_uuid
)
- if not target_uuid:
- target_uuid = self.resolve_node_by_name_labels(
- brain, object_labels, object_name
+ target_uuid, target_payload = self._ensure_relationship_endpoint(
+ brain, object_labels, object_name, object_uuid
)
- if not source_uuid or not target_uuid:
- return None
- props = dict(rel_props)
- props["rel_type"] = rel_type
- rel_uuid = props.get("uuid") or f"{source_uuid}-{rel_type}-{target_uuid}"
- props["uuid"] = rel_uuid
- if brain.graph.has_edge(source_uuid, target_uuid, key=rel_uuid):
- existing = dict(brain.graph.get_edge_data(source_uuid, target_uuid, key=rel_uuid))
- existing.update(props)
- props = existing
- props["uuid"] = rel_uuid
+ if source_payload is not None and source_uuid:
+ pending_nodes.append((source_uuid, source_payload))
+ if target_payload is not None and target_uuid:
+ pending_nodes.append((target_uuid, target_payload))
+ if not source_uuid or not target_uuid:
+ raise GraphDatabaseError(
+ "Cannot persist relationship "
+ f"{rel_type!r}: missing endpoint nodes "
+ f"(subject_uuid={subject_uuid!r}, object_uuid={object_uuid!r}, "
+ f"subject_name={subject_name!r}, object_name={object_name!r})"
+ )
+ props = dict(rel_props)
props["rel_type"] = rel_type
- brain.graph.add_edge(source_uuid, target_uuid, key=rel_uuid, **props)
- self._persist_relationship(
- brain_id, rel_uuid, rel_type, source_uuid, target_uuid, props
- )
- return brain.node_data(source_uuid), brain.node_data(target_uuid)
+ rel_uuid = props.get("uuid") or f"{source_uuid}-{rel_type}-{target_uuid}"
+ props["uuid"] = rel_uuid
+ if brain.graph.has_edge(source_uuid, target_uuid, key=rel_uuid):
+ existing = dict(
+ brain.graph.get_edge_data(source_uuid, target_uuid, key=rel_uuid)
+ )
+ existing.update(props)
+ props = existing
+ props["uuid"] = rel_uuid
+ props["rel_type"] = rel_type
+ brain.graph.add_edge(source_uuid, target_uuid, key=rel_uuid, **props)
+ source_data = dict(brain.node_data(source_uuid))
+ target_data = dict(brain.node_data(target_uuid))
+ rel_payload = (
+ rel_uuid,
+ rel_type,
+ source_uuid,
+ target_uuid,
+ dict(props),
+ )
+
+ for node_id, payload in pending_nodes:
+ self._persist_node(brain_id, node_id, payload)
+ self._persist_relationship(brain_id, *rel_payload)
+ return source_data, target_data
def resolve_node_by_name_labels(
self, brain: _BrainGraph, labels: list[str], name: Any
diff --git a/src/lib/postgresql/networkx_client.py b/src/lib/postgresql/networkx_client.py
index 2a6e92d..988ffde 100644
--- a/src/lib/postgresql/networkx_client.py
+++ b/src/lib/postgresql/networkx_client.py
@@ -53,19 +53,39 @@ class NetworkXGraphClient(GraphClient):
@property
def graphdb_description(self) -> str:
return (
- "The graph database is PostgreSQL. Use read-only SQL (SELECT or WITH ... SELECT) "
- "as db_query; Cypher is not supported. "
- "Tables: kg_nodes(uuid TEXT PRIMARY KEY, data JSONB) where data holds name, labels, "
+ "The graph database is PostgreSQL (NetworkX-backed). "
+ "You MUST write read-only SQL only. Cypher is NOT supported. "
+ "Never use MATCH, RETURN, labels(n), type(r), or EXPLAIN MATCH. "
+ "Never query a table named nodes or relationships — use kg_nodes and kg_relationships. "
+ "IMPORTANT: for text filters use data->>'field' (text), never data->'field' (jsonb) with ILIKE/LIKE. "
+ "Always send one complete SQL statement in a single tool call "
+ "(never split a WITH RECURSIVE CTE across multiple calls). "
+ "Tables: "
+ "kg_nodes(uuid TEXT PRIMARY KEY, data JSONB) where data holds name, labels, "
"description and other node properties; "
"kg_relationships(uuid TEXT PRIMARY KEY, rel_type TEXT, source_uuid TEXT, "
"target_uuid TEXT, data JSONB). "
- "Filter nodes by label with data->'labels' ? 'PERSON', by name with data->>'name' ILIKE '%alice%'. "
- "Multi-hop traversal example: WITH RECURSIVE walk AS ("
- "SELECT source_uuid, target_uuid, rel_type, 1 AS depth FROM kg_relationships "
- "WHERE source_uuid = '<uuid>' "
- "UNION ALL SELECT r.source_uuid, r.target_uuid, r.rel_type, w.depth + 1 "
+ "Examples: "
+ "SELECT COUNT(*) AS total FROM kg_nodes; "
+ "SELECT uuid, data->>'name' AS name, data->'labels' AS labels FROM kg_nodes "
+ "WHERE data->>'name' ILIKE '%Counsel%' OR data->>'description' ILIKE '%Mental Health%' LIMIT 50; "
+ "SELECT uuid, data FROM kg_nodes WHERE uuid = '<uuid>'; "
+ "SELECT n.uuid, n.data->>'name' AS name, r.rel_type, m.uuid AS target_uuid, "
+ "m.data->>'name' AS target_name, m.data->'labels' AS target_labels "
+ "FROM kg_relationships r "
+ "JOIN kg_nodes n ON n.uuid = r.source_uuid "
+ "JOIN kg_nodes m ON m.uuid = r.target_uuid "
+ "WHERE n.uuid = '<uuid>'; "
+ "Filter nodes by label with data->'labels' ? 'PERSON'. "
+ "Prefer 1-hop joins first. For multi-hop use this single complete statement: "
+ "WITH RECURSIVE walk AS ("
+ "SELECT source_uuid, target_uuid, rel_type, 1 AS depth "
+ "FROM kg_relationships WHERE source_uuid = '<uuid>' "
+ "UNION ALL "
+ "SELECT r.source_uuid, r.target_uuid, r.rel_type, w.depth + 1 "
"FROM walk w JOIN kg_relationships r ON r.source_uuid = w.target_uuid "
- "WHERE w.depth < 3) SELECT * FROM walk LIMIT 50. "
+ "WHERE w.depth < 3"
+ ") SELECT * FROM walk LIMIT 50; "
"Only read queries are allowed; results are capped at 100 rows."
)
@@ -951,6 +971,101 @@ class NetworkXGraphClient(GraphClient):
return neighbors
+ def _node_from_brain(self, brain, uuid: str) -> Node:
+ data = brain.node_data(uuid) if uuid in brain.graph else {}
+ node_kwargs: Dict[str, Any] = {
+ "uuid": uuid,
+ "name": data.get("name", "") or "",
+ "labels": data.get("labels", []) or [],
+ "description": data.get("description", "") or "",
+ "properties": always_dict(data),
+ "polarity": data.get("polarity", "neutral") or "neutral",
+ "metadata": always_dict(data.get("metadata", {})),
+ "happened_at": data.get("happened_at", "") or "",
+ "observations": data.get("observations", []) or [],
+ }
+ last_updated = data.get("last_updated")
+ if last_updated:
+ node_kwargs["last_updated"] = last_updated
+ return Node(**node_kwargs)
+
+
+ def _predicate_from_edge(self, edge_data: dict, direction: str) -> Predicate:
+ edge_data = always_dict(edge_data)
+ predicate_kwargs: Dict[str, Any] = {
+ "uuid": edge_data.get("uuid", "") or "",
+ "name": edge_data.get("rel_type", "") or edge_data.get("name", "") or "",
+ "description": edge_data.get("description", "") or "",
+ "direction": direction,
+ "properties": always_dict(edge_data),
+ "flow_key": edge_data.get("flow_key", "") or "",
+ "observations": edge_data.get("observations", []) or [],
+ "amount": edge_data.get("amount"),
+ }
+ last_updated = edge_data.get("last_updated")
+ if last_updated:
+ predicate_kwargs["last_updated"] = last_updated
+ return Predicate(**predicate_kwargs)
+
+
+ def _incident_edges(
+ self, brain, node_uuid: str
+ ) -> List[Tuple[str, str, dict, str]]:
+ if node_uuid not in brain.graph:
+ return []
+ edges: List[Tuple[str, str, dict, str]] = []
+ for _, neighbor, key, data in brain.graph.out_edges(
+ node_uuid, keys=True, data=True
+ ):
+ edges.append((neighbor, key, dict(data), "out"))
+ for neighbor, _, key, data in brain.graph.in_edges(
+ node_uuid, keys=True, data=True
+ ):
+ edges.append((neighbor, key, dict(data), "in"))
+ return edges
+
+
+ def _event_path_records(
+ self,
+ brain,
+ node_uuids: list[str],
+ predicate_uuid: Optional[str] = None,
+ flow_key: Optional[str] = None,
+ ):
+ for n_uuid in node_uuids:
+ if n_uuid not in brain.graph:
+ continue
+ for m_uuid, r1_key, r1_data, r1_direction in self._incident_edges(
+ brain, n_uuid
+ ):
+ if (
+ predicate_uuid is not None
+ and (r1_data.get("uuid") or r1_key) != predicate_uuid
+ ):
+ continue
+ required_flow_key = (
+ flow_key if flow_key is not None else r1_data.get("flow_key")
+ )
+ if not required_flow_key:
+ continue
+ for b_uuid, r2_key, r2_data, r2_direction in self._incident_edges(
+ brain, m_uuid
+ ):
+ if r2_key == r1_key:
+ continue
+ if r2_data.get("flow_key") != required_flow_key:
+ continue
+ yield (
+ n_uuid,
+ m_uuid,
+ b_uuid,
+ r1_data,
+ r2_data,
+ r1_direction,
+ r2_direction,
+ )
+
+
def get_event_centric_neighbors(
self,
nodes: list[Node | str],
@@ -962,9 +1077,7 @@ class NetworkXGraphClient(GraphClient):
self._store.ensure_database(brain_id)
brain = self._store.get_brain(brain_id)
results = []
- for n_uuid, m_uuid, b_uuid, r1_data, r2_data, _, _ in self._event_path_records(brain, node_uuids):
- r_direction = "out"
- r2_direction = "out"
+ for n_uuid, m_uuid, b_uuid, r1_data, r2_data, r_direction, r2_direction in self._event_path_records(brain, node_uuids):
results.append(
(
self._node_from_brain(brain, n_uuid),
diff --git a/src/lib/postgresql/read_query.py b/src/lib/postgresql/read_query.py
index aefc8c4..3e71abd 100644
--- a/src/lib/postgresql/read_query.py
+++ b/src/lib/postgresql/read_query.py
@@ -17,12 +17,54 @@ _FORBIDDEN_SQL = re.compile(
r")\b",
re.IGNORECASE,
)
+_CYPHER_MARKERS = re.compile(
+ r"\b(MATCH|RETURN|OPTIONAL\s+MATCH|CREATE\s*\(|DETACH\s+DELETE|SET\s+n\.|REMOVE\s+n\.)\b",
+ re.IGNORECASE,
+)
+_LEGACY_TABLE_NAMES = re.compile(
+ r"\b(FROM|JOIN|TABLE|INTO|UPDATE)\s+(nodes|relationships)\b",
+ re.IGNORECASE,
+)
+_WITH_RECURSIVE = re.compile(r"\bWITH\s+RECURSIVE\b", re.IGNORECASE)
class ReadQueryValidationError(Exception):
pass
+def _paren_balance(sql: str) -> int:
+ """Return open-paren count minus close-paren count, ignoring quoted strings."""
+ depth = 0
+ in_single = False
+ in_double = False
+ i = 0
+ while i < len(sql):
+ ch = sql[i]
+ if in_single:
+ if ch == "'" and i + 1 < len(sql) and sql[i + 1] == "'":
+ i += 2
+ continue
+ if ch == "'":
+ in_single = False
+ i += 1
+ continue
+ if in_double:
+ if ch == '"':
+ in_double = False
+ i += 1
+ continue
+ if ch == "'":
+ in_single = True
+ elif ch == '"':
+ in_double = True
+ elif ch == "(":
+ depth += 1
+ elif ch == ")":
+ depth -= 1
+ i += 1
+ return depth
+
+
def validate_read_only_sql(query: str) -> str:
normalized = query.strip()
if not normalized:
@@ -31,10 +73,52 @@ def validate_read_only_sql(query: str) -> str:
if len(parts) != 1:
raise ReadQueryValidationError("Multiple statements are not allowed")
normalized = parts[0]
+ if _CYPHER_MARKERS.search(normalized):
+ raise ReadQueryValidationError(
+ "Cypher is not supported on PostgreSQL. Use SQL against kg_nodes "
+ "and kg_relationships (for example: SELECT uuid, data->>'name' AS name "
+ "FROM kg_nodes WHERE data->>'name' ILIKE '%alice%' LIMIT 50)."
+ )
+ if _LEGACY_TABLE_NAMES.search(normalized):
+ raise ReadQueryValidationError(
+ "Unknown table. Use kg_nodes and kg_relationships "
+ "(not nodes/relationships)."
+ )
if _FORBIDDEN_SQL.search(normalized):
raise ReadQueryValidationError("Only read-only SELECT queries are allowed")
if not _READ_QUERY_START.match(normalized):
raise ReadQueryValidationError(
"Query must start with SELECT, WITH, EXPLAIN, or TABLE"
)
+ # EXPLAIN of Cypher still starts with EXPLAIN; catch residual Cypher body.
+ if normalized.upper().startswith("EXPLAIN") and _CYPHER_MARKERS.search(
+ normalized[7:]
+ ):
+ raise ReadQueryValidationError(
+ "Cypher is not supported on PostgreSQL. Use SQL EXPLAIN SELECT ..."
+ )
+
+ balance = _paren_balance(normalized)
+ if balance != 0:
+ raise ReadQueryValidationError(
+ "Incomplete SQL (unbalanced parentheses). Send one complete statement "
+ "in a single tool call — do not split WITH RECURSIVE CTEs across calls. "
+ "Example: WITH RECURSIVE walk AS ("
+ "SELECT source_uuid, target_uuid, rel_type, 1 AS depth "
+ "FROM kg_relationships WHERE source_uuid = '<uuid>' "
+ "UNION ALL "
+ "SELECT r.source_uuid, r.target_uuid, r.rel_type, w.depth + 1 "
+ "FROM walk w JOIN kg_relationships r ON r.source_uuid = w.target_uuid "
+ "WHERE w.depth < 3"
+ ") SELECT * FROM walk LIMIT 50"
+ )
+ if _WITH_RECURSIVE.search(normalized):
+ # A recursive CTE must both define the CTE and select from it.
+ if not re.search(r"\)\s*SELECT\b", normalized, re.IGNORECASE):
+ raise ReadQueryValidationError(
+ "Incomplete WITH RECURSIVE query. Include the outer SELECT after "
+ "the CTE, e.g. ') SELECT * FROM walk LIMIT 50'. Do not send only "
+ "the CTE body or a fragment of the UNION ALL branch."
+ )
+
return normalized
diff --git a/src/services/api/constants/requests.py b/src/services/api/constants/requests.py
index f1cb571..8c0311e 100644
--- a/src/services/api/constants/requests.py
+++ b/src/services/api/constants/requests.py
@@ -393,6 +393,7 @@ class GetContextRequestBody(BaseModel):
text: str
brain_id: str = "default"
historical_limit: int = 10
+ max_facts: int = 40
class GetContextTriple(BaseModel):
diff --git a/src/services/api/controllers/retrieve.py b/src/services/api/controllers/retrieve.py
index 8fd2e0d..94d2457 100644
--- a/src/services/api/controllers/retrieve.py
+++ b/src/services/api/controllers/retrieve.py
@@ -10,7 +10,8 @@ Modified By: Christian Nonis <[email protected]>
import asyncio
import json
-from typing import Optional
+import threading
+from typing import Any, Optional
from fastapi import HTTPException
from starlette.responses import JSONResponse
@@ -18,6 +19,7 @@ from starlette.responses import JSONResponse
from src.constants.embeddings import Vector
from src.constants.kg import IdentificationParams, Node, Predicate
from src.core.search.entities import search_entities
+from src.core.search.entity_info import EventSynergyRetriever, MatchPath
from src.core.search.relationships import search_relationships
from src.utils.vector_search import VectorSearchFacade
from src.services.api.constants.requests import (
@@ -36,6 +38,10 @@ from src.utils.nlp.ner import _entity_extractor
vector_search = VectorSearchFacade(vector_store_adapter)
+_MAX_NOUN_CHUNKS = 5
+_MAX_DOSSIER_ENTITIES = 5
+_DOSSIER_MAX_DEPTH = 3
+
async def retrieve_data(
text: str, limit: int, preferred_entities: str, brain_id: str = "default"
@@ -381,6 +387,115 @@ async def get_entities(
)
+def _format_kg_item(item: Any) -> str:
+ name = getattr(item, "name", None) or ""
+ description = getattr(item, "description", None) or ""
+ happened_at = getattr(item, "happened_at", None) or ""
+ base = f"{name}: {description}" if description else name
+ if happened_at:
+ return f"{base} @{happened_at}"
+ return base
+
+
+def _format_event_fact(n: Node, r: Predicate, m: Node, r2: Predicate, b: Node) -> str:
+ return " | ".join(_format_kg_item(item) for item in (n, r, m, r2, b))
+
+
+def _fact_key(r: Predicate, r2: Predicate) -> tuple[str, str]:
+ return (getattr(r, "uuid", None) or "", getattr(r2, "uuid", None) or "")
+
+
+def _collect_query_variants(text: str, elements) -> list[str]:
+ seen: set[str] = set()
+ variants: list[str] = []
+
+ def _add(value: str | None) -> None:
+ if not value:
+ return
+ cleaned = value.strip()
+ if not cleaned:
+ return
+ key = cleaned.lower()
+ if key in seen:
+ return
+ seen.add(key)
+ variants.append(cleaned)
+
+ _add(text)
+ for token in getattr(elements, "tokens", None) or []:
+ if isinstance(token, dict):
+ _add(token.get("text"))
+ noun_chunks = getattr(elements, "noun_chunks", None) or []
+ added_chunks = 0
+ for chunk in noun_chunks:
+ if added_chunks >= _MAX_NOUN_CHUNKS:
+ break
+ before = len(variants)
+ _add(chunk if isinstance(chunk, str) else None)
+ if len(variants) > before:
+ added_chunks += 1
+ return variants
+
+
+def _seed_nodes_for_text(text: str, brain_id: str) -> list[tuple[str, float, str]]:
+ """Return (node_uuid, distance, entity_name) seeds from nodes + relationships."""
+ text_embeddings = embeddings_adapter.embed_text(text)
+ embeddings = text_embeddings.embeddings
+ seeds: list[tuple[str, float, str]] = []
+
+ node_vectors = vector_search.search_nodes(
+ embeddings,
+ brain_id=brain_id,
+ )
+ for vector in node_vectors:
+ meta = vector.metadata or {}
+ uuid = meta.get("uuid")
+ if not uuid:
+ continue
+ distance = (
+ vector.distance if vector.distance is not None else float("inf")
+ )
+ name = meta.get("name") or text
+ seeds.append((uuid, float(distance), str(name)))
+
+ rel_vectors = vector_search.search_relationships(
+ embeddings,
+ brain_id=brain_id,
+ )
+ for vector in rel_vectors:
+ meta = vector.metadata or {}
+ distance = (
+ vector.distance if vector.distance is not None else float("inf")
+ )
+ node_ids = meta.get("node_ids") or []
+ for node_id in node_ids:
+ if node_id:
+ seeds.append((str(node_id), float(distance), text))
+
+ return seeds
+
+
+def _flatten_match_path(path: MatchPath) -> list[str]:
+ lines: list[str] = []
+ current: MatchPath | None = path
+ while current is not None:
+ predicate, node = current.path
+ if node is None and not getattr(predicate, "name", None):
+ break
+ parts = []
+ if predicate and getattr(predicate, "name", None):
+ parts.append(_format_kg_item(predicate))
+ if node is not None:
+ parts.append(_format_kg_item(node))
+ if parts:
+ lines.append(" -> ".join(parts))
+ if current.children:
+ current = current.children[0]
+ else:
+ current = None
+ return lines
+
+
async def get_context(request: GetContextRequestBody) -> GetContextResponse:
"""
Retrieve contextual information for a text.
@@ -394,57 +509,66 @@ async def get_context(request: GetContextRequestBody) -> GetContextResponse:
GetContextResponse: Response containing the context information.
"""
- embeddings_map = {}
- futures = []
-
elements = _entity_extractor.extract_elements(request.text)
- relationships = []
- historical_context = []
- text_context = ""
-
- def _find_vs_nodes(text: str):
- nonlocal text_context, relationships
- if text in embeddings_map:
- return embeddings_map[text]
- text_embeddings = embeddings_adapter.embed_text(text)
- data_vectors = vector_search.search_nodes(
- text_embeddings.embeddings,
- brain_id=request.brain_id,
- )
+ variants = _collect_query_variants(request.text, elements)
+ historical_context: list[str] = []
+ candidate_lock = threading.Lock()
+ candidates: list[dict[str, Any]] = []
+ seed_hits: list[tuple[str, float, str]] = []
+
+ def _collect_facts_for_variant(text: str) -> None:
+ seeds = _seed_nodes_for_text(text, request.brain_id)
+ if not seeds:
+ return
+ with candidate_lock:
+ seed_hits.extend(seeds)
+ uuid_to_distance: dict[str, float] = {}
+ for uuid, distance, _ in seeds:
+ prev = uuid_to_distance.get(uuid)
+ if prev is None or distance < prev:
+ uuid_to_distance[uuid] = distance
+ seed_uuids = list(uuid_to_distance.keys())
neighbors = graph_adapter.get_event_centric_neighbors(
- [v.metadata.get("uuid") for v in data_vectors], brain_id=request.brain_id
+ seed_uuids, brain_id=request.brain_id
)
- if len(neighbors) > 0:
- for n, r, m, r2, b in neighbors:
- embeddings_map[text] = (text, n, r, m, r2, b)
- relationships.append((text, n, r, m, r2, b))
- text_context += (
- "\n".join(
- f"{item.name}: {item.description}" for item in (n, r, m, r2, b)
- )
- + "\n"
+ local: list[dict[str, Any]] = []
+ for n, r, m, r2, b in neighbors:
+ distances = [
+ uuid_to_distance[u]
+ for u in (
+ getattr(n, "uuid", None),
+ getattr(m, "uuid", None),
+ getattr(b, "uuid", None),
)
- return GetContextTriple(identified_entity=text, triple=(n, r, m, r2, b))
- return ()
+ if u in uuid_to_distance
+ ]
+ score = min(distances) if distances else float("inf")
+ local.append(
+ {
+ "identified_entity": text,
+ "triple": (n, r, m, r2, b),
+ "score": score,
+ "key": _fact_key(r, r2),
+ }
+ )
+ if local:
+ with candidate_lock:
+ candidates.extend(local)
async def _get_historical_context():
nonlocal historical_context
- _futures = []
- _futures.append(
+ text_chunks, structured_data = await asyncio.gather(
asyncio.to_thread(
data_adapter.get_last_text_chunks,
brain_id=request.brain_id,
limit=request.historical_limit,
- )
- )
- _futures.append(
+ ),
asyncio.to_thread(
data_adapter.get_last_structured_data,
brain_id=request.brain_id,
limit=request.historical_limit,
- )
+ ),
)
- text_chunks, structured_data = await asyncio.gather(*_futures)
historical_context = [text_chunk.text for text_chunk in text_chunks] + [
json.dumps(structured_data.data)
for structured_data in structured_data
@@ -452,21 +576,82 @@ async def get_context(request: GetContextRequestBody) -> GetContextResponse:
]
return historical_context
- futures.append(asyncio.to_thread(_find_vs_nodes, request.text))
- for chunk in elements.tokens:
- futures.append(asyncio.to_thread(_find_vs_nodes, chunk["text"]))
+ futures = [
+ asyncio.to_thread(_collect_facts_for_variant, variant) for variant in variants
+ ]
futures.append(_get_historical_context())
-
await asyncio.gather(*futures)
- return GetContextResponse(
- text_context=text_context,
- triples=[
+ deduped: dict[tuple[str, str], dict[str, Any]] = {}
+ for candidate in candidates:
+ key = candidate["key"]
+ existing = deduped.get(key)
+ if existing is None or candidate["score"] < existing["score"]:
+ deduped[key] = candidate
+
+ ranked = sorted(deduped.values(), key=lambda c: c["score"])
+ max_facts = max(1, int(getattr(request, "max_facts", 40) or 40))
+ curated = ranked[:max_facts]
+
+ text_lines: list[str] = []
+ triples: list[GetContextTriple] = []
+ for candidate in curated:
+ n, r, m, r2, b = candidate["triple"]
+ text_lines.append(_format_event_fact(n, r, m, r2, b))
+ triples.append(
GetContextTriple(
- identified_entity=t[0],
- triple=(t[1], t[2], t[3], t[4], t[5]),
+ identified_entity=candidate["identified_entity"],
+ triple=(n, r, m, r2, b),
)
- for t in relationships
- ],
+ )
+
+ entity_scores: dict[str, float] = {}
+ for _uuid, distance, entity_name in seed_hits:
+ name = (entity_name or "").strip()
+ if not name:
+ continue
+ if name.lower() == request.text.strip().lower():
+ continue
+ prev = entity_scores.get(name)
+ if prev is None or distance < prev:
+ entity_scores[name] = distance
+ top_entities = [
+ name
+ for name, _ in sorted(entity_scores.items(), key=lambda item: item[1])[
+ :_MAX_DOSSIER_ENTITIES
+ ]
+ ]
+
+ dossier_lines: list[str] = []
+ if top_entities:
+
+ def _run_dossiers() -> list[str]:
+ lines: list[str] = []
+ retriever = EventSynergyRetriever(request.brain_id)
+ for entity_name in top_entities:
+ try:
+ path = retriever.retrieve_matches(
+ entity_name,
+ request.text,
+ max_depth=_DOSSIER_MAX_DEPTH,
+ )
+ except Exception:
+ continue
+ if path is None or path.target_node is None:
+ continue
+ flattened = _flatten_match_path(path)
+ if not flattened:
+ continue
+ target_name = getattr(path.target_node, "name", None) or entity_name
+ lines.append(f"[dossier:{target_name}] " + " | ".join(flattened))
+ return lines
+
+ dossier_lines = await asyncio.to_thread(_run_dossiers)
+
+ text_context = "\n".join(text_lines + dossier_lines)
+
+ return GetContextResponse(
+ text_context=text_context,
+ triples=triples,
historical_context=historical_context,
)
diff --git a/src/utils/normalization/list_reduction.py b/src/utils/normalization/list_reduction.py
index 3f3fcc4..5d622ae 100644
--- a/src/utils/normalization/list_reduction.py
+++ b/src/utils/normalization/list_reduction.py
@@ -12,11 +12,6 @@ from typing import List, Optional, TypeVar
from src.utils.similarity.vectors import cosine_similarity
from pydantic import BaseModel
-try:
- from src.lib.embeddings.client_small import EmbeddingsClientSmall
-except ModuleNotFoundError:
- EmbeddingsClientSmall = None
-
T = TypeVar("T")
@@ -70,10 +65,22 @@ def _fallback_embed_text(text: str) -> list[float]:
return [value / divisor for value in buckets]
+def _resolve_embeddings_client():
+ try:
+ from src.core.instances import embeddings_small_adapter
+
+ return embeddings_small_adapter
+ except Exception:
+ return None
+
+
def _embed_text(embeddings_client, text: str) -> list[float]:
if embeddings_client is None:
return _fallback_embed_text(text)
- return embeddings_client.embed_text(text)
+ result = embeddings_client.embed_text(text)
+ if hasattr(result, "embeddings"):
+ return list(result.embeddings)
+ return list(result)
def reduce_list(
@@ -95,7 +102,7 @@ def reduce_list(
if not list_:
return []
- embeddings_client = EmbeddingsClientSmall() if EmbeddingsClientSmall else None
+ embeddings_client = _resolve_embeddings_client()
reduced_list = []
item_embeddings = []
diff --git a/src/workers/tasks/ingestion.py b/src/workers/tasks/ingestion.py
index aad19ae..460d5f6 100644
--- a/src/workers/tasks/ingestion.py
+++ b/src/workers/tasks/ingestion.py
@@ -864,6 +864,10 @@ def process_architect_relationships(self, args: dict):
if len(item_errors) < len(relationships_data)
else "failed"
)
+ print(
+ f"[!] Relationship persistence failures ({len(item_errors)}): "
+ f"{item_errors}"
+ )
set_ingestion_task_status(
status_task_id,
brain_id,
diff --git a/tests/test_context_retrieval.py b/tests/test_context_retrieval.py
new file mode 100644
index 0000000..0534fcd
--- /dev/null
+++ b/tests/test_context_retrieval.py
@@ -0,0 +1,537 @@
+import asyncio
+import os
+import sys
+import types
+import unittest
+from unittest.mock import MagicMock, patch
+
+
+ENV_DEFAULTS = {
+ "BRAINPAT_TOKEN": "test-token",
+ "MODELS_MODE": "local",
+ "EMBEDDINGS_LOCAL_MODEL": "local-model",
+ "EMBEDDINGS_SMALL_MODEL": "small-model",
+ "EMBEDDING_NODES_DIMENSION": "3",
+ "EMBEDDING_TRIPLETS_DIMENSION": "3",
+ "EMBEDDING_OBSERVATIONS_DIMENSION": "3",
+ "EMBEDDING_DATA_DIMENSION": "3",
+ "EMBEDDING_RELATIONSHIPS_DIMENSION": "3",
+ "REDIS_HOST": "localhost",
+ "REDIS_PORT": "6379",
+ "NEO4J_HOST": "localhost",
+ "NEO4J_PORT": "7687",
+ "NEO4J_USERNAME": "neo4j",
+ "NEO4J_PASSWORD": "password",
+ "MILVUS_HOST": "localhost",
+ "MILVUS_PORT": "19530",
+ "MONGO_CONNECTION_STRING": "mongodb://localhost:27017",
+ "CELERY_WORKER_CONCURRENCY": "1",
+ "OLLAMA_HOST": "localhost",
+ "OLLAMA_PORT": "11434",
+ "OLLAMA_LLM_SMALL_MODEL": "small",
+ "OLLAMA_LLM_LARGE_MODEL": "large",
+ "POSTGRES_HOST": "localhost",
+ "POSTGRES_PORT": "5432",
+ "POSTGRES_USERNAME": "postgres",
+ "POSTGRES_PASSWORD": "postgres",
+}
+for key, value in ENV_DEFAULTS.items():
+ os.environ.setdefault(key, value)
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from src.constants.embeddings import Vector
+from src.constants.kg import Node, Predicate
+from src.core.search.entity_info import EventSynergyRetriever, MatchPath
+from src.services.api.constants.requests import GetContextRequestBody
+from src.utils.nlp.ner import ExtractElementsResponse
+
+
+def _node(uuid: str, name: str, *, happened_at: str | None = None, labels=None):
+ return Node(
+ uuid=uuid,
+ name=name,
+ labels=labels or ["PERSON"],
+ description=f"desc-{name}",
+ happened_at=happened_at,
+ )
+
+
+def _pred(uuid: str, name: str):
+ return Predicate(uuid=uuid, name=name, description=f"desc-{name}")
+
+
+class CollectQueryVariantsTests(unittest.TestCase):
+ def test_includes_noun_chunks_deduped_and_capped(self):
+ from src.services.api.controllers import retrieve as retrieve_mod
+
+ elements = ExtractElementsResponse(
+ tokens=[{"text": "Alice", "lemma": "Alice", "pos": "PROPN"}],
+ noun_chunks=[
+ "closest friend",
+ "Alice",
+ "charity event",
+ "weekend trip",
+ "old school",
+ "extra chunk",
+ "another one",
+ ],
+ )
+ variants = retrieve_mod._collect_query_variants(
+ "What did Alice do with her closest friend?", elements
+ )
+ self.assertEqual(variants[0], "What did Alice do with her closest friend?")
+ self.assertIn("Alice", variants)
+ self.assertIn("closest friend", variants)
+ self.assertIn("charity event", variants)
+ noun_only = [
+ v
+ for v in variants
+ if v
+ not in {
+ "What did Alice do with her closest friend?",
+ "Alice",
+ }
+ ]
+ self.assertLessEqual(len(noun_only), retrieve_mod._MAX_NOUN_CHUNKS)
+
+
+class FormatHelpersTests(unittest.TestCase):
+ def test_format_includes_happened_at(self):
+ from src.services.api.controllers import retrieve as retrieve_mod
+
+ n = _node("a", "Alice")
+ r = _pred("r1", "ATTENDED")
+ m = _node("e", "Wedding", happened_at="2024-05-01", labels=["EVENT"])
+ r2 = _pred("r2", "INVOLVED")
+ b = _node("b", "Bob")
+ line = retrieve_mod._format_event_fact(n, r, m, r2, b)
+ self.assertIn("@2024-05-01", line)
+ self.assertIn("Wedding", line)
+
+
+class GetContextRetrievalTests(unittest.IsolatedAsyncioTestCase):
+ async def test_retains_all_event_facts_not_just_first(self):
+ from src.services.api.controllers import retrieve as retrieve_mod
+
+ facts = [
+ (
+ _node("a", "Alice"),
+ _pred("r1", "ATTENDED"),
+ _node("e1", "Event1", happened_at="2024-01-01", labels=["EVENT"]),
+ _pred("r2", "WITH"),
+ _node("b", "Bob"),
+ ),
+ (
+ _node("a", "Alice"),
+ _pred("r3", "VISITED"),
+ _node("e2", "Event2", happened_at="2024-02-01", labels=["EVENT"]),
+ _pred("r4", "IN"),
+ _node("c", "Paris", labels=["LOCATION"]),
+ ),
+ (
+ _node("a", "Alice"),
+ _pred("r5", "BOUGHT"),
+ _node("e3", "Event3", labels=["EVENT"]),
+ _pred("r6", "ITEM"),
+ _node("d", "Book", labels=["OBJECT"]),
+ ),
+ (
+ _node("a", "Alice"),
+ _pred("r7", "MET"),
+ _node("e4", "Event4", labels=["EVENT"]),
+ _pred("r8", "WITH"),
+ _node("f", "Carol"),
+ ),
+ (
+ _node("a", "Alice"),
+ _pred("r9", "MOVED"),
+ _node("e5", "Event5", labels=["EVENT"]),
+ _pred("r10", "TO"),
+ _node("g", "London", labels=["LOCATION"]),
+ ),
+ ]
+
+ mock_embeddings = MagicMock()
+ mock_embeddings.embed_text.return_value = Vector(
+ id="q", embeddings=[0.1, 0.2, 0.3], metadata={}
+ )
+
+ mock_vs = MagicMock()
+ mock_vs.search_nodes.return_value = [
+ Vector(
+ id="n1",
+ embeddings=[0.1, 0.2, 0.3],
+ metadata={"uuid": "a", "name": "Alice"},
+ distance=0.1,
+ )
+ ]
+ mock_vs.search_relationships.return_value = []
+
+ mock_graph = MagicMock()
+ mock_graph.get_event_centric_neighbors.return_value = facts
+
+ mock_data = MagicMock()
+ mock_data.get_last_text_chunks.return_value = []
+ mock_data.get_last_structured_data.return_value = []
+
+ elements = ExtractElementsResponse(tokens=[], noun_chunks=[])
+
+ with (
+ patch.object(retrieve_mod, "embeddings_adapter", mock_embeddings),
+ patch.object(retrieve_mod, "vector_search", mock_vs),
+ patch.object(retrieve_mod, "graph_adapter", mock_graph),
+ patch.object(retrieve_mod, "data_adapter", mock_data),
+ patch.object(
+ retrieve_mod._entity_extractor,
+ "extract_elements",
+ return_value=elements,
+ ),
+ patch.object(
+ retrieve_mod,
+ "EventSynergyRetriever",
+ return_value=MagicMock(
+ retrieve_matches=MagicMock(
+ return_value=MatchPath(
+ target_node=None,
+ path=(_pred("", ""), None),
+ similarity=0.0,
+ )
+ )
+ ),
+ ),
+ ):
+ response = await retrieve_mod.get_context(
+ GetContextRequestBody(text="Alice", brain_id="brain-a", max_facts=40)
+ )
+
+ self.assertEqual(len(response.triples), 5)
+ self.assertIn("@2024-01-01", response.text_context)
+ self.assertIn("Paris", response.text_context)
+
+ async def test_relationship_vector_search_seeds_facts(self):
+ from src.services.api.controllers import retrieve as retrieve_mod
+
+ fact = (
+ _node("tail", "Alice"),
+ _pred("r1", "ORGANIZED"),
+ _node("mid", "Fundraiser", happened_at="2023-11-11", labels=["EVENT"]),
+ _pred("r2", "FOR"),
+ _node("tip", "Charity", labels=["ORG"]),
+ )
+
+ mock_embeddings = MagicMock()
+ mock_embeddings.embed_text.return_value = Vector(
+ id="q", embeddings=[0.1, 0.2, 0.3], metadata={}
+ )
+
+ mock_vs = MagicMock()
+ mock_vs.search_nodes.return_value = []
+ mock_vs.search_relationships.return_value = [
+ Vector(
+ id="rel1",
+ embeddings=[0.1, 0.2, 0.3],
+ metadata={"uuid": "r1", "node_ids": ["tail", "mid"]},
+ distance=0.05,
+ )
+ ]
+
+ mock_graph = MagicMock()
+ mock_graph.get_event_centric_neighbors.return_value = [fact]
+
+ mock_data = MagicMock()
+ mock_data.get_last_text_chunks.return_value = []
+ mock_data.get_last_structured_data.return_value = []
+
+ elements = ExtractElementsResponse(tokens=[], noun_chunks=[])
+
+ with (
+ patch.object(retrieve_mod, "embeddings_adapter", mock_embeddings),
+ patch.object(retrieve_mod, "vector_search", mock_vs),
+ patch.object(retrieve_mod, "graph_adapter", mock_graph),
+ patch.object(retrieve_mod, "data_adapter", mock_data),
+ patch.object(
+ retrieve_mod._entity_extractor,
+ "extract_elements",
+ return_value=elements,
+ ),
+ patch.object(
+ retrieve_mod,
+ "EventSynergyRetriever",
+ return_value=MagicMock(
+ retrieve_matches=MagicMock(
+ return_value=MatchPath(
+ target_node=None,
+ path=(_pred("", ""), None),
+ similarity=0.0,
+ )
+ )
+ ),
+ ),
+ ):
+ response = await retrieve_mod.get_context(
+ GetContextRequestBody(
+ text="who organized the fundraiser",
+ brain_id="brain-a",
+ )
+ )
+
+ mock_graph.get_event_centric_neighbors.assert_called()
+ seed_uuids = mock_graph.get_event_centric_neighbors.call_args[0][0]
+ self.assertIn("tail", seed_uuids)
+ self.assertIn("mid", seed_uuids)
+ self.assertEqual(len(response.triples), 1)
+ self.assertIn("Fundraiser", response.text_context)
+
+ async def test_max_facts_cap_and_dedup(self):
+ from src.services.api.controllers import retrieve as retrieve_mod
+
+ shared = (
+ _node("a", "Alice"),
+ _pred("r1", "ATTENDED"),
+ _node("e1", "Event1", labels=["EVENT"]),
+ _pred("r2", "WITH"),
+ _node("b", "Bob"),
+ )
+ other = (
+ _node("a", "Alice"),
+ _pred("r3", "VISITED"),
+ _node("e2", "Event2", labels=["EVENT"]),
+ _pred("r4", "IN"),
+ _node("c", "Paris", labels=["LOCATION"]),
+ )
+
+ mock_embeddings = MagicMock()
+ mock_embeddings.embed_text.return_value = Vector(
+ id="q", embeddings=[0.1, 0.2, 0.3], metadata={}
+ )
+ mock_vs = MagicMock()
+ mock_vs.search_nodes.return_value = [
+ Vector(
+ id="n1",
+ embeddings=[0.1, 0.2, 0.3],
+ metadata={"uuid": "a", "name": "Alice"},
+ distance=0.2,
+ )
+ ]
+ mock_vs.search_relationships.return_value = []
+ mock_graph = MagicMock()
+ mock_graph.get_event_centric_neighbors.return_value = [shared, shared, other]
+ mock_data = MagicMock()
+ mock_data.get_last_text_chunks.return_value = []
+ mock_data.get_last_structured_data.return_value = []
+ elements = ExtractElementsResponse(tokens=[], noun_chunks=[])
+
+ with (
+ patch.object(retrieve_mod, "embeddings_adapter", mock_embeddings),
+ patch.object(retrieve_mod, "vector_search", mock_vs),
+ patch.object(retrieve_mod, "graph_adapter", mock_graph),
+ patch.object(retrieve_mod, "data_adapter", mock_data),
+ patch.object(
+ retrieve_mod._entity_extractor,
+ "extract_elements",
+ return_value=elements,
+ ),
+ patch.object(
+ retrieve_mod,
+ "EventSynergyRetriever",
+ return_value=MagicMock(
+ retrieve_matches=MagicMock(
+ return_value=MatchPath(
+ target_node=None,
+ path=(_pred("", ""), None),
+ similarity=0.0,
+ )
+ )
+ ),
+ ),
+ ):
+ response = await retrieve_mod.get_context(
+ GetContextRequestBody(text="Alice", brain_id="brain-a", max_facts=1)
+ )
+
+ self.assertEqual(len(response.triples), 1)
+
+ async def test_dossier_lines_appended_to_text_context(self):
+ from src.services.api.controllers import retrieve as retrieve_mod
+
+ fact = (
+ _node("a", "Alice"),
+ _pred("r1", "KNOWS"),
+ _node("e1", "Meeting", labels=["EVENT"]),
+ _pred("r2", "WITH"),
+ _node("b", "Bob"),
+ )
+ dossier = MatchPath(
+ target_node=_node("a", "Alice"),
+ path=(_pred("d1", "FRIEND_OF"), _node("b", "Bob")),
+ similarity=0.9,
+ children=[
+ MatchPath(
+ target_node=_node("a", "Alice"),
+ path=(
+ _pred("d2", "LIVES_IN"),
+ _node("c", "Chicago", labels=["LOCATION"]),
+ ),
+ similarity=0.8,
+ )
+ ],
+ )
+
+ mock_embeddings = MagicMock()
+ mock_embeddings.embed_text.return_value = Vector(
+ id="q", embeddings=[0.1, 0.2, 0.3], metadata={}
+ )
+ mock_vs = MagicMock()
+ mock_vs.search_nodes.return_value = [
+ Vector(
+ id="n1",
+ embeddings=[0.1, 0.2, 0.3],
+ metadata={"uuid": "a", "name": "Alice"},
+ distance=0.1,
+ )
+ ]
+ mock_vs.search_relationships.return_value = []
+ mock_graph = MagicMock()
+ mock_graph.get_event_centric_neighbors.return_value = [fact]
+ mock_data = MagicMock()
+ mock_data.get_last_text_chunks.return_value = []
+ mock_data.get_last_structured_data.return_value = []
+ elements = ExtractElementsResponse(
+ tokens=[{"text": "Alice", "lemma": "Alice", "pos": "PROPN"}],
+ noun_chunks=[],
+ )
+ retriever = MagicMock()
+ retriever.retrieve_matches.return_value = dossier
+
+ with (
+ patch.object(retrieve_mod, "embeddings_adapter", mock_embeddings),
+ patch.object(retrieve_mod, "vector_search", mock_vs),
+ patch.object(retrieve_mod, "graph_adapter", mock_graph),
+ patch.object(retrieve_mod, "data_adapter", mock_data),
+ patch.object(
+ retrieve_mod._entity_extractor,
+ "extract_elements",
+ return_value=elements,
+ ),
+ patch.object(retrieve_mod, "EventSynergyRetriever", return_value=retriever),
+ ):
+ response = await retrieve_mod.get_context(
+ GetContextRequestBody(
+ text="Where does Alice's friend live?",
+ brain_id="brain-a",
+ )
+ )
+
+ self.assertIn("[dossier:Alice]", response.text_context)
+ self.assertIn("Chicago", response.text_context)
+ self.assertEqual(len(response.triples), 1)
+
+
+class EventSynergyHardeningTests(unittest.TestCase):
+ def test_explores_top_k_branches_not_only_greedy_best(self):
+ alice = _node("a", "Alice")
+ wrong = _node("w", "Wrong")
+ mid = _node("m", "Bridge")
+ goal = _node("g", "Goal")
+ rel_wrong = _pred("rw", "DISTRACTOR")
+ rel_wrong.properties = {"v_id": "v_wrong"}
+ rel_bridge = _pred("rb", "VIA")
+ rel_bridge.properties = {"v_id": "v_bridge"}
+ rel_goal = _pred("rg", "REACHES")
+ rel_goal.properties = {"v_id": "v_goal"}
+
+ neighbors = {
+ "a": [(rel_wrong, wrong), (rel_bridge, mid)],
+ "m": [(rel_goal, goal)],
+ "w": [],
+ "g": [],
+ }
+
+ embeddings = {
+ "v_wrong": [1.0, 0.0, 0.0],
+ "v_bridge": [0.8, 0.2, 0.0],
+ "v_goal": [0.0, 1.0, 0.0],
+ }
+ query_embedding = [0.0, 1.0, 0.0]
+
+ def fake_get_by_ids(ids, store="relationships", brain_id="default"):
+ vid = ids[0]
+ return [
+ Vector(id=vid, embeddings=embeddings[vid], metadata={})
+ ]
+
+ with (
+ patch(
+ "src.core.search.entity_info.graph_adapter.get_neighbors",
+ side_effect=lambda nodes, brain_id="default", **kwargs: {
+ nodes[0]: neighbors.get(nodes[0], [])
+ },
+ ),
+ patch(
+ "src.core.search.entity_info.vector_store_adapter.get_by_ids",
+ side_effect=fake_get_by_ids,
+ ),
+ patch(
+ "src.core.search.entity_info.cosine_similarity",
+ side_effect=lambda a, b: sum(x * y for x, y in zip(a, b)),
+ ),
+ ):
+ retriever = EventSynergyRetriever("brain-a")
+ path = retriever._recursive_explorer(
+ "a",
+ query_embedding,
+ depth=3,
+ visited_ids=set(),
+ work_counter=[0],
+ branch_factor=3,
+ )
+
+ self.assertGreaterEqual(len(path), 2)
+ hop_names = [hop[0][0].name for hop in path]
+ self.assertIn("VIA", hop_names)
+ self.assertIn("REACHES", hop_names)
+
+ def test_work_cap_stops_exploration(self):
+ nodes = {f"n{i}": _node(f"n{i}", f"N{i}") for i in range(20)}
+ rels = {}
+ for i in range(19):
+ pred = _pred(f"r{i}", f"REL{i}")
+ pred.properties = {"v_id": f"v{i}"}
+ rels[f"n{i}"] = [(pred, nodes[f"n{i+1}"])]
+ rels["n19"] = []
+
+ with (
+ patch(
+ "src.core.search.entity_info.graph_adapter.get_neighbors",
+ side_effect=lambda nodes_arg, brain_id="default", **kwargs: {
+ nodes_arg[0]: rels.get(nodes_arg[0], [])
+ },
+ ),
+ patch(
+ "src.core.search.entity_info.vector_store_adapter.get_by_ids",
+ side_effect=lambda ids, store="relationships", brain_id="default": [
+ Vector(id=ids[0], embeddings=[1.0, 0.0], metadata={})
+ ],
+ ),
+ patch(
+ "src.core.search.entity_info.cosine_similarity",
+ return_value=0.5,
+ ),
+ patch("src.core.search.entity_info._MAX_EXPLORATION_WORK", 3),
+ ):
+ retriever = EventSynergyRetriever("brain-a")
+ work = [0]
+ retriever._recursive_explorer(
+ "n0",
+ [1.0, 0.0],
+ depth=20,
+ visited_ids=set(),
+ work_counter=work,
+ branch_factor=1,
+ )
+ self.assertLessEqual(work[0], 3)
+
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/tests/test_graph_upsert_contract.py b/tests/test_graph_upsert_contract.py
index 9999516..ddd2626 100644
--- a/tests/test_graph_upsert_contract.py
+++ b/tests/test_graph_upsert_contract.py
@@ -82,7 +82,9 @@ class NetworkXUpsertContractTests(unittest.TestCase):
brain.upsert_node(["PERSON"], {"uuid": "a"}, {"uuid": "a", "name": "Alice"})
brain.upsert_node(["PRODUCT"], {"uuid": "b"}, {"uuid": "b", "name": "Widget"})
store._brains = {"brain-a": brain}
+ store._brains_lock = __import__("threading").RLock()
store._persist_relationship = MagicMock()
+ store._persist_node = MagicMock()
first = store.merge_relationship(
"brain-a",
@@ -113,6 +115,65 @@ class NetworkXUpsertContractTests(unittest.TestCase):
self.assertEqual(edges[0][2], "rel-1")
self.assertEqual(edges[0][3]["description"], "v2")
+ def test_relationship_upsert_creates_missing_endpoint_nodes(self):
+ from src.lib.postgresql.graph_store import PostgreSQLGraphStore
+ from src.lib.postgresql.graph_store import _BrainGraph
+
+ store = PostgreSQLGraphStore.__new__(PostgreSQLGraphStore)
+ brain = _BrainGraph("brain-a")
+ store._brains = {"brain-a": brain}
+ store._brains_lock = __import__("threading").RLock()
+ store._persist_node = MagicMock()
+ store._persist_relationship = MagicMock()
+
+ result = store.merge_relationship(
+ "brain-a",
+ ["PERSON"],
+ "Alice",
+ ["PRODUCT"],
+ "Widget",
+ "OWNS",
+ {"uuid": "rel-1", "description": "created"},
+ subject_uuid="a",
+ object_uuid="b",
+ )
+ self.assertIsNotNone(result)
+ self.assertEqual(set(brain.graph.nodes), {"a", "b"})
+ self.assertEqual(len(list(brain.graph.edges(keys=True))), 1)
+ store._persist_node.assert_called()
+ store._persist_relationship.assert_called_once()
+
+ def test_normalize_read_sql_rewrites_jsonb_ilike(self):
+ from src.lib.postgresql.graph_store import normalize_read_sql
+
+ sql = (
+ "SELECT uuid, data FROM kg_nodes WHERE data->>'name' ILIKE '%Reminding%' "
+ "OR data->'description' ILIKE '%remind%' LIMIT 50"
+ )
+ fixed = normalize_read_sql(sql)
+ self.assertIn("data->>'description' ILIKE", fixed)
+ self.assertNotIn("data->'description' ILIKE", fixed)
+
+ def test_stale_cache_reloads_when_db_counts_diverge(self):
+ from src.lib.postgresql.graph_store import PostgreSQLGraphStore
+ from src.lib.postgresql.graph_store import _BrainGraph
+
+ store = PostgreSQLGraphStore.__new__(PostgreSQLGraphStore)
+ stale = _BrainGraph("brain-a")
+ store._brains = {"brain-a": stale}
+ store._brains_lock = __import__("threading").RLock()
+ store._db_graph_counts = MagicMock(return_value=(2, 1))
+
+ fresh = _BrainGraph("brain-a")
+ fresh.graph.add_node("a", name="Alice", labels=["PERSON"])
+ fresh.graph.add_node("b", name="Bob", labels=["PERSON"])
+ fresh.graph.add_edge("a", "b", key="r1", rel_type="KNOWS")
+ store._reload_brain_locked = MagicMock(return_value=fresh)
+
+ loaded = store._load_brain("brain-a")
+ self.assertIs(loaded, fresh)
+ store._reload_brain_locked.assert_called_once_with("brain-a")
+
class Neo4jUpsertContractTests(unittest.TestCase):
def test_add_nodes_merges_by_uuid_not_name(self):
diff --git a/tests/test_networkx_event_centric.py b/tests/test_networkx_event_centric.py
new file mode 100644
index 0000000..1ecd7df
--- /dev/null
+++ b/tests/test_networkx_event_centric.py
@@ -0,0 +1,127 @@
+import os
+import sys
+import unittest
+
+ENV_DEFAULTS = {
+ "BRAINPAT_TOKEN": "test-token",
+ "MODELS_MODE": "local",
+ "EMBEDDINGS_LOCAL_MODEL": "local-model",
+ "EMBEDDINGS_SMALL_MODEL": "small-model",
+ "EMBEDDING_NODES_DIMENSION": "3",
+ "EMBEDDING_TRIPLETS_DIMENSION": "3",
+ "EMBEDDING_OBSERVATIONS_DIMENSION": "3",
+ "EMBEDDING_DATA_DIMENSION": "3",
+ "EMBEDDING_RELATIONSHIPS_DIMENSION": "3",
+ "REDIS_HOST": "localhost",
+ "REDIS_PORT": "6379",
+ "NEO4J_HOST": "localhost",
+ "NEO4J_PORT": "7687",
+ "NEO4J_USERNAME": "neo4j",
+ "NEO4J_PASSWORD": "password",
+ "MILVUS_HOST": "localhost",
+ "MILVUS_PORT": "19530",
+ "MONGO_CONNECTION_STRING": "mongodb://localhost:27017",
+ "CELERY_WORKER_CONCURRENCY": "1",
+ "OLLAMA_HOST": "localhost",
+ "OLLAMA_PORT": "11434",
+ "OLLAMA_LLM_SMALL_MODEL": "small",
+ "OLLAMA_LLM_LARGE_MODEL": "large",
+ "POSTGRES_HOST": "localhost",
+ "POSTGRES_PORT": "5432",
+ "POSTGRES_USERNAME": "postgres",
+ "POSTGRES_PASSWORD": "postgres",
+}
+for key, value in ENV_DEFAULTS.items():
+ os.environ.setdefault(key, value)
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+try:
+ from src.lib.postgresql.graph_store import _BrainGraph
+ from src.lib.postgresql.networkx_client import NetworkXGraphClient
+
+ DEPS_AVAILABLE = True
+except ImportError:
+ _BrainGraph = None
+ NetworkXGraphClient = None
+ DEPS_AVAILABLE = False
+
+
+class _FakeStore:
+ def __init__(self, brain):
+ self._brain = brain
+
+ def ensure_database(self, brain_id):
+ return None
+
+ def get_brain(self, brain_id):
+ return self._brain
+
+
+def _build_brain():
+ brain = _BrainGraph("test-brain")
+ brain.graph.add_node("A", uuid="A", name="Alice", labels=["Person"])
+ brain.graph.add_node("B", uuid="B", name="Meeting", labels=["Event"])
+ brain.graph.add_node("C", uuid="C", name="Bob", labels=["Person"])
+ brain.graph.add_node("D", uuid="D", name="Elsewhere", labels=["Event"])
+ brain.graph.add_edge(
+ "A", "B", key="r1", uuid="r1", rel_type="CAUSED", flow_key="flow1"
+ )
+ brain.graph.add_edge(
+ "B", "C", key="r2", uuid="r2", rel_type="LED_TO", flow_key="flow1"
+ )
+ brain.graph.add_edge(
+ "B", "D", key="r3", uuid="r3", rel_type="UNRELATED", flow_key="other"
+ )
+ return brain
+
+
[email protected](DEPS_AVAILABLE, "postgresql/networkx dependencies unavailable")
+class NetworkXEventCentricTests(unittest.TestCase):
+ def _client(self):
+ client = object.__new__(NetworkXGraphClient)
+ client._store = _FakeStore(_build_brain())
+ return client
+
+ def test_event_path_records_helper_exists(self):
+ self.assertTrue(hasattr(NetworkXGraphClient, "_event_path_records"))
+ self.assertTrue(hasattr(NetworkXGraphClient, "_node_from_brain"))
+ self.assertTrue(hasattr(NetworkXGraphClient, "_predicate_from_edge"))
+
+ def test_get_event_centric_neighbors_returns_matching_flow_paths(self):
+ client = self._client()
+ results = client.get_event_centric_neighbors(["A"], "test-brain")
+ self.assertEqual(len(results), 1)
+ n, r1, m, r2, b = results[0]
+ self.assertEqual(n.uuid, "A")
+ self.assertEqual(m.uuid, "B")
+ self.assertEqual(b.uuid, "C")
+ self.assertEqual(r1.name, "CAUSED")
+ self.assertEqual(r2.name, "LED_TO")
+ self.assertEqual(r1.flow_key, "flow1")
+ self.assertEqual(r2.flow_key, "flow1")
+
+ def test_get_event_centric_neighbors_empty_for_unknown_node(self):
+ client = self._client()
+ self.assertEqual(client.get_event_centric_neighbors(["Z"], "test-brain"), [])
+
+ def test_get_event_centric_neighbors_empty_input(self):
+ client = self._client()
+ self.assertEqual(client.get_event_centric_neighbors([], "test-brain"), [])
+
+ def test_get_nexts_by_flow_key_returns_next_triple(self):
+ client = self._client()
+ res = client.get_nexts_by_flow_key(
+ [{"predicate_uuid": "r1", "flow_key": "flow1"}], "test-brain"
+ )
+ self.assertIn("r1", res)
+ triples = res["r1"]
+ self.assertEqual(len(triples), 1)
+ m, r2, b = triples[0]
+ self.assertEqual(m.uuid, "B")
+ self.assertEqual(r2.name, "LED_TO")
+ self.assertEqual(b.uuid, "C")
+
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/tests/test_postgresql_graph_read_query.py b/tests/test_postgresql_graph_read_query.py
index 56aca2f..9daa639 100644
--- a/tests/test_postgresql_graph_read_query.py
+++ b/tests/test_postgresql_graph_read_query.py
@@ -106,6 +106,60 @@ class ValidateReadOnlySqlTests(unittest.TestCase):
with self.assertRaises(ReadQueryValidationError):
validate_read_only_sql("UPDATE kg_nodes SET data = '{}'")
+ def test_rejects_cypher_match(self):
+ with self.assertRaises(ReadQueryValidationError) as ctx:
+ validate_read_only_sql(
+ "MATCH (n:PERSON) WHERE n.uuid = 'abc' RETURN n"
+ )
+ self.assertIn("Cypher", str(ctx.exception))
+
+ def test_rejects_explain_cypher(self):
+ with self.assertRaises(ReadQueryValidationError) as ctx:
+ validate_read_only_sql(
+ "EXPLAIN MATCH (n) RETURN n.uuid AS uuid LIMIT 50"
+ )
+ self.assertIn("Cypher", str(ctx.exception))
+
+ def test_rejects_legacy_nodes_table(self):
+ with self.assertRaises(ReadQueryValidationError) as ctx:
+ validate_read_only_sql("SELECT COUNT(*) AS total FROM nodes")
+ self.assertIn("kg_nodes", str(ctx.exception))
+
+ def test_rejects_table_nodes_shorthand(self):
+ with self.assertRaises(ReadQueryValidationError) as ctx:
+ validate_read_only_sql("TABLE nodes")
+ self.assertIn("kg_nodes", str(ctx.exception))
+
+ def test_rejects_incomplete_with_recursive_unbalanced_parens(self):
+ with self.assertRaises(ReadQueryValidationError) as ctx:
+ validate_read_only_sql(
+ "WITH RECURSIVE walk AS ("
+ "SELECT source_uuid, target_uuid, rel_type, 1 AS depth "
+ "FROM kg_relationships WHERE source_uuid = 'abc'"
+ )
+ self.assertIn("Incomplete SQL", str(ctx.exception))
+
+ def test_rejects_recursive_cte_fragment_without_outer_select(self):
+ with self.assertRaises(ReadQueryValidationError) as ctx:
+ validate_read_only_sql(
+ "WITH RECURSIVE walk AS ("
+ "SELECT source_uuid, target_uuid, rel_type, 1 AS depth "
+ "FROM kg_relationships WHERE source_uuid = 'abc')"
+ )
+ msg = str(ctx.exception)
+ self.assertTrue(
+ "Incomplete WITH RECURSIVE" in msg or "Incomplete SQL" in msg
+ )
+
+ def test_rejects_orphaned_union_all_branch(self):
+ with self.assertRaises(ReadQueryValidationError) as ctx:
+ validate_read_only_sql(
+ "SELECT r.source_uuid, r.target_uuid, r.rel_type, w.depth + 1 "
+ "FROM walk w JOIN kg_relationships r ON r.source_uuid = w.target_uuid "
+ "WHERE w.depth < 3) SELECT DISTINCT w.source_uuid FROM walk w LIMIT 50"
+ )
+ self.assertIn("Incomplete SQL", str(ctx.exception))
+
class ExecuteReadQueryTests(unittest.TestCase):
@unittest.skipIf(PostgreSQLGraphStore is None, "postgresql dependencies unavailable")
diff --git a/tui/AGENTS.md b/tui/AGENTS.md
new file mode 100644
index 0000000..e69de29
diff --git a/tui/CLAUDE.md b/tui/CLAUDE.md
new file mode 100644
index 0000000..43c994c
--- /dev/null
+++ b/tui/CLAUDE.md
@@ -0,0 +1 @@
diff --git a/tui/README.md b/tui/README.md
index 6d1af3b..373c64d 100644
--- a/tui/README.md
+++ b/tui/README.md
@@ -30,6 +30,10 @@ brainapi doctor # check Python, Docker, Ollama, and configured services
| `brainapi plugins install <name>` | Install a plugin from the registry (e.g. `brainapi plugins install chatbot`). |
| `brainapi plugins list` | List installed plugins (`--remote` to browse the registry). |
| `brainapi plugins uninstall <name>` | Remove an installed plugin. |
+| `brainapi reset` | Purge Celery queues and/or wipe Redis + Postgres brain state (interactive). |
+| `brainapi reset --brain <id> -y` | Wipe one brain's Redis keys and DROP its `brain_*` Postgres database. |
+| `brainapi reset --all -y` | Redis `FLUSHDB` + DROP all `brain_*` databases + clear `data_brains`. |
+| `brainapi reset --queues-only -y` | Purge Celery queue keys only. |
### `brainapi start` options
diff --git a/tui/src/commands/reset.ts b/tui/src/commands/reset.ts
new file mode 100644
index 0000000..c7a57e6
--- /dev/null
+++ b/tui/src/commands/reset.ts
@@ -0,0 +1,220 @@
+import * as p from "@clack/prompts";
+import pc from "picocolors";
+import { readEnvFile, getEnvValue } from "../lib/env.js";
+import { envFilePath } from "../lib/paths.js";
+import { readState } from "../lib/state.js";
+import { ENV_KEYS } from "../constants.js";
+import {
+ buildResetPlan,
+ brainDbName,
+ executeReset,
+ listBrainDatabases,
+ type ResetScope,
+} from "../lib/reset-state.js";
+
+export interface ResetCommandOptions {
+ brain?: string;
+ all?: boolean;
+ queuesOnly?: boolean;
+ redisOnly?: boolean;
+ dbOnly?: boolean;
+ yes?: boolean;
+}
+
+async function ensureInstalled(): Promise<void> {
+ const state = await readState();
+ if (!state || !state.envWritten) {
+ p.cancel(
+ "No brainapi install detected. Run " + pc.cyan("brainapi init") + " first.",
+ );
+ process.exit(1);
+ }
+}
+
+function summarizePlan(
+ scope: ResetScope,
+ redisOnly: boolean,
+ dbOnly: boolean,
+): string[] {
+ const lines: string[] = [];
+ if (scope.kind === "queues") {
+ lines.push("Purge Celery queues in Redis (leave brain data intact)");
+ } else if (scope.kind === "brain") {
+ lines.push(`Reset brain ${pc.cyan(scope.brainId)}`);
+ if (!dbOnly) {
+ lines.push(`- Redis: delete ${scope.brainId}:* keys + purge Celery queues`);
+ }
+ if (!redisOnly) {
+ lines.push(
+ `- Postgres: DROP ${brainDbName(scope.brainId)} + registry row`,
+ );
+ }
+ } else {
+ lines.push("Reset ALL brains");
+ if (!dbOnly) lines.push("- Redis: FLUSHDB");
+ if (!redisOnly) {
+ lines.push("- Postgres: DROP every brain_* database + clear data_brains");
+ }
+ }
+ lines.push("");
+ lines.push(
+ pc.yellow("Stop `brainapi start` first so workers are not holding DB connections."),
+ );
+ return lines;
+}
+
+async function resolveScope(
+ opts: ResetCommandOptions,
+): Promise<ResetScope> {
+ if (opts.queuesOnly) return { kind: "queues" };
+ if (opts.all) return { kind: "all" };
+ if (opts.brain) {
+ const brainId = opts.brain.trim();
+ if (!brainId) {
+ throw new Error("Brain id cannot be empty.");
+ }
+ return { kind: "brain", brainId };
+ }
+
+ const mode = await p.select({
+ message: "What should be cleared?",
+ options: [
+ {
+ value: "brain",
+ label: "One brain",
+ hint: "Redis keys + DROP that brain's Postgres DB",
+ },
+ {
+ value: "all",
+ label: "All brains",
+ hint: "Redis FLUSHDB + DROP all brain_* databases",
+ },
+ {
+ value: "queues",
+ label: "Celery queues only",
+ hint: "Purge pending/running queue keys in Redis",
+ },
+ ],
+ });
+ if (p.isCancel(mode)) {
+ p.cancel("Reset cancelled.");
+ process.exit(0);
+ }
+
+ if (mode === "queues") return { kind: "queues" };
+ if (mode === "all") return { kind: "all" };
+
+ const env = await readEnvFile(envFilePath());
+ let hint = "e.g. locomoconv26";
+ try {
+ const brains = await listBrainDatabases(env);
+ if (brains.length > 0) {
+ hint = brains.map((b) => b.brainId).slice(0, 8).join(", ");
+ if (brains.length > 8) hint += ", …";
+ }
+ } catch {
+ // listing is optional for the prompt
+ }
+
+ const brain = await p.text({
+ message: "Brain id to reset",
+ placeholder: hint,
+ validate: (value) => {
+ if (!value.trim()) return "Brain id is required";
+ if (!/^[A-Za-z0-9_-]+$/.test(value.trim())) {
+ return "Use alphanumeric brain ids (hyphen/underscore ok)";
+ }
+ },
+ });
+ if (p.isCancel(brain)) {
+ p.cancel("Reset cancelled.");
+ process.exit(0);
+ }
+ return { kind: "brain", brainId: String(brain).trim() };
+}
+
+export async function runReset(opts: ResetCommandOptions = {}): Promise<void> {
+ await ensureInstalled();
+ p.intro(pc.bgCyan(pc.black(" brainapi ")) + " " + pc.dim("reset"));
+
+ if (opts.redisOnly && opts.dbOnly) {
+ p.log.error("Use only one of --redis-only / --db-only.");
+ process.exit(1);
+ }
+ const exclusive = [opts.brain, opts.all, opts.queuesOnly].filter(Boolean).length;
+ if (exclusive > 1) {
+ p.log.error("Use only one of --brain / --all / --queues-only.");
+ process.exit(1);
+ }
+
+ const env = await readEnvFile(envFilePath());
+ const scope = await resolveScope(opts);
+ const plan = buildResetPlan(env, scope);
+
+ p.note(summarizePlan(scope, Boolean(opts.redisOnly), Boolean(opts.dbOnly)).join("\n"), "Plan");
+
+ if (plan.postgres) {
+ p.log.info(
+ `Postgres ${plan.postgres.host}:${plan.postgres.port} (system db ${plan.postgres.systemDatabase})`,
+ );
+ }
+ p.log.info(`Redis ${plan.redisHost}:${plan.redisPort}`);
+
+ const celeryBackend = getEnvValue(env, ENV_KEYS.celeryBackend) ?? "redis";
+ if (celeryBackend !== "redis" && !opts.dbOnly) {
+ p.log.error(
+ `reset supports CELERY_BACKEND=redis only (current: ${celeryBackend}). Use --db-only to wipe Postgres.`,
+ );
+ process.exit(1);
+ }
+
+ if (!opts.yes) {
+ const ok = await p.confirm({
+ message: "This is destructive. Continue?",
+ initialValue: false,
+ });
+ if (p.isCancel(ok) || !ok) {
+ p.cancel("Reset cancelled.");
+ process.exit(0);
+ }
+ }
+
+ const spinner = p.spinner();
+ spinner.start("Clearing state…");
+ try {
+ const result = await executeReset(env, scope, {
+ yes: opts.yes,
+ redisOnly: opts.redisOnly,
+ dbOnly: opts.dbOnly,
+ });
+ spinner.stop("Done.");
+
+ const lines: string[] = [];
+ if (result.redisDeletedKeys === -1) {
+ lines.push("Redis: FLUSHDB completed");
+ } else if (!opts.dbOnly) {
+ lines.push(`Redis: deleted ~${result.redisDeletedKeys} key(s)`);
+ }
+ if (result.databasesDropped.length > 0) {
+ lines.push(`Postgres: dropped ${result.databasesDropped.join(", ")}`);
+ }
+ if (result.registryRowsDeleted > 0) {
+ lines.push(`Registry: removed ${result.registryRowsDeleted} data_brains row(s)`);
+ }
+ for (const warning of result.warnings) {
+ lines.push(pc.yellow(warning));
+ }
+ if (lines.length > 0) {
+ p.note(lines.join("\n"), "Result");
+ }
+ p.outro(
+ pc.green("State cleared.") +
+ " Restart with " +
+ pc.cyan("brainapi start") +
+ " when ready.",
+ );
+ } catch (err) {
+ spinner.stop("Failed.");
+ throw err;
+ }
+}
diff --git a/tui/src/index.ts b/tui/src/index.ts
index 5ba7ab0..e8d5c8e 100644
--- a/tui/src/index.ts
+++ b/tui/src/index.ts
@@ -6,10 +6,19 @@ import { runConfig } from "./commands/config.js";
import { runDoctor } from "./commands/doctor.js";
import { runUpdate } from "./commands/update.js";
import { runPlugins, type PluginsOptions, type PluginsSubcommand } from "./commands/plugins.js";
+import { runReset, type ResetCommandOptions } from "./commands/reset.js";
import { readState } from "./lib/state.js";
import { isPipelineMode, type PipelineMode } from "./types.js";
-type Command = "init" | "start" | "config" | "doctor" | "update" | "plugins" | "help";
+type Command =
+ | "init"
+ | "start"
+ | "config"
+ | "doctor"
+ | "update"
+ | "plugins"
+ | "reset"
+ | "help";
interface ParsedArgs {
command: Command;
@@ -24,6 +33,7 @@ interface ParsedArgs {
pipeline?: PipelineMode;
invalidPipeline?: string;
plugins?: PluginsOptions;
+ reset?: ResetCommandOptions;
unknown: string[];
}
@@ -94,12 +104,31 @@ function parseArgs(argv: string[]): ParsedArgs {
case "doctor":
case "update":
case "plugins":
+ case "reset":
case "help":
out.command = arg;
break;
default:
unknown.push(arg);
}
+ } else if (out.command === "reset") {
+ if (!out.reset) out.reset = {};
+ const reset = out.reset;
+ if (arg === "--brain") {
+ reset.brain = args.shift();
+ } else if (arg === "--all") {
+ reset.all = true;
+ } else if (arg === "--queues-only") {
+ reset.queuesOnly = true;
+ } else if (arg === "--redis-only") {
+ reset.redisOnly = true;
+ } else if (arg === "--db-only") {
+ reset.dbOnly = true;
+ } else if (arg === "--yes" || arg === "-y") {
+ reset.yes = true;
+ } else {
+ unknown.push(arg);
+ }
} else if (out.command === "plugins") {
if (!out.plugins) out.plugins = {};
const plugins = out.plugins;
@@ -143,6 +172,7 @@ function printHelp(): void {
" doctor Check Python, Docker, Ollama, GCP credentials, services",
" update Fetch latest source + reinstall Python dependencies",
" plugins <cmd> Manage plugins (install, uninstall, list, info, update)",
+ " reset [...] Purge Celery queues and/or wipe Redis + Postgres brain state",
" help Show this message",
"",
pc.bold("Options:"),
@@ -155,6 +185,12 @@ function printHelp(): void {
" --no-worker For 'start': skip the celery worker",
" --only api,mcp,worker For 'start': run only the listed processes (shortcut)",
" --pipeline accurate|lightweight Set PIPELINE_MODE in ~/.brainapi/source/.env before start",
+ " --brain <id> For 'reset': wipe one brain (Redis keys + DROP brain_* DB)",
+ " --all For 'reset': Redis FLUSHDB + DROP all brain_* databases",
+ " --queues-only For 'reset': purge Celery queue keys only",
+ " --redis-only For 'reset': skip Postgres",
+ " --db-only For 'reset': skip Redis",
+ " -y, --yes For 'reset': skip confirmation prompt",
" -v, --version Print the CLI version",
" -h, --help Show this message",
"",
@@ -245,6 +281,9 @@ async function main(): Promise<void> {
case "plugins":
await runPlugins(parsed.plugins ?? {});
return;
+ case "reset":
+ await runReset(parsed.reset ?? {});
+ return;
case "help":
printHelp();
return;
diff --git a/tui/src/lib/reset-state.ts b/tui/src/lib/reset-state.ts
new file mode 100644
index 0000000..8222b24
--- /dev/null
+++ b/tui/src/lib/reset-state.ts
@@ -0,0 +1,481 @@
+import { createHash } from "node:crypto";
+import { ENV_KEYS, SERVICE_DEFAULTS } from "../constants.js";
+import type { EnvFile } from "./env.js";
+import { getEnvValue } from "./env.js";
+import { runQuiet } from "./exec.js";
+import { which } from "./exec.js";
+
+const CELERY_QUEUE_KEYS = [
+ "ingest_data",
+ "process_architect_relationships",
+ "ingest_structured_data",
+ "consolidate_graph",
+ "finalize_ingestion",
+ "ingest_file",
+ "unacked",
+ "unacked_index",
+ "unacked_mutex",
+] as const;
+
+const BRAIN_DB_PREFIX = "brain_";
+const MAX_DBNAME_LEN = 63;
+const HASH_SUFFIX_LEN = 12;
+
+export type ResetScope =
+ | { kind: "queues" }
+ | { kind: "brain"; brainId: string }
+ | { kind: "all" };
+
+export interface ResetOptions {
+ yes?: boolean;
+ redisOnly?: boolean;
+ dbOnly?: boolean;
+}
+
+export interface ResetPlan {
+ scope: ResetScope;
+ redisHost: string;
+ redisPort: number;
+ postgres?: {
+ host: string;
+ port: number;
+ username: string;
+ password: string;
+ systemDatabase: string;
+ maintenanceDatabase: string;
+ };
+ brainDatabases: string[];
+ redisActions: string[];
+ dbActions: string[];
+}
+
+export interface ResetResult {
+ redisDeletedKeys: number;
+ databasesDropped: string[];
+ registryRowsDeleted: number;
+ warnings: string[];
+}
+
+function requireEnv(env: EnvFile, key: string, fallback?: string): string {
+ return getEnvValue(env, key) ?? fallback ?? "";
+}
+
+export function brainDbName(brainId: string): string {
+ if (!brainId.trim()) {
+ throw new Error("brain_id must be a non-empty string");
+ }
+ const sanitized =
+ brainId
+ .toLowerCase()
+ .replace(/[^a-z0-9_]+/g, "_")
+ .replace(/^_+|_+$/g, "") || "x";
+ const candidate = `${BRAIN_DB_PREFIX}${sanitized}`;
+ if (candidate.length <= MAX_DBNAME_LEN) return candidate;
+
+ const digest = createHash("sha256")
+ .update(brainId)
+ .digest("hex")
+ .slice(0, HASH_SUFFIX_LEN);
+ const headBudget = MAX_DBNAME_LEN - BRAIN_DB_PREFIX.length - digest.length - 1;
+ const head = sanitized.slice(0, headBudget).replace(/_+$/g, "");
+ return `${BRAIN_DB_PREFIX}${head}_${digest}`;
+}
+
+export function brainIdFromDbName(dbName: string): string | null {
+ if (!dbName.startsWith(BRAIN_DB_PREFIX)) return null;
+ const suffix = dbName.slice(BRAIN_DB_PREFIX.length);
+ return suffix || null;
+}
+
+export function usesPostgres(env: EnvFile): boolean {
+ const graph = getEnvValue(env, ENV_KEYS.graphDb);
+ const data = getEnvValue(env, ENV_KEYS.dataDb);
+ const vector = getEnvValue(env, ENV_KEYS.vectorDb);
+ return (
+ graph === "networkx" || data === "postgresql" || vector === "postgresql"
+ );
+}
+
+export function buildResetPlan(env: EnvFile, scope: ResetScope): ResetPlan {
+ const redisHost =
+ requireEnv(env, ENV_KEYS.redisHost, SERVICE_DEFAULTS.redis.host) ||
+ SERVICE_DEFAULTS.redis.host;
+ const redisPort = Number(
+ requireEnv(env, ENV_KEYS.redisPort, String(SERVICE_DEFAULTS.redis.port)) ||
+ SERVICE_DEFAULTS.redis.port,
+ );
+
+ const plan: ResetPlan = {
+ scope,
+ redisHost,
+ redisPort,
+ brainDatabases: [],
+ redisActions: [],
+ dbActions: [],
+ };
+
+ if (scope.kind === "queues") {
+ plan.redisActions.push("Purge Celery queue keys and unacked buffers");
+ } else if (scope.kind === "brain") {
+ plan.redisActions.push(`Delete Redis keys for brain "${scope.brainId}"`);
+ plan.redisActions.push("Purge Celery queue keys and unacked buffers");
+ plan.brainDatabases = [brainDbName(scope.brainId)];
+ plan.dbActions.push(`DROP DATABASE ${plan.brainDatabases[0]}`);
+ plan.dbActions.push(
+ `DELETE FROM data_brains WHERE name_key/id = "${scope.brainId}"`,
+ );
+ } else {
+ plan.redisActions.push("FLUSHDB (clear entire Redis database)");
+ plan.dbActions.push("DROP all brain_* databases");
+ plan.dbActions.push("DELETE all rows from data_brains");
+ }
+
+ if (usesPostgres(env)) {
+ plan.postgres = {
+ host:
+ requireEnv(env, ENV_KEYS.postgresHost, SERVICE_DEFAULTS.postgresql.host) ||
+ SERVICE_DEFAULTS.postgresql.host,
+ port: Number(
+ requireEnv(
+ env,
+ ENV_KEYS.postgresPort,
+ String(SERVICE_DEFAULTS.postgresql.port),
+ ) || SERVICE_DEFAULTS.postgresql.port,
+ ),
+ username:
+ requireEnv(
+ env,
+ ENV_KEYS.postgresUsername,
+ SERVICE_DEFAULTS.postgresql.username,
+ ) || SERVICE_DEFAULTS.postgresql.username,
+ password:
+ requireEnv(
+ env,
+ ENV_KEYS.postgresPassword,
+ SERVICE_DEFAULTS.postgresql.password,
+ ) || SERVICE_DEFAULTS.postgresql.password,
+ systemDatabase:
+ requireEnv(env, ENV_KEYS.postgresSystemDatabase, "brainapi") ||
+ "brainapi",
+ maintenanceDatabase:
+ requireEnv(env, ENV_KEYS.postgresMaintenanceDatabase, "postgres") ||
+ "postgres",
+ };
+ } else {
+ plan.dbActions = [];
+ plan.brainDatabases = [];
+ }
+
+ return plan;
+}
+
+async function ensureBin(bin: string): Promise<void> {
+ if (!(await which(bin))) {
+ throw new Error(
+ `Required binary "${bin}" was not found on PATH. Install it or add it to PATH.`,
+ );
+ }
+}
+
+async function redisCli(
+ host: string,
+ port: number,
+ args: string[],
+): Promise<{ ok: boolean; stdout: string; stderr: string }> {
+ return runQuiet("redis-cli", ["-h", host, "-p", String(port), ...args]);
+}
+
+async function purgeCeleryQueues(
+ host: string,
+ port: number,
+): Promise<number> {
+ const result = await redisCli(host, port, ["DEL", ...CELERY_QUEUE_KEYS]);
+ if (!result.ok) {
+ throw new Error(
+ `Failed to purge Celery queues: ${result.stderr || result.stdout}`,
+ );
+ }
+ const n = Number(result.stdout.trim() || "0");
+ return Number.isFinite(n) ? n : 0;
+}
+
+async function deleteKeysByPattern(
+ host: string,
+ port: number,
+ pattern: string,
+): Promise<number> {
+ const scanned = await redisCli(host, port, ["--scan", "--pattern", pattern]);
+ if (!scanned.ok) {
+ throw new Error(
+ `Failed to scan Redis keys (${pattern}): ${scanned.stderr || scanned.stdout}`,
+ );
+ }
+ const keys = scanned.stdout
+ .split(/\r?\n/)
+ .map((k) => k.trim())
+ .filter(Boolean);
+ if (keys.length === 0) return 0;
+
+ let deleted = 0;
+ const chunkSize = 100;
+ for (let i = 0; i < keys.length; i += chunkSize) {
+ const chunk = keys.slice(i, i + chunkSize);
+ const result = await redisCli(host, port, ["DEL", ...chunk]);
+ if (!result.ok) {
+ throw new Error(
+ `Failed to delete Redis keys: ${result.stderr || result.stdout}`,
+ );
+ }
+ deleted += Number(result.stdout.trim() || "0") || chunk.length;
+ }
+ return deleted;
+}
+
+async function psql(
+ pg: NonNullable<ResetPlan["postgres"]>,
+ database: string,
+ sql: string,
+): Promise<{ ok: boolean; stdout: string; stderr: string }> {
+ return runQuiet(
+ "psql",
+ [
+ "-h",
+ pg.host,
+ "-p",
+ String(pg.port),
+ "-U",
+ pg.username,
+ "-d",
+ database,
+ "-v",
+ "ON_ERROR_STOP=1",
+ "-t",
+ "-A",
+ "-c",
+ sql,
+ ],
+ {
+ env: {
+ ...process.env,
+ PGPASSWORD: pg.password,
+ },
+ },
+ );
+}
+
+export async function listBrainDatabases(
+ env: EnvFile,
+): Promise<Array<{ brainId: string; database: string }>> {
+ if (!usesPostgres(env)) return [];
+ await ensureBin("psql");
+ const plan = buildResetPlan(env, { kind: "all" });
+ if (!plan.postgres) return [];
+
+ const result = await psql(
+ plan.postgres,
+ plan.postgres.maintenanceDatabase,
+ `SELECT datname FROM pg_database WHERE datname LIKE 'brain\\_%' ESCAPE '\\' AND datname <> '${plan.postgres.systemDatabase.replace(/'/g, "''")}' ORDER BY 1;`,
+ );
+ if (!result.ok) {
+ throw new Error(
+ `Failed to list brain databases: ${result.stderr || result.stdout}`,
+ );
+ }
+ return result.stdout
+ .split(/\r?\n/)
+ .map((line) => line.trim())
+ .filter(Boolean)
+ .map((database) => ({
+ database,
+ brainId: brainIdFromDbName(database) ?? database,
+ }));
+}
+
+async function dropBrainDatabase(
+ pg: NonNullable<ResetPlan["postgres"]>,
+ database: string,
+): Promise<void> {
+ if (!/^[a-z0-9_]+$/.test(database)) {
+ throw new Error(`Refusing to drop unsafe database name: ${database}`);
+ }
+ if (database === pg.systemDatabase || database === pg.maintenanceDatabase) {
+ throw new Error(
+ `Refusing to drop protected database "${database}" (system/maintenance).`,
+ );
+ }
+ if (!database.startsWith(BRAIN_DB_PREFIX) || database === "brainapi") {
+ throw new Error(
+ `Refusing to drop non-brain database "${database}". Expected prefix ${BRAIN_DB_PREFIX}.`,
+ );
+ }
+ const terminate = await psql(
+ pg,
+ pg.maintenanceDatabase,
+ `SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = '${database}' AND pid <> pg_backend_pid();`,
+ );
+ if (!terminate.ok) {
+ throw new Error(
+ `Failed to terminate connections to ${database}: ${terminate.stderr || terminate.stdout}`,
+ );
+ }
+ const drop = await psql(
+ pg,
+ pg.maintenanceDatabase,
+ `DROP DATABASE IF EXISTS ${database};`,
+ );
+ if (!drop.ok) {
+ throw new Error(
+ `Failed to drop ${database}: ${drop.stderr || drop.stdout}`,
+ );
+ }
+}
+
+async function ensureSystemDatabase(
+ pg: NonNullable<ResetPlan["postgres"]>,
+): Promise<void> {
+ if (!/^[a-z0-9_]+$/.test(pg.systemDatabase)) {
+ throw new Error(`Unsafe system database name: ${pg.systemDatabase}`);
+ }
+ const result = await psql(
+ pg,
+ pg.maintenanceDatabase,
+ `SELECT 1 FROM pg_database WHERE datname = '${pg.systemDatabase}';`,
+ );
+ if (!result.ok) {
+ throw new Error(
+ `Failed to check system database: ${result.stderr || result.stdout}`,
+ );
+ }
+ if (result.stdout.trim()) return;
+ const created = await psql(
+ pg,
+ pg.maintenanceDatabase,
+ `CREATE DATABASE ${pg.systemDatabase};`,
+ );
+ if (!created.ok) {
+ throw new Error(
+ `Failed to recreate system database ${pg.systemDatabase}: ${created.stderr || created.stdout}`,
+ );
+ }
+}
+
+async function deleteBrainRegistryRows(
+ pg: NonNullable<ResetPlan["postgres"]>,
+ brainId: string | null,
+): Promise<number> {
+ const sql =
+ brainId === null
+ ? "DELETE FROM data_brains RETURNING id;"
+ : `DELETE FROM data_brains WHERE name_key = '${brainId.replace(/'/g, "''")}' OR id = '${brainId.replace(/'/g, "''")}' RETURNING id;`;
+ const result = await psql(pg, pg.systemDatabase, sql);
+ if (!result.ok) {
+ // Table may not exist yet on fresh installs.
+ if (/data_brains/i.test(result.stderr) && /does not exist/i.test(result.stderr)) {
+ return 0;
+ }
+ throw new Error(
+ `Failed to clean data_brains registry: ${result.stderr || result.stdout}`,
+ );
+ }
+ return result.stdout
+ .split(/\r?\n/)
+ .map((line) => line.trim())
+ .filter(Boolean).length;
+}
+
+export async function executeReset(
+ env: EnvFile,
+ scope: ResetScope,
+ options: ResetOptions = {},
+): Promise<ResetResult> {
+ const celeryBackend = getEnvValue(env, ENV_KEYS.celeryBackend) ?? "redis";
+ if (celeryBackend !== "redis" && !options.dbOnly) {
+ throw new Error(
+ `reset currently supports CELERY_BACKEND=redis only (got "${celeryBackend}").`,
+ );
+ }
+
+ const plan = buildResetPlan(env, scope);
+ const warnings: string[] = [];
+ let redisDeletedKeys = 0;
+ const databasesDropped: string[] = [];
+ let registryRowsDeleted = 0;
+
+ const doRedis = !options.dbOnly;
+ const doDb = !options.redisOnly && Boolean(plan.postgres) && scope.kind !== "queues";
+
+ if (doRedis) {
+ await ensureBin("redis-cli");
+ if (scope.kind === "all") {
+ const flush = await redisCli(plan.redisHost, plan.redisPort, ["FLUSHDB"]);
+ if (!flush.ok) {
+ throw new Error(`Redis FLUSHDB failed: ${flush.stderr || flush.stdout}`);
+ }
+ redisDeletedKeys = -1; // unknown count
+ } else {
+ if (scope.kind === "brain") {
+ redisDeletedKeys += await deleteKeysByPattern(
+ plan.redisHost,
+ plan.redisPort,
+ `${scope.brainId}:*`,
+ );
+ const systemKey = await redisCli(plan.redisHost, plan.redisPort, [
+ "DEL",
+ `system:brain:${scope.brainId}`,
+ ]);
+ if (systemKey.ok) {
+ redisDeletedKeys += Number(systemKey.stdout.trim() || "0") || 0;
+ }
+ }
+ redisDeletedKeys += await purgeCeleryQueues(plan.redisHost, plan.redisPort);
+ }
+ }
+
+ if (doDb && plan.postgres) {
+ await ensureBin("psql");
+ await ensureSystemDatabase(plan.postgres);
+ let targets = plan.brainDatabases;
+ if (scope.kind === "all") {
+ const listed = await listBrainDatabases(env);
+ targets = listed.map((row) => row.database);
+ }
+ for (const database of targets) {
+ if (
+ database === plan.postgres.systemDatabase ||
+ database === plan.postgres.maintenanceDatabase ||
+ database === "brainapi"
+ ) {
+ warnings.push(`Skipped protected database "${database}".`);
+ continue;
+ }
+ await dropBrainDatabase(plan.postgres, database);
+ databasesDropped.push(database);
+ }
+ if (scope.kind === "brain") {
+ registryRowsDeleted = await deleteBrainRegistryRows(
+ plan.postgres,
+ scope.brainId,
+ );
+ } else if (scope.kind === "all") {
+ registryRowsDeleted = await deleteBrainRegistryRows(plan.postgres, null);
+ }
+ } else if (!options.redisOnly && !plan.postgres && scope.kind !== "queues") {
+ warnings.push(
+ "Postgres is not configured for this install (GRAPH_DB/DATA_DB/VECTOR_DB); skipped database wipe.",
+ );
+ }
+
+ if (options.redisOnly) {
+ warnings.push("Skipped Postgres (--redis-only).");
+ }
+ if (options.dbOnly) {
+ warnings.push("Skipped Redis (--db-only).");
+ }
+
+ return {
+ redisDeletedKeys,
+ databasesDropped,
+ registryRowsDeleted,
+ warnings,
+ };
+}