BrainAPI is a knowledge graph–powered AI memory layer that transforms unstructured data into structured knowledge, enabling intelligent search, recommendations, and contextual memory for AI agents and applications.
10 — Ingest cost and latency (without quality trade-offs)
Workstream: make PIPELINE_MODE=accurate ingestion cheaper and faster while preserving event-hub graph shape and LoCoMo/LongMemEval quality. Constraints in 00-scope-and-constraints.md are binding. Extraction quality work remains owned by 01-ingestion-extraction.md; this document owns token and wall-clock levers that must not regress quality.
Quality bar: accurate-mode event hubs; no McNemar-significant judge drop; Phase-0 extraction metrics non-regressing. lightweight is out of scope.
Cost map (accurate mode)
| Rank | Stage | Lever shipped |
|---|---|---|
| 1 | Architect full-text tool loop | Architect now runs per Scout unit (+ optional prior-unit window ≤4), not unbounded session text |
| 2 | Atomic Janitor per create | Batched post-Architect Janitor + cheap grounding precheck skips |
| 3 | Scout per 6k chunk | Unchanged (instrumented) |
| 4 | Observations | Gated by RUN_OBSERVATIONS (default false) — never wrote the graph |
| 5 | Consolidation | RUN_GRAPH_CONSOLIDATOR default false until audited (01 Phase 6) |
| 6 | Hub/topic rebuild | Entity-scoped hub refresh; topic rebuild deferred to finalize |
Phases
C0 — Measure (shipped)
src/core/saving/ingest_cost.py— per-stage ledger; contextvar accumulation from custom agent invoke loop- Task status
costfield viaset_ingestion_task_status(..., cost=...) - LoCoMo / LongMemEval ingest records + report mean tokens/unit and stage breakdown
C1 — Waste removal (shipped)
| Knob | Default | Effect |
|---|---|---|
RUN_OBSERVATIONS |
false |
Skip ObservationsAgent LLM |
RUN_GRAPH_CONSOLIDATOR |
false |
Skip consolidator LLM mutations |
JANITOR_BATCH_SIZE |
20 |
Relationships per Janitor call |
C2 — Bound and batch (shipped)
auto_kg.py: Scout chunks → Architect per unit with prior context;reset=Falseacross chunksarchitect_agent.py:defer_janitor=True,run_batched_janitor- Hub:
refresh_hub_bridges_for_entities; topic rebuild on finalize only
C3 — Gate remaining LLM (shipped)
grounding.py: span align precheck before Janitor- Ambiguous entity resolution routes to
KGAgent.verify_entity_existenceinstead of silent duplicate
Distillation (deferred)
Distill-SynthKG (2410.16597) only after 01 Phases 0–3 stabilize the teacher. Premature distillation freezes defects.
Literature (access 2026-07-29)
| Paper | Role |
|---|---|
| ATOM 2510.22590 | Atomic units + parallel merge → latency cut |
| Distill-SynthKG 2410.16597 | Deferred student extractor |
| Zep/Graphiti 2501.13956 | Prior-context window; uncertainty-gated ER |
| UCCI 2605.18796 | Cascade routing at fixed quality |
| SafePassage 2510.00276 | Span grounding without LLM scorer |
Verification
# Unit
pytest tests/test_ingest_cost.py tests/test_grounding.py -v
# After a live ingest, task status should include cost.stages
# Benchmark report.md Ingest section should show mean tokens when cost is present
Measured A/B (2026-07-30): benchmarks/runs/ingest-cost-baseline.json — conv-26 sessions 1–5.
| Arm | mean wait | mean LLM tokens | LLM÷source | type-named | events |
|---|---|---|---|---|---|
Legacy hotpath (ingest-cost-legacy) |
630 s | 1.19M | — | 0 | 107 |
Cheap defaults (ingest-cost-cheap2) |
223 s | 0.73M | ~886× | 0 | 100 |
CP2 batch (architect-cp2-batch) |
41 s | 15.4k | 18.8× | 0 | 60 |
CP3 batch+scratchpad (architect-cp3-scratchpad) |
34 s | 12.6k | 15.3× | 0 | 65 |
C0–C3 shipping: ≈65% lower mean wait, ≈39% fewer mean tokens vs legacy. Architect redesign: old cheap ~886× → batch+scratchpad ~15.3× (same five sessions; source denom 4,114 cl100k_base). Details: architect-cp2-NOTES.md, architect-cp3-NOTES.md. No McNemar in these cost measurements.
What not to do
- Do not use
PIPELINE_MODE=lightweightas the cost lever - Do not add reflection passes for cost goals
- Do not claim accuracy wins without paired A/B (noise floor in
00)
Follow-on: Architect loop efficiency workstream and ship defaults (batch + auto/scratchpad) in 11-architect-loop-efficiency-plan.md.