#1 Persistent memory for AI coding agents based on real-world benchmarks
TypeScript agentmemoryagentsaiclaudeclaudecodecodexcopilotcursorgenaiharnesshermesmemoryompopenclawopencodepi
fix(graph): bound retrieval reads and resume explicit index migrations (#1559)
Rohit Ghumare857cf1babf5b
diff --git a/.github/workflows/release-gate.yml b/.github/workflows/release-gate.ymlindex 76abbffb..81b34232 100644--- a/.github/workflows/release-gate.yml+++ b/.github/workflows/release-gate.yml@@ -45,6 +45,12 @@ jobs:- run: npm install --legacy-peer-deps --no-audit --no-fund- run: npm run release:gate -- --out "${{ runner.temp }}/release-gate"- run: npm run test:startup:packaged -- --skip-build --out "${{ runner.temp }}/startup-gate"+ - name: Verify bounded graph migration with the packaged CLI+ run: |+ mkdir -p "$RUNNER_TEMP/graph-index-pack"+ npm pack --ignore-scripts --pack-destination "$RUNNER_TEMP/graph-index-pack"+ archives=("$RUNNER_TEMP"/graph-index-pack/*.tgz)+ npm run test:graph-index:packaged -- --tarball "${archives[0]}" --out "$RUNNER_TEMP/graph-index-gate"- if: always()uses: actions/upload-artifact@v4with:@@ -52,5 +58,6 @@ jobs:path: |${{ runner.temp }}/release-gate${{ runner.temp }}/startup-gate+ ${{ runner.temp }}/graph-index-gateif-no-files-found: ignoreretention-days: 14diff --git a/AGENTS.md b/AGENTS.mdindex 1e8b63b6..2f862508 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -101,7 +101,7 @@ Hook scripts in `src/hooks/` are standalone Node.js scripts (no iii-sdk import).## Testing-- Before every commit run, in order: `npm run build`, `npm run skills:gen`, `npm run docs:sync`, `npm run skills:check`, `npm test` (3,300+ tests). Commit anything the generators changed. CI runs the same build, skills check and tests.+- Before every commit run, in order: `npm run build`, `npm run skills:gen`, `npm run docs:sync`, `npm run skills:check`, `npm test` (3,500+ tests). Commit anything the generators changed. CI runs the same build, skills check and tests.- Before any npm publish: `npm run release:gate` must pass. It installs the packed tarballs into a clean home and runs capture, recovery, restart, export/import, MCP and status checks against the installed CLI (see CONTRIBUTING.md, Release process)- Mock pattern: `vi.mock("iii-sdk")` with mock `sdk.trigger`, `kv.get/set/list`- Test files go in `test/` with `.test.ts` extension@@ -110,8 +110,8 @@ Hook scripts in `src/hooks/` are standalone Node.js scripts (no iii-sdk import).## Current Stats (v0.9.30)- 54 MCP tools (8 visible by default, `AGENTMEMORY_TOOLS=all` for all)-- 138 REST endpoints+- 141 REST endpoints- 6 MCP resources, 3 MCP prompts- 12 hooks, 17 skills-- 312+ iii functions-- 3,300+ tests+- 318+ iii functions+- 3,500+ testsdiff --git a/README.md b/README.mdindex 38183721..947334e4 100644--- a/README.md+++ b/README.md@@ -72,7 +72,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-tools.svg"><img src="assets/tags/stat-tools.svg" alt="54 MCP tools" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-hooks.svg"><img src="assets/tags/stat-hooks.svg" alt="12 auto hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-deps.svg"><img src="assets/tags/stat-deps.svg" alt="0 external DBs" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-tests.svg"><img src="assets/tags/stat-tests.svg" alt="3,300+ tests passing" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-tests.svg"><img src="assets/tags/stat-tests.svg" alt="3,500+ tests passing" height="38" /></picture></p><p align="center">@@ -1435,7 +1435,7 @@ fi| Prometheus / Grafana | iii OTEL + health monitor || Custom plugin systems | `iii worker add <name>` |-**230 source files · ~55,000 LOC · 3,300+ tests · 312 functions · 62 KV scopes**, all on three primitives. No `agentmemory plugin install`. The plugin system is iii itself.+**238 source files · ~56,000 LOC · 3,500+ tests · 318 functions · 66 KV scopes**, all on three primitives. No `agentmemory plugin install`. The plugin system is iii itself.---@@ -1781,7 +1781,7 @@ Create `~/.agentmemory/.env`:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/section-api.svg"><img src="assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpoints on port `3111`. The REST API binds to `127.0.0.1` by default. Protected endpoints require `Authorization: Bearer <secret>`, and mesh sync endpoints require an explicitly set `AGENTMEMORY_SECRET` on both peers.+141 endpoints on port `3111`. The REST API binds to `127.0.0.1` by default. Protected endpoints require `Authorization: Bearer <secret>`, and mesh sync endpoints require an explicitly set `AGENTMEMORY_SECRET` on both peers.**Authentication is on by default.** When `AGENTMEMORY_SECRET` is not set (in the shell or in `~/.agentmemory/.env`), the server generates a random secret on first start and stores it in `~/.agentmemory/secret` with mode `0600`. Most bundled clients read it from there when they talk to a local server: the CLI, the viewer, the hooks under `plugin/scripts`, the MCP server and the `@agentmemory/mcp` shim, the configs written by `agentmemory connect`, and the bundled OpenCode, Pi, OpenClaw and filesystem-watcher integrations. The Hermes provider and its optional MCP connection instead use credentials managed by Hermes; configure them with `hermes memory setup`. The stored secret is only sent to loopback URLs (`localhost`, `127.0.0.0/8`, `::1`). An explicit `AGENTMEMORY_SECRET` always wins, and remote clients still need it set. Docker and the `deploy/` entrypoints already generate and export their own secret. To call the API by hand:@@ -1867,7 +1867,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.ar-SA.md b/READMEs/README.ar-SA.mdindex 0d2c4855..9409f425 100644--- a/READMEs/README.ar-SA.md+++ b/READMEs/README.ar-SA.md@@ -73,7 +73,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP tools" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 auto hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 external DBs" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ tests passing" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ tests passing" height="38" /></picture></p><p align="center">@@ -1439,7 +1439,7 @@ fi| Prometheus / Grafana | OTEL في iii + مراقب الحالة الصحية || أنظمة إضافات مخصصة | `iii worker add <name>` |-**230 ملف مصدري · ~55,000 سطر برمجي · 3,300+ اختبار · 312 دالة · 62 نطاق KV**، كل ذلك على ثلاث بنى أساسية. لا يوجد أمر `agentmemory plugin install`. نظام الإضافات هو iii نفسه.+**238 ملف مصدري · ~56,000 سطر برمجي · 3,500+ اختبار · 318 دالة · 66 نطاق KV**، كل ذلك على ثلاث بنى أساسية. لا يوجد أمر `agentmemory plugin install`. نظام الإضافات هو iii نفسه.---@@ -1782,7 +1782,7 @@ GRAPH_EXTRACTION_ENABLED=true<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 نقطة نهاية على المنفذ `3111`. ترتبط REST API بـ `127.0.0.1` افتراضيًا. تتطلب نقاط النهاية المحمية ترويسة `Authorization: Bearer <secret>`، وتتطلب نقاط نهاية مزامنة mesh ضبط `AGENTMEMORY_SECRET` صريحًا على كلا الطرفين.+141 نقطة نهاية على المنفذ `3111`. ترتبط REST API بـ `127.0.0.1` افتراضيًا. تتطلب نقاط النهاية المحمية ترويسة `Authorization: Bearer <secret>`، وتتطلب نقاط نهاية مزامنة mesh ضبط `AGENTMEMORY_SECRET` صريحًا على كلا الطرفين.**المصادقة مُفعَّلة افتراضيًا.** عندما لا يكون `AGENTMEMORY_SECRET` مضبوطًا (في الصدفة أو في `~/.agentmemory/.env`)، يُولِّد الخادم سرًا عشوائيًا عند أول تشغيل ويخزِّنه في `~/.agentmemory/secret` بوضع صلاحيات `0600`. ويقرؤه كل عميل مُضمَّن من هناك عند التحدث إلى خادم محلي: الواجهة الطرفية CLI، والعارض، والخطافات تحت `plugin/scripts`، وخادم MCP والوسيط `@agentmemory/mcp`، والإعدادات التي يكتبها `agentmemory connect`، وتكاملات OpenCode وPi وOpenClaw وHermes ومراقب نظام الملفات (filesystem-watcher) المُضمَّنة. ولا يُرسَل السر المخزَّن إلا إلى عناوين loopback (`localhost`، و`127.0.0.0/8`، و`::1`). ويتغلّب `AGENTMEMORY_SECRET` الصريح دائمًا، ولا يزال العملاء البعيدون بحاجة إلى ضبطه. وتُولِّد Docker وقوالب [`deploy/`](../deploy/) سرها الخاص وتُصدِّره مسبقًا. ولاستدعاء الواجهة البرمجية يدويًا:@@ -1863,7 +1863,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.bn-BD.md b/READMEs/README.bn-BD.mdindex 170027bb..b2d0f890 100644--- a/READMEs/README.bn-BD.md+++ b/READMEs/README.bn-BD.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP টুল" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 অটো হুক" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 এক্সটার্নাল ডিবি" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ টেস্ট পাস" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ টেস্ট পাস" height="38" /></picture></p><p align="center">@@ -1437,7 +1437,7 @@ fi| Prometheus / Grafana | iii OTEL + হেলথ মনিটর || কাস্টম প্লাগইন সিস্টেম | `iii worker add <name>` |-**230 সোর্স ফাইল · ~55,000 LOC · 3,300+ টেস্ট · 312 ফাংশন · 62 KV স্কোপ**, সবকিছু তিনটি প্রিমিটিভের উপর। কোনো `agentmemory plugin install` নেই। প্লাগইন সিস্টেমটাই iii নিজে।+**238 সোর্স ফাইল · ~56,000 LOC · 3,500+ টেস্ট · 318 ফাংশন · 66 KV স্কোপ**, সবকিছু তিনটি প্রিমিটিভের উপর। কোনো `agentmemory plugin install` নেই। প্লাগইন সিস্টেমটাই iii নিজে।---@@ -1780,7 +1780,7 @@ GRAPH_EXTRACTION_ENABLED=true<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-`3111` পোর্টে 138 এন্ডপয়েন্ট। REST API ডিফল্টভাবে `127.0.0.1`-এ বাইন্ড করে। প্রোটেক্টেড এন্ডপয়েন্টে `Authorization: Bearer <secret>` লাগে, এবং mesh sync এন্ডপয়েন্টে দুই পিয়ারেই স্পষ্টভাবে সেট করা `AGENTMEMORY_SECRET` লাগে।+`3111` পোর্টে 141 এন্ডপয়েন্ট। REST API ডিফল্টভাবে `127.0.0.1`-এ বাইন্ড করে। প্রোটেক্টেড এন্ডপয়েন্টে `Authorization: Bearer <secret>` লাগে, এবং mesh sync এন্ডপয়েন্টে দুই পিয়ারেই স্পষ্টভাবে সেট করা `AGENTMEMORY_SECRET` লাগে।**অথেন্টিকেশন ডিফল্টভাবে চালু।** `AGENTMEMORY_SECRET` সেট না থাকলে (শেলে বা `~/.agentmemory/.env`-এ), সার্ভার প্রথম স্টার্টে একটি র্যান্ডম সিক্রেট জেনারেট করে এবং `0600` মোডে `~/.agentmemory/secret`-এ জমা রাখে। লোকাল সার্ভারের সাথে কথা বলার সময় প্রতিটি বান্ডলড ক্লায়েন্ট সেখান থেকেই এটা পড়ে: CLI, ভিউয়ার, `plugin/scripts`-এর নিচের হুক, MCP সার্ভার এবং `@agentmemory/mcp` শিম, `agentmemory connect`-এ লেখা কনফিগ, এবং বান্ডলড OpenCode, Pi, OpenClaw, Hermes, এবং filesystem-watcher ইন্টিগ্রেশন। জমা রাখা সিক্রেটটি কেবল লুপব্যাক URL-এ (`localhost`, `127.0.0.0/8`, `::1`) পাঠানো হয়। একটি এক্সপ্লিসিট `AGENTMEMORY_SECRET` সবসময় জেতে, এবং রিমোট ক্লায়েন্টদেরও এটা সেট করা লাগে। Docker এবং `deploy/` এন্ট্রিপয়েন্ট নিজেরাই নিজেদের সিক্রেট জেনারেট ও এক্সপোর্ট করে। হাতে API কল করতে:@@ -1861,7 +1861,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.cs-CZ.md b/READMEs/README.cs-CZ.mdindex 2c674034..9028b2e2 100644--- a/READMEs/README.cs-CZ.md+++ b/READMEs/README.cs-CZ.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP nástrojů" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 automatických hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 externích databází" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ testů úspěšně proběhlo" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ testů úspěšně proběhlo" height="38" /></picture></p><p align="center">@@ -1446,7 +1446,7 @@ fi| Prometheus / Grafana | iii OTEL + health monitor || Vlastní systémy pluginů | `iii worker add <name>` |-**230 zdrojových souborů · ~55,000 řádků kódu · 3,300+ testů · 312 funkcí · 62 KV scopes**, vše na třech primitivech. Žádné `agentmemory plugin install`. Systém pluginů je samo iii.+**238 zdrojových souborů · ~56,000 řádků kódu · 3,500+ testů · 318 funkcí · 66 KV scopes**, vše na třech primitivech. Žádné `agentmemory plugin install`. Systém pluginů je samo iii.---@@ -1789,7 +1789,7 @@ Vytvořte `~/.agentmemory/.env`:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpointů na portu `3111`. REST API se ve výchozím stavu váže na `127.0.0.1`. Chráněné endpointy vyžadují `Authorization: Bearer <secret>`, a endpointy pro mesh sync vyžadují explicitně nastavené `AGENTMEMORY_SECRET` na obou stranách.+141 endpointů na portu `3111`. REST API se ve výchozím stavu váže na `127.0.0.1`. Chráněné endpointy vyžadují `Authorization: Bearer <secret>`, a endpointy pro mesh sync vyžadují explicitně nastavené `AGENTMEMORY_SECRET` na obou stranách.**Autentizace je ve výchozím stavu zapnutá.** Když `AGENTMEMORY_SECRET` není nastaveno (v shellu ani v `~/.agentmemory/.env`), server při prvním startu vygeneruje náhodný secret a uloží ho do `~/.agentmemory/secret` s režimem `0600`. Každý zabalený klient ho odtud čte, když mluví s lokálním serverem: CLI, viewer, hooks pod `plugin/scripts`, server MCP a shim `@agentmemory/mcp`, konfigurace zapsané příkazem `agentmemory connect` a zabalené integrace OpenCode, Pi, OpenClaw, Hermes a filesystem-watcher. Uložený secret se posílá jen na loopback URL (`localhost`, `127.0.0.0/8`, `::1`). Explicitní `AGENTMEMORY_SECRET` má vždy přednost, a vzdálení klienti ho stejně potřebují nastavený. Docker a entrypointy v `deploy/` už svůj vlastní secret generují a exportují. Pro manuální volání API:@@ -1870,7 +1870,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.da-DK.md b/READMEs/README.da-DK.mdindex 9035239a..0d6234a5 100644--- a/READMEs/README.da-DK.md+++ b/READMEs/README.da-DK.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP-værktøjer" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 automatiske hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 eksterne databaser" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ beståede tests" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ beståede tests" height="38" /></picture></p><p align="center">@@ -1436,7 +1436,7 @@ fi| Prometheus / Grafana | iii OTEL + health monitor || Custom plugin-systemer | `iii worker add <name>` |-**230 kildefiler · ~55,000 LOC · 3,300+ tests · 312 funktioner · 62 KV-scopes**, alt på tre primitiver. Intet `agentmemory plugin install`. Plugin-systemet er iii selv.+**238 kildefiler · ~56,000 LOC · 3,500+ tests · 318 funktioner · 66 KV-scopes**, alt på tre primitiver. Intet `agentmemory plugin install`. Plugin-systemet er iii selv.---@@ -1779,7 +1779,7 @@ Opret `~/.agentmemory/.env`:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpoints på port `3111`. REST API'et binder til `127.0.0.1` som standard. Beskyttede endpoints kræver `Authorization: Bearer <secret>`, og mesh-sync-endpoints kræver en eksplicit sat `AGENTMEMORY_SECRET` på begge peers.+141 endpoints på port `3111`. REST API'et binder til `127.0.0.1` som standard. Beskyttede endpoints kræver `Authorization: Bearer <secret>`, og mesh-sync-endpoints kræver en eksplicit sat `AGENTMEMORY_SECRET` på begge peers.**Autentificering er slået til som standard.** Når `AGENTMEMORY_SECRET` ikke er sat (i shellen eller i `~/.agentmemory/.env`), genererer serveren en tilfældig secret ved første opstart og lagrer den i `~/.agentmemory/secret` med rettigheden `0600`. Hver medfølgende klient læser den fra der, når den taler med en lokal server: CLI'en, viewer, hooks under `plugin/scripts`, MCP-serveren og `@agentmemory/mcp`-shimmen, konfigurationerne skrevet af `agentmemory connect`, og de medfølgende OpenCode-, Pi-, OpenClaw-, Hermes- og filesystem-watcher-integrationer. Den lagrede secret sendes kun til loopback-URL'er (`localhost`, `127.0.0.0/8`, `::1`). En eksplicit `AGENTMEMORY_SECRET` vinder altid, og eksterne klienter skal stadig have den sat. Docker og `deploy/`-entrypoints genererer og eksporterer allerede deres egen secret. For at kalde API'et manuelt:@@ -1860,7 +1860,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.de-DE.md b/READMEs/README.de-DE.mdindex 19a2d1d6..56842167 100644--- a/READMEs/README.de-DE.md+++ b/READMEs/README.de-DE.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP-Tools" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 Auto-Hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 externe DBs" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ bestandene Tests" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ bestandene Tests" height="38" /></picture></p><p align="center">@@ -1446,7 +1446,7 @@ fi| Prometheus / Grafana | iii OTEL + Health-Monitor || Eigene Plugin-Systeme | `iii worker add <name>` |-**230 Quelldateien · ~55,000 LOC · 3,300+ Tests · 312 Funktionen · 62 KV-Scopes**, alles auf drei Primitiven. Kein `agentmemory plugin install`. Das Plugin-System ist iii selbst.+**238 Quelldateien · ~56,000 LOC · 3,500+ Tests · 318 Funktionen · 66 KV-Scopes**, alles auf drei Primitiven. Kein `agentmemory plugin install`. Das Plugin-System ist iii selbst.---@@ -1789,7 +1789,7 @@ GRAPH_EXTRACTION_ENABLED=true<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 Endpunkte auf Port `3111`. Die REST API bindet sich standardmäßig an `127.0.0.1`. Geschützte Endpunkte verlangen `Authorization: Bearer <secret>`, und Mesh-Sync-Endpunkte erfordern ein explizit gesetztes `AGENTMEMORY_SECRET` auf beiden Peers.+141 Endpunkte auf Port `3111`. Die REST API bindet sich standardmäßig an `127.0.0.1`. Geschützte Endpunkte verlangen `Authorization: Bearer <secret>`, und Mesh-Sync-Endpunkte erfordern ein explizit gesetztes `AGENTMEMORY_SECRET` auf beiden Peers.**Authentifizierung ist standardmäßig aktiv.** Wenn `AGENTMEMORY_SECRET` nicht gesetzt ist (weder in der Shell noch in `~/.agentmemory/.env`), erzeugt der Server beim ersten Start ein zufälliges Secret und speichert es in `~/.agentmemory/secret` mit dem Modus `0600`. Jeder mitgelieferte Client liest es von dort, wenn er mit einem lokalen Server spricht: die CLI, der Viewer, die Hooks unter `plugin/scripts`, der MCP-Server und das `@agentmemory/mcp`-Shim, die von `agentmemory connect` geschriebenen Konfigs sowie die mitgelieferten Integrationen für OpenCode, Pi, OpenClaw, Hermes und den Filesystem-Watcher. Das gespeicherte Secret wird nur an Loopback-URLs gesendet (`localhost`, `127.0.0.0/8`, `::1`). Ein explizites `AGENTMEMORY_SECRET` gewinnt immer, und Remote-Clients benötigen es trotzdem gesetzt. Docker und die `deploy/`-Entrypoints erzeugen und exportieren bereits ihr eigenes Secret. Um die API von Hand aufzurufen:@@ -1870,7 +1870,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.el-GR.md b/READMEs/README.el-GR.mdindex ae57444d..9c8681b3 100644--- a/READMEs/README.el-GR.md+++ b/READMEs/README.el-GR.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 εργαλεία MCP" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 αυτόματα hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 εξωτερικές βάσεις δεδομένων" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ επιτυχημένες δοκιμές" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ επιτυχημένες δοκιμές" height="38" /></picture></p><p align="center">@@ -1438,7 +1438,7 @@ fi| Prometheus / Grafana | iii OTEL + monitor υγείας || Προσαρμοσμένα συστήματα plugin | `iii worker add <name>` |-**230 αρχεία πηγαίου κώδικα · ~55,000 LOC · 3,300+ δοκιμές · 312 functions · 62 KV scopes**, όλα πάνω σε τρία primitives. Δεν υπάρχει `agentmemory plugin install`. Το σύστημα plugin είναι το ίδιο το iii.+**238 αρχεία πηγαίου κώδικα · ~56,000 LOC · 3,500+ δοκιμές · 318 functions · 66 KV scopes**, όλα πάνω σε τρία primitives. Δεν υπάρχει `agentmemory plugin install`. Το σύστημα plugin είναι το ίδιο το iii.---@@ -1781,7 +1781,7 @@ GRAPH_EXTRACTION_ENABLED=true<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpoints στη θύρα `3111`. Το REST API συνδέεται στο `127.0.0.1` από προεπιλογή. Τα προστατευμένα endpoints απαιτούν `Authorization: Bearer <secret>`, και τα endpoints mesh sync απαιτούν ένα ρητά ρυθμισμένο `AGENTMEMORY_SECRET` και στους δύο peers.+141 endpoints στη θύρα `3111`. Το REST API συνδέεται στο `127.0.0.1` από προεπιλογή. Τα προστατευμένα endpoints απαιτούν `Authorization: Bearer <secret>`, και τα endpoints mesh sync απαιτούν ένα ρητά ρυθμισμένο `AGENTMEMORY_SECRET` και στους δύο peers.**Η πιστοποίηση είναι ενεργή από προεπιλογή.** Όταν το `AGENTMEMORY_SECRET` δεν έχει ρυθμιστεί (στο shell ή στο `~/.agentmemory/.env`), ο διακομιστής δημιουργεί ένα τυχαίο secret στην πρώτη εκκίνηση και το αποθηκεύει στο `~/.agentmemory/secret` με mode `0600`. Κάθε ενσωματωμένος client το διαβάζει από εκεί όταν μιλά με έναν τοπικό διακομιστή: το CLI, το viewer, τα hooks κάτω από το `plugin/scripts`, ο διακομιστής MCP και το shim `@agentmemory/mcp`, οι ρυθμίσεις που γράφει το `agentmemory connect`, και οι ενσωματωμένες ενσωματώσεις OpenCode, Pi, OpenClaw, Hermes, και filesystem-watcher. Το αποθηκευμένο secret αποστέλλεται μόνο σε loopback URLs (`localhost`, `127.0.0.0/8`, `::1`). Ένα ρητό `AGENTMEMORY_SECRET` υπερισχύει πάντα, και οι απομακρυσμένοι clients εξακολουθούν να το χρειάζονται ρυθμισμένο. Το Docker και τα entrypoints του `deploy/` δημιουργούν και εξάγουν ήδη το δικό τους secret. Για να καλέσετε το API χειροκίνητα:@@ -1862,7 +1862,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.es-ES.md b/READMEs/README.es-ES.mdindex a77a6fc1..c249f88e 100644--- a/READMEs/README.es-ES.md+++ b/READMEs/README.es-ES.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 tools MCP" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 hooks automáticos" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 bases de datos externas" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ tests superados" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ tests superados" height="38" /></picture></p><p align="center">@@ -1446,7 +1446,7 @@ fi| Prometheus / Grafana | iii OTEL + monitor de salud || Sistemas de plugins propios | `iii worker add <name>` |-**230 ficheros de código · ~55,000 LOC · 3,300+ tests · 312 funciones · 62 scopes KV**, todo sobre tres primitivos. No hay `agentmemory plugin install`. El sistema de plugins es iii mismo.+**238 ficheros de código · ~56,000 LOC · 3,500+ tests · 318 funciones · 66 scopes KV**, todo sobre tres primitivos. No hay `agentmemory plugin install`. El sistema de plugins es iii mismo.---@@ -1789,7 +1789,7 @@ Crea `~/.agentmemory/.env`:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpoints en el puerto `3111`. La REST API se enlaza a `127.0.0.1` por defecto. Los endpoints protegidos requieren `Authorization: Bearer <secret>`, y los endpoints de mesh sync requieren un `AGENTMEMORY_SECRET` definido explícitamente en ambos peers.+141 endpoints en el puerto `3111`. La REST API se enlaza a `127.0.0.1` por defecto. Los endpoints protegidos requieren `Authorization: Bearer <secret>`, y los endpoints de mesh sync requieren un `AGENTMEMORY_SECRET` definido explícitamente en ambos peers.**La autenticación está activada por defecto.** Cuando `AGENTMEMORY_SECRET` no está definida (ni en el shell ni en `~/.agentmemory/.env`), el servidor genera un secreto aleatorio en el primer arranque y lo guarda en `~/.agentmemory/secret` con modo `0600`. Todos los clientes empaquetados lo leen desde ahí cuando hablan con un servidor local: la CLI, el visor, los hooks bajo `plugin/scripts`, el servidor MCP y el shim `@agentmemory/mcp`, las configs escritas por `agentmemory connect`, y las integraciones empaquetadas de OpenCode, Pi, OpenClaw, Hermes y filesystem-watcher. El secreto guardado solo se envía a URLs de loopback (`localhost`, `127.0.0.0/8`, `::1`). Un `AGENTMEMORY_SECRET` explícito siempre gana, y los clientes remotos igualmente necesitan tenerlo definido. Docker y los entrypoints de `deploy/` ya generan y exportan su propio secreto. Para llamar a la API a mano:@@ -1870,7 +1870,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.fi-FI.md b/READMEs/README.fi-FI.mdindex e700bb37..81f439be 100644--- a/READMEs/README.fi-FI.md+++ b/READMEs/README.fi-FI.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP-työkalua" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 automaattista hookia" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 ulkoista tietokantaa" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ testiä läpäisty" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ testiä läpäisty" height="38" /></picture></p><p align="center">@@ -1446,7 +1446,7 @@ fi| Prometheus / Grafana | iii OTEL + terveysmonitori || Mukautetut plugin-järjestelmät | `iii worker add <name>` |-**230 lähdetiedostoa · ~55,000 LOC · 3,300+ testiä · 312 funktiota · 62 KV-scopea**, kaikki kolmen primitiivin päällä. Ei `agentmemory plugin install` -komentoa. Plugin-järjestelmä on iii itse.+**238 lähdetiedostoa · ~56,000 LOC · 3,500+ testiä · 318 funktiota · 66 KV-scopea**, kaikki kolmen primitiivin päällä. Ei `agentmemory plugin install` -komentoa. Plugin-järjestelmä on iii itse.---@@ -1789,7 +1789,7 @@ Luo `~/.agentmemory/.env`:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpointia portissa `3111`. REST-rajapinta sitoutuu oletuksena osoitteeseen `127.0.0.1`. Suojatut endpointit vaativat `Authorization: Bearer <secret>`, ja mesh-synkronointiendpointit vaativat nimenomaisesti asetetun `AGENTMEMORY_SECRET`-arvon molemmilla vertaisilla.+141 endpointia portissa `3111`. REST-rajapinta sitoutuu oletuksena osoitteeseen `127.0.0.1`. Suojatut endpointit vaativat `Authorization: Bearer <secret>`, ja mesh-synkronointiendpointit vaativat nimenomaisesti asetetun `AGENTMEMORY_SECRET`-arvon molemmilla vertaisilla.**Autentikointi on oletuksena päällä.** Kun `AGENTMEMORY_SECRET` ei ole asetettu (shellissä tai tiedostossa `~/.agentmemory/.env`), palvelin generoi satunnaisen salaisuuden ensimmäisellä käynnistyksellä ja tallentaa sen polkuun `~/.agentmemory/secret` oikeuksilla `0600`. Joka mukana tuleva asiakas lukee sen sieltä puhuessaan paikalliselle palvelimelle: CLI, katseluohjelma, hookit kansiossa `plugin/scripts`, MCP-palvelin ja `@agentmemory/mcp`-shim, `agentmemory connect`-komennon kirjoittamat asetukset, ja mukana tulevat OpenCode-, Pi-, OpenClaw-, Hermes- ja tiedostojärjestelmän tarkkailijaintegraatiot. Tallennettu salaisuus lähetetään vain loopback-URL-osoitteisiin (`localhost`, `127.0.0.0/8`, `::1`). Nimenomainen `AGENTMEMORY_SECRET` voittaa aina, ja etäasiakkaat tarvitsevat sen yhä asetettuna. Docker ja `deploy/`-entrypointit generoivat ja vievät jo omansa. Kutsuaksesi rajapintaa käsin:@@ -1870,7 +1870,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.fr-FR.md b/READMEs/README.fr-FR.mdindex 289892fc..20a469c4 100644--- a/READMEs/README.fr-FR.md+++ b/READMEs/README.fr-FR.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP tools" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 auto hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 external DBs" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ tests passing" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ tests passing" height="38" /></picture></p><p align="center">@@ -1448,7 +1448,7 @@ fi| Prometheus / Grafana | iii OTEL + moniteur de santé || Systèmes de plugins custom | `iii worker add <name>` |-**230 fichiers sources · ~55,000 LOC · 3,300+ tests · 312 fonctions · 62 scopes KV**, tout sur trois primitives. Pas de `agentmemory plugin install`. Le système de plugins, c'est iii lui-même.+**238 fichiers sources · ~56,000 LOC · 3,500+ tests · 318 fonctions · 66 scopes KV**, tout sur trois primitives. Pas de `agentmemory plugin install`. Le système de plugins, c'est iii lui-même.---@@ -1791,7 +1791,7 @@ Créez `~/.agentmemory/.env` :<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpoints sur le port `3111`. L'API REST se lie à `127.0.0.1` par défaut. Les endpoints protégés requièrent `Authorization: Bearer <secret>`, et les endpoints de mesh sync requièrent un `AGENTMEMORY_SECRET` explicitement défini sur les deux pairs.+141 endpoints sur le port `3111`. L'API REST se lie à `127.0.0.1` par défaut. Les endpoints protégés requièrent `Authorization: Bearer <secret>`, et les endpoints de mesh sync requièrent un `AGENTMEMORY_SECRET` explicitement défini sur les deux pairs.**L'authentification est activée par défaut.** Quand `AGENTMEMORY_SECRET` n'est pas défini (dans le shell ou dans `~/.agentmemory/.env`), le serveur génère un secret aléatoire au premier démarrage et le stocke dans `~/.agentmemory/secret` avec le mode `0600`. Chaque client embarqué le lit depuis là quand il parle à un serveur local : la CLI, le visualiseur, les hooks sous `plugin/scripts`, le serveur MCP et le shim `@agentmemory/mcp`, les configs écrites par `agentmemory connect`, et les intégrations embarquées OpenCode, Pi, OpenClaw, Hermes et filesystem-watcher. Le secret stocké n'est envoyé qu'à des URLs loopback (`localhost`, `127.0.0.0/8`, `::1`). Un `AGENTMEMORY_SECRET` explicite prime toujours, et les clients distants en ont toujours besoin. Docker et les points d'entrée `deploy/` génèrent et exportent déjà leur propre secret. Pour appeler l'API à la main :@@ -1872,7 +1872,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.he-IL.md b/READMEs/README.he-IL.mdindex 0a0de75a..ff6a3aa9 100644--- a/READMEs/README.he-IL.md+++ b/READMEs/README.he-IL.md@@ -73,7 +73,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 כלי MCP" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 hooks אוטומטיים" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 מסדי נתונים חיצוניים" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ בדיקות עוברות" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ בדיקות עוברות" height="38" /></picture></p><p align="center">@@ -1439,7 +1439,7 @@ fi| Prometheus / Grafana | iii OTEL + מוניטור health || מערכות plugin מותאמות אישית | `iii worker add <name>` |-**230 קבצי מקור · ~55,000 LOC · 3,300+ בדיקות · 312 פונקציות · 62 תחומי KV**, הכול על שלושה primitives. אין `agentmemory plugin install`. מערכת ה-plugin היא iii עצמו.+**238 קבצי מקור · ~56,000 LOC · 3,500+ בדיקות · 318 פונקציות · 66 תחומי KV**, הכול על שלושה primitives. אין `agentmemory plugin install`. מערכת ה-plugin היא iii עצמו.---@@ -1782,7 +1782,7 @@ GRAPH_EXTRACTION_ENABLED=true<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpoints בפורט `3111`. ה-REST API נקשר ל-`127.0.0.1` כברירת מחדל. endpoints מוגנים דורשים `Authorization: Bearer <secret>`, ו-endpoints של mesh sync דורשים `AGENTMEMORY_SECRET` מוגדר באופן מפורש בשני ה-peers.+141 endpoints בפורט `3111`. ה-REST API נקשר ל-`127.0.0.1` כברירת מחדל. endpoints מוגנים דורשים `Authorization: Bearer <secret>`, ו-endpoints של mesh sync דורשים `AGENTMEMORY_SECRET` מוגדר באופן מפורש בשני ה-peers.**אימות (Authentication) פעיל כברירת מחדל.** כש-`AGENTMEMORY_SECRET` לא מוגדר (ב-shell או ב-`~/.agentmemory/.env`), השרת מייצר secret אקראי בהפעלה הראשונה ושומר אותו ב-`~/.agentmemory/secret` במוד `0600`. כל לקוח מצורף קורא אותו משם כשהוא מדבר עם שרת מקומי: ה-CLI, המציג, ה-hooks תחת `plugin/scripts`, שרת ה-MCP וה-shim `@agentmemory/mcp`, התצורות שנכתבות על ידי `agentmemory connect`, והאינטגרציות המצורפות של OpenCode, Pi, OpenClaw, Hermes וה-filesystem-watcher. ה-secret המאוחסן נשלח רק ל-URLs של loopback (`localhost`, `127.0.0.0/8`, `::1`). `AGENTMEMORY_SECRET` מפורש גובר תמיד, ולקוחות מרוחקים עדיין צריכים אותו מוגדר. Docker וה-entrypoints של `deploy/` כבר מייצרים ומייצאים את ה-secret שלהם. כדי לקרוא ל-API ביד:@@ -1863,7 +1863,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.hi-IN.md b/READMEs/README.hi-IN.mdindex 90eaabff..55fc9f1e 100644--- a/READMEs/README.hi-IN.md+++ b/READMEs/README.hi-IN.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP tools" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 auto hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 external DBs" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ tests passing" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ tests passing" height="38" /></picture></p><p align="center">@@ -1449,7 +1449,7 @@ fi| Prometheus / Grafana | iii OTEL + health monitor || Custom plugin systems | `iii worker add <name>` |-**230 source files · ~55,000 LOC · 3,300+ tests · 312 functions · 62 KV scopes**, सब कुछ तीन primitives पर। कोई `agentmemory plugin install` नहीं। Plugin system iii स्वयं है।+**238 source files · ~56,000 LOC · 3,500+ tests · 318 functions · 66 KV scopes**, सब कुछ तीन primitives पर। कोई `agentmemory plugin install` नहीं। Plugin system iii स्वयं है।---@@ -1792,7 +1792,7 @@ GRAPH_EXTRACTION_ENABLED=true<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-Port `3111` पर 138 endpoints। REST API default रूप से `127.0.0.1` से bind होता है। Protected endpoints `Authorization: Bearer <secret>` की आवश्यकता रखते हैं, और mesh sync endpoints दोनों peers पर एक explicitly set `AGENTMEMORY_SECRET` की आवश्यकता रखते हैं।+Port `3111` पर 141 endpoints। REST API default रूप से `127.0.0.1` से bind होता है। Protected endpoints `Authorization: Bearer <secret>` की आवश्यकता रखते हैं, और mesh sync endpoints दोनों peers पर एक explicitly set `AGENTMEMORY_SECRET` की आवश्यकता रखते हैं।**Authentication डिफ़ॉल्ट रूप से on है।** जब `AGENTMEMORY_SECRET` set नहीं है (shell में या `~/.agentmemory/.env` में), server पहले start पर एक random secret generate करता है और इसे mode `0600` के साथ `~/.agentmemory/secret` में store करता है। हर bundled client किसी local server से बात करते समय इसे वहाँ से पढ़ता है: CLI, viewer, `plugin/scripts` के अंतर्गत hooks, MCP server और `@agentmemory/mcp` shim, `agentmemory connect` द्वारा लिखी गई configs, और bundled OpenCode, Pi, OpenClaw, Hermes, और filesystem-watcher integrations। Stored secret केवल loopback URLs (`localhost`, `127.0.0.0/8`, `::1`) पर भेजा जाता है। एक explicit `AGENTMEMORY_SECRET` हमेशा जीतता है, और remote clients को फिर भी इसे set करना होता है। Docker और `deploy/` entrypoints पहले से अपना स्वयं का secret generate और export करते हैं। API को हाथ से call करने के लिए:@@ -1873,7 +1873,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.hu-HU.md b/READMEs/README.hu-HU.mdindex bb3a7a05..4bbab84b 100644--- a/READMEs/README.hu-HU.md+++ b/READMEs/README.hu-HU.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP eszköz" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 automatikus hook" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 külső adatbázis" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ sikeres teszt" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ sikeres teszt" height="38" /></picture></p><p align="center">@@ -1446,7 +1446,7 @@ A `/agentmemory/export` emellett elfogadja a `?maxSessions=`-t és a `?offset=`-| Prometheus / Grafana | iii OTEL + egészségfigyelő || Egyéni plugin-rendszerek | `iii worker add <name>` |-**230 forrásfájl · ~55,000 LOC · 3,300+ teszt · 312 funkció · 62 KV-hatókör**, mindez három primitíven. Nincs `agentmemory plugin install`. A plugin-rendszer maga az iii.+**238 forrásfájl · ~56,000 LOC · 3,500+ teszt · 318 funkció · 66 KV-hatókör**, mindez három primitíven. Nincs `agentmemory plugin install`. A plugin-rendszer maga az iii.---@@ -1789,7 +1789,7 @@ Hozd létre a `~/.agentmemory/.env`-et:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpoint a `3111`-es porton. A REST API alapértelmezetten a `127.0.0.1`-hez kötődik. A védett végpontok `Authorization: Bearer <secret>`-et igényelnek, és a mesh-sync végpontok mindkét oldalon explicit módon beállított `AGENTMEMORY_SECRET`-et igényelnek.+141 endpoint a `3111`-es porton. A REST API alapértelmezetten a `127.0.0.1`-hez kötődik. A védett végpontok `Authorization: Bearer <secret>`-et igényelnek, és a mesh-sync végpontok mindkét oldalon explicit módon beállított `AGENTMEMORY_SECRET`-et igényelnek.**A hitelesítés alapértelmezetten be van kapcsolva.** Amikor az `AGENTMEMORY_SECRET` nincs beállítva (sem a shellben, sem a `~/.agentmemory/.env`-ben), a szerver az első indításkor egy véletlen secretet generál, és `0600` móddal tárolja a `~/.agentmemory/secret`-ben. Minden becsomagolt kliens onnan olvassa, amikor egy lokális szerverrel beszél: a CLI, a megjelenítő, a `plugin/scripts` alatti hookok, az MCP szerver és a `@agentmemory/mcp` shim, az `agentmemory connect` által írt konfigurációk, és a becsomagolt OpenCode, Pi, OpenClaw, Hermes és fájlrendszer-figyelő integrációk. A tárolt secret csak loopback URL-eknek kerül elküldésre (`localhost`, `127.0.0.0/8`, `::1`). Egy explicit `AGENTMEMORY_SECRET` mindig elsőbbséget kap, és a távoli klienseknek továbbra is be kell állítaniuk. A Docker és a `deploy/` entrypointok már generálnak és exportálnak egy saját secretet. Az API kézi hívásához:@@ -1870,7 +1870,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.id-ID.md b/READMEs/README.id-ID.mdindex c76478fa..aa44f1ba 100644--- a/READMEs/README.id-ID.md+++ b/READMEs/README.id-ID.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 tool MCP" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 hook otomatis" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 DB eksternal" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ pengujian lolos" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ pengujian lolos" height="38" /></picture></p><p align="center">@@ -1447,7 +1447,7 @@ fi| Prometheus / Grafana | iii OTEL + health monitor || Sistem plugin custom | `iii worker add <name>` |-**230 file sumber · ~55,000 LOC · 3,300+ pengujian · 312 fungsi · 62 scope KV**, semua di atas tiga primitif. Tidak ada `agentmemory plugin install`. Sistem plugin-nya adalah iii itu sendiri.+**238 file sumber · ~56,000 LOC · 3,500+ pengujian · 318 fungsi · 66 scope KV**, semua di atas tiga primitif. Tidak ada `agentmemory plugin install`. Sistem plugin-nya adalah iii itu sendiri.---@@ -1790,7 +1790,7 @@ Buat `~/.agentmemory/.env`:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpoint di port `3111`. REST API terbind ke `127.0.0.1` secara default. Endpoint yang terlindungi membutuhkan `Authorization: Bearer <secret>`, dan endpoint mesh sync membutuhkan `AGENTMEMORY_SECRET` yang diset secara eksplisit di kedua peer.+141 endpoint di port `3111`. REST API terbind ke `127.0.0.1` secara default. Endpoint yang terlindungi membutuhkan `Authorization: Bearer <secret>`, dan endpoint mesh sync membutuhkan `AGENTMEMORY_SECRET` yang diset secara eksplisit di kedua peer.**Autentikasi aktif secara default.** Saat `AGENTMEMORY_SECRET` tidak diset (di shell atau di `~/.agentmemory/.env`), server menghasilkan secret acak pada start pertama dan menyimpannya di `~/.agentmemory/secret` dengan mode `0600`. Setiap klien bundel membacanya dari sana saat berbicara dengan server lokal: CLI, viewer, hook di bawah `plugin/scripts`, server MCP dan shim `@agentmemory/mcp`, konfigurasi yang ditulis `agentmemory connect`, serta integrasi OpenCode, Pi, OpenClaw, Hermes, dan filesystem-watcher bundel. Secret yang tersimpan hanya dikirim ke URL loopback (`localhost`, `127.0.0.0/8`, `::1`). `AGENTMEMORY_SECRET` eksplisit selalu menang, dan klien remote tetap membutuhkannya diset. Docker dan entrypoint `deploy/` sudah menghasilkan dan mengekspor secret-nya sendiri. Untuk memanggil API secara manual:@@ -1871,7 +1871,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.it-IT.md b/READMEs/README.it-IT.mdindex eb2426b4..2672b955 100644--- a/READMEs/README.it-IT.md+++ b/READMEs/README.it-IT.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP tools" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 auto hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 external DBs" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ tests passing" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ tests passing" height="38" /></picture></p><p align="center">@@ -1446,7 +1446,7 @@ fi| Prometheus / Grafana | iii OTEL + monitor di salute || Sistemi di plugin personalizzati | `iii worker add <name>` |-**230 file sorgente · ~55,000 righe di codice · 3,300+ test · 312 funzioni · 62 scope KV**, tutto su tre primitive. Nessun `agentmemory plugin install`. Il sistema di plugin è iii stesso.+**238 file sorgente · ~56,000 righe di codice · 3,500+ test · 318 funzioni · 66 scope KV**, tutto su tre primitive. Nessun `agentmemory plugin install`. Il sistema di plugin è iii stesso.---@@ -1789,7 +1789,7 @@ Crea `~/.agentmemory/.env`:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpoint sulla porta `3111`. L'API REST si collega a `127.0.0.1` per impostazione predefinita. Gli endpoint protetti richiedono `Authorization: Bearer <secret>`, e gli endpoint di sincronizzazione mesh richiedono un `AGENTMEMORY_SECRET` impostato esplicitamente su entrambi i peer.+141 endpoint sulla porta `3111`. L'API REST si collega a `127.0.0.1` per impostazione predefinita. Gli endpoint protetti richiedono `Authorization: Bearer <secret>`, e gli endpoint di sincronizzazione mesh richiedono un `AGENTMEMORY_SECRET` impostato esplicitamente su entrambi i peer.**L'autenticazione è attiva per default.** Quando `AGENTMEMORY_SECRET` non è impostato (nella shell o in `~/.agentmemory/.env`), il server genera un secret casuale al primo avvio e lo memorizza in `~/.agentmemory/secret` con modalità `0600`. Ogni client incluso lo legge da lì quando parla con un server locale: la CLI, il viewer, gli hook sotto `plugin/scripts`, il server MCP e lo shim `@agentmemory/mcp`, le configurazioni scritte da `agentmemory connect`, e le integrazioni incluse OpenCode, Pi, OpenClaw, Hermes e filesystem-watcher. Il secret memorizzato viene inviato solo a URL loopback (`localhost`, `127.0.0.0/8`, `::1`). Un `AGENTMEMORY_SECRET` esplicito vince sempre, e i client remoti devono comunque averlo impostato. Docker e gli entrypoint di `deploy/` generano ed esportano già il proprio secret. Per chiamare l'API a mano:@@ -1870,7 +1870,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.ja-JP.md b/READMEs/README.ja-JP.mdindex b3dbee41..cd234b20 100644--- a/READMEs/README.ja-JP.md+++ b/READMEs/README.ja-JP.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP ツール" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 自動 hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="外部 DB 0 個" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ 件のテストが成功" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ 件のテストが成功" height="38" /></picture></p><p align="center">@@ -1445,7 +1445,7 @@ fi| Prometheus / Grafana | iii OTEL + ヘルスモニタ || カスタムプラグインシステム | `iii worker add <name>` |-**230 ソースファイル · ~55,000 LOC · 3,300+ テスト · 312 functions · 62 KV スコープ** — すべて 3 つのプリミティブの上に。`agentmemory plugin install` はありません。プラグインシステムは iii そのものです。+**238 ソースファイル · ~56,000 LOC · 3,500+ テスト · 318 functions · 66 KV スコープ** — すべて 3 つのプリミティブの上に。`agentmemory plugin install` はありません。プラグインシステムは iii そのものです。---@@ -1788,7 +1788,7 @@ GRAPH_EXTRACTION_ENABLED=true<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-ポート `3111` 上の 138 エンドポイント。REST API はデフォルトで `127.0.0.1` にバインドします。保護されたエンドポイントは `Authorization: Bearer <secret>` を要求し、mesh sync エンドポイントは両ピアで明示的に設定された `AGENTMEMORY_SECRET` を要求します。+ポート `3111` 上の 141 エンドポイント。REST API はデフォルトで `127.0.0.1` にバインドします。保護されたエンドポイントは `Authorization: Bearer <secret>` を要求し、mesh sync エンドポイントは両ピアで明示的に設定された `AGENTMEMORY_SECRET` を要求します。**認証はデフォルトでオンです。** `AGENTMEMORY_SECRET` が(シェルにも `~/.agentmemory/.env` にも)設定されていない場合、サーバーは初回起動時にランダムなシークレットを生成し、モード `0600` で `~/.agentmemory/secret` に保存します。同梱のすべてのクライアント — CLI、ビューワー、`plugin/scripts` 配下の hooks、MCP サーバーと `@agentmemory/mcp` の shim、`agentmemory connect` が書き込む設定、同梱の OpenCode、Pi、OpenClaw、Hermes、ファイルシステムウォッチャーの各統合 — は、ローカルサーバーと話すときにそこから読み取ります。保存されたシークレットは、ループバックの URL(`localhost`、`127.0.0.0/8`、`::1`)にのみ送信されます。明示的な `AGENTMEMORY_SECRET` は常に優先され、リモートクライアントではそれでも設定が必要です。Docker と `deploy/` のエントリポイントは、既に独自のシークレットを生成・export しています。手動で API を呼ぶには:@@ -1869,7 +1869,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.ko-KR.md b/READMEs/README.ko-KR.mdindex 472bfc63..027dd6a4 100644--- a/READMEs/README.ko-KR.md+++ b/READMEs/README.ko-KR.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP tools" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 auto hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 external DBs" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ tests passing" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ tests passing" height="38" /></picture></p><p align="center">@@ -1446,7 +1446,7 @@ fi| Prometheus / Grafana | iii OTEL + health 모니터 || 커스텀 플러그인 시스템 | `iii worker add <name>` |-**소스 파일 230개 · LOC ~55,000줄 · 테스트 3,300+개 · 함수 312개 · KV 스코프 62개**, 모두 세 가지 프리미티브 위에서 동작합니다. `agentmemory plugin install` 같은 것은 없습니다. 플러그인 시스템은 곧 iii 자체입니다.+**소스 파일 238개 · LOC ~56,000줄 · 테스트 3,500+개 · 함수 318개 · KV 스코프 66개**, 모두 세 가지 프리미티브 위에서 동작합니다. `agentmemory plugin install` 같은 것은 없습니다. 플러그인 시스템은 곧 iii 자체입니다.---@@ -1789,7 +1789,7 @@ GRAPH_EXTRACTION_ENABLED=true<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-포트 `3111`의 엔드포인트 138개. REST API는 기본적으로 `127.0.0.1`에 바인딩됩니다. 보호된 엔드포인트는 `Authorization: Bearer <secret>`을 요구하며, mesh sync 엔드포인트는 양쪽 피어 모두에 `AGENTMEMORY_SECRET`이 명시적으로 설정되어 있어야 합니다.+포트 `3111`의 엔드포인트 141개. REST API는 기본적으로 `127.0.0.1`에 바인딩됩니다. 보호된 엔드포인트는 `Authorization: Bearer <secret>`을 요구하며, mesh sync 엔드포인트는 양쪽 피어 모두에 `AGENTMEMORY_SECRET`이 명시적으로 설정되어 있어야 합니다.**인증은 기본적으로 켜져 있습니다.** `AGENTMEMORY_SECRET`이(셸이나 `~/.agentmemory/.env`에) 설정되어 있지 않으면, 서버는 첫 시작 시 무작위 시크릿을 생성해 `~/.agentmemory/secret`에 `0600` 권한으로 저장합니다. 번들된 모든 클라이언트는 로컬 서버와 통신할 때 그곳에서 이를 읽습니다: CLI, 뷰어, `plugin/scripts` 아래의 hook, MCP 서버와 `@agentmemory/mcp` shim, `agentmemory connect`가 작성하는 설정, 그리고 번들된 OpenCode, Pi, OpenClaw, Hermes, filesystem-watcher 통합까지입니다. 저장된 시크릿은 loopback URL(`localhost`, `127.0.0.0/8`, `::1`)에만 전송됩니다. 명시적인 `AGENTMEMORY_SECRET`은 항상 우선하며, 원격 클라이언트는 여전히 이를 설정해야 합니다. Docker와 `deploy/` 진입점은 이미 자체 시크릿을 생성하고 export합니다. API를 직접 호출하려면:@@ -1870,7 +1870,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.nb-NO.md b/READMEs/README.nb-NO.mdindex 7ff308a4..6a93cb8c 100644--- a/READMEs/README.nb-NO.md+++ b/READMEs/README.nb-NO.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP tools" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 auto hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 external DBs" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ tests passing" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ tests passing" height="38" /></picture></p><p align="center">@@ -1447,7 +1447,7 @@ fi| Prometheus / Grafana | iii OTEL + helseovervåker || Egendefinerte plugin-systemer | `iii worker add <name>` |-**230 kildefiler · ~55,000 LOC · 3,300+ tester · 312 funksjoner · 62 KV-scopes**, alt på tre primitiver. Ingen `agentmemory plugin install`. Plugin-systemet er iii selv.+**238 kildefiler · ~56,000 LOC · 3,500+ tester · 318 funksjoner · 66 KV-scopes**, alt på tre primitiver. Ingen `agentmemory plugin install`. Plugin-systemet er iii selv.---@@ -1790,7 +1790,7 @@ Opprett `~/.agentmemory/.env`:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endepunkter på port `3111`. REST-API-en bindes til `127.0.0.1` som standard. Beskyttede endepunkter krever `Authorization: Bearer <secret>`, og mesh-synkroniseringsendepunkter krever en uttrykkelig satt `AGENTMEMORY_SECRET` på begge sider.+141 endepunkter på port `3111`. REST-API-en bindes til `127.0.0.1` som standard. Beskyttede endepunkter krever `Authorization: Bearer <secret>`, og mesh-synkroniseringsendepunkter krever en uttrykkelig satt `AGENTMEMORY_SECRET` på begge sider.**Autentisering er på som standard.** Når `AGENTMEMORY_SECRET` ikke er satt (i skallet eller i `~/.agentmemory/.env`), genererer serveren en tilfeldig hemmelighet ved første oppstart og lagrer den i `~/.agentmemory/secret` med modus `0600`. Hver medfølgende klient leser den fra der når den snakker med en lokal server: CLI-en, viewer, hookene under `plugin/scripts`, MCP-serveren og `@agentmemory/mcp`-shimmen, konfigurasjonene skrevet av `agentmemory connect`, og de medfølgende OpenCode-, Pi-, OpenClaw-, Hermes-, og filsystem-overvåker-integrasjonene. Den lagrede hemmeligheten sendes bare til loopback-URL-er (`localhost`, `127.0.0.0/8`, `::1`). En uttrykkelig `AGENTMEMORY_SECRET` vinner alltid, og eksterne klienter trenger den fortsatt satt. Docker og `deploy/`-entrypointene genererer og eksporterer allerede sin egen hemmelighet. For å kalle API-en manuelt:@@ -1871,7 +1871,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.nl-NL.md b/READMEs/README.nl-NL.mdindex 17956ec1..d2e81104 100644--- a/READMEs/README.nl-NL.md+++ b/READMEs/README.nl-NL.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP tools" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 auto hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 external DBs" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ tests passing" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ tests passing" height="38" /></picture></p><p align="center">@@ -1446,7 +1446,7 @@ fi| Prometheus / Grafana | iii OTEL + health monitor || Custom plugin-systemen | `iii worker add <name>` |-**230 bronbestanden · ~55,000 LOC · 3,300+ tests · 312 functies · 62 KV-scopes**, allemaal op drie primitieven. Geen `agentmemory plugin install`. Het pluginsysteem is iii zelf.+**238 bronbestanden · ~56,000 LOC · 3,500+ tests · 318 functies · 66 KV-scopes**, allemaal op drie primitieven. Geen `agentmemory plugin install`. Het pluginsysteem is iii zelf.---@@ -1789,7 +1789,7 @@ Maak `~/.agentmemory/.env` aan:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpoints op poort `3111`. De REST API bindt standaard aan `127.0.0.1`. Beschermde endpoints vereisen `Authorization: Bearer <secret>`, en mesh-sync-endpoints vereisen een expliciet ingestelde `AGENTMEMORY_SECRET` op beide peers.+141 endpoints op poort `3111`. De REST API bindt standaard aan `127.0.0.1`. Beschermde endpoints vereisen `Authorization: Bearer <secret>`, en mesh-sync-endpoints vereisen een expliciet ingestelde `AGENTMEMORY_SECRET` op beide peers.**Authenticatie staat standaard aan.** Wanneer `AGENTMEMORY_SECRET` niet is ingesteld (in de shell of in `~/.agentmemory/.env`), genereert de server bij de eerste start een willekeurig secret en bewaart dit in `~/.agentmemory/secret` met modus `0600`. Elke gebundelde client leest dit vandaar wanneer hij met een lokale server praat: de CLI, de viewer, de hooks onder `plugin/scripts`, de MCP-server en de `@agentmemory/mcp`-shim, de configs geschreven door `agentmemory connect`, en de gebundelde OpenCode-, Pi-, OpenClaw-, Hermes- en filesystem-watcher-integraties. Het opgeslagen secret wordt alleen verstuurd naar loopback-URL's (`localhost`, `127.0.0.0/8`, `::1`). Een expliciete `AGENTMEMORY_SECRET` wint altijd, en remote clients moeten deze nog steeds instellen. Docker en de `deploy/`-entrypoints genereren en exporteren al hun eigen secret. Om de API handmatig aan te roepen:@@ -1870,7 +1870,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.pl-PL.md b/READMEs/README.pl-PL.mdindex b14af39e..d0bdff02 100644--- a/READMEs/README.pl-PL.md+++ b/READMEs/README.pl-PL.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 narzędzia MCP" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 automatycznych hooków" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 zewnętrznych baz danych" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ zaliczonych testów" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ zaliczonych testów" height="38" /></picture></p><p align="center">@@ -1446,7 +1446,7 @@ fi| Prometheus / Grafana | iii OTEL + monitor stanu || Niestandardowe systemy pluginów | `iii worker add <name>` |-**230 plików źródłowych · ~55,000 LOC · 3,300+ testów · 312 funkcji · 62 zakresy KV**, wszystko na trzech prymitywach. Nie ma `agentmemory plugin install`. Systemem pluginów jest sam iii.+**238 plików źródłowych · ~56,000 LOC · 3,500+ testów · 318 funkcji · 66 zakresów KV**, wszystko na trzech prymitywach. Nie ma `agentmemory plugin install`. Systemem pluginów jest sam iii.---@@ -1789,7 +1789,7 @@ Utwórz `~/.agentmemory/.env`:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpointów na porcie `3111`. REST API domyślnie wiąże się z `127.0.0.1`. Chronione endpointy wymagają `Authorization: Bearer <secret>`, a endpointy synchronizacji mesh wymagają wyraźnie ustawionego `AGENTMEMORY_SECRET` na obu peerach.+141 endpointów na porcie `3111`. REST API domyślnie wiąże się z `127.0.0.1`. Chronione endpointy wymagają `Authorization: Bearer <secret>`, a endpointy synchronizacji mesh wymagają wyraźnie ustawionego `AGENTMEMORY_SECRET` na obu peerach.**Autoryzacja jest domyślnie włączona.** Gdy `AGENTMEMORY_SECRET` nie jest ustawione (w shellu albo w `~/.agentmemory/.env`), serwer generuje losowy sekret przy pierwszym starcie i zapisuje go w `~/.agentmemory/secret` z uprawnieniami `0600`. Każdy dołączony klient czyta go z tego miejsca, gdy rozmawia z lokalnym serwerem: CLI, podgląd, hooki w `plugin/scripts`, serwer MCP i shim `@agentmemory/mcp`, konfiguracje zapisywane przez `agentmemory connect` oraz dołączone integracje OpenCode, Pi, OpenClaw, Hermes i filesystem-watchera. Zapisany sekret jest wysyłany tylko na adresy loopback (`localhost`, `127.0.0.0/8`, `::1`). Wyraźnie podane `AGENTMEMORY_SECRET` zawsze wygrywa, a zdalni klienci wciąż muszą mieć je ustawione. Docker i punkty wejścia w `deploy/` już generują i eksportują swój własny sekret. Aby wywołać API ręcznie:@@ -1870,7 +1870,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.pt-BR.md b/READMEs/README.pt-BR.mdindex b63aa3d0..18c0956b 100644--- a/READMEs/README.pt-BR.md+++ b/READMEs/README.pt-BR.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP tools" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 auto hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 external DBs" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ tests passing" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ tests passing" height="38" /></picture></p><p align="center">@@ -1447,7 +1447,7 @@ fi| Prometheus / Grafana | iii OTEL + monitor de saúde || Sistemas de plugin customizados | `iii worker add <name>` |-**230 arquivos de código · ~55,000 LOC · 3,300+ testes · 312 funções · 62 escopos KV**, tudo em cima de três primitivos. Sem `agentmemory plugin install`. O sistema de plugins é o próprio iii.+**238 arquivos de código · ~56,000 LOC · 3,500+ testes · 318 funções · 66 escopos KV**, tudo em cima de três primitivos. Sem `agentmemory plugin install`. O sistema de plugins é o próprio iii.---@@ -1790,7 +1790,7 @@ Crie `~/.agentmemory/.env`:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpoints na porta `3111`. A REST API faz bind em `127.0.0.1` por padrão. Endpoints protegidos exigem `Authorization: Bearer <secret>`, e endpoints de mesh sync exigem um `AGENTMEMORY_SECRET` explicitamente definido em ambos os peers.+141 endpoints na porta `3111`. A REST API faz bind em `127.0.0.1` por padrão. Endpoints protegidos exigem `Authorization: Bearer <secret>`, e endpoints de mesh sync exigem um `AGENTMEMORY_SECRET` explicitamente definido em ambos os peers.**A autenticação vem ativada por padrão.** Quando `AGENTMEMORY_SECRET` não está definido (no shell ou em `~/.agentmemory/.env`), o servidor gera um secret aleatório no primeiro início e o guarda em `~/.agentmemory/secret` com modo `0600`. Todo cliente empacotado o lê de lá quando fala com um servidor local: o CLI, o viewer, os hooks em `plugin/scripts`, o servidor MCP e o shim `@agentmemory/mcp`, as configs escritas por `agentmemory connect`, e as integrações empacotadas de OpenCode, Pi, OpenClaw, Hermes e filesystem-watcher. O secret guardado só é enviado para URLs de loopback (`localhost`, `127.0.0.0/8`, `::1`). Um `AGENTMEMORY_SECRET` explícito sempre prevalece, e clientes remotos ainda precisam dele definido. O Docker e os entrypoints de `deploy/` já geram e exportam o próprio secret. Para chamar a API manualmente:@@ -1871,7 +1871,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.pt-PT.md b/READMEs/README.pt-PT.mdindex 8e74ba62..6fc2a81e 100644--- a/READMEs/README.pt-PT.md+++ b/READMEs/README.pt-PT.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 ferramentas MCP" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 hooks automáticos" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 bases de dados externas" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ testes aprovados" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ testes aprovados" height="38" /></picture></p><p align="center">@@ -1446,7 +1446,7 @@ O `/agentmemory/export` também aceita `?maxSessions=` e `?offset=` para dividir| Prometheus / Grafana | iii OTEL + monitor de saúde || Sistemas de plugins personalizados | `iii worker add <name>` |-**230 ficheiros de código-fonte · ~55,000 LOC · 3,300+ testes · 312 funções · 62 âmbitos de KV**, tudo sobre três primitivos. Sem `agentmemory plugin install`. O sistema de plugins é o próprio iii.+**238 ficheiros de código-fonte · ~56,000 LOC · 3,500+ testes · 318 funções · 66 âmbitos de KV**, tudo sobre três primitivos. Sem `agentmemory plugin install`. O sistema de plugins é o próprio iii.---@@ -1789,7 +1789,7 @@ Crie `~/.agentmemory/.env`:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpoints na porta `3111`. A REST API faz bind em `127.0.0.1` por predefinição. Os endpoints protegidos exigem `Authorization: Bearer <secret>`, e os endpoints de sincronização mesh exigem uma `AGENTMEMORY_SECRET` explicitamente definida em ambos os pares.+141 endpoints na porta `3111`. A REST API faz bind em `127.0.0.1` por predefinição. Os endpoints protegidos exigem `Authorization: Bearer <secret>`, e os endpoints de sincronização mesh exigem uma `AGENTMEMORY_SECRET` explicitamente definida em ambos os pares.**A autenticação está ativa por predefinição.** Quando `AGENTMEMORY_SECRET` não está definida (na shell ou em `~/.agentmemory/.env`), o servidor gera um segredo aleatório no primeiro arranque e guarda-o em `~/.agentmemory/secret` com o modo `0600`. Todo o cliente incluído lê-o a partir daí quando fala com um servidor local: o CLI, o visualizador, os hooks em `plugin/scripts`, o servidor MCP e o shim `@agentmemory/mcp`, as configurações escritas por `agentmemory connect`, e as integrações incluídas do OpenCode, Pi, OpenClaw, Hermes e do filesystem-watcher. O segredo guardado só é enviado para URLs de loopback (`localhost`, `127.0.0.0/8`, `::1`). Uma `AGENTMEMORY_SECRET` explícita vence sempre, e os clientes remotos continuam a precisar de a ter definida. O Docker e os entrypoints de `deploy/` já geram e exportam o seu próprio segredo. Para chamar a API manualmente:@@ -1870,7 +1870,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.ro-RO.md b/READMEs/README.ro-RO.mdindex f79e9cdf..c6acda9c 100644--- a/READMEs/README.ro-RO.md+++ b/READMEs/README.ro-RO.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 de instrumente MCP" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 hook-uri automate" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 baze de date externe" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ teste trecute" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ teste trecute" height="38" /></picture></p><p align="center">@@ -1430,7 +1430,7 @@ fi| Prometheus / Grafana | iii OTEL + monitor de sănătate || Sisteme de plugin-uri personalizate | `iii worker add <name>` |-**230 fișiere sursă · ~55,000 LOC · 3,300+ teste · 312 funcții · 62 de domenii KV**, toate pe trei primitive. Niciun `agentmemory plugin install`. Sistemul de plugin-uri este chiar iii.+**238 fișiere sursă · ~56,000 LOC · 3,500+ teste · 318 funcții · 66 de domenii KV**, toate pe trei primitive. Niciun `agentmemory plugin install`. Sistemul de plugin-uri este chiar iii.---@@ -1773,7 +1773,7 @@ Creează `~/.agentmemory/.env`:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 de endpoint-uri pe portul `3111`. API-ul REST se leagă implicit la `127.0.0.1`. Endpoint-urile protejate necesită `Authorization: Bearer <secret>`, iar endpoint-urile de sincronizare mesh necesită un `AGENTMEMORY_SECRET` setat explicit pe ambii parteneri (peers).+141 de endpoint-uri pe portul `3111`. API-ul REST se leagă implicit la `127.0.0.1`. Endpoint-urile protejate necesită `Authorization: Bearer <secret>`, iar endpoint-urile de sincronizare mesh necesită un `AGENTMEMORY_SECRET` setat explicit pe ambii parteneri (peers).**Autentificarea este activată implicit.** Când `AGENTMEMORY_SECRET` nu este setat (în shell sau în `~/.agentmemory/.env`), serverul generează un secret aleatoriu la prima pornire și îl stochează în `~/.agentmemory/secret`, cu modul `0600`. Fiecare client inclus îl citește de acolo, când vorbește cu un server local: CLI-ul, vizualizatorul, hook-urile din `plugin/scripts`, serverul MCP și shim-ul `@agentmemory/mcp`, configurațiile scrise de `agentmemory connect`, și integrările incluse OpenCode, Pi, OpenClaw, Hermes și filesystem-watcher. Secretul stocat este trimis doar către URL-uri loopback (`localhost`, `127.0.0.0/8`, `::1`). Un `AGENTMEMORY_SECRET` explicit are întotdeauna prioritate, iar clienții la distanță au nevoie, în continuare, să îl aibă setat. Docker și entrypoint-urile din `deploy/` își generează și exportă deja propriul secret. Pentru a apela API-ul manual:@@ -1854,7 +1854,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.ru-RU.md b/READMEs/README.ru-RU.mdindex a3e44316..9c964170 100644--- a/READMEs/README.ru-RU.md+++ b/READMEs/README.ru-RU.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP tools" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 auto hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 external DBs" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ tests passing" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ tests passing" height="38" /></picture></p><p align="center">@@ -1446,7 +1446,7 @@ fi| Prometheus / Grafana | iii OTEL + монитор состояния || Самописные плагинные системы | `iii worker add <name>` |-**230 исходных файлов · ~55,000 LOC · 3,300+ тестов · 312 функций · 62 KV-scope**, всё на трёх примитивах. Никакого `agentmemory plugin install`. Плагинная система — это сам iii.+**238 исходных файлов · ~56,000 LOC · 3,500+ тестов · 318 функций · 66 KV-scope**, всё на трёх примитивах. Никакого `agentmemory plugin install`. Плагинная система — это сам iii.---@@ -1789,7 +1789,7 @@ GRAPH_EXTRACTION_ENABLED=true<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 эндпоинтов на порту `3111`. REST API по умолчанию слушает на `127.0.0.1`. Защищённые эндпоинты требуют `Authorization: Bearer <secret>`, а эндпоинты mesh-синхронизации требуют явно заданного `AGENTMEMORY_SECRET` на обоих узлах.+141 эндпоинтов на порту `3111`. REST API по умолчанию слушает на `127.0.0.1`. Защищённые эндпоинты требуют `Authorization: Bearer <secret>`, а эндпоинты mesh-синхронизации требуют явно заданного `AGENTMEMORY_SECRET` на обоих узлах.**Аутентификация включена по умолчанию.** Когда `AGENTMEMORY_SECRET` не задан (ни в shell, ни в `~/.agentmemory/.env`), сервер генерирует случайный секрет при первом запуске и хранит его в `~/.agentmemory/secret` с правами `0600`. Каждый встроенный клиент читает его оттуда, когда говорит с локальным сервером: CLI, просмотрщик, хуки из `plugin/scripts`, MCP-сервер и shim `@agentmemory/mcp`, конфиги, записанные `agentmemory connect`, и встроенные интеграции OpenCode, Pi, OpenClaw, Hermes и filesystem-watcher. Сохранённый секрет отправляется только на loopback-адреса (`localhost`, `127.0.0.0/8`, `::1`). Явно заданный `AGENTMEMORY_SECRET` всегда выигрывает, и удалённым клиентам он всё равно нужен. Docker и entrypoint'ы `deploy/` уже генерируют и экспортируют собственный секрет. Чтобы вызвать API вручную:@@ -1870,7 +1870,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.sv-SE.md b/READMEs/README.sv-SE.mdindex e3154061..90e646f4 100644--- a/READMEs/README.sv-SE.md+++ b/READMEs/README.sv-SE.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP-verktyg" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 automatiska hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 externa databaser" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ godkända tester" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ godkända tester" height="38" /></picture></p><p align="center">@@ -1437,7 +1437,7 @@ fi| Prometheus / Grafana | iii OTEL + hälsoövervakare || Anpassade pluginsystem | `iii worker add <name>` |-**230 källkodsfiler · ~55,000 LOC · 3,300+ tester · 312 funktioner · 62 KV-scopes**, allt på tre primitiv. Inget `agentmemory plugin install`. Pluginsystemet är iii självt.+**238 källkodsfiler · ~56,000 LOC · 3,500+ tester · 318 funktioner · 66 KV-scopes**, allt på tre primitiv. Inget `agentmemory plugin install`. Pluginsystemet är iii självt.---@@ -1780,7 +1780,7 @@ Skapa `~/.agentmemory/.env`:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpoints på port `3111`. REST API:et binder till `127.0.0.1` som standard. Skyddade endpoints kräver `Authorization: Bearer <secret>`, och mesh-synk-endpoints kräver en explicit satt `AGENTMEMORY_SECRET` på båda parterna.+141 endpoints på port `3111`. REST API:et binder till `127.0.0.1` som standard. Skyddade endpoints kräver `Authorization: Bearer <secret>`, och mesh-synk-endpoints kräver en explicit satt `AGENTMEMORY_SECRET` på båda parterna.**Autentisering är aktiverad som standard.** När `AGENTMEMORY_SECRET` inte är satt (i skalet eller i `~/.agentmemory/.env`), genererar servern en slumpmässig hemlighet vid första start och lagrar den i `~/.agentmemory/secret` med rättigheterna `0600`. Varje medföljande klient läser den därifrån när den talar med en lokal server: CLI:n, viewern, hookarna under `plugin/scripts`, MCP-servern och `@agentmemory/mcp`-shimmen, konfigurationerna skrivna av `agentmemory connect`, och de medföljande integrationerna för OpenCode, Pi, OpenClaw, Hermes och filsystemsbevakaren. Den lagrade hemligheten skickas bara till loopback-URL:er (`localhost`, `127.0.0.0/8`, `::1`). En explicit `AGENTMEMORY_SECRET` vinner alltid, och fjärrklienter måste fortfarande ha den satt. Docker och startpunkterna i `deploy/` genererar och exporterar redan sin egen hemlighet. För att anropa API:et för hand:@@ -1861,7 +1861,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.th-TH.md b/READMEs/README.th-TH.mdindex f2acec87..02acd012 100644--- a/READMEs/README.th-TH.md+++ b/READMEs/README.th-TH.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 เครื่องมือ MCP" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 hook อัตโนมัติ" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="ไม่ต้องพึ่ง external DB เลย" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="ผ่านการทดสอบ 3,300+ รายการ" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="ผ่านการทดสอบ 3,500+ รายการ" height="38" /></picture></p><p align="center">@@ -1444,7 +1444,7 @@ fi| Prometheus / Grafana | iii OTEL + health monitor || Custom plugin systems | `iii worker add <name>` |-**230 source file · ~55,000 LOC · 3,300+ การทดสอบ · 312 function · 62 KV scope** ทั้งหมดอยู่บน primitive สามตัว ไม่มี `agentmemory plugin install` ระบบ plugin ก็คือ iii เองนั่นแหละ+**238 source file · ~56,000 LOC · 3,500+ การทดสอบ · 318 function · 66 KV scope** ทั้งหมดอยู่บน primitive สามตัว ไม่มี `agentmemory plugin install` ระบบ plugin ก็คือ iii เองนั่นแหละ---@@ -1787,7 +1787,7 @@ GRAPH_EXTRACTION_ENABLED=true<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpoint บนพอร์ต `3111` REST API จะ bind กับ `127.0.0.1` โดย default endpoint ที่ป้องกันไว้ต้องมี `Authorization: Bearer <secret>` และ endpoint ของ mesh sync ต้องตั้งค่า `AGENTMEMORY_SECRET` ไว้อย่างชัดเจนทั้งสองฝั่ง+141 endpoint บนพอร์ต `3111` REST API จะ bind กับ `127.0.0.1` โดย default endpoint ที่ป้องกันไว้ต้องมี `Authorization: Bearer <secret>` และ endpoint ของ mesh sync ต้องตั้งค่า `AGENTMEMORY_SECRET` ไว้อย่างชัดเจนทั้งสองฝั่ง**Authentication เปิดอยู่โดย default** เมื่อไม่ได้ตั้งค่า `AGENTMEMORY_SECRET` (ทั้งใน shell หรือใน `~/.agentmemory/.env`) server จะสร้าง secret แบบสุ่มตอนเริ่มครั้งแรก และเก็บไว้ที่ `~/.agentmemory/secret` ด้วย mode `0600` client ที่ bundle มาทุกตัวจะอ่านมันจากที่นั่นเมื่อคุยกับ server ในเครื่อง: CLI, viewer, hook ใต้ `plugin/scripts`, MCP server และ shim `@agentmemory/mcp`, config ที่เขียนโดย `agentmemory connect`, และ integration ของ OpenCode, Pi, OpenClaw, Hermes และ filesystem-watcher ที่ bundle มา secret ที่เก็บไว้จะถูกส่งไปที่ loopback URL เท่านั้น (`localhost`, `127.0.0.0/8`, `::1`) `AGENTMEMORY_SECRET` แบบ explicit จะมีสิทธิ์เหนือเสมอ และ client แบบ remote ยังต้องตั้งค่ามันไว้ Docker และ entrypoint ของ [deploy](../deploy/) สร้างและ export secret ของตัวเองไว้แล้ว เรียก API ด้วยมือ:@@ -1868,7 +1868,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.tl-PH.md b/READMEs/README.tl-PH.mdindex 390c4c64..04984bc4 100644--- a/READMEs/README.tl-PH.md+++ b/READMEs/README.tl-PH.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP tool" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 awtomatikong hook" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 panlabas na DB" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ pumapasang test" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ pumapasang test" height="38" /></picture></p><p align="center">@@ -1446,7 +1446,7 @@ Tinatanggap din ng `/agentmemory/export` ang `?maxSessions=` at `?offset=` para| Prometheus / Grafana | iii OTEL + health monitor || Custom na plugin system | `iii worker add <name>` |-**230 na source file · ~55,000 LOC · 3,300+ na test · 312 na function · 62 na KV scope**, lahat sa tatlong primitive. Walang `agentmemory plugin install`. Ang plugin system mismo ay ang iii.+**238 na source file · ~56,000 LOC · 3,500+ na test · 318 na function · 66 na KV scope**, lahat sa tatlong primitive. Walang `agentmemory plugin install`. Ang plugin system mismo ay ang iii.---@@ -1789,7 +1789,7 @@ Gawin ang `~/.agentmemory/.env`:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 na endpoint sa port `3111`. Nagbibind ang REST API sa `127.0.0.1` bilang default. Kinakailangan ng mga protected endpoint ang `Authorization: Bearer <secret>`, at kinakailangan ng mga mesh sync endpoint ang tahasang naka-set na `AGENTMEMORY_SECRET` sa dalawang peer.+141 na endpoint sa port `3111`. Nagbibind ang REST API sa `127.0.0.1` bilang default. Kinakailangan ng mga protected endpoint ang `Authorization: Bearer <secret>`, at kinakailangan ng mga mesh sync endpoint ang tahasang naka-set na `AGENTMEMORY_SECRET` sa dalawang peer.**Naka-on ang Authentication bilang default.** Kapag hindi naka-set ang `AGENTMEMORY_SECRET` (sa shell o sa `~/.agentmemory/.env`), gumagawa ang server ng random na secret sa unang start at itinatago ito sa `~/.agentmemory/secret` na may mode `0600`. Babasahin ito mula roon ng bawat bundled na client kapag nakikipag-usap ito sa isang local server: ang CLI, ang viewer, ang hooks sa ilalim ng `plugin/scripts`, ang MCP server at ang `@agentmemory/mcp` shim, ang mga config na isinulat ng `agentmemory connect`, at ang bundled na OpenCode, Pi, OpenClaw, Hermes, at filesystem-watcher integrations. Ipinapadala lamang ang itinagong secret sa mga loopback URL (`localhost`, `127.0.0.0/8`, `::1`). Palaging mananaig ang isang tahasang `AGENTMEMORY_SECRET`, at kailangan pa rin itong naka-set para sa mga remote client. Gumagawa at nag-export na ng sariling secret ang Docker at ang mga entrypoint ng `deploy/`. Para tawagin ang API nang manu-mano:@@ -1870,7 +1870,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.tr-TR.md b/READMEs/README.tr-TR.mdindex cc43d8d9..3ab21441 100644--- a/READMEs/README.tr-TR.md+++ b/READMEs/README.tr-TR.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP tools" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 auto hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 external DBs" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ tests passing" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ tests passing" height="38" /></picture></p><p align="center">@@ -1437,7 +1437,7 @@ fi| Prometheus / Grafana | iii OTEL + health monitor || Custom eklenti sistemleri | `iii worker add <name>` |-**230 kaynak dosya · ~55,000 LOC · 3,300+ test · 312 fonksiyon · 62 KV scope**, hepsi üç primitif üzerinde. `agentmemory plugin install` yok. Eklenti sistemi, iii'nin kendisidir.+**238 kaynak dosya · ~56,000 LOC · 3,500+ test · 318 fonksiyon · 66 KV scope**, hepsi üç primitif üzerinde. `agentmemory plugin install` yok. Eklenti sistemi, iii'nin kendisidir.---@@ -1780,7 +1780,7 @@ GRAPH_EXTRACTION_ENABLED=true<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-`3111` portunda 138 endpoint. REST API, varsayılan olarak `127.0.0.1`'e bağlanır. Korumalı endpoint'ler `Authorization: Bearer <secret>` gerektirir ve mesh sync endpoint'leri her iki peer'da da açıkça ayarlanmış bir `AGENTMEMORY_SECRET` gerektirir.+`3111` portunda 141 endpoint. REST API, varsayılan olarak `127.0.0.1`'e bağlanır. Korumalı endpoint'ler `Authorization: Bearer <secret>` gerektirir ve mesh sync endpoint'leri her iki peer'da da açıkça ayarlanmış bir `AGENTMEMORY_SECRET` gerektirir.**Authentication varsayılan olarak açıktır.** `AGENTMEMORY_SECRET` ayarlanmadığında (shell'de veya `~/.agentmemory/.env` içinde), sunucu ilk başlatmada rastgele bir secret üretir ve onu `0600` mod ile `~/.agentmemory/secret`'a saklar. Yerel bir sunucuyla konuştuğunda paketlenmiş her istemci bunu oradan okur: CLI, görüntüleyici, `plugin/scripts` altındaki hook'lar, MCP sunucusu ve `@agentmemory/mcp` shim'i, `agentmemory connect` tarafından yazılan config'ler ve paketlenmiş OpenCode, Pi, OpenClaw, Hermes ve filesystem-watcher entegrasyonları. Saklanan secret, yalnızca loopback URL'lerine (`localhost`, `127.0.0.0/8`, `::1`) gönderilir. Açık bir `AGENTMEMORY_SECRET` her zaman kazanır ve uzak istemcilerin de bunu ayarlamış olması gerekir. Docker ve `deploy/` entrypoint'leri zaten kendi secret'larını üretir ve export eder. API'yi elle çağırmak için:@@ -1861,7 +1861,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.uk-UA.md b/READMEs/README.uk-UA.mdindex d8013319..c6dbd1b1 100644--- a/READMEs/README.uk-UA.md+++ b/READMEs/README.uk-UA.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 інструменти MCP" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 автоматичних хуків" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 зовнішніх БД" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ тестів пройдено" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ тестів пройдено" height="38" /></picture></p><p align="center">@@ -1447,7 +1447,7 @@ fi| Prometheus / Grafana | iii OTEL + монітор стану здоров'я || Власні системи плагінів | `iii worker add <name>` |-**230 вихідних файлів · ~55,000 LOC · 3,300+ тестів · 312 функцій · 62 KV-скоупів**, усе на трьох примітивах. Жодного `agentmemory plugin install`. Система плагінів — це сам iii.+**238 вихідних файлів · ~56,000 LOC · 3,500+ тестів · 318 функцій · 66 KV-скоупів**, усе на трьох примітивах. Жодного `agentmemory plugin install`. Система плагінів — це сам iii.---@@ -1790,7 +1790,7 @@ GRAPH_EXTRACTION_ENABLED=true<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 ендпоінтів на порту `3111`. REST API за замовчуванням прив'язується до `127.0.0.1`. Захищені ендпоінти вимагають `Authorization: Bearer <secret>`, а ендпоінти синхронізації mesh вимагають явно встановленого `AGENTMEMORY_SECRET` на обох сторонах.+141 ендпоінтів на порту `3111`. REST API за замовчуванням прив'язується до `127.0.0.1`. Захищені ендпоінти вимагають `Authorization: Bearer <secret>`, а ендпоінти синхронізації mesh вимагають явно встановленого `AGENTMEMORY_SECRET` на обох сторонах.**Автентифікація увімкнена за замовчуванням.** Коли `AGENTMEMORY_SECRET` не встановлено (ні в оболонці, ні в `~/.agentmemory/.env`), сервер генерує випадковий секрет під час першого запуску та зберігає його в `~/.agentmemory/secret` з режимом `0600`. Кожен вбудований клієнт зчитує його звідти, коли спілкується з локальним сервером: CLI, переглядач, хуки в `plugin/scripts`, MCP-сервер та шим `@agentmemory/mcp`, конфігурації, записані `agentmemory connect`, а також вбудовані інтеграції OpenCode, Pi, OpenClaw, Hermes та спостерігача за файловою системою. Збережений секрет надсилається лише на loopback-URL (`localhost`, `127.0.0.0/8`, `::1`). Явний `AGENTMEMORY_SECRET` завжди переважає, і віддаленим клієнтам все одно потрібно його встановити. Docker та точки входу `deploy/` уже генерують і експортують власний секрет. Щоб викликати API вручну:@@ -1871,7 +1871,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.ur-PK.md b/READMEs/README.ur-PK.mdindex 6ef0c63c..72fac16e 100644--- a/READMEs/README.ur-PK.md+++ b/READMEs/README.ur-PK.md@@ -73,7 +73,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP ٹولز" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 خودکار ہکس" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 خارجی ڈیٹابیسز" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ ٹیسٹس پاس ہو رہے ہیں" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ ٹیسٹس پاس ہو رہے ہیں" height="38" /></picture></p><p align="center">@@ -1432,7 +1432,7 @@ fi| Prometheus / Grafana | iii OTEL + health monitor || کسٹم plugin سسٹمز | `iii worker add <name>` |-**230 سورس فائلز · ~55,000 لائنز کوڈ · 3,300+ ٹیسٹس · 312 فنکشنز · 62 KV اسکوپس**، سب تین primitives پر۔ کوئی `agentmemory plugin install` نہیں۔ plugin سسٹم خود iii ہی ہے۔+**238 سورس فائلز · ~56,000 لائنز کوڈ · 3,500+ ٹیسٹس · 318 فنکشنز · 66 KV اسکوپس**، سب تین primitives پر۔ کوئی `agentmemory plugin install` نہیں۔ plugin سسٹم خود iii ہی ہے۔---@@ -1773,7 +1773,7 @@ GRAPH_EXTRACTION_ENABLED=true<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-پورٹ `3111` پر 138 اینڈ پوائنٹس۔ REST API ڈیفالٹ کے طور پر `127.0.0.1` سے bind ہوتا ہے۔ محفوظ اینڈ پوائنٹس کو `Authorization: Bearer <secret>` درکار ہے، اور mesh sync اینڈ پوائنٹس کو دونوں peers پر واضح طور پر سیٹ کیا گیا `AGENTMEMORY_SECRET` درکار ہے۔+پورٹ `3111` پر 141 اینڈ پوائنٹس۔ REST API ڈیفالٹ کے طور پر `127.0.0.1` سے bind ہوتا ہے۔ محفوظ اینڈ پوائنٹس کو `Authorization: Bearer <secret>` درکار ہے، اور mesh sync اینڈ پوائنٹس کو دونوں peers پر واضح طور پر سیٹ کیا گیا `AGENTMEMORY_SECRET` درکار ہے۔**Authentication ڈیفالٹ کے طور پر آن ہے۔** جب `AGENTMEMORY_SECRET` سیٹ نہ ہو (shell میں یا `~/.agentmemory/.env` میں)، سرور پہلے اسٹارٹ پر ایک random secret تیار کرتا ہے اور اسے `0600` موڈ کے ساتھ `~/.agentmemory/secret` میں محفوظ کرتا ہے۔ ہر bundled کلائنٹ جب کسی مقامی سرور سے بات کرتا ہے تو اسے وہاں سے پڑھتا ہے: CLI، ویور، `plugin/scripts` کے تحت ہکس، MCP سرور اور `@agentmemory/mcp` shim، `agentmemory connect` کی لکھی گئی configs، اور bundled OpenCode، Pi، OpenClaw، Hermes اور filesystem-watcher integrations۔ محفوظ شدہ secret صرف loopback URLs (`localhost`، `127.0.0.0/8`، `::1`) کو بھیجا جاتا ہے۔ ایک واضح `AGENTMEMORY_SECRET` ہمیشہ جیتتا ہے، اور ریموٹ کلائنٹس کو بھی اسے سیٹ کرنا ضروری ہے۔ Docker اور `deploy/` entrypoints پہلے سے اپنا secret تیار اور export کرتے ہیں۔ API کو دستی طور پر کال کرنے کے لیے:@@ -1854,7 +1854,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.vi-VN.md b/READMEs/README.vi-VN.mdindex c79cd17f..2c52a0cc 100644--- a/READMEs/README.vi-VN.md+++ b/READMEs/README.vi-VN.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 MCP tools" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 auto hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 external DBs" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ tests passing" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ tests passing" height="38" /></picture></p><p align="center">@@ -1437,7 +1437,7 @@ fi| Prometheus / Grafana | iii OTEL + health monitor || Hệ thống plugin tùy chỉnh | `iii worker add <name>` |-**230 file source · ~55,000 LOC · 3,300+ test · 312 function · 62 KV scope**, tất cả trên ba primitive. Không có `agentmemory plugin install`. Hệ thống plugin chính là iii.+**238 file source · ~56,000 LOC · 3,500+ test · 318 function · 66 KV scope**, tất cả trên ba primitive. Không có `agentmemory plugin install`. Hệ thống plugin chính là iii.---@@ -1780,7 +1780,7 @@ Tạo `~/.agentmemory/.env`:<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-138 endpoint trên port `3111`. REST API bind vào `127.0.0.1` theo mặc định. Các endpoint được bảo vệ cần `Authorization: Bearer <secret>`, và các endpoint mesh sync cần `AGENTMEMORY_SECRET` được đặt rõ ràng trên cả hai phía.+141 endpoint trên port `3111`. REST API bind vào `127.0.0.1` theo mặc định. Các endpoint được bảo vệ cần `Authorization: Bearer <secret>`, và các endpoint mesh sync cần `AGENTMEMORY_SECRET` được đặt rõ ràng trên cả hai phía.**Authentication được bật theo mặc định.** Khi `AGENTMEMORY_SECRET` chưa được đặt (trong shell hay trong `~/.agentmemory/.env`), server sinh một secret ngẫu nhiên ở lần khởi động đầu tiên và lưu nó vào `~/.agentmemory/secret` với mode `0600`. Mọi client đi kèm đọc nó từ đó khi nói chuyện với một server cục bộ: CLI, viewer, các hook dưới `plugin/scripts`, MCP server và shim `@agentmemory/mcp`, các config do `agentmemory connect` viết ra, và các integration OpenCode, Pi, OpenClaw, Hermes và filesystem-watcher đi kèm. Secret đã lưu chỉ được gửi tới các URL loopback (`localhost`, `127.0.0.0/8`, `::1`). Một `AGENTMEMORY_SECRET` rõ ràng luôn thắng, và các client remote vẫn cần đặt nó. Docker và các entrypoint của `deploy/` đã tự sinh và export secret riêng của chúng. Để gọi API bằng tay:@@ -1861,7 +1861,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.zh-CN.md b/READMEs/README.zh-CN.mdindex 8aedec90..3197fa84 100644--- a/READMEs/README.zh-CN.md+++ b/READMEs/README.zh-CN.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 个 MCP 工具" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 个自动 hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 个外部数据库" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ 项测试通过" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ 项测试通过" height="38" /></picture></p><p align="center">@@ -1444,7 +1444,7 @@ fi| Prometheus / Grafana | iii OTEL + 健康监控 || Custom plugin systems | `iii worker add <name>` |-**230 个源文件 · ~55,000 行代码 · 3,300+ 个测试 · 312 个函数 · 62 个 KV 作用域**,全部基于三种原语。没有 `agentmemory plugin install`。插件系统就是 iii 本身。+**238 个源文件 · ~56,000 行代码 · 3,500+ 个测试 · 318 个函数 · 66 个 KV 作用域**,全部基于三种原语。没有 `agentmemory plugin install`。插件系统就是 iii 本身。---@@ -1787,7 +1787,7 @@ GRAPH_EXTRACTION_ENABLED=true<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-端口 `3111` 上有 138 个端点。REST API 默认绑定到 `127.0.0.1`。受保护的端点需要 `Authorization: Bearer <secret>`,mesh 同步端点则需要双方都显式设置 `AGENTMEMORY_SECRET`。+端口 `3111` 上有 141 个端点。REST API 默认绑定到 `127.0.0.1`。受保护的端点需要 `Authorization: Bearer <secret>`,mesh 同步端点则需要双方都显式设置 `AGENTMEMORY_SECRET`。**身份验证默认是开启的。** 当 `AGENTMEMORY_SECRET`(在 shell 或 `~/.agentmemory/.env` 中)没有设置时,服务器会在首次启动时生成一个随机密钥,并以 `0600` 权限存储在 `~/.agentmemory/secret` 中。每个捆绑的客户端在与本地服务器通信时都会从那里读取它:CLI、查看器、`plugin/scripts` 下的 hooks、MCP 服务器和 `@agentmemory/mcp` shim、由 `agentmemory connect` 写入的配置,以及捆绑的 OpenCode、Pi、OpenClaw、Hermes 和文件系统监视集成。存储的密钥只会发送到回环地址(`localhost`、`127.0.0.0/8`、`::1`)。显式设置的 `AGENTMEMORY_SECRET` 始终优先,远程客户端仍然需要设置它。Docker 和 `deploy/` 的 entrypoint 已经会生成并导出自己的密钥。要手动调用 API:@@ -1868,7 +1868,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/READMEs/README.zh-TW.md b/READMEs/README.zh-TW.mdindex 6dd5ecc7..27e946cb 100644--- a/READMEs/README.zh-TW.md+++ b/READMEs/README.zh-TW.md@@ -71,7 +71,7 @@<picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tools.svg"><img src="../assets/tags/stat-tools.svg" alt="54 個 MCP 工具" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-hooks.svg"><img src="../assets/tags/stat-hooks.svg" alt="12 個自動 hooks" height="38" /></picture><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-deps.svg"><img src="../assets/tags/stat-deps.svg" alt="0 個外部資料庫" height="38" /></picture>- <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,300+ 測試通過" height="38" /></picture>+ <picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/stat-tests.svg"><img src="../assets/tags/stat-tests.svg" alt="3,500+ 測試通過" height="38" /></picture></p><p align="center">@@ -1430,7 +1430,7 @@ fi| Prometheus / Grafana | iii OTEL + 健康監控 || 自訂外掛系統 | `iii worker add <name>` |-**230 個原始檔 · ~55,000 行程式碼 · 3,300+ 測試 · 312 個函式 · 62 個 KV 範圍**,全部基於三種原語。沒有 `agentmemory plugin install`。外掛系統就是 iii 本身。+**238 個原始檔 · ~56,000 行程式碼 · 3,500+ 測試 · 318 個函式 · 66 個 KV 範圍**,全部基於三種原語。沒有 `agentmemory plugin install`。外掛系統就是 iii 本身。---@@ -1773,7 +1773,7 @@ GRAPH_EXTRACTION_ENABLED=true<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="../assets/tags/light/section-api.svg"><img src="../assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>-連接埠 `3111` 上有 138 個端點。REST API 預設綁定 `127.0.0.1`。受保護的端點需要 `Authorization: Bearer <secret>`,而網狀同步端點要求兩端都明確設定 `AGENTMEMORY_SECRET`。+連接埠 `3111` 上有 141 個端點。REST API 預設綁定 `127.0.0.1`。受保護的端點需要 `Authorization: Bearer <secret>`,而網狀同步端點要求兩端都明確設定 `AGENTMEMORY_SECRET`。**驗證預設是開啟的。** 當 `AGENTMEMORY_SECRET` 未設定時(無論是在殼層中或 `~/.agentmemory/.env` 中),伺服器會在第一次啟動時產生一個隨機密鑰,並以 `0600` 權限存進 `~/.agentmemory/secret`。每個捆綁的客戶端在與本機伺服器溝通時都會從那裡讀取它:CLI、檢視器、`plugin/scripts` 下的 hooks、MCP 伺服器和 `@agentmemory/mcp` shim、由 `agentmemory connect` 寫入的設定,以及捆綁的 OpenCode、Pi、OpenClaw、Hermes 和檔案系統監看整合。儲存的密鑰只會被送到 loopback 的 URL(`localhost`、`127.0.0.0/8`、`::1`)。明確設定的 `AGENTMEMORY_SECRET` 永遠優先,而遠端客戶端仍需要設定它。Docker 和 `deploy/` 的 entrypoint 已經會自行產生並匯出自己的密鑰。要手動呼叫 API:@@ -1854,7 +1854,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: a```bashnpm run dev # Hot reloadnpm run build # Production build-npm test # 3,300+ tests+npm test # 3,500+ testsnpm run test:integration # API tests (requires running services)```diff --git a/assets/tags/light/stat-tests.svg b/assets/tags/light/stat-tests.svgindex e76f85a0..e19f7612 100644--- a/assets/tags/light/stat-tests.svg+++ b/assets/tags/light/stat-tests.svg@@ -1,5 +1,5 @@-<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 140 48" fill="none" role="img" aria-label="TESTS PASSING: 3300+">+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 140 48" fill="none" role="img" aria-label="TESTS PASSING: 3500+"><rect x="0" y="0" width="140" height="48" rx="10" fill="#FFFFFF" stroke="#E5E7EB" stroke-width="1"/>- <text x="70" y="24" fill="#16A34A" font-family='ui-sans-serif, -apple-system, "Segoe UI", Inter, system-ui, sans-serif' font-size="18" font-weight="900" text-anchor="middle" letter-spacing="-0.3">3300+</text>+ <text x="70" y="24" fill="#16A34A" font-family='ui-sans-serif, -apple-system, "Segoe UI", Inter, system-ui, sans-serif' font-size="18" font-weight="900" text-anchor="middle" letter-spacing="-0.3">3500+</text><text x="70" y="38" fill="#64748B" font-family='ui-sans-serif, -apple-system, "Segoe UI", Inter, system-ui, sans-serif' font-size="9" font-weight="600" text-anchor="middle" letter-spacing="1.5">TESTS PASSING</text></svg>diff --git a/assets/tags/stat-tests.svg b/assets/tags/stat-tests.svgindex ef2d19a0..610a4dca 100644--- a/assets/tags/stat-tests.svg+++ b/assets/tags/stat-tests.svg@@ -1,5 +1,5 @@-<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 140 48" fill="none" role="img" aria-label="TESTS PASSING: 3300+">+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 140 48" fill="none" role="img" aria-label="TESTS PASSING: 3500+"><rect x="0" y="0" width="140" height="48" rx="10" fill="#1A1A1A" stroke="#2A2A2A" stroke-width="1"/>- <text x="70" y="24" fill="#00D26A" font-family='ui-sans-serif, -apple-system, "Segoe UI", Inter, system-ui, sans-serif' font-size="18" font-weight="900" text-anchor="middle" letter-spacing="-0.3">3300+</text>+ <text x="70" y="24" fill="#00D26A" font-family='ui-sans-serif, -apple-system, "Segoe UI", Inter, system-ui, sans-serif' font-size="18" font-weight="900" text-anchor="middle" letter-spacing="-0.3">3500+</text><text x="70" y="38" fill="#9CA3AF" font-family='ui-sans-serif, -apple-system, "Segoe UI", Inter, system-ui, sans-serif' font-size="9" font-weight="600" text-anchor="middle" letter-spacing="1.5">TESTS PASSING</text></svg>diff --git a/docs/reliability/graph-index-migration.md b/docs/reliability/graph-index-migration.mdnew file mode 100644index 00000000..389e1126--- /dev/null+++ b/docs/reliability/graph-index-migration.md@@ -0,0 +1,51 @@+# Bounded graph indexes and legacy migration++Graph retrieval uses generation-scoped indexes maintained with canonical graph writes. Startup, graph search and graph-index status do not list legacy node, edge or history scopes. Any uninitialized store without an index, including a fresh installation, reports `needs-migration`; keyword search, vector search and capture remain available. Missing metadata cannot prove a store is empty without a scan, so the initialization command below is explicit for both fresh and existing stores.++Inspect the selected server without starting it or modifying graph data:++```sh+agentmemory graph-index status --json+```++To build indexes for existing graph data, schedule a maintenance window and explicitly authorize discovery:++```sh+agentmemory graph-index migrate --allow-legacy-scan --json+```++**Initial discovery is not bounded.** The pinned engine cannot paginate its state listings. Migration reads each complete legacy node, edge and history scope once, including full record values. Large stores can consume substantial memory and delay other requests during those reads. An HTTP timeout does not cancel an engine allocation already in progress. No ordinary search, status request or startup silently authorizes this scan.++The command returns a job ID after saving its intent. It runs in the background in an isolated index build within the current graph generation; canonical records are preserved. Capture, imports, mesh updates, temporal changes and snapshot restores maintain that build while migration runs. Status reports discovered and processed totals, readiness and partial coverage.++Discovery writes an ID manifest with at most 128 entries and 32 KiB per page, then seals it. The build rereads each current canonical record under the graph writer lock before updating its index, and saves progress after the index write. Replaying an interrupted cursor is idempotent. Writes made after discovery are also indexed, and records removed during migration are not resurrected.++After a restart, a sealed manifest resumes without listing a legacy scope or reading the old snapshot. If a pending legacy graph transaction needs snapshot recovery, the job pauses until you run an explicit resume. If that transaction has no recoverable journal, start a new `migrate --allow-legacy-scan` build. To resume immediately:++```sh+agentmemory graph-index resume --json+```++If discovery did not finish, startup does not resume it. After an unclean shutdown the saved state can still say `discovering`; the status action explains how to continue. Repeating discovery needs fresh, explicit consent:++```sh+agentmemory graph-index resume --allow-legacy-scan --json+```++A sealed build with a missing control cannot resume; it requires a new explicit migration so orphaned journals are never reused. A corrupt derived index control can be replaced by a new explicit `migrate --allow-legacy-scan` build. Storage failures do not authorize replacement. Missing or corrupt manifest pages stop the build before publication. Restore the saved manifest before resuming. A paused index caused by an unsupported canonical ID requires repairing the source record and starting `migrate --allow-legacy-scan` again. Graph reset or replacement supersedes an older migration; its late writes stay in the old generation. Reset deliberately clears the logical graph and is not a migration shortcut.++Progress and recovery journals use the configured state backend. An acknowledged state write is not a filesystem flush guarantee: the engine's persistence window still applies. Crash recovery tests wait for the sealed manifest to reach disk before terminating the worker and engine. They do not establish power-loss atomicity across separate state scopes.++Oversized source fields remain in canonical storage. Bounded projections record partial coverage when they cannot represent a field fully, and status exposes the affected record count. A ready index with partial coverage does not claim every graph fact is searchable. Search budgets and pagination can also truncate a request; callers should check its graph diagnostics.++Indexes add storage and write overhead. A replacement build isolates previous derived records by build ID; it does not reclaim their disk space. Repeated rebuilds and graph generations can therefore accumulate retained data. Bounded records and reads do not imply bounded total storage.++The CLI honors `AGENTMEMORY_URL`, local `--port` and `--instance` selection, and the existing credential resolution. Named instances sharing a home directory share its saved API credential by default. To use separate credentials, set a different `AGENTMEMORY_SECRET` for each instance and its clients, or use separate home directories. Equivalent authenticated endpoints are `GET /agentmemory/graph/index`, `POST /agentmemory/graph/index/migrate`, and `POST /agentmemory/graph/index/resume`. Mutation bodies accept only the boolean `allowLegacyScan` flag. Repeated start requests join the running job. Use one AgentMemory graph worker per state store; the writer lock coordinates operations within that worker.++The local packaged gate runs the installed CLI against the pinned native engine. It checks separate data directories for named instances sharing a home with explicit separate credentials, and generated credentials across separate homes:++```sh+npm run test:graph-index:packaged -- --tarball /path/to/agentmemory.tgz --engine /path/to/iii+```++It starts with 100 nodes and 300 edges. Use `--nodes 1000` for a larger run after checking the smaller run's timing and graph scope file sizes. `--dependencies /path/to/node_modules` explicitly reuses cached dependencies without modifying the archive or installed CLI. The report distinguishes that mode from a fresh dependency installation. It records native migration throughput, disk sizes, bounded read latency and wire requests, and tests sealed crash recovery, writer reconciliation, credential isolation and restart recall using synthetic memories with model calls disabled.diff --git a/docs/reliability/graph-search-budget.md b/docs/reliability/graph-search-budget.mdnew file mode 100644index 00000000..3939f8da--- /dev/null+++ b/docs/reliability/graph-search-budget.md@@ -0,0 +1,13 @@+# Graph enrichment budget++Hybrid search allows 500 ms for optional graph enrichment after keyword and vector retrieval. When that timer expires, it returns the available keyword/vector results without graph context and records a warning at most once per minute. A subsequent search can use graph enrichment again. Enrichment uses a request-scoped index reader with bounded point reads, decoded bytes, concurrency and traversal work.++Cancellation prevents additional graph work after the request expires. Name, character, observation and adjacency postings avoid listing canonical graph scopes. Every request checks the active generation and index revision before returning graph evidence. Pending writes, import, mesh updates, reset or restore invalidate an in-flight view. Optional model context contains only titles of current observations authorized for the requested scope; graph properties and intermediate labels do not become model evidence.++The graph view returns at most 500 nodes and uses a separate budget of 2,048 reads and 4 MiB. Default pages follow indexed inventory order, not a global degree ranking. Filtered counts describe examined records when the response is truncated. Temporal and graph-node lookups also report partial results when their budgets or projection limits are reached. Graph statistics use maintained index counts; an unavailable index reports unknown counts rather than an empty graph.++Existing stores require an [explicit legacy migration](graph-index-migration.md). Its initial discovery still reads complete scopes because the pinned engine cannot paginate them. Startup, indexed search, graph views, graph-node lookup and server status never start that scan, and automatic legacy boot compaction is deferred. Explicit maintenance, export, mesh push and snapshot creation remain bulk operations. The standalone MCP graph-stats resource still enumerates canonical graph scopes and requires a separate update; the bounded server graph-stats function does not. The index adds write and storage overhead, so a large migration needs a maintenance window and disk capacity.++The optional enrichment timer is not a strict end-to-end response limit. It cannot cancel an engine allocation already in progress, response decoding can delay timers, and independent requests have independent budgets. Index limits bound application-level reads; they do not change the engine's internal storage implementation.++`AGENTMEMORY_GRAPH_WEIGHT=0` skips graph enrichment entirely while preserving keyword and configured vector search. It reduces graph recall and is an operational workaround, not a replacement for the storage-level fix.diff --git a/package.json b/package.jsonindex b1de467b..e8418464 100644--- a/package.json+++ b/package.json@@ -26,6 +26,7 @@"test:integration": "vitest run test/integration.test.ts","test:all": "vitest run","test:startup:packaged": "node scripts/release-gate/startup.mjs",+ "test:graph-index:packaged": "node scripts/release-gate/graph-index.mjs","test:skills:packaged": "node scripts/release-gate/skills.mjs","test:openclaw:packaged": "node scripts/release-gate/openclaw.mjs","release:gate": "node scripts/release-gate/run.mjs",@@ -101,7 +102,8 @@"tsdown": "^0.21.10","tsx": "^4.19.0","typescript": "^6.0.3",- "vitest": "^4.1.6"+ "vitest": "^4.1.6",+ "ws": "^8.21.0"},"overrides": {"qs": "^6.15.2",diff --git a/plugin/skills/agentmemory-rest-api/REFERENCE.md b/plugin/skills/agentmemory-rest-api/REFERENCE.mdindex ae5b371f..77191714 100644--- a/plugin/skills/agentmemory-rest-api/REFERENCE.md+++ b/plugin/skills/agentmemory-rest-api/REFERENCE.md@@ -5,7 +5,7 @@ Generated from `src/triggers/api.ts`. Do not edit the block below by hand; run `<!-- AUTOGEN:rest START - generated by scripts/skills/generate.ts, do not edit by hand -->The REST API is the primary surface. All paths are under `http://localhost:3111` (override with `--port`). Send `Authorization: Bearer <secret>`: the value of `AGENTMEMORY_SECRET`, or the one generated into `~/.agentmemory/secret` on first start when it is unset. Writes must use `Content-Type: application/json`.-138 registered endpoints:+141 registered endpoints:| Method | Path || --- | --- |@@ -61,6 +61,9 @@ The REST API is the primary surface. All paths are under `http://localhost:3111`| POST | `/agentmemory/graph/compact` || POST | `/agentmemory/graph/extract` || POST | `/agentmemory/graph/import-graphify` |+| GET | `/agentmemory/graph/index` |+| POST | `/agentmemory/graph/index/migrate` |+| POST | `/agentmemory/graph/index/resume` || GET | `/agentmemory/graph/node` || POST | `/agentmemory/graph/query` || POST | `/agentmemory/graph/reset` |diff --git a/scripts/docs-sync.state.json b/scripts/docs-sync.state.jsonindex c2b5a59e..9d368517 100644--- a/scripts/docs-sync.state.json+++ b/scripts/docs-sync.state.json@@ -11,7 +11,7 @@"7"],"restEndpoints": [- "138"+ "141"],"skills": ["17"@@ -20,19 +20,19 @@"12"],"tests": [- "3300+"+ "3500+"],"functions": [- "312"+ "318"],"sourceFiles": [- "230"+ "238"],"loc": [- "55000"+ "56000"],"kvScopes": [- "62"+ "66"]}}diff --git a/scripts/release-gate/graph-index.mjs b/scripts/release-gate/graph-index.mjsnew file mode 100644index 00000000..55b4394e--- /dev/null+++ b/scripts/release-gate/graph-index.mjs@@ -0,0 +1,508 @@+#!/usr/bin/env node+import assert from "node:assert/strict";+import { spawn, execFileSync } from "node:child_process";+import { createHash, randomBytes } from "node:crypto";+import { appendFileSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, statSync, symlinkSync, writeFileSync } from "node:fs";+import net from "node:net";+import { arch, homedir, platform, tmpdir } from "node:os";+import { delimiter, dirname, join, resolve } from "node:path";+import { registerWorker } from "iii-sdk";+import WebSocket, { WebSocketServer } from "ws";+import { assertCommandCompleted, assertSupportedPlatform, runCommand } from "./command.mjs";+import { decodeRkyvJsonScope } from "./recovery.mjs";++assertSupportedPlatform();+const options = {};+for (let i = 2; i < process.argv.length; i++) {+ const key = process.argv[i];+ assert(["--tarball", "--engine", "--out", "--dependencies", "--nodes"].includes(key), `Unknown argument ${key}`);+ const value = process.argv[++i];+ assert(value && !value.startsWith("--"), `Missing value for ${key}`);+ options[key.slice(2)] = key === "--nodes" ? Number(value) : resolve(value);+}+assert(options.tarball, "Pass --tarball with the built npm package to test");+const nodeCount = options.nodes ?? 100;+assert(Number.isSafeInteger(nodeCount) && nodeCount >= 3 && nodeCount <= 100_000, "--nodes must be an integer between 3 and 100000");+const edgeCount = nodeCount * 3;+const scopes = {+ nodes: "mem:graph:nodes", edges: "mem:graph:edges", history: "mem:graph:edge-history",+ snapshot: "mem:graph:snapshot", control: "mem:graph:retrieval-control", migration: "mem:graph:index-migration", manifest: "mem:graph:index-manifest",+ index: "mem:graph:retrieval-data",+};+const canonicalScopes = [scopes.nodes, scopes.edges, scopes.history];+const out = options.out ?? mkdtempSync(join(tmpdir(), "agentmemory-graph-index-"));+mkdirSync(out, { recursive: true });+const root = mkdtempSync(join(out, "run-"));+console.log(`Artifacts: ${root}`);+const prefix = join(root, "prefix"), modules = join(prefix, "node_modules"), pkg = join(modules, "@agentmemory/agentmemory");+mkdirSync(prefix, { recursive: true });+const children = new Set(), profiles = [], results = [];+const hash = (path) => createHash("sha256").update(readFileSync(path)).digest("hex");+const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));+const free = (port) => new Promise((resolve) => {+ const server = net.createServer(); server.once("error", () => resolve(false));+ server.listen(port, "127.0.0.1", () => server.close(() => resolve(true)));+});+async function poll(label, check, timeout = 30_000) {+ const end = Date.now() + timeout;+ let error;+ do {+ try { const value = await check(); if (value) return value; } catch (cause) { error = cause; }+ await sleep(100);+ } while (Date.now() < end);+ throw new Error(`${label} timed out: ${error?.message ?? "not ready"}`);+}+function note(name, details = {}) {+ const result = { name, ...details }; results.push(result); console.log(JSON.stringify(result));+}+function kill(child, signal = "SIGKILL") {+ if (!child || child.exitCode !== null || child.signalCode !== null) return;+ try { process.kill(-child.pid, signal); } catch { try { child.kill(signal); } catch {} }+}+process.on("exit", () => { for (const child of children) kill(child); });+for (const signal of ["SIGINT", "SIGTERM", "SIGHUP"]) {+ process.on(signal, () => { for (const child of children) kill(child); process.exit(130); });+}+let manifest, engine, cli, worker, originals, fatal, cleanupError;+async function install() {+ if (options.dependencies) {+ mkdirSync(pkg, { recursive: true });+ execFileSync("tar", ["-xzf", options.tarball, "--strip-components=1", "-C", pkg]);+ for (const entry of readdirSync(options.dependencies, { withFileTypes: true })) {+ if (entry.name.startsWith(".")) continue;+ if (entry.name.startsWith("@")) {+ mkdirSync(join(modules, entry.name), { recursive: true });+ for (const name of readdirSync(join(options.dependencies, entry.name))) {+ const target = join(modules, entry.name, name);+ if (!existsSync(target)) symlinkSync(join(options.dependencies, entry.name, name), target);+ }+ } else symlinkSync(join(options.dependencies, entry.name), join(modules, entry.name));+ }+ } else {+ writeFileSync(join(prefix, "package.json"), '{"name":"graph-index-gate","private":true}\n');+ const installHome = join(root, "install-home"); mkdirSync(installHome);+ execFileSync("npm", ["install", "--ignore-scripts", "--omit=optional", "--no-audit", "--no-fund", options.tarball], {+ cwd: prefix, stdio: "inherit", timeout: 180_000,+ env: { HOME: installHome, USERPROFILE: installHome, PATH: [dirname(process.execPath), "/usr/bin", "/bin"].join(delimiter), npm_config_cache: join(root, "npm-cache") },+ });+ }+ manifest = JSON.parse(readFileSync(join(pkg, "package.json"), "utf8"));+ engine = options.engine ?? process.env.AGENTMEMORY_GATE_ENGINE ?? join(homedir(), ".cache/agentmemory-release-gate", `iii-${manifest.dependencies["iii-sdk"]}-${platform()}-${arch()}`);+ assert(execFileSync(engine, ["--version"], { encoding: "utf8" }).includes(manifest.dependencies["iii-sdk"]), "Pinned native engine version required");+ cli = join(pkg, "dist/cli.mjs"); worker = join(pkg, "dist/index.mjs");+ originals = [options.tarball, cli, worker].map(hash);+}+async function profile(name, instance = 0, sharedHome) {+ const dir = join(root, name), home = sharedHome ?? join(dir, "home"), data = join(dir, "data"), runtime = instance ? data : join(home, ".agentmemory");+ for (const path of [runtime, data]) mkdirSync(path, { recursive: true });+ let port;+ for (let i = 0; i < 100; i++) {+ const candidate = 12_000 + Math.floor(Math.random() * 7_000);+ const block = [0, 1, 2, 3, 6353, 46023].map((n) => candidate + n);+ if (block.some((n) => profiles.some((p) => p.ports.includes(n)))) continue;+ if ((await Promise.all(block.map(free))).every(Boolean)) { port = candidate; break; }+ }+ assert(port, "No free port block");+ const p = { name, instance, dir, home, runtime, data, port, ports: [0, 1, 2, 3, 6353, 46023].map((n) => port + n), secret: "", trace: [], traceSummary: new Map(), phase: "startup", sockets: new Set(), holdManifestAt: 0, manifestReads: 0, held: false };+ profiles.push(p);+ let config = readFileSync(join(pkg, "iii-config.yaml"), "utf8").split(" - name: iii-exec")[0];+ config = config.replaceAll("3111", String(port)).replaceAll("3112", String(port + 1)).replaceAll("3113", String(port + 2)).replaceAll("./data/", `${data}/`);+ config += ` - name: iii-worker-manager\n config:\n port: ${port + 46023}\n host: 127.0.0.1\n`;+ p.config = join(dir, "iii-config.yaml"); writeFileSync(p.config, config);+ return p;+}+function environment(p) {+ return {+ HOME: p.home, USERPROFILE: p.home, PATH: [dirname(process.execPath), "/usr/bin", "/bin", "/usr/sbin", "/sbin"].join(delimiter), TMPDIR: root,+ CI: "1", NO_COLOR: "1", LANG: "C.UTF-8", OTEL_ENABLED: "false", III_TELEMETRY_ENABLED: "false",+ AGENTMEMORY_DATA_DIR: p.data, AGENTMEMORY_RUNTIME_DIR: p.runtime,+ ...(p.explicitSecret ? { AGENTMEMORY_SECRET: p.explicitSecret } : {}),+ III_REST_PORT: String(p.port), III_STREAM_PORT: String(p.port + 1), III_VIEWER_PORT: String(p.port + 2),+ III_ENGINE_PORT: String(p.port + 3), III_ENGINE_URL: `ws://127.0.0.1:${p.port + 3}`, AGENTMEMORY_METRICS_PORT: String(p.port + 6353),+ LLM_PROVIDER: "none", EMBEDDING_PROVIDER: "none", AGENTMEMORY_GRAPH_WEIGHT: "0.3",+ AGENTMEMORY_GRAPH_COMPACT_ON_BOOT: "true", AGENTMEMORY_AUTO_COMPRESS: "false", AGENTMEMORY_SESSION_SWEEP_ENABLED: "false", AGENTMEMORY_FINALIZE_IDLE_MS: "0",+ GRAPH_EXTRACTION_ENABLED: "false", CONSOLIDATION_ENABLED: "false", AGENTMEMORY_REFLECT: "false",+ SNAPSHOT_ENABLED: "true", SNAPSHOT_DIR: join(p.dir, "snapshots"), SNAPSHOT_INTERVAL: "3600",+ };+}+function launch(p, executable, args, name) {+ const child = spawn(executable, args, { env: environment(p), cwd: p.dir, detached: true, stdio: ["ignore", "pipe", "pipe"] });+ children.add(child); child.text = "";+ const log = (chunk) => { child.text = (child.text + chunk).slice(-65536); appendFileSync(join(p.dir, `${name}.log`), chunk); };+ child.stdout.on("data", log); child.stderr.on("data", log); child.on("error", (error) => log(String(error)));+ child.done = new Promise((resolve) => child.on("close", (code, signal) => { children.delete(child); resolve({ code, signal }); }));+ return child;+}+async function relay(p) {+ p.manifestReads = 0;+ p.relay = new WebSocketServer({ host: "127.0.0.1", port: p.port + 3 });+ await new Promise((resolve, reject) => { p.relay.once("listening", resolve); p.relay.once("error", reject); });+ p.relay.on("connection", (downstream) => {+ const upstream = new WebSocket(`ws://127.0.0.1:${p.port + 46023}`);+ p.sockets.add(downstream); p.sockets.add(upstream);+ const queued = [], inFlight = new Map();+ const forward = (bytes) => upstream.readyState === WebSocket.OPEN ? upstream.send(bytes) : queued.push(bytes);+ upstream.on("open", () => { for (const bytes of queued.splice(0)) upstream.send(bytes); });+ downstream.on("message", (bytes) => {+ const message = JSON.parse(bytes.toString());+ if (message.type === "invokefunction" && message.function_id?.startsWith("state::") && message.data?.scope?.startsWith("mem:graph:")) {+ const record = { phase: p.phase, operation: message.function_id, scope: message.data.scope, key: message.data.key, requestBytes: bytes.length };+ if (p.trace.length < 20_000) p.trace.push(record);+ let summary = p.traceSummary.get(p.phase);+ if (!summary) p.traceSummary.set(p.phase, summary = { graphOperations: 0, maxGraphResponseBytes: 0, maxReadLatencyMs: 0, canonicalLists: [], snapshotOperations: 0 });+ summary.graphOperations++;+ if (record.operation === "state::list" && canonicalScopes.some((scope) => record.scope.startsWith(scope))) summary.canonicalLists.push(record.scope);+ if (record.scope.startsWith(scopes.snapshot)) summary.snapshotOperations++;+ inFlight.set(message.invocation_id, { record, summary, started: performance.now() });+ // A durable sealed descriptor exists before the first manifest read. Hold only this request to make the crash deterministic.+ if (message.function_id === "state::get" && message.data.scope === scopes.manifest) {+ p.manifestReads++;+ if (p.manifestReads === p.holdManifestAt) { p.holdManifestAt = 0; p.held = true; return; }+ }+ }+ forward(bytes);+ });+ upstream.on("message", (bytes) => {+ const message = JSON.parse(bytes.toString());+ const tracked = inFlight.get(message.invocation_id);+ if (message.type === "invocationresult" && tracked) {+ tracked.record.responseBytes = bytes.length; tracked.record.latencyMs = performance.now() - tracked.started;+ tracked.summary.maxGraphResponseBytes = Math.max(tracked.summary.maxGraphResponseBytes, bytes.length);+ if (tracked.record.operation === "state::get") tracked.summary.maxReadLatencyMs = Math.max(tracked.summary.maxReadLatencyMs, tracked.record.latencyMs);+ inFlight.delete(message.invocation_id);+ }+ if (downstream.readyState === WebSocket.OPEN) downstream.send(bytes);+ });+ for (const socket of [upstream, downstream]) {+ socket.on("error", () => { upstream.terminate(); downstream.terminate(); });+ socket.on("close", () => { p.sockets.delete(socket); upstream.terminate(); downstream.terminate(); });+ }+ });+}+async function startEngine(p, name) {+ p.engine = launch(p, engine, ["--config", p.config, "--no-update-check"], `${name}-engine`);+ await poll("native engine", async () => (await fetch(`http://127.0.0.1:${p.port}/`, { signal: AbortSignal.timeout(500) })).status > 0);+ await relay(p);+ p.client = registerWorker(`ws://127.0.0.1:${p.port + 46023}`, { workerName: `graph-index-fixture-${p.name}`, invocationTimeoutMs: 30_000, enableMetricsReporting: false, otel: { enabled: false } });+}+async function startWorker(p, name) {+ p.worker = launch(p, process.execPath, [cli, "--no-engine", "--port", String(p.port), "--data-dir", p.data, ...(p.instance ? ["--instance", String(p.instance)] : [])], name);+ await poll("packaged CLI", async () => {+ if (p.worker.exitCode !== null) throw new Error(`CLI exited: ${p.worker.text}`);+ if (!p.worker.text.includes("REST API")) return false;+ const value = await rest(p, "/livez"); return value.viewerPort || value.viewerSkipped;+ });+ p.secret = p.explicitSecret ?? readFileSync(join(p.home, ".agentmemory", "secret"), "utf8").trim();+}+async function start(p, name) { await startEngine(p, name); await startWorker(p, name); }+async function stop(p, graceful = false) {+ if (graceful && p.worker) { kill(p.worker, "SIGTERM"); await Promise.race([p.worker.done, sleep(5000)]); }+ kill(p.worker); if (p.worker) await Promise.race([p.worker.done, sleep(2000)]); p.worker = undefined;+ if (graceful) await sleep(2500);+ await p.client?.shutdown(); p.client = undefined;+ for (const socket of p.sockets) socket.terminate();+ if (p.relay) await new Promise((resolve) => p.relay.close(resolve)); p.relay = undefined;+ kill(p.engine); if (p.engine) await Promise.race([p.engine.done, sleep(2000)]); p.engine = undefined;+ await poll("ports closed", async () => (await Promise.all(p.ports.map(free))).every(Boolean), 5000);+}+async function response(p, path, body, secret = p.secret) {+ return fetch(`http://127.0.0.1:${p.port}/agentmemory${path}`, {+ method: body === undefined ? "GET" : "POST", headers: { "content-type": "application/json", ...(secret ? { authorization: `Bearer ${secret}` } : {}) },+ ...(body === undefined ? {} : { body: JSON.stringify(body) }), signal: AbortSignal.timeout(30_000),+ });+}+async function rest(p, path, body) {+ const res = await response(p, path, body); const text = await res.text();+ assert(res.ok, `${path}: HTTP ${res.status}: ${text.slice(0, 1000)}`); return JSON.parse(text);+}+async function saveStatusHtml(p, name, expectedCounts) {+ const res = await response(p, "/status?format=html");+ assert(res.ok, `Status HTML: HTTP ${res.status}`);+ const html = await res.text();+ assert(html.includes(`<th>Nodes / edges</th><td>${expectedCounts}</td>`), `Status HTML must report ${expectedCounts}`);+ writeFileSync(join(p.dir, `${name}-status.html`), html);+ writeFileSync(join(p.dir, `${name}-status.json`), JSON.stringify(await rest(p, "/status"), null, 2) + "\n");+ const viewer = await fetch(`http://127.0.0.1:${p.port + 2}/`, { signal: AbortSignal.timeout(5000) });+ assert(viewer.ok, `Viewer HTML: HTTP ${viewer.status}`);+ writeFileSync(join(p.dir, `${name}-viewer.html`), await viewer.text());+ writeFileSync(join(p.dir, `${name}-viewer-snapshot.json`), JSON.stringify(await rest(p, "/viewer/snapshot"), null, 2) + "\n");+}+async function command(p, args, code = 0, extraEnv = {}) {+ const result = await runCommand(process.execPath, [cli, "graph-index", ...args, "--port", String(p.port), ...(p.instance ? ["--instance", String(p.instance)] : [])], { env: { ...environment(p), ...extraEnv }, cwd: p.dir, timeoutMs: 15_000 });+ assertCommandCompleted(result); assert.equal(result.code, code, result.output);+ return JSON.parse(result.output);+}+const state = (p, id, payload) => p.client.trigger({ function_id: id, payload });+async function assertEdgeKey(p, source, target, type, expectedIds) {+ const control = await rest(p, "/graph/index");+ assert(control.ready);+ const scope = control.generation ? `${scopes.index}:generation:${control.generation}` : scopes.index;+ const posting = createHash("sha256").update(JSON.stringify(`edgeKey\0${JSON.stringify([source, target, type])}`)).digest("hex");+ const key = `${control.buildId}:root:${posting}`;+ const root = await state(p, "state::get", { scope, key });+ assert.equal(root?.count, expectedIds.length);+ const ids = [];+ let next = root.head;+ while (next !== null) {+ assert(ids.length < expectedIds.length, "Exact edge posting contains extra or cyclic entries");+ const entry = await state(p, "state::get", { scope, key: `${control.buildId}:entry:${posting}:${next}` });+ assert(entry, "Missing exact edge posting entry");+ ids.push(entry.id); next = entry.next;+ }+ assert.deepEqual(ids.sort(), [...expectedIds].sort());+}+function persisted(p, scope) {+ try { return decodeRkyvJsonScope(readFileSync(join(p.data, "state_store.db", `${encodeURIComponent(scope)}.bin`))); } catch { return {}; }+}+function graphFiles(p) {+ const dir = join(p.data, "state_store.db");+ return readdirSync(dir).filter((file) => decodeURIComponent(file).startsWith("mem:graph:")).map((file) => ({ scope: decodeURIComponent(file).replace(/\.bin$/, ""), bytes: statSync(join(dir, file)).size }));+}+function ordinaryReads(p, phase, requestCount = 10) {+ const summary = p.traceSummary.get(phase);+ assert(summary?.graphOperations > 0, "Wire trace must observe actual graph operations");+ assert.equal(summary.canonicalLists.length, 0, `${phase}: implicit canonical scope scan`);+ assert.equal(summary.snapshotOperations, 0, `${phase}: legacy snapshot read`);+ assert(summary.graphOperations <= requestCount * 2080, `${phase}: graph read request budget exceeded`);+ assert(summary.maxGraphResponseBytes <= 64 * 1024, `${phase}: graph response exceeds bounded record envelope`);+ return summary;+}+async function measuredReads(p, label, tasks) {+ p.phase = label;+ const timings = await Promise.all(tasks.map(async ([name, task]) => {+ const started = performance.now(); await task();+ const durationMs = performance.now() - started;+ assert(durationMs < 15_000, `${label}/${name}: request exceeded 15 second local gate budget`);+ return { name, durationMs };+ }));+ note(label, { requests: timings, ...ordinaryReads(p, label, tasks.length) });+}+const timestamp = "2026-10-14T00:00:00.000Z";+const node = (id, name = id, extra = {}) => ({ id, name, type: "concept", properties: {}, sourceObservationIds: ["mem_gate_source"], createdAt: timestamp, ...extra });+const edge = (id, from, to, extra = {}) => ({ id, sourceNodeId: from, targetNodeId: to, type: "related_to", weight: 0.8, sourceObservationIds: ["mem_gate_source"], createdAt: timestamp, ...extra });+const exported = (nodes = [], edges = []) => ({ version: manifest.version, exportedAt: timestamp, sessions: [], observations: {}, memories: [], summaries: [], graphNodes: nodes, graphEdges: edges });+async function find(p, query) { return (await rest(p, "/graph/query", { query, maxDepth: 0, limit: 20 })).nodes; }+function snapshot(p, value) {+ const dir = join(p.dir, "snapshots"); mkdirSync(dir, { recursive: true });+ writeFileSync(join(dir, "state.json"), JSON.stringify(value));+ const git = (...args) => execFileSync("git", ["-C", dir, ...args], { env: environment(p), encoding: "utf8", stdio: ["ignore", "pipe", "pipe"] }).trim();+ git("init", "--quiet"); git("add", "state.json"); git("-c", "user.name=Local fixture", "-c", "user.email=fixture@localhost", "commit", "--quiet", "-m", "Synthetic graph snapshot");+ return git("rev-parse", "HEAD");+}+try {+ await install();+ const primary = await profile("primary");+ await startEngine(primary, "legacy-seed");+ const seedStart = performance.now();+ await state(primary, "state::set", { scope: "mem:memories", key: "mem_gate_source", value: {+ id: "mem_gate_source", type: "fact", title: "Fixture evidence", content: "Graph transport uses durable identity checks", sessionIds: [], concepts: [], files: [], sourceObservationIds: [],+ strength: 5, version: 1, isLatest: true, createdAt: timestamp, updatedAt: timestamp, project: "fixture-primary", agentId: "fixture-agent",+ } });+ for (let i = 0; i < nodeCount; i++) await state(primary, "state::set", { scope: scopes.nodes, key: `node-${i}`, value: node(`node-${i}`, `Cobalt${i}`) });+ for (let i = 0; i < edgeCount; i++) await state(primary, "state::set", { scope: scopes.edges, key: `edge-${i}`, value: edge(`edge-${i}`, `node-${i % nodeCount}`, `node-${(i + 1) % nodeCount}`) });+ await state(primary, "state::set", { scope: scopes.history, key: "past", value: edge("past", "node-0", "node-1", { isLatest: false, version: 1 }) });+ // A large legacy snapshot makes accidental fallback reads visible in the wire trace.+ await state(primary, "state::set", { scope: scopes.snapshot, key: "current", value: { version: 1, topNodes: [], topEdges: [], topDegrees: {}, stats: { totalNodes: nodeCount, totalEdges: edgeCount, nodesByType: { concept: nodeCount }, edgesByType: { related_to: edgeCount } }, updatedAt: timestamp, dirty: false, marker: "x".repeat(512 * 1024) } });+ note("native-legacy-fixture", { nodes: nodeCount, edges: edgeCount, history: 1, seedMs: performance.now() - seedStart });+ primary.phase = "legacy-startup-status-search";+ await startWorker(primary, "legacy-startup");+ assert.equal((await command(primary, ["status", "--json"])).state, "needs-migration");+ const plainStatus = await runCommand(process.execPath, [cli, "status", "--port", String(primary.port)], { env: environment(primary), cwd: primary.dir, timeoutMs: 15_000 });+ assertCommandCompleted(plainStatus); assert.equal(plainStatus.code, 0, plainStatus.output);+ assert(plainStatus.output.includes("unknown nodes, unknown edges"), "CLI status must not present unavailable graph counts as zero");+ assert.equal((await command(primary, ["migrate", "--json"], 2)).error.includes("--allow-legacy-scan"), true);+ assert.equal((await response(primary, "/graph/index", undefined, "wrong-secret")).status, 401);+ assert.equal((await response(primary, "/graph/index/migrate", { allowLegacyScan: true }, "wrong-secret")).status, 401);+ assert.equal((await response(primary, "/graph/index/migrate", { allowLegacyScan: "true" })).status, 400);+ assert.equal((await response(primary, "/graph/index/migrate", {})).status, 400);+ assert.equal((await rest(primary, "/graph/query", { query: "Cobalt0" })).nodes.length, 0);+ await rest(primary, "/status"); await rest(primary, "/search", { query: "Cobalt0", limit: 3 });+ await sleep(4000);+ await saveStatusHtml(primary, "legacy", "unknown / unknown");+ note("legacy-startup-status-search", ordinaryReads(primary, primary.phase));++ primary.phase = "explicit-discovery"; primary.holdManifestAt = 1;+ const migrationStart = performance.now();+ const accepted = await command(primary, ["migrate", "--allow-legacy-scan", "--json"]);+ assert(accepted.jobId);+ await poll("sealed manifest held before replay", () => primary.held);+ assert.equal((await command(primary, ["resume", "--json"])).jobId, accepted.jobId);+ await poll("sealed migration durable", () => persisted(primary, scopes.migration).current?.sealed === true);+ assert.deepEqual(primary.traceSummary.get(primary.phase).canonicalLists, canonicalScopes);+ const sealed = persisted(primary, scopes.migration).current;+ assert.equal(sealed.processed, 0); assert.equal(sealed.totals.nodes, nodeCount); assert.equal(sealed.totals.edges, edgeCount);+ await stop(primary);+ note("sealed-crash-before-replay", { jobId: accepted.jobId, processed: sealed.processed, total: nodeCount + edgeCount + 1, discoveryAndCrashMs: performance.now() - migrationStart });++ primary.phase = "sealed-partial-restart";+ primary.held = false; primary.holdManifestAt = 2;+ const allResumeStart = performance.now();+ await start(primary, "sealed-partial-restart");+ await poll("second manifest held after replay checkpoint", () => primary.held);+ const checkpointProcessed = Math.min(nodeCount, 128);+ await poll("partial migration checkpoint durable across scopes", () => {+ const job = persisted(primary, scopes.migration).current;+ const control = persisted(primary, scopes.control).current;+ const page = persisted(primary, scopes.manifest)[`${accepted.jobId}:nodes:0`];+ const data = persisted(primary, scopes.index);+ return job?.processed === checkpointProcessed && control?.buildId === accepted.buildId && control.nodes === checkpointProcessed &&+ !control.pending && !control.batchPending && page?.ids?.length === checkpointProcessed && page.ids.every((id) =>+ data[`${accepted.buildId}:record:node:${createHash("sha256").update(JSON.stringify(id)).digest("hex")}`]?.row?.id === id);+ });+ assert.equal(primary.traceSummary.get(primary.phase).canonicalLists.length, 0);+ assert.equal(primary.traceSummary.get(primary.phase).snapshotOperations, 0);+ await stop(primary);+ note("partial-checkpoint-crash", { processed: checkpointProcessed, checkpointAcrossControlManifestAndRecords: true, elapsedMs: performance.now() - allResumeStart });++ primary.phase = "sealed-restart";+ const resumeStart = performance.now();+ await start(primary, "sealed-restart");+ const ready = await poll("sealed migration resumed", async () => {+ const status = await rest(primary, "/graph/index");+ return status.ready || status.state === "paused" ? status : null;+ }, 300_000);+ assert(ready.ready, ready.error ?? ready.action);+ assert.equal(ready.jobId, accepted.jobId); assert.equal(ready.nodes, nodeCount); assert.equal(ready.edges, edgeCount); assert.equal(ready.historyEdges, 1);+ assert.equal(ready.processed, nodeCount + edgeCount + 1); assert.equal(ready.coverage, "complete");+ await assertEdgeKey(primary, "node-0", "node-1", "related_to", ["edge-0", `edge-${nodeCount}`, `edge-${nodeCount * 2}`]);+ const migrationMs = performance.now() - resumeStart;+ assert.equal(primary.traceSummary.get(primary.phase).canonicalLists.length, 0);+ assert.equal(primary.traceSummary.get(primary.phase).snapshotOperations, 0);+ await poll("ready index flushed", () => persisted(primary, scopes.migration).current?.state === "ready");+ note("sealed-restart", { nodes: ready.nodes, edges: ready.edges, processed: ready.processed, resumeWallMs: migrationMs, includesEngineAndWorkerStartup: true, recordsResumed: ready.processed - checkpointProcessed, resumedRecordsPerSecond: (ready.processed - checkpointProcessed) / (migrationMs / 1000), totalReplayAndCheckpointCrashWallMs: performance.now() - allResumeStart, graphFiles: graphFiles(primary) });++ primary.phase = "indexed-ordinary-reads";+ await saveStatusHtml(primary, "indexed", `${nodeCount} / ${edgeCount}`);+ assert((await find(primary, "Cobalt0")).some((row) => row.id === "node-0"));+ const detail = await rest(primary, "/graph/node?id=node-0&limit=1");+ assert.equal(detail.node.id, "node-0"); assert(detail.relations.length <= 1); assert(detail.relationsCursor);+ const next = await rest(primary, `/graph/node?id=node-0&limit=1&cursor=${encodeURIComponent(detail.relationsCursor)}`);+ assert(next.relations.length <= 1);+ assert.equal((await response(primary, "/graph/node?id=node-0&cursor=not-a-cursor")).status, 400);+ assert.equal((await rest(primary, "/graph/stats")).totalNodes, nodeCount);+ await rest(primary, "/status"); await rest(primary, "/search", { query: "Cobalt0", limit: 3 });+ note("indexed-ordinary-reads", ordinaryReads(primary, primary.phase));+ await measuredReads(primary, "warm-repeat", [["graph-query", async () => assert((await find(primary, "Cobalt0")).some((row) => row.id === "node-0"))]]);+ await measuredReads(primary, "parallel-reads-and-health", [+ ["graph-query", async () => assert((await find(primary, "Cobalt0")).some((row) => row.id === "node-0"))],+ ["node-detail", async () => assert.equal((await rest(primary, "/graph/node?id=node-0&limit=5")).node.id, "node-0")],+ ["memory-search", async () => { await rest(primary, "/search", { query: "Cobalt0", limit: 3 }); }],+ ["health", async () => { await rest(primary, "/health"); }],+ ["livez", async () => { await rest(primary, "/livez"); }],+ ]);++ primary.phase = "canonical-writer-reconciliation";+ assert.equal((await rest(primary, "/import", { strategy: "merge", exportData: exported([node("node-0", "AmberRenamed")]) })).success, true);+ assert(!(await find(primary, "Cobalt0")).some((row) => row.id === "node-0"));+ assert((await find(primary, "AmberRenamed")).some((row) => row.id === "node-0"));+ assert.equal((await response(primary, `/graph/node?id=node-0&cursor=${encodeURIComponent(detail.relationsCursor)}`)).status, 400);+ assert.equal((await rest(primary, "/mesh/receive", { graphNodes: [node("node-0", "ZirconMesh", { updatedAt: "2026-10-15T00:00:00.000Z" })] })).success, true);+ assert(!(await find(primary, "AmberRenamed")).some((row) => row.id === "node-0"));+ assert((await find(primary, "ZirconMesh")).some((row) => row.id === "node-0"));+ assert.equal((await rest(primary, "/mesh/receive", { graphNodes: [node("node-0", "ZirconMesh", { updatedAt: "2026-10-16T00:00:00.000Z", stale: true })] })).success, true);+ assert(!(await find(primary, "ZirconMesh")).some((row) => row.id === "node-0"));+ const commitHash = snapshot(primary, { graphNodes: [node("node-0", "OnyxRestored")], graphEdges: [edge("edge-0", "node-0", "node-2", { type: "uses" })] });+ assert.equal((await rest(primary, "/snapshot/restore", { commitHash })).success, true);+ assert((await find(primary, "OnyxRestored")).some((row) => row.id === "node-0"));+ const restored = await rest(primary, "/graph/node?id=node-0&limit=200");+ assert(restored.relations.some((relation) => relation.edgeId === "edge-0" && relation.neighbor?.id === "node-2" && relation.type === "uses"));+ await assertEdgeKey(primary, "node-0", "node-1", "related_to", [`edge-${nodeCount}`, `edge-${nodeCount * 2}`]);+ await assertEdgeKey(primary, "node-0", "node-2", "uses", ["edge-0"]);+ const remembered = await rest(primary, "/remember", { content: "Indigo durable capture survives graph migration restart", project: "fixture-primary", agentId: "fixture-agent" });+ assert.equal(remembered.success, true);+ assert(remembered.memory?.id);+ assert.equal((await rest(primary, "/mesh/receive", { graphNodes: [node("node-0", "OnyxRestored", { updatedAt: "2026-10-17T00:00:00.000Z", sourceObservationIds: [remembered.memory.id] })] })).success, true);+ const graphRecall = () => rest(primary, "/search", { query: "OnyxRestored", project: "fixture-primary", agentId: "fixture-agent", targetLayer: "memory", limit: 5, format: "full" });+ assert((await graphRecall()).results.some((row) => row.observation?.id === remembered.memory.id), "Graph-only query must find captured memory through current provenance");+ assert.equal((await rest(primary, "/search", { query: "OnyxRestored", project: "different-project", targetLayer: "memory", limit: 5 })).results.length, 0);+ assert((await rest(primary, "/search", { query: "Indigo durable capture", project: "fixture-primary", agentId: "fixture-agent", limit: 5 })).results.length > 0);+ note("canonical-writer-reconciliation", { importRename: true, meshRename: true, staleExcluded: true, snapshotOverlay: true, capturedMemory: true, graphOnlyMemoryRecall: true });+ await stop(primary, true);++ primary.phase = "completed-restart";+ await start(primary, "completed-restart");+ assert.equal((await command(primary, ["status", "--json"])).ready, true);+ await assertEdgeKey(primary, "node-0", "node-1", "related_to", [`edge-${nodeCount}`, `edge-${nodeCount * 2}`]);+ await assertEdgeKey(primary, "node-0", "node-2", "uses", ["edge-0"]);+ note("completed-restart-startup", ordinaryReads(primary, primary.phase));+ await measuredReads(primary, "cold-engine-query", [["graph-query", async () => assert((await find(primary, "OnyxRestored")).some((row) => row.id === "node-0"))]]);+ await measuredReads(primary, "repeated-engine-query", [["graph-query", async () => assert((await find(primary, "OnyxRestored")).some((row) => row.id === "node-0"))]]);+ primary.phase = "completed-restart-recall";+ assert((await graphRecall()).results.some((row) => row.observation?.id === remembered.memory.id), "Graph-only memory recall must survive a restart");+ assert((await rest(primary, "/search", { query: "Indigo durable capture", project: "fixture-primary", agentId: "fixture-agent", limit: 5 })).results.length > 0);+ assert.equal((await rest(primary, "/search", { query: "Indigo durable capture", project: "different-project", limit: 5 })).results.length, 0);+ note("completed-restart-recall", ordinaryReads(primary, primary.phase));++ const indexed = await rest(primary, "/graph/index");+ const controlScope = indexed.generation ? `${scopes.control}:generation:${indexed.generation}` : scopes.control;+ const oldControl = await state(primary, "state::get", { scope: controlScope, key: "current" });+ await state(primary, "state::set", { scope: controlScope, key: "current", value: { ...oldControl, schema: 1 } });+ await stop(primary, true);+ primary.phase = "older-index-startup";+ await start(primary, "older-index-startup");+ const unavailable = await command(primary, ["status", "--json"]);+ assert.equal(unavailable.state, "unavailable");+ assert(unavailable.error.includes("explicit graph-index migration"));+ note("older-index-startup", ordinaryReads(primary, primary.phase));+ primary.phase = "explicit-index-rebuild";+ await command(primary, ["migrate", "--allow-legacy-scan", "--json"]);+ const rebuilt = await poll("older index rebuilt explicitly", async () => {+ const status = await rest(primary, "/graph/index");+ return status.ready || status.state === "paused" ? status : null;+ }, 300_000);+ assert(rebuilt.ready, rebuilt.error ?? rebuilt.action);+ assert.notEqual(rebuilt.buildId, indexed.buildId);+ assert.equal(rebuilt.nodes, nodeCount);+ await assertEdgeKey(primary, "node-0", "node-2", "uses", ["edge-0"]);+ assert((await graphRecall()).results.some((row) => row.observation?.id === remembered.memory.id));+ note("explicit-index-rebuild", { canonicalMemoryPreserved: true, exactEdgeKeyRestored: true });++ const secondary = await profile("secondary", 1, primary.home); secondary.phase = "named-instance-startup-status-search";+ secondary.explicitSecret = randomBytes(32).toString("hex");+ await start(secondary, "fresh-profile");+ assert.equal(primary.home, secondary.home);+ assert.equal(secondary.runtime, secondary.data);+ assert.notEqual(primary.secret, secondary.secret);+ assert.equal((await command(secondary, ["status", "--json"], 1, { AGENTMEMORY_SECRET: primary.secret })).httpStatus, 401);+ assert.equal((await response(secondary, "/graph/index", undefined, primary.secret)).status, 401);+ assert.equal((await command(secondary, ["status", "--json"])).state, "needs-migration");+ assert.equal((await rest(secondary, "/search", { query: "Indigo durable capture", limit: 5 })).results.length, 0);+ note("named-instance-startup-status-search", ordinaryReads(secondary, secondary.phase));+ assert.equal((await rest(secondary, "/graph/reset", {})).success, true);+ assert.equal((await command(secondary, ["status", "--json"])).ready, true);+ assert.equal((await rest(primary, "/graph/index")).nodes, nodeCount);+ await stop(secondary);++ const separateHome = await profile("separate-home"); separateHome.phase = "separate-home-startup-status";+ await start(separateHome, "fresh-home");+ assert.notEqual(separateHome.home, primary.home);+ assert.notEqual(separateHome.secret, primary.secret);+ assert.equal((await response(separateHome, "/graph/index", undefined, primary.secret)).status, 401);+ assert.equal((await command(separateHome, ["status", "--json"])).state, "needs-migration");+ note("separate-home-startup-status", { credentialMode: "generated", ...ordinaryReads(separateHome, separateHome.phase) });+ await stop(separateHome);++ assert.equal((await rest(primary, "/graph/reset", {})).success, true);+ const reset = await command(primary, ["status", "--json"]);+ assert.equal(reset.ready, true); assert.equal(reset.nodes, 0); assert.equal(reset.jobId, undefined);+ assert.equal((await find(primary, "OnyxRestored")).length, 0);+ note("profile-isolation-and-reset", { namedInstance: secondary.instance, sharedHome: true, credentialMode: "explicit", separateCredentials: true, separateHomeGeneratedCredentials: true, separateData: true, readyEmptyGeneration: true, oldMigrationHidden: true });+} catch (error) { fatal = error.stack ?? String(error); console.error(fatal); }+finally {+ for (const p of profiles) {+ try { await stop(p); } catch (error) { cleanupError = `${cleanupError ?? ""}\n${error.stack ?? error}`; }+ }+ for (const child of children) kill(child);+ await Promise.all([...children].map((child) => Promise.race([child.done, sleep(2000)])));+}+const unchanged = Boolean(originals && [options.tarball, cli, worker].every((path, i) => hash(path) === originals[i]));+const portsClosed = (await Promise.all(profiles.flatMap((p) => p.ports).map(free))).every(Boolean);+const summary = {+ ok: !fatal && !cleanupError && unchanged && portsClosed, root, tarball: options.tarball, tarballSha256: originals?.[0], installedArtifactsUnmodified: unchanged,+ cachedDependencies: Boolean(options.dependencies), node: process.version, platform: `${platform()}-${arch()}`, engine: manifest?.dependencies["iii-sdk"],+ nodes: nodeCount, edges: edgeCount, realModelConversations: false, results, fatal, cleanupError, portsClosed,+};+for (const p of profiles) writeFileSync(join(p.dir, "graph-wire-trace.json"), JSON.stringify({ phases: Object.fromEntries(p.traceSummary), firstOperations: p.trace, retainedOperationsLimit: 20_000 }, null, 2) + "\n");+writeFileSync(join(root, "summary.json"), JSON.stringify(summary, null, 2) + "\n");+console.log(`Summary: ${join(root, "summary.json")}`);+process.exit(summary.ok ? 0 : 1);diff --git a/scripts/skills/generate.ts b/scripts/skills/generate.tsindex 88865927..35efcb12 100644--- a/scripts/skills/generate.ts+++ b/scripts/skills/generate.ts@@ -3,6 +3,7 @@ import { join, dirname } from "node:path";import { fileURLToPath } from "node:url";import { getAllTools, ESSENTIAL_TOOLS } from "../../src/mcp/tools-registry.js";import { ADAPTERS } from "../../src/cli/connect/index.js";+import { collectRestEndpoints } from "./rest-endpoints.js";const HERE = dirname(fileURLToPath(import.meta.url));const ROOT = join(HERE, "..", "..");@@ -86,25 +87,7 @@ function tools(): string {function rest(): string {const text = readFileSync(join(SRC, "triggers", "api.ts"), "utf8");- const found: { path: string; method: string }[] = [];- const re = /api_path:\s*"([^"]+)"/g;- let m: RegExpExecArray | null;- while ((m = re.exec(text)) !== null) {- const path = m[1];- const win = text.slice(Math.max(0, m.index - 140), m.index + 140);- const mm = /http_method:\s*"([A-Z]+)"/.exec(win);- found.push({ path, method: mm ? mm[1] : "POST" });- }- // Dedupe on method+path, not path alone: ten paths register both GET and- // POST, and a path-only dedupe hid the second method and undercounted the- // surface (119 listed vs 130 registered).- const seen = new Set<string>();- const rows = found- .filter((e) => {- const key = `${e.method} ${e.path}`;- return seen.has(key) ? false : (seen.add(key), true);- })- .sort((a, b) => a.path.localeCompare(b.path) || a.method.localeCompare(b.method));+ const rows = collectRestEndpoints(text);const lines = [`The REST API is the primary surface. All paths are under \`http://localhost:3111\` (override with \`--port\`). Send \`Authorization: Bearer <secret>\`: the value of \`AGENTMEMORY_SECRET\`, or the one generated into \`~/.agentmemory/secret\` on first start when it is unset. Writes must use \`Content-Type: application/json\`.`,"",diff --git a/scripts/skills/rest-endpoints.ts b/scripts/skills/rest-endpoints.tsnew file mode 100644index 00000000..1a27c196--- /dev/null+++ b/scripts/skills/rest-endpoints.ts@@ -0,0 +1,32 @@+import ts from "typescript";++function property(object: ts.ObjectLiteralExpression, name: string): ts.Expression | undefined {+ const field = object.properties.find((item): item is ts.PropertyAssignment =>+ ts.isPropertyAssignment(item) && (ts.isIdentifier(item.name) || ts.isStringLiteralLike(item.name)) && item.name.text === name);+ return field?.initializer;+}++function literal(value: ts.Node | undefined): string | undefined {+ return value && ts.isStringLiteralLike(value) ? value.text : undefined;+}++export function collectRestEndpoints(text: string): { path: string; method: string }[] {+ const rows = new Map<string, { path: string; method: string }>();+ const source = ts.createSourceFile("api.ts", text, ts.ScriptTarget.Latest, false, ts.ScriptKind.TS);+ function visit(node: ts.Node): void {+ if (ts.isCallExpression(node) && ts.isPropertyAccessExpression(node.expression) && node.expression.name.text === "registerTrigger") {+ const registration = node.arguments[0];+ if (registration && ts.isObjectLiteralExpression(registration) && literal(property(registration, "type")) === "http") {+ const config = property(registration, "config");+ if (config && ts.isObjectLiteralExpression(config)) {+ const path = literal(property(config, "api_path"));+ const method = literal(property(config, "http_method")) ?? "POST";+ if (path) rows.set(`${method} ${path}`, { path, method });+ }+ }+ }+ ts.forEachChild(node, visit);+ }+ visit(source);+ return [...rows.values()].sort((a, b) => a.path.localeCompare(b.path) || a.method.localeCompare(b.method));+}diff --git a/src/cli.ts b/src/cli.tsindex 34e2c79e..356625a1 100644--- a/src/cli.ts+++ b/src/cli.ts@@ -105,6 +105,7 @@ const ALL_TOOLS_COUNT = getAllTools().length;const CORE_TOOLS_COUNT = getAllTools().filter((t) => ESSENTIAL_TOOLS.has(t.name)).length;import { resolveDataDir } from "./cli-data-dir.js";import { runCaptureCommand } from "./cli/capture.js";+import { runGraphIndexCommand } from "./cli/graph-index.js";import { bearerHeaders, resolveClientSecret } from "./secret-store.js";const __dirname = dirname(fileURLToPath(import.meta.url));@@ -239,6 +240,9 @@ Commands:capture [--drain] Show capture health: the local offline spool and the serverinbox (pending, retrying, dead letters). --drain sendsspooled observations now. --json for machine output.+ graph-index Inspect bounded graph index status. --json for machine output.+ migrate --allow-legacy-scan explicitly scans legacy graph data.+ resume continues a sealed migration without another full scan.import-jsonl [p] Import Claude Code JSONL transcripts (default: ~/.claude/projects)--max-files <N> | --max-files=<N>: override scan cap (default 200, max 1000;out-of-range is rejected; for trees >1000 files, batch by subdirectory)@@ -302,7 +306,7 @@ if (toolsIdx !== -1 && args[toolsIdx + 1]) {process.env["AGENTMEMORY_TOOLS"] = toolsMode;}-const URL_CLIENT_COMMANDS = new Set(["status", "doctor", "mcp", "capture"]);+const URL_CLIENT_COMMANDS = new Set(["status", "doctor", "mcp", "capture", "graph-index"]);let hasExplicitLocalPortOverride = false;let selectedInstance = 0;@@ -2348,8 +2352,8 @@ async function runStatus() {const version = healthRes?.version || "?";const sessionList = Array.isArray(sessionsRes?.sessions) ? sessionsRes.sessions : [];const sessions = sessionList.length;- const nodes = Number(graphRes?.totalNodes ?? graphRes?.nodes ?? graphRes?.nodeCount ?? 0);- const edges = Number(graphRes?.totalEdges ?? graphRes?.edges ?? graphRes?.edgeCount ?? 0);+ const nodes = graphRes?.available === false ? "unknown" : graphRes?.totalNodes ?? graphRes?.nodes ?? graphRes?.nodeCount ?? "unknown";+ const edges = graphRes?.available === false ? "unknown" : graphRes?.totalEdges ?? graphRes?.edges ?? graphRes?.edgeCount ?? "unknown";const cb = healthRes?.circuitBreaker?.state || "closed";const heapMB = h?.memory ? Math.round(h.memory.heapUsed / 1048576) : 0;const uptime = h?.uptimeSeconds ? Math.round(h.uptimeSeconds) : 0;@@ -2652,10 +2656,8 @@ async function passiveServerChecks(): Promise<DoctorCheck[]> {const hasLlm = flags?.provider === "llm";const hasEmbed = flags?.embeddingProvider === "embeddings";- const graphNodeCount = Number(- graph?.totalNodes ?? graph?.nodes ?? graph?.nodeCount ?? 0,- );- const graphHas = graphNodeCount > 0;+ const graphNodeCount = graph?.available === false ? null : graph?.totalNodes ?? graph?.nodes ?? graph?.nodeCount ?? null;+ const graphHas = typeof graphNodeCount === "number" && graphNodeCount > 0;checks.push({@@ -2723,7 +2725,9 @@ async function passiveServerChecks(): Promise<DoctorCheck[]> {ok: graphHas,hint: graphHas? undefined- : "Graph is empty. Run a session with GRAPH_EXTRACTION_ENABLED=true.",+ : graphNodeCount === null+ ? "Graph counts are unavailable. Run agentmemory graph-index status."+ : "Graph is empty. Run a session with GRAPH_EXTRACTION_ENABLED=true.",});return checks;@@ -4230,6 +4234,11 @@ async function runCapture(): Promise<void> {process.exit(code);}+async function runGraphIndex(): Promise<void> {+ const base = getBaseUrl();+ process.exit(await runGraphIndexCommand({ base, args: args.slice(1), secret: resolveClientSecret(base) }));+}+const commands: Record<string, () => Promise<void>> = {init: runInit,connect: runConnectCmd,@@ -4243,6 +4252,7 @@ const commands: Record<string, () => Promise<void>> = {mcp: runMcp,"import-jsonl": runImportJsonl,capture: runCapture,+ "graph-index": runGraphIndex,};const first = args[0] ?? "";diff --git a/src/cli/graph-index.ts b/src/cli/graph-index.tsnew file mode 100644index 00000000..6022319f--- /dev/null+++ b/src/cli/graph-index.ts@@ -0,0 +1,63 @@+interface GraphIndexCommandOptions {+ base: string;+ args: string[];+ secret?: string;+ log?: (line: string) => void;+}++const VALUE_FLAGS = new Set(["--port", "--instance", "--data-dir", "--tools"]);++export async function runGraphIndexCommand(options: GraphIndexCommandOptions): Promise<number> {+ const log = options.log ?? console.log;+ const json = options.args.includes("--json");+ const output = (value: Record<string, unknown>): void => {+ if (json) { log(JSON.stringify(value, null, 2)); return; }+ if (value.error) { log(String(value.error)); return; }+ log(`Graph index: ${value.state ?? "unknown"}`);+ if (value.jobId) log(`Job: ${value.jobId}, processed ${value.processed ?? 0} of ${value.total ?? "unknown"}`);+ if (value.action) log(String(value.action));+ };+ let command: string | undefined;+ let allowLegacyScan = false;+ try {+ for (let i = 0; i < options.args.length; i++) {+ const arg = options.args[i]!;+ if (arg === "--json") continue;+ if (arg === "--allow-legacy-scan") { allowLegacyScan = true; continue; }+ if (VALUE_FLAGS.has(arg)) {+ if (!options.args[++i] || options.args[i]!.startsWith("--")) throw new Error(`Missing value for ${arg}`);+ continue;+ }+ if ([...VALUE_FLAGS].some((flag) => arg.startsWith(`${flag}=`) && arg.length > flag.length + 1)) continue;+ if (arg === "--verbose" || arg === "-v") continue;+ if (!command && ["status", "migrate", "resume"].includes(arg)) { command = arg; continue; }+ throw new Error(`Unknown graph-index argument: ${arg}`);+ }+ command ??= "status";+ if (command === "status" && allowLegacyScan) throw new Error("status does not accept --allow-legacy-scan");+ if (command === "migrate" && !allowLegacyScan) {+ throw new Error("Migration requires --allow-legacy-scan. Initial discovery reads each complete legacy graph scope and can use substantial memory; run it during maintenance.");+ }+ } catch (error) {+ output({ error: error instanceof Error ? error.message : String(error) });+ return 2;+ }+ try {+ const headers: Record<string, string> = { Accept: "application/json" };+ if (options.secret) headers.Authorization = `Bearer ${options.secret}`;+ if (command !== "status") headers["Content-Type"] = "application/json";+ const response = await fetch(`${options.base}/agentmemory/graph/index${command === "status" ? "" : `/${command}`}`, {+ method: command === "status" ? "GET" : "POST",+ headers,+ ...(command === "status" ? {} : { body: JSON.stringify({ allowLegacyScan }) }),+ signal: AbortSignal.timeout(10_000),+ });+ const value: unknown = await response.json();+ if (!value || typeof value !== "object" || Array.isArray(value)) throw new Error("Server returned an invalid graph-index response");+ output({ ...(value as Record<string, unknown>), ...(!response.ok ? { httpStatus: response.status } : {}) });+ return response.ok ? 0 : 1;+ } catch (error) {+ output({ error: error instanceof Error ? error.message : String(error) });+ return 1;+ }+}diff --git a/src/functions/cascade.ts b/src/functions/cascade.tsindex cfa65720..da922659 100644--- a/src/functions/cascade.ts+++ b/src/functions/cascade.ts@@ -1,19 +1,21 @@-import { pinGraphState } from "../state/graph-generation.js";+import { createHash } from "node:crypto";+import { setImmediate as yieldToEventLoop } from "node:timers/promises";+import { getGraphGeneration, pinGraphState } from "../state/graph-generation.js";import type { IIIClient } from "iii-sdk";import type { StateKV } from "../state/kv.js";import { KV } from "../state/schema.js";-import { withKeyedLock } from "../state/keyed-mutex.js";-import type { Memory, GraphNode, GraphEdge } from "../types.js";-import { recordAudit } from "./audit.js";+import { createGraphIndexReader, mutateIndexedGraphRecord } from "../state/graph-retrieval-index.js";+import type { Memory, GraphCascadeProgress } from "../types.js";+import { safeAudit } from "./audit.js";export function registerCascadeFunction(sdk: IIIClient, state: StateKV): void {sdk.registerFunction("mem::cascade-update",async (data: { supersededMemoryId: string }) => {- const kv = await pinGraphState(state);- if (!data.supersededMemoryId || typeof data.supersededMemoryId !== "string") {+ if (!data || !data.supersededMemoryId || typeof data.supersededMemoryId !== "string") {return { success: false, error: "supersededMemoryId is required" };}+ const kv = await pinGraphState(state);const superseded = await kv.get<Memory>(KV.memories, data.supersededMemoryId);if (!superseded) {return { success: false, error: "superseded memory not found" };@@ -23,51 +25,91 @@ export function registerCascadeFunction(sdk: IIIClient, state: StateKV): void {let flaggedEdges = 0;let flaggedMemories = 0;- const obsIds = new Set(superseded.sourceObservationIds || []);-+ const sourceIds = [...new Set(superseded.sourceObservationIds ?? [])];+ const obsIds = new Set(sourceIds);+ let graphDeferred = false;+ let graphReason: string | undefined;+ let graphTruncated = false;+ let progressSaved = true;+ const progressKey = `cascade:${createHash("sha256").update(data.supersededMemoryId).digest("hex")}`;if (obsIds.size > 0) {- const now = new Date().toISOString();- const nodes = await kv.list<GraphNode>(KV.graphNodes);- for (const node of nodes) {- if (node.stale) continue;- const overlap = (node.sourceObservationIds ?? []).some((id) => obsIds.has(id));- if (!overlap) continue;- const flagged = await withKeyedLock("graph:persist", async () => {- const latest = (await kv.get<GraphNode>(KV.graphNodes, node.id)) ?? node;- if (latest.stale) return false;- await kv.set(KV.graphNodes, node.id, { ...latest, stale: true, updatedAt: now });- return true;- });- if (flagged) {- await recordAudit(kv, "consolidate", "mem::cascade-update", [node.id], {- resourceType: "GraphNode",- change: "marked stale from superseded memory",- supersededMemoryId: data.supersededMemoryId,- });- flaggedNodes++;- }- }-- const edges = await kv.list<GraphEdge>(KV.graphEdges);- for (const edge of edges) {- if (edge.stale) continue;- const overlap = (edge.sourceObservationIds ?? []).some((id) => obsIds.has(id));- if (!overlap) continue;- const flagged = await withKeyedLock("graph:persist", async () => {- const latest = (await kv.get<GraphEdge>(KV.graphEdges, edge.id)) ?? edge;- if (latest.stale) return false;- await kv.set(KV.graphEdges, edge.id, { ...latest, stale: true });- return true;- });- if (flagged) {- await recordAudit(kv, "consolidate", "mem::cascade-update", [edge.id], {- resourceType: "GraphEdge",- change: "marked stale from superseded memory",- supersededMemoryId: data.supersededMemoryId,- });- flaggedEdges++;+ const generation = await getGraphGeneration(kv);+ const sourceFingerprint = createHash("sha256").update(JSON.stringify(sourceIds)).digest("hex");+ let offset = 0;+ let continueBatches = true;+ while (continueBatches) {+ continueBatches = false;+ graphDeferred = false; graphTruncated = false; graphReason = undefined;+ const priorCount = flaggedNodes + flaggedEdges;+ try {+ const saved = await kv.get<GraphCascadeProgress>(KV.graphControl, progressKey);+ if (saved?.schema === 1 && saved.generation === generation && saved.sourceFingerprint === sourceFingerprint &&+ Number.isSafeInteger(saved.offset) && saved.offset >= 0 && saved.offset < sourceIds.length) offset = saved.offset;+ const reader = await createGraphIndexReader(kv, { maxReads: 512 });+ if (!reader) {+ graphDeferred = true;+ graphReason = "Graph index is unavailable. Complete graph-index migration or recovery, then retry cascade-update.";+ } else {+ const previousOffset = offset;+ const candidates = new Map<string, { kind: "node" | "edge"; id: string }>();+ let nextOffset = offset;+ for (let i = offset; i < sourceIds.length; i++) {+ if (i - offset >= 64 || candidates.size >= 64 || reader.remainingReads < 4) { graphTruncated = true; break; }+ const observationId = sourceIds[i]!;+ if (typeof observationId !== "string" || Buffer.byteLength(observationId) > 1024) {+ graphDeferred = true; graphReason = "A source observation ID exceeds the graph index limit."; break;+ }+ const limit = Math.max(1, Math.floor((64 - candidates.size) / 2));+ const [nodes, edges] = await Promise.all([+ reader.posting("observation", observationId, { limit }),+ reader.posting("edgeObservation", observationId, { limit }),+ ]);+ for (const id of nodes.ids) candidates.set(`node:${id}`, { kind: "node", id });+ for (const id of edges.ids) candidates.set(`edge:${id}`, { kind: "edge", id });+ if (nodes.cursor || edges.cursor || reader.exhausted) { graphTruncated = true; nextOffset = i; break; }+ nextOffset = i + 1;+ }+ if (!await reader.finish()) {+ graphDeferred = true;+ graphReason = "Graph changed during cascade discovery; retry cascade-update.";+ } else {+ const now = new Date().toISOString();+ for (const { kind, id } of candidates.values()) {+ const changed = await mutateIndexedGraphRecord(kv, kind, id, (latest) => {+ if (!latest || latest.stale || !(latest.sourceObservationIds ?? []).some((sourceId) => obsIds.has(sourceId))) return undefined;+ return { ...latest, stale: true, ...(kind === "node" ? { updatedAt: now } : {}) };+ });+ if (!changed) continue;+ if (kind === "node") flaggedNodes++; else flaggedEdges++;+ await safeAudit(kv, "consolidate", "mem::cascade-update", [id], {+ resourceType: kind === "node" ? "GraphNode" : "GraphEdge",+ change: "marked stale from superseded memory", supersededMemoryId: data.supersededMemoryId,+ });+ }+ offset = nextOffset;+ continueBatches = graphTruncated && !graphReason && reader.status.incompleteRecords === 0 &&+ (offset > previousOffset || flaggedNodes + flaggedEdges > priorCount);+ graphDeferred ||= graphTruncated || offset < sourceIds.length || reader.status.incompleteRecords > 0;+ if (reader.status.incompleteRecords > 0) graphReason = "Graph index coverage is partial. Repair oversized records and retry cascade-update.";+ else if (graphDeferred && !graphReason) graphReason = "Bounded cascade batch completed; retry cascade-update to continue saved progress.";+ if (offset >= sourceIds.length && graphDeferred) offset = 0;+ }+ }+ } catch {+ continueBatches = false;+ graphDeferred = true;+ graphReason = "Graph cascade stopped after a storage or consistency failure; retry after graph index recovery.";}+ try {+ if (graphDeferred) {+ await kv.set<GraphCascadeProgress>(KV.graphControl, progressKey, { schema: 1, generation, sourceFingerprint,+ offset, reason: graphReason!, updatedAt: new Date().toISOString() });+ } else await kv.delete(KV.graphControl, progressKey);+ } catch { continueBatches = false; progressSaved = false; graphDeferred = true; graphReason = "Graph cascade progress could not be saved; retry after storage recovery."; }+ if (continueBatches) await yieldToEventLoop();}+ } else {+ await kv.delete(KV.graphControl, progressKey).catch(() => {});}const supersededConcepts = new Set(@@ -90,6 +132,9 @@ export function registerCascadeFunction(sdk: IIIClient, state: StateKV): void {return {success: true,+ graphDeferred,+ graphTruncated,+ ...(graphDeferred ? { graphReason, retryable: true, progressSaved } : {}),flagged: {nodes: flaggedNodes,edges: flaggedEdges,diff --git a/src/functions/graph-compact-boot.ts b/src/functions/graph-compact-boot.tsindex 7b5391b2..d71fa259 100644--- a/src/functions/graph-compact-boot.ts+++ b/src/functions/graph-compact-boot.ts@@ -44,7 +44,7 @@ export interface GraphCompactBootSlicePlan {total: number | null;}-export type GraphCompactBootState = "off" | "pending" | "running" | "done" | "failed";+export type GraphCompactBootState = "off" | "pending" | "running" | "done" | "failed" | "deferred";export interface GraphCompactBootStatus {state: GraphCompactBootState;@@ -60,6 +60,7 @@ export interface GraphCompactBootStatus {}export interface GraphCompactBootOptions {+ allowLegacyScan?: boolean;sliceSize?: number;pauseMs?: number;maxAttempts?: number;@@ -133,6 +134,7 @@ function scannedOf(scope: SlicedScope, r: GraphCompactResult): [number, number]export function describeGraphCompactBoot(s: GraphCompactBootStatus): string {if (s.state === "off") return "off (AGENTMEMORY_GRAPH_COMPACT_ON_BOOT=false)";if (s.state === "pending") return "pending";+ if (s.state === "deferred") return "deferred (legacy scan requires explicit maintenance)";const where =s.phase === "snapshot" || s.total === null || s.total === undefined? `${s.phase ?? "nodes"}`@@ -146,6 +148,10 @@ export function runGraphCompactOnBoot(kv: StateKV,opts: GraphCompactBootOptions = {},): Promise<GraphCompactBootStatus> {+ if (opts.allowLegacyScan !== true) {+ current = { state: "deferred" };+ return Promise.resolve(getGraphCompactBootStatus());+ }if (running) return running;running = runOnce(kv, opts).finally(() => {running = null;diff --git a/src/functions/graph-index-query.ts b/src/functions/graph-index-query.tsnew file mode 100644index 00000000..7016db39--- /dev/null+++ b/src/functions/graph-index-query.ts@@ -0,0 +1,115 @@+import type { StateKV } from "../state/kv.js";+import { createGraphIndexReader, normalizeGraphIndexText } from "../state/graph-retrieval-index.js";+import type { GraphEdge, GraphNode, GraphQueryResult } from "../types.js";++export interface IndexedGraphQuery {+ startNodeId?: string;+ nodeType?: string;+ maxDepth?: number;+ query?: string;+ limit?: number;+ offset?: number;+}++export async function queryGraphIndex(state: StateKV, data: IndexedGraphQuery): Promise<GraphQueryResult> {+ const integer = (value: number | undefined, fallback: number) =>+ typeof value === "number" && Number.isFinite(value) ? Math.floor(value) : fallback;+ const limit = Math.max(1, Math.min(integer(data.limit, 500), 500));+ const offset = Math.max(0, integer(data.offset, 0));+ const maxDepth = Math.max(0, Math.min(integer(data.maxDepth, 3), 5));+ const empty = (warning: string): GraphQueryResult => ({+ nodes: [], edges: [], depth: 0, totalNodes: 0, totalEdges: 0,+ truncated: true, limit, offset, warning,+ });+ try {+ const reader = await createGraphIndexReader(state, { maxReads: 2048, maxBytes: 4 * 1024 * 1024 });+ if (!reader) return empty("Graph index is unavailable. Run agentmemory graph-index status; legacy stores require an explicit migration.");+ const nodes: GraphNode[] = [];+ const edges = new Map<string, GraphEdge>();+ let truncated = reader.status.incompleteRecords > 0;+ let fullyScanned = false;+ let depth = 0;+ const query = data.query ? normalizeGraphIndexText(data.query) : "";+ const matches = (node: GraphNode) => (!data.nodeType || node.type === data.nodeType) && (!query ||+ normalizeGraphIndexText(node.name).includes(query) ||+ Object.values(node.properties ?? {}).some(value => typeof value === "string" && normalizeGraphIndexText(value).includes(query)));++ if (data.startNodeId && !query) {+ const queue = [{ id: data.startNodeId, depth: 0 }];+ const queued = new Set([data.startNodeId]);+ const visited = new Set<string>();+ for (let head = 0; head < queue.length && !reader.exhausted; head++) {+ const item = queue[head]!;+ if (visited.has(item.id) || item.depth > maxDepth) continue;+ visited.add(item.id);+ const node = await reader.getNode(item.id);+ if (!node || node.stale) continue;+ depth = Math.max(depth, item.depth);+ if (matches(node)) nodes.push(node);+ let cursor: string | undefined;+ do {+ const page = await reader.posting("adjacency", item.id, { cursor, limit: 64 });+ for (const id of page.ids) {+ const edge = await reader.getEdge(id);+ if (!edge || edge.stale || edge.isLatest === false) continue;+ edges.set(edge.id, edge);+ const next = edge.sourceNodeId === item.id ? edge.targetNodeId : edge.sourceNodeId;+ if (item.depth < maxDepth && !queued.has(next)) {+ if (queue.length < 512) {+ queue.push({ id: next, depth: item.depth + 1 });+ queued.add(next);+ }+ else truncated = true;+ }+ }+ cursor = page.cursor ?? undefined;+ } while (cursor && !reader.exhausted);+ if (cursor) truncated = true;+ }+ fullyScanned = !reader.exhausted && !truncated;+ } else {+ let cursor: string | undefined;+ do {+ const page = await reader.posting(query ? "character" : "nodes", query ? Array.from(query)[0]! : "", { cursor, limit: 64 });+ for (const id of page.ids) {+ const node = await reader.getNode(id);+ if (node && !node.stale && matches(node)) nodes.push(node);+ }+ cursor = page.cursor ?? undefined;+ fullyScanned = !cursor && !reader.exhausted;+ } while (cursor && nodes.length < offset + limit && !reader.exhausted);+ if (cursor) truncated = true;+ }++ const pageNodes = nodes.slice(offset, offset + limit);+ const ids = new Set(pageNodes.map(node => node.id));+ if (!data.startNodeId || query) {+ for (const node of pageNodes) {+ let cursor: string | undefined;+ do {+ const page = await reader.posting("adjacency", node.id, { cursor, limit: 64 });+ for (const id of page.ids) {+ if (edges.has(id)) continue;+ const edge = await reader.getEdge(id);+ if (edge && !edge.stale && edge.isLatest !== false && ids.has(edge.sourceNodeId) && ids.has(edge.targetNodeId)) edges.set(id, edge);+ }+ cursor = page.cursor ?? undefined;+ } while (cursor && !reader.exhausted);+ if (cursor || reader.exhausted) { truncated = true; break; }+ }+ }+ if (!await reader.finish()) return empty("Graph changed during the query. Retry to read a consistent index revision.");+ const pageEdges = [...edges.values()].filter(edge => ids.has(edge.sourceNodeId) && ids.has(edge.targetNodeId));+ const allNodes = !query && !data.startNodeId && !data.nodeType;+ truncated ||= reader.exhausted || !fullyScanned || nodes.length > pageNodes.length;+ return {+ nodes: pageNodes, edges: pageEdges, depth: data.startNodeId && !query ? maxDepth : depth,+ totalNodes: allNodes ? reader.status.nodes : nodes.length,+ totalEdges: allNodes ? reader.status.edges : pageEdges.length,+ truncated, limit, offset, fromSnapshot: false,+ ...(truncated ? { warning: "Graph response is bounded. Filtered counts describe examined records; some nodes, edges or oversized fields may be omitted." } : {}),+ };+ } catch {+ return empty("Graph index could not be read consistently. Run agentmemory graph-index status before retrying.");+ }+}diff --git a/src/functions/graph-index-temporal.ts b/src/functions/graph-index-temporal.tsnew file mode 100644index 00000000..e4045195--- /dev/null+++ b/src/functions/graph-index-temporal.ts@@ -0,0 +1,53 @@+import type { StateKV } from "../state/kv.js";+import { createGraphIndexReader, normalizeGraphIndexText } from "../state/graph-retrieval-index.js";+import type { GraphEdge, GraphNode } from "../types.js";++export async function readIndexedEntity(state: StateKV, name: string, includeAliases = true): Promise<{+ entity: GraphNode;+ relatedEdges: GraphEdge[];+ entityHistory: GraphEdge[];+ truncated: boolean;+} | { error: string }> {+ if (typeof name !== "string" || !name.trim()) return { error: "Entity name is required" };+ try {+ const reader = await createGraphIndexReader(state, { maxReads: 2048, maxBytes: 4 * 1024 * 1024 });+ if (!reader) return { error: "Graph index unavailable. Run agentmemory graph-index status; legacy stores require an explicit migration." };+ const normalized = normalizeGraphIndexText(name);+ const matches = await reader.posting("name", normalized, { limit: 64 });+ let entity: GraphNode | null = null;+ for (const id of matches.ids) {+ const node = await reader.getNode(id);+ if (node && !node.stale && (normalizeGraphIndexText(node.name) === normalized ||+ includeAliases && node.aliases?.some(alias => normalizeGraphIndexText(alias) === normalized))) {+ if (!entity || node.id < entity.id) entity = node;+ }+ }+ if (!entity) return { error: matches.cursor || reader.exhausted+ ? "Entity lookup exceeded the graph read budget" : `Entity "${name}" not found` };+ let truncated = !!matches.cursor || reader.status.incompleteRecords > 0;+ const readEdges = async (history: boolean) => {+ const edges = new Map<string, GraphEdge>();+ let cursor: string | undefined;+ do {+ const page = await reader.posting(history ? "history" : "adjacency", entity!.id, { cursor, limit: 64 });+ for (const id of page.ids) {+ const edge = history ? await reader.getHistoryEdge(id) : await reader.getEdge(id);+ if (!edge || edge.stale) continue;+ if (edge.sourceNodeId !== entity!.id && edge.targetNodeId !== entity!.id) continue;+ const other = edge.sourceNodeId === entity!.id ? edge.targetNodeId : edge.sourceNodeId;+ const neighbor = await reader.getNode(other);+ if (neighbor && !neighbor.stale) edges.set(id, edge);+ }+ cursor = page.cursor ?? undefined;+ } while (cursor && !reader.exhausted && edges.size < 512);+ if (cursor || reader.exhausted) truncated = true;+ return [...edges.values()];+ };+ const relatedEdges = await readEdges(false);+ const entityHistory = await readEdges(true);+ if (!await reader.finish()) return { error: "Graph changed during temporal lookup; retry the request" };+ return { entity, relatedEdges, entityHistory, truncated };+ } catch {+ return { error: "Graph index could not be read consistently. Run agentmemory graph-index status." };+ }+}diff --git a/src/functions/graph-node.ts b/src/functions/graph-node.tsindex 5c782c6c..8a754369 100644--- a/src/functions/graph-node.ts+++ b/src/functions/graph-node.ts@@ -1,10 +1,10 @@import type { StateKV } from "../state/kv.js";-import { pinGraphState } from "../state/graph-generation.js";+import { createGraphIndexReader, getGraphIndexStatus } from "../state/graph-retrieval-index.js";+import { lookupObservationSession } from "../state/obs-index.js";import { KV } from "../state/schema.js";import type { CompressedObservation, GraphEdge, GraphNode, Memory, Session } from "../types.js";import { getSearchIndex } from "./search.js";-const EDGE_SCAN_BUDGET_MS = 6000;const MAX_RELATIONS = 200;const MAX_OBSERVATIONS = 40;const MAX_MEMORIES = 25;@@ -23,8 +23,12 @@ export type GraphNodeDetail = {degree: number;relations: GraphNodeRelation[] | null;relationsTotal: number;+ relationsTotalExact: boolean;+ relationsCursor: string | null;+ relationsTruncated: boolean;relationsWarning?: string;provenance: {+ truncated: boolean;observationIds: string[];observations: Array<{ id: string; sessionId: string; title?: string; type?: string; timestamp?: string }>;sessions: Array<{ id: string; project?: string; agentId?: string; status?: string; startedAt?: string }>;@@ -34,128 +38,109 @@ export type GraphNodeDetail = {};};-function withBudget<T>(p: Promise<T>, ms: number): Promise<T | null> {- return new Promise((resolve) => {- const timer = setTimeout(() => resolve(null), ms);- p.then(- (v) => { clearTimeout(timer); resolve(v); },- () => { clearTimeout(timer); resolve(null); },- );- });-}+export type GraphNodeUnavailable = { graphIndex: { status: string }; error: string };function matchesName(values: string[] | undefined, name: string): boolean {- if (!values || values.length === 0) return false;const lower = name.toLowerCase();- return values.some((v) => {- const s = String(v).toLowerCase();- return s === lower || s.endsWith("/" + lower) || lower.endsWith("/" + s);+ return (values ?? []).some((value) => {+ const text = String(value).toLowerCase();+ return text === lower || text.endsWith("/" + lower) || lower.endsWith("/" + text);});}-export async function describeGraphNode(kv: StateKV, nodeId: string): Promise<GraphNodeDetail | null> {- kv = await pinGraphState(kv);- const node = await kv.get<GraphNode>(KV.graphNodes, nodeId);- if (!node) return null;-- const edges = await withBudget(kv.list<GraphEdge>(KV.graphEdges), EDGE_SCAN_BUDGET_MS);- let relations: GraphNodeRelation[] | null = null;- let relationsTotal = 0;- let relationsWarning: string | undefined;- const edgeObsIds: string[] = [];- if (edges) {- const touching = edges.filter(- (e) => !e.stale && e.isLatest !== false && (e.sourceNodeId === nodeId || e.targetNodeId === nodeId),- );- relationsTotal = touching.length;- touching.sort((a, b) => (b.weight ?? 0) - (a.weight ?? 0));- const kept = touching.slice(0, MAX_RELATIONS);- const neighborIds = [...new Set(kept.map((e) => (e.sourceNodeId === nodeId ? e.targetNodeId : e.sourceNodeId)))];- const neighbors = new Map<string, GraphNode | null>();- await Promise.all(- neighborIds.map(async (id) => neighbors.set(id, await kv.get<GraphNode>(KV.graphNodes, id).catch(() => null))),- );- relations = kept.map((e) => {- const out = e.sourceNodeId === nodeId;- const n = neighbors.get(out ? e.targetNodeId : e.sourceNodeId) ?? null;- for (const id of e.sourceObservationIds ?? []) edgeObsIds.push(id);- return {- edgeId: e.id,- type: e.type,- direction: out ? "out" : "in",- weight: typeof e.weight === "number" ? e.weight : 0.5,- observationCount: (e.sourceObservationIds ?? []).length,- neighbor: n ? { id: n.id, name: n.name, type: n.type } : null,- };- });- } else {- relationsWarning = "Listing relations took too long on this graph; showing the node without them.";+export async function describeGraphNode(+ kv: StateKV,+ nodeId: string,+ options: { cursor?: string; limit?: number } = {},+): Promise<GraphNodeDetail | GraphNodeUnavailable | null> {+ const reader = await createGraphIndexReader(kv);+ if (!reader) return { graphIndex: (await getGraphIndexStatus(kv)) ?? { status: "needs-migration" }, error: "Graph index is unavailable; run agentmemory graph-index status." };+ const node = await reader.getNode(nodeId);+ if (!node || node.stale) return null;+ const limit = Math.max(1, Math.min(MAX_RELATIONS, options.limit ?? MAX_RELATIONS));+ const page = await reader.posting("adjacency", nodeId, { cursor: options.cursor, limit });+ const edges: GraphEdge[] = [];+ for (const id of page.ids) {+ const edge = await reader.getEdge(id);+ if (edge && !edge.stale && edge.isLatest !== false && (edge.sourceNodeId === nodeId || edge.targetNodeId === nodeId)) edges.push(edge);+ }+ edges.sort((a, b) => b.weight - a.weight || (a.id < b.id ? -1 : a.id > b.id ? 1 : 0));+ const relations: GraphNodeRelation[] = [];+ const sourceIds = new Set(node.sourceObservationIds ?? []);+ for (const edge of edges) {+ const out = edge.sourceNodeId === nodeId;+ const neighbor = await reader.getNode(out ? edge.targetNodeId : edge.sourceNodeId);+ if (!neighbor || neighbor.stale) continue;+ for (const id of edge.sourceObservationIds ?? []) {+ if (sourceIds.size >= MAX_OBSERVATIONS) break;+ sourceIds.add(id);+ }+ relations.push({ edgeId: edge.id, type: edge.type, direction: out ? "out" : "in", weight: edge.weight,+ observationCount: edge.sourceObservationIds?.length ?? 0,+ neighbor: { id: neighbor.id, name: neighbor.name, type: neighbor.type } });}- const storedDegree = await kv.get<number>(KV.graphNodeDegree, nodeId).catch(() => null);- const degree = typeof storedDegree === "number" ? storedDegree : relationsTotal;-- const observationIds = [...new Set([...(node.sourceObservationIds ?? []), ...edgeObsIds])];const index = getSearchIndex();+ const observationIds: string[] = [];const observations: GraphNodeDetail["provenance"]["observations"] = [];const sessionIds = new Set<string>();- for (const id of observationIds.slice(0, MAX_OBSERVATIONS)) {- const sessionId = index.sessionOf(id);- if (!sessionId || sessionId === "lesson" || id.startsWith("mem_")) continue;- sessionIds.add(sessionId);- const obs = await kv.get<CompressedObservation>(KV.observations(sessionId), id).catch(() => null);- observations.push({- id,- sessionId,- ...(obs?.title ? { title: obs.title } : {}),- ...(obs?.type ? { type: obs.type } : {}),- ...(obs?.timestamp ? { timestamp: obs.timestamp } : {}),- });+ const memories: GraphNodeDetail["provenance"]["memories"] = [];+ const read = reader.read.bind(reader);+ for (const id of [...sourceIds].slice(0, MAX_OBSERVATIONS)) {+ const layer = index.layerOf(id);+ if (layer === "lesson" || layer === "insight") continue;+ if (layer !== "memory") {+ const sessionId = index.sessionOf(id) ?? await lookupObservationSession({ get: read } as StateKV, id);+ if (sessionId) {+ const obs = await read<CompressedObservation>(KV.observations(sessionId), id);+ if (obs?.id === id && obs.sessionId === sessionId) {+ observationIds.push(id);+ sessionIds.add(sessionId);+ observations.push({ id, sessionId, title: obs.title, type: obs.type, timestamp: obs.timestamp });+ continue;+ }+ if (layer === "observation") continue;+ }+ }+ const memory = await read<Memory>(KV.memories, id);+ if (memory?.isLatest !== false && memory?.id === id) {+ observationIds.push(id);+ memories.push({ id, title: memory.title, type: memory.type, project: memory.project, via: "observation" });+ }+ }+ if ((node.sourceObservationIds?.length ?? 0) > 0 && !observationIds.some((id) => node.sourceObservationIds.includes(id))) {+ if (node.sourceObservationIds.length > MAX_OBSERVATIONS || reader.exhausted || node.indexIncomplete) {+ return { graphIndex: { status: "partial" }, error: "Node provenance could not be verified within the read budget." };+ }+ return null;}-const sessions: GraphNodeDetail["provenance"]["sessions"] = [];- for (const sid of sessionIds) {- const s = await kv.get<Session>(KV.sessions, sid).catch(() => null);- sessions.push({- id: sid,- ...(s?.project ? { project: s.project } : {}),- ...(s?.agentId ? { agentId: s.agentId } : {}),- ...(s?.status ? { status: s.status } : {}),- ...(s?.startedAt ? { startedAt: s.startedAt } : {}),- });+ for (const id of sessionIds) {+ const session = await reader.read<Session>(KV.sessions, id);+ if (session) sessions.push({ id, project: session.project, agentId: session.agentId, status: session.status, startedAt: session.startedAt });}- const projects = [...new Set(sessions.map((s) => s.project).filter((p): p is string => !!p))].sort();-- const obsSet = new Set(observationIds);- const memories: GraphNodeDetail["provenance"]["memories"] = [];- const allMemories = await kv.list<Memory>(KV.memories).catch(() => [] as Memory[]);- for (const m of allMemories) {- if (!m || m.isLatest === false) continue;- const viaObs = (m.sourceObservationIds ?? []).some((id) => obsSet.has(id));- const viaName = !viaObs && (matchesName(m.concepts, node.name) || matchesName(m.files, node.name));- if (!viaObs && !viaName) continue;- memories.push({- id: m.id,- ...(m.title ? { title: m.title } : {}),- type: m.type,- ...(m.project ? { project: m.project } : {}),- via: viaObs ? "observation" : "name",- });+ const projects = [...new Set(sessions.map((session) => session.project).filter((project): project is string => !!project))].sort();+ const sourceSet = new Set(observationIds);+ const basename = node.name.split(/[\/\\]/).filter(Boolean).at(-1) ?? node.name;+ for (const hit of index.search(`${node.name} ${basename.replace(/\.[^.]+$/, "")}`, MAX_MEMORIES, "memory")) {+ if (memories.some((memory) => memory.id === hit.obsId)) continue;+ const memory = await reader.read<Memory>(KV.memories, hit.obsId);+ if (!memory || memory.isLatest === false) continue;+ const viaObservation = (memory.sourceObservationIds ?? []).some((id) => sourceSet.has(id));+ if (!viaObservation && !matchesName(memory.concepts, node.name) && !matchesName(memory.files, node.name)) continue;+ memories.push({ id: memory.id, title: memory.title, type: memory.type, project: memory.project, via: viaObservation ? "observation" : "name" });if (memories.length >= MAX_MEMORIES) break;}-const props = node.properties ?? {};- const structural =- typeof props.source === "string"- ? { source: props.source, ...(typeof props.sourceFile === "string" ? { sourceFile: props.sourceFile } : {}) }- : null;-+ const structural = typeof props.source === "string"+ ? { source: props.source, ...(typeof props.sourceFile === "string" ? { sourceFile: props.sourceFile } : {}) } : null;+ const truncated = Boolean(page.cursor || options.cursor || reader.exhausted || node.indexIncomplete);+ if (!await reader.finish()) return { graphIndex: { status: "changed" }, error: "Graph changed during retrieval; retry the request." };return {- node,- degree,- relations,- relationsTotal,- ...(relationsWarning ? { relationsWarning } : {}),- provenance: { observationIds, observations, sessions, projects, memories, structural },+ node: { ...node, sourceObservationIds: observationIds.filter((id) => node.sourceObservationIds.includes(id)) },+ degree: page.total, relations, relationsTotal: truncated ? page.total : relations.length,+ relationsTotalExact: !truncated, relationsCursor: page.cursor, relationsTruncated: truncated,+ ...(truncated ? { relationsWarning: "Relations are a bounded page; counts include unfiltered adjacency entries when truncated." } : {}),+ provenance: { truncated: true, observationIds, observations, sessions, projects, memories, structural },};}diff --git a/src/functions/graph-retrieval.ts b/src/functions/graph-retrieval.tsindex 5f545880..115f523b 100644--- a/src/functions/graph-retrieval.ts+++ b/src/functions/graph-retrieval.ts@@ -1,11 +1,10 @@-import { pinGraphState } from "../state/graph-generation.js";-import type {- GraphNode,- GraphEdge,-} from "../types.js";-import { matchesSearchLayer, type SearchLayer, type SearchLayerResolver } from "../state/search-layer.js";+import type { CompressedObservation, GraphNode, GraphEdge, Memory } from "../types.js";+import { matchesSearchLayer, type SearchLayer, type SearchLayerResolver, type SearchResultLayer } from "../state/search-layer.js";import { KV } from "../state/schema.js";import type { StateKV } from "../state/kv.js";+import { lookupObservationSession } from "../state/obs-index.js";+import { memoryToObservation } from "../state/memory-utils.js";+import { createGraphIndexReader, getGraphIndexStatus, normalizeGraphIndexText, type GraphIndexReader } from "../state/graph-retrieval-index.js";export interface GraphRetrievalResult {obsId: string;@@ -15,344 +14,354 @@ export interface GraphRetrievalResult {pathLength: number;}-function buildGraphContext(- path: Array<{ node: GraphNode; edge?: GraphEdge }>,-): string {- const parts: string[] = [];- for (const step of path) {- const props = Object.entries(step.node.properties)- .slice(0, 3)- .map(([k, v]) => `${k}=${v}`)- .join(", ");- let line = `[${step.node.type}] ${step.node.name}`;- if (props) line += ` (${props})`;- if (step.edge) {- line += ` --${step.edge.type}-->`;- if (step.edge.context?.reasoning) {- line += ` [${step.edge.context.reasoning}]`;- }- if (step.edge.tvalid) {- line += ` @${step.edge.tvalid}`;- }- }- parts.push(line);- }- return parts.join(" ");-}+export type GraphObservationAuthorization = (+ record: CompressedObservation | Memory,+ sessionId: string,+ read: GraphIndexReader["read"],+) => boolean | Promise<boolean>;-const MAX_START_NODES = 25;-const MAX_DISCOVERED_NODES = 2000;-const MAX_EXAMINED_EDGES = 10_000;--type GraphView = {- nodes: GraphNode[];- edges: GraphEdge[];- byId: Map<string, GraphNode>;- adjacency: Map<string, Array<{ neighborId: string; edge: GraphEdge }>>;-};-type TraversalBudget = { nodes: number; edges: number };--function compareText(a: string, b: string): number {- if (a === b) return 0;- return a < b ? -1 : 1;-}+export type GraphQueryDiagnostic = { status: string; truncated: boolean; reason?: string };-async function readGraph(kv: StateKV): Promise<GraphView> {- const pinned = await pinGraphState(kv);- const nodes = (await pinned.list<GraphNode>(KV.graphNodes)).filter((node) => !node.stale);- const edges = (await pinned.list<GraphEdge>(KV.graphEdges)).filter((edge) => !edge.stale);- const byId = new Map(nodes.map((node) => [node.id, node]));- const adjacency: GraphView["adjacency"] = new Map();- for (const edge of edges) {- if (!byId.has(edge.sourceNodeId) || !byId.has(edge.targetNodeId)) continue;- for (const [from, to] of [[edge.sourceNodeId, edge.targetNodeId], [edge.targetNodeId, edge.sourceNodeId]]) {- let neighbors = adjacency.get(from);- if (!neighbors) adjacency.set(from, neighbors = []);- neighbors.push({ neighborId: to, edge });- }- }- for (const neighbors of adjacency.values()) {- neighbors.sort((a, b) =>- b.edge.weight - a.edge.weight ||- compareText(a.neighborId, b.neighborId) ||- compareText(a.edge.id, b.edge.id),- );- }- return { nodes, edges, byId, adjacency };+type Evidence = { observation: CompressedObservation; layer: SearchResultLayer };+type Path = Array<{ node: GraphNode; edge?: GraphEdge }>;+const MAX_START_NODES = 25;+const MAX_CANDIDATES = 128;+const MAX_STATES = 512;+const MAX_EDGES = 1024;+const MAX_EVIDENCE = 256;+const MAX_OPERATIONS = 32;+const MAX_RESULTS = 32;+const FINAL_EVIDENCE_READS = 64;++function compareText(a: string, b: string): number { return a === b ? 0 : a < b ? -1 : 1; }+function nodeNames(node: GraphNode): string[] {+ return [node.name, ...(node.aliases ?? [])].map(normalizeGraphIndexText).filter(Boolean);}export class GraphRetrieval {- constructor(private kv: StateKV, private resolveLayer?: SearchLayerResolver) {}-- // The caller owns this query's lifetime; never retain its view across searches.- createQuery(): GraphQuery {- let view: Promise<GraphView> | undefined;- return new GraphQuery(() => view ??= readGraph(this.kv), this.resolveLayer);+ constructor(+ private kv: StateKV,+ private resolveLayer?: SearchLayerResolver,+ private sessionOf?: (id: string) => string | undefined,+ ) {}++ createQuery(authorize?: GraphObservationAuthorization): GraphQuery {+ return new GraphQuery(this.kv, this.resolveLayer, this.sessionOf, authorize);}searchByEntities(...args: Parameters<GraphQuery["searchByEntities"]>): Promise<GraphRetrievalResult[]> {return this.createQuery().searchByEntities(...args);}-expandFromChunks(...args: Parameters<GraphQuery["expandFromChunks"]>): Promise<GraphRetrievalResult[]> {return this.createQuery().expandFromChunks(...args);}-temporalQuery(...args: Parameters<GraphQuery["temporalQuery"]>): ReturnType<GraphQuery["temporalQuery"]> {return this.createQuery().temporalQuery(...args);}}export class GraphQuery {- constructor(private loadGraph: () => Promise<GraphView>, private resolveLayer?: SearchLayerResolver) {}-- async searchByEntities(- entityNames: string[],- maxDepth = 2,- maxResults = 20,- targetLayer: SearchLayer = "all",- ): Promise<GraphRetrievalResult[]> {- const exactNames = new Set(entityNames.map((name) => name.trim().toLowerCase()).filter(Boolean));- const names = [...exactNames];- if (names.length === 0 || maxResults <= 0) return [];- const graph = await this.loadGraph();- const matchingNodes = graph.nodes- .map((node) => ({ node, name: node.name.toLowerCase() }))- .filter(({ name }) => name.length > 0 && names.some((entity) => name.includes(entity) || entity.includes(name)))- .sort((a, b) =>- Number(exactNames.has(b.name)) - Number(exactNames.has(a.name)) ||- compareText(a.name, b.name) || compareText(a.node.id, b.node.id),- )- .slice(0, MAX_START_NODES)- .map(({ node }) => node);-- if (matchingNodes.length === 0) return [];-- const results: GraphRetrievalResult[] = [];- const visitedObs = new Set<string>();-- const budget = { nodes: MAX_DISCOVERED_NODES, edges: MAX_EXAMINED_EDGES };- for (const startNode of matchingNodes) {- const paths = this.dijkstraTraversal(startNode, graph, maxDepth, budget);-- for (const path of paths) {- const lastNode = path[path.length - 1].node;- for (const obsId of lastNode.sourceObservationIds) {- if (visitedObs.has(obsId) || !matchesSearchLayer(obsId, "", targetLayer, this.resolveLayer?.(obsId, ""))) continue;- visitedObs.add(obsId);+ private controller = new AbortController();+ private reader?: Promise<GraphIndexReader | null>;+ private evidence = new Map<string, Promise<Evidence | null>>();+ private operations = new Map<string, Promise<GraphRetrievalResult[]>>();+ private states = 0;+ private edges = 0;+ private examinedEvidence = 0;+ private diagnostic: GraphQueryDiagnostic = { status: "idle", truncated: false };++ constructor(+ private kv: StateKV,+ private resolveLayer?: SearchLayerResolver,+ private sessionOf?: (id: string) => string | undefined,+ private authorize?: GraphObservationAuthorization,+ ) {}++ cancel(): void { this.controller.abort(); this.diagnostic = { ...this.diagnostic, status: "cancelled", truncated: true }; }+ diagnostics(): GraphQueryDiagnostic { return { ...this.diagnostic }; }++ private async load(): Promise<GraphIndexReader | null> {+ this.reader ??= createGraphIndexReader(this.kv, { signal: this.controller.signal }).then(async (reader) => {+ this.controller.signal.throwIfAborted();+ if (!reader) {+ const status = await getGraphIndexStatus(this.kv);+ this.diagnostic = { status: status?.pending || status?.batchPending ? "dirty" : status?.status ?? "needs-migration", truncated: false, reason: "Graph index is unavailable; run graph-index status for recovery instructions." };+ } else {+ this.diagnostic.status = "ready";+ this.diagnostic.truncated ||= reader.status.incompleteRecords > 0;+ }+ return reader;+ });+ this.controller.signal.throwIfAborted();+ return this.reader;+ }- const pathLength = path.length;- const edgeWeights = path- .filter((s) => s.edge)- .map((s) => s.edge!.weight);- const avgWeight =- edgeWeights.length > 0- ? edgeWeights.reduce((a, b) => a + b, 0) / edgeWeights.length- : 0.5;- const score = avgWeight * (1 / pathLength);+ private truncated(): void { this.diagnostic.truncated = true; }++ private async finish(reader: GraphIndexReader, results: GraphRetrievalResult[]): Promise<GraphRetrievalResult[]> {+ this.controller.signal.throwIfAborted();+ if (reader.exhausted) this.truncated();+ const current: GraphRetrievalResult[] = [];+ const read = reader.read.bind(reader) as GraphIndexReader["read"];+ for (const result of results.slice(0, MAX_RESULTS)) {+ const prior = await this.evidence.get(result.obsId);+ if (!prior) continue;+ const record = prior.layer === "memory"+ ? await read<Memory>(KV.memories, result.obsId)+ : await read<CompressedObservation>(KV.observations(result.sessionId), result.obsId);+ if (!record || record.id !== result.obsId || ("isLatest" in record && record.isLatest === false)) continue;+ if ("sessionId" in record && record.sessionId !== result.sessionId) continue;+ if (this.authorize && !await this.authorize(record, result.sessionId, read)) continue;+ current.push({ ...result, graphContext: `[graph evidence] ${record.title.slice(0, 512)}` });+ }+ if (reader.exhausted || results.length > MAX_RESULTS) this.truncated();+ if (!await reader.finish()) {+ this.diagnostic = { status: "changed", truncated: true, reason: "Graph changed during retrieval; graph enrichment was skipped." };+ return [];+ }+ return current;+ }- results.push({- obsId,- sessionId: "",- score,- graphContext: buildGraphContext(path),- pathLength,- });+ private async source(reader: GraphIndexReader, id: string): Promise<Evidence | null> {+ let pending = this.evidence.get(id);+ if (pending) return pending;+ if (++this.examinedEvidence > MAX_EVIDENCE) { this.truncated(); return null; }+ pending = (async () => {+ const read = reader.read.bind(reader) as GraphIndexReader["read"];+ let sessionId = this.sessionOf?.(id);+ const layer = this.resolveLayer?.(id, sessionId ?? "");+ if (layer === "lesson" || layer === "insight") return null;+ if (layer !== "memory") {+ sessionId ??= await lookupObservationSession({ get: read } as StateKV, id) ?? undefined;+ if (sessionId) {+ const obs = await read<CompressedObservation>(KV.observations(sessionId), id);+ if (obs?.id === id && obs.sessionId === sessionId) {+ if (this.authorize && !await this.authorize(obs, sessionId, read)) return null;+ return { observation: obs, layer: "observation" as const };+ }+ if (layer === "observation") return null;}}+ const memory = await read<Memory>(KV.memories, id);+ if (!memory || memory.id !== id || memory.isLatest === false) return null;+ const observation = memoryToObservation(memory);+ if (this.authorize && !await this.authorize(memory, observation.sessionId, read)) return null;+ return { observation, layer: "memory" as const };+ })().catch(() => null);+ this.evidence.set(id, pending);+ return pending;+ }- for (const obsId of startNode.sourceObservationIds) {- if (visitedObs.has(obsId) || !matchesSearchLayer(obsId, "", targetLayer, this.resolveLayer?.(obsId, ""))) continue;- visitedObs.add(obsId);- results.push({- obsId,- sessionId: "",- score: 1.0,- graphContext: `[${startNode.type}] ${startNode.name}`,- pathLength: 0,- });- }+ private async backed(reader: GraphIndexReader, row: GraphNode | GraphEdge): Promise<boolean> {+ if ("indexIncomplete" in row && row.indexIncomplete) this.truncated();+ for (const id of row.sourceObservationIds ?? []) {+ if (await this.source(reader, id)) return true;+ if (reader.exhausted || this.controller.signal.aborted) return false;}-- results.sort((a, b) => b.score - a.score);- return results.slice(0, maxResults);+ return false;}- async expandFromChunks(- obsIds: string[],- maxDepth = 1,- maxResults = 10,- targetLayer: SearchLayer = "all",- ): Promise<GraphRetrievalResult[]> {- if (obsIds.length === 0 || maxResults <= 0) return [];- const graph = await this.loadGraph();- const wanted = new Set(obsIds);- const linkedNodes = graph.nodes- .filter((node) => node.sourceObservationIds.some((id) => wanted.has(id)))- .sort((a, b) => compareText(a.id, b.id))- .slice(0, MAX_START_NODES);-- const results: GraphRetrievalResult[] = [];- const visitedObs = new Set<string>(obsIds);-- const budget = { nodes: MAX_DISCOVERED_NODES, edges: MAX_EXAMINED_EDGES };- for (const node of linkedNodes) {- const paths = this.dijkstraTraversal(node, graph, maxDepth, budget);- for (const path of paths) {- const lastNode = path[path.length - 1].node;- for (const obsId of lastNode.sourceObservationIds) {- if (visitedObs.has(obsId) || !matchesSearchLayer(obsId, "", targetLayer, this.resolveLayer?.(obsId, ""))) continue;- visitedObs.add(obsId);+ private async append(+ reader: GraphIndexReader, node: GraphNode, results: GraphRetrievalResult[], seen: Set<string>,+ score: number, pathLength: number, layer: SearchLayer,+ ): Promise<void> {+ for (const id of node.sourceObservationIds ?? []) {+ if (seen.has(id)) continue;+ const evidence = await this.source(reader, id);+ if (!evidence || !matchesSearchLayer(id, evidence.observation.sessionId, layer, evidence.layer)) continue;+ seen.add(id);+ results.push({ obsId: id, sessionId: evidence.observation.sessionId, score, pathLength,+ graphContext: `[graph evidence] ${evidence.observation.title.slice(0, 512)}` });+ }+ }- const pathLength = path.length;- const score = 0.5 * (1 / (pathLength + 1));+ private run(key: string, work: () => Promise<GraphRetrievalResult[]>): Promise<GraphRetrievalResult[]> {+ if (this.controller.signal.aborted) return Promise.reject(this.controller.signal.reason);+ const existing = this.operations.get(key);+ if (existing) return existing;+ if (this.operations.size >= MAX_OPERATIONS) { this.truncated(); return Promise.resolve([]); }+ const pending = work().catch((error) => {+ if (!this.controller.signal.aborted) this.diagnostic = { status: "unavailable", truncated: true, reason: "Graph index could not be read; graph enrichment was skipped." };+ throw error;+ });+ this.operations.set(key, pending);+ return pending;+ }- results.push({- obsId,- sessionId: "",- score,- graphContext: buildGraphContext(path),- pathLength,- });+ async searchByEntities(entityNames: string[], maxDepth = 2, maxResults = 20, targetLayer: SearchLayer = "all"): Promise<GraphRetrievalResult[]> {+ if (entityNames.length > MAX_START_NODES || entityNames.some((name) => name.length > 512)) this.truncated();+ const names = [...new Set(entityNames.slice(0, MAX_START_NODES).map((name) => normalizeGraphIndexText(name.slice(0, 512).trim())).filter(Boolean))];+ if (!names.length || maxResults <= 0) return [];+ return this.run(JSON.stringify(["entities", names, maxDepth, maxResults, targetLayer]), async () => {+ const reader = await this.load();+ if (!reader) return [];+ const selected = new Map<string, GraphNode>();+ const exactNames = new Set(names);+ for (const name of names) {+ const page = await reader.posting("name", name, { limit: MAX_START_NODES });+ if (page.cursor) this.truncated();+ for (const id of page.ids) {+ if (selected.size >= MAX_START_NODES) { this.truncated(); break; }+ const node = await reader.getNode(id);+ if (node && !node.stale && nodeNames(node).some((value) => exactNames.has(value)) && await this.backed(reader, node)) selected.set(id, node);}}- }-- results.sort((a, b) => b.score - a.score);- return results.slice(0, maxResults);+ const partialCandidates = new Map<string, GraphNode>();+ const examined = new Set<string>();+ for (const name of names) {+ if (selected.size >= MAX_START_NODES || reader.exhausted || reader.remainingReads <= FINAL_EVIDENCE_READS) break;+ const characters = [...new Set([...name].filter((value) => value.trim()))];+ if (characters.length > 16) this.truncated();+ for (const [position, character] of characters.slice(0, 16).entries()) {+ if (reader.remainingReads <= FINAL_EVIDENCE_READS || selected.size >= MAX_START_NODES) { this.truncated(); break; }+ const page = await reader.posting("character", character, { limit: position === 0 ? MAX_CANDIDATES : 16 });+ if (page.cursor) this.truncated();+ for (const id of page.ids) {+ if (selected.has(id) || examined.has(id)) continue;+ if (examined.size >= MAX_CANDIDATES || reader.remainingReads <= FINAL_EVIDENCE_READS) { this.truncated(); break; }+ examined.add(id);+ const node = await reader.getNode(id);+ if (node && !node.stale && nodeNames(node).some((value) => value.includes(name) || name.includes(value))) partialCandidates.set(id, node);+ }+ }+ }+ const orderedPartials = [...partialCandidates.values()].sort((a, b) => compareText(normalizeGraphIndexText(a.name), normalizeGraphIndexText(b.name)) || compareText(a.id, b.id));+ for (const node of orderedPartials) {+ if (selected.size >= MAX_START_NODES) { this.truncated(); break; }+ if (await this.backed(reader, node)) selected.set(node.id, node);+ }+ const nodes = [...selected.values()].sort((a, b) =>+ Number(nodeNames(b).some((n) => exactNames.has(n))) - Number(nodeNames(a).some((n) => exactNames.has(n))) || compareText(normalizeGraphIndexText(a.name), normalizeGraphIndexText(b.name)) || compareText(a.id, b.id));+ const results: GraphRetrievalResult[] = [];+ const seen = new Set<string>();+ for (const node of nodes) await this.append(reader, node, results, seen, 1, 0, targetLayer);+ for (const node of nodes) {+ for (const path of await this.traverse(reader, node, maxDepth)) {+ const weights = path.slice(1).map((step) => step.edge!.weight);+ const score = weights.reduce((sum, weight) => sum + weight, 0) / weights.length / path.length;+ await this.append(reader, path[path.length - 1].node, results, seen, score, path.length, targetLayer);+ }+ }+ results.sort((a, b) => b.score - a.score);+ return this.finish(reader, results.slice(0, maxResults));+ });}- async temporalQuery(- entityName: string,- asOf?: string,- ): Promise<{- entity: GraphNode | null;- currentState: GraphEdge[];- history: GraphEdge[];- }> {- const { nodes: allNodes, edges: allEdges } = await this.loadGraph();-- const entity = allNodes.find(- (n) => n.name.toLowerCase() === entityName.toLowerCase(),- );- if (!entity) return { entity: null, currentState: [], history: [] };-- const relatedEdges = allEdges.filter(- (e) => e.sourceNodeId === entity.id || e.targetNodeId === entity.id,- );-- if (!asOf) {- const latestEdges = this.getLatestEdges(relatedEdges);- const historicalEdges = relatedEdges.filter(- (e) => !latestEdges.some((le) => le.id === e.id),- );- return { entity, currentState: latestEdges, history: historicalEdges };- }-- const asOfDate = new Date(asOf).getTime();- const validEdges = relatedEdges.filter((e) => {- const commitDate = new Date(e.tcommit || e.createdAt).getTime();- if (commitDate > asOfDate) return false;- if (e.tvalid) {- const validDate = new Date(e.tvalid).getTime();- if (validDate > asOfDate) return false;+ async expandFromChunks(obsIds: string[], maxDepth = 1, maxResults = 10, targetLayer: SearchLayer = "all"): Promise<GraphRetrievalResult[]> {+ if (!obsIds.length || maxResults <= 0) return [];+ const ids = [...new Set(obsIds.slice(0, MAX_START_NODES))];+ return this.run(JSON.stringify(["chunks", ids, maxDepth, maxResults, targetLayer]), async () => {+ const reader = await this.load();+ if (!reader) return [];+ const nodes = new Map<string, GraphNode>();+ for (const id of ids) {+ if (!await this.source(reader, id)) continue;+ const page = await reader.posting("observation", id, { limit: MAX_START_NODES });+ if (page.cursor) this.truncated();+ for (const nodeId of page.ids) {+ const node = await reader.getNode(nodeId);+ if (node && !node.stale && await this.backed(reader, node)) nodes.set(nodeId, node);+ if (nodes.size >= MAX_START_NODES) { this.truncated(); break; }+ }+ if (nodes.size >= MAX_START_NODES) break;}- if (e.tvalidEnd) {- const endDate = new Date(e.tvalidEnd).getTime();- if (endDate < asOfDate) return false;+ const results: GraphRetrievalResult[] = [];+ const seen = new Set(ids);+ for (const node of [...nodes.values()].sort((a, b) => compareText(a.id, b.id))) {+ for (const path of await this.traverse(reader, node, maxDepth)) {+ await this.append(reader, path[path.length - 1].node, results, seen, 0.5 / (path.length + 1), path.length, targetLayer);+ }}- return true;+ results.sort((a, b) => b.score - a.score);+ return this.finish(reader, results.slice(0, maxResults));});-- return {- entity,- currentState: this.getLatestEdges(validEdges),- history: validEdges,- };}- private getLatestEdges(edges: GraphEdge[]): GraphEdge[] {- const byKey = new Map<string, GraphEdge[]>();- for (const e of edges) {- const key = `${e.sourceNodeId}|${e.targetNodeId}|${e.type}`;- if (!byKey.has(key)) byKey.set(key, []);- byKey.get(key)!.push(e);+ async temporalQuery(entityName: string, asOf?: string): Promise<{ entity: GraphNode | null; currentState: GraphEdge[]; history: GraphEdge[] }> {+ const empty = { entity: null, currentState: [], history: [] };+ const reader = await this.load();+ if (!reader) return empty;+ const page = await reader.posting("name", normalizeGraphIndexText(entityName), { limit: MAX_START_NODES });+ let entity: GraphNode | null = null;+ for (const id of page.ids) {+ const node = await reader.getNode(id);+ if (node && !node.stale) { entity = node; break; }}-- const latest: GraphEdge[] = [];- for (const group of byKey.values()) {- if (group.length === 0) continue;- group.sort(- (a, b) =>- new Date(b.tcommit || b.createdAt).getTime() -- new Date(a.tcommit || a.createdAt).getTime(),- );- const newest = group.find((e) => e.isLatest !== false) || group[0];- latest.push(newest);+ if (!entity) return empty;+ const adjacent = await reader.posting("adjacency", entity.id, { limit: MAX_CANDIDATES });+ const historyPage = await reader.posting("history", entity.id, { limit: MAX_CANDIDATES });+ if (adjacent.cursor || historyPage.cursor) this.truncated();+ const edges = new Map<string, GraphEdge>();+ const keepEdge = async (edge: GraphEdge | null) => {+ if (!edge || edge.stale || (edge.sourceNodeId !== entity!.id && edge.targetNodeId !== entity!.id)) return;+ const neighbor = await reader.getNode(edge.sourceNodeId === entity!.id ? edge.targetNodeId : edge.sourceNodeId);+ if (neighbor && !neighbor.stale && !edges.has(edge.id)) edges.set(edge.id, edge);+ };+ for (const id of adjacent.ids) await keepEdge(await reader.getEdge(id));+ for (const id of historyPage.ids) await keepEdge(await reader.getHistoryEdge(id));+ const at = asOf === undefined ? undefined : Date.parse(asOf);+ if (at !== undefined && !Number.isFinite(at)) return empty;+ const valid = [...edges.values()].filter((edge) => at === undefined ||+ (Date.parse(edge.tcommit || edge.createdAt) <= at && (!edge.tvalid || Date.parse(edge.tvalid) <= at) && (!edge.tvalidEnd || Date.parse(edge.tvalidEnd) >= at)));+ const latest = new Map<string, GraphEdge>();+ for (const edge of valid) {+ if (at === undefined && edge.isLatest === false) continue;+ const key = `${edge.sourceNodeId}|${edge.targetNodeId}|${edge.type}`;+ const prior = latest.get(key);+ if (!prior || Date.parse(edge.tcommit || edge.createdAt) > Date.parse(prior.tcommit || prior.createdAt)) latest.set(key, edge);}- return latest;+ if (!await reader.finish()) return empty;+ const currentState = [...latest.values()];+ return { entity, currentState, history: at === undefined ? valid.filter((edge) => !currentState.includes(edge)) : valid };}- private dijkstraTraversal(- startNode: GraphNode,- graph: GraphView,- maxDepth: number,- budget: TraversalBudget,- ): Array<Array<{ node: GraphNode; edge?: GraphEdge }>> {- if (budget.nodes <= 0 || budget.edges <= 0 || maxDepth <= 0) return [];- budget.nodes--;- const { byId: nodeIndex, adjacency } = graph;-- const dist = new Map<string, number>();- const pathTo = new Map<string, Array<{ node: GraphNode; edge?: GraphEdge }>>();- dist.set(startNode.id, 0);- pathTo.set(startNode.id, [{ node: startNode }]);-- const heap = new MinHeap<{ nodeId: string; depth: number; cost: number }>(- (a, b) => a.cost - b.cost,- );- heap.push({ nodeId: startNode.id, depth: 0, cost: 0 });-- while (heap.size() > 0 && budget.edges > 0) {- const { nodeId, depth, cost } = heap.pop()!;- // Skip stale heap entries (cost beaten by a later push).- if (cost > (dist.get(nodeId) ?? Infinity)) continue;- if (depth >= maxDepth) continue;-- const neighbors = adjacency.get(nodeId) ?? [];- for (const { neighborId, edge } of neighbors) {- if (budget.edges <= 0) break;- budget.edges--;- const nextNode = nodeIndex.get(neighborId);- if (!nextNode) continue;- // Clamp weight to avoid division-by-zero on malformed edges;- // 0.01 is below the documented 0.1 floor.- const edgeCost = 1 / Math.max(edge.weight, 0.01);- const newCost = cost + edgeCost;- if (newCost < (dist.get(neighborId) ?? Infinity)) {- if (!dist.has(neighborId)) {- if (budget.nodes <= 0) continue;- budget.nodes--;- }- dist.set(neighborId, newCost);- pathTo.set(neighborId, [- ...pathTo.get(nodeId)!,- { node: nextNode, edge },- ]);- heap.push({ nodeId: neighborId, depth: depth + 1, cost: newCost });- }+ private async traverse(reader: GraphIndexReader, start: GraphNode, maxDepth: number): Promise<Path[]> {+ const depthLimit = Math.max(0, Math.min(8, Math.floor(maxDepth)));+ if (!depthLimit || this.states >= MAX_STATES) return [];+ const stateKey = (id: string, depth: number) => JSON.stringify([id, depth]);+ const initial = stateKey(start.id, 0);+ const distance = new Map<string, number>([[initial, 0]]);+ const paths = new Map<string, Path>([[initial, [{ node: start }]]]);+ const best = new Map<string, { cost: number; path: Path }>();+ const heap = new MinHeap<{ id: string; depth: number; cost: number }>((a, b) => a.cost - b.cost || a.depth - b.depth || compareText(a.id, b.id));+ heap.push({ id: start.id, depth: 0, cost: 0 });+ this.states++;+ while (heap.size() && this.edges < MAX_EDGES && !reader.exhausted && reader.remainingReads > FINAL_EVIDENCE_READS) {+ this.controller.signal.throwIfAborted();+ const current = heap.pop()!;+ const key = stateKey(current.id, current.depth);+ if (current.cost > (distance.get(key) ?? Infinity) || current.depth >= depthLimit) continue;+ const page = await reader.posting("adjacency", current.id, { limit: MAX_CANDIDATES });+ if (page.cursor) this.truncated();+ const adjacent: GraphEdge[] = [];+ for (const id of page.ids) {+ if (reader.remainingReads <= FINAL_EVIDENCE_READS) { this.truncated(); break; }+ if (++this.edges > MAX_EDGES) { this.truncated(); break; }+ const edge = await reader.getEdge(id);+ if (edge && !edge.stale && edge.isLatest !== false && Number.isFinite(edge.weight)) adjacent.push(edge);+ }+ adjacent.sort((a, b) => b.weight - a.weight || compareText(a.id, b.id));+ for (const edge of adjacent) {+ if (reader.remainingReads <= FINAL_EVIDENCE_READS) { this.truncated(); break; }+ if (edge.sourceNodeId !== current.id && edge.targetNodeId !== current.id) continue;+ const id = edge.sourceNodeId === current.id ? edge.targetNodeId : edge.sourceNodeId;+ const path = paths.get(key)!;+ if (path.some((step) => step.node.id === id)) continue;+ const node = await reader.getNode(id);+ if (!node || node.stale || !await this.backed(reader, edge) || !await this.backed(reader, node)) continue;+ const cost = current.cost + 1 / Math.max(edge.weight, 0.01);+ const nextKey = stateKey(id, current.depth + 1);+ if (cost >= (distance.get(nextKey) ?? Infinity)) continue;+ if (!distance.has(nextKey) && ++this.states > MAX_STATES) { this.truncated(); continue; }+ const nextPath = [...path, { node, edge }];+ distance.set(nextKey, cost);+ paths.set(nextKey, nextPath);+ if (cost < (best.get(id)?.cost ?? Infinity)) best.set(id, { cost, path: nextPath });+ heap.push({ id, depth: current.depth + 1, cost });}}-- // Drop the startNode's own entry before returning: callers- // (searchByEntities, expandFromChunks) score start-node- // observations via a dedicated fallback loop with score=1.0. If- // we leave it in here, the start-path (length 1, no edges) goes- // through the generic path-scoring loop first — pathLength=1 +- // empty edgeWeights makes avgWeight fall to 0.5, the obs get- // marked visited, and the score=1.0 fallback becomes dead code.- pathTo.delete(startNode.id);- return Array.from(pathTo.values());+ if (reader.exhausted || heap.size()) this.truncated();+ return [...best.values()].map((entry) => entry.path);}}diff --git a/src/functions/graph.ts b/src/functions/graph.tsindex 993e87c2..0dfbad0c 100644--- a/src/functions/graph.ts+++ b/src/functions/graph.ts@@ -1,3 +1,5 @@+import { getGraphIndexStatus } from "../state/graph-retrieval-index.js";+import { queryGraphIndex } from "./graph-index-query.js";import { isLegacyGraphState, pinGraphState, resetGraphGeneration } from "../state/graph-generation.js";import type { IIIClient } from "iii-sdk";import type {@@ -21,16 +23,11 @@ import { logger } from "../logger.js";import { scrubRecord } from "./privacy.js";const DEFAULT_GRAPH_QUERY_LIMIT = 500;-const MAX_GRAPH_QUERY_LIMIT = 5000;const SNAPSHOT_TOP_NODES = DEFAULT_GRAPH_QUERY_LIMIT;const SNAPSHOT_KEY = "current";-// `state::list` over a 75K-node scope can exceed the iii invocation-// timeout. The query handler races the enumeration against this budget-// and falls back to the snapshot (or a warning envelope) when the live-// path is too slow. 6000ms leaves headroom under the default 8s engine-// invocation deadline.+// This deadline applies only to explicitly requested legacy snapshot rebuilds.const LIVE_ENUMERATION_BUDGET_MS = 6000;function withTimeout<T>(p: Promise<T>, ms: number, label: string): Promise<T> {@@ -154,44 +151,6 @@ function buildSnapshotFromArrays(};}-function paginateFromSnapshot(- snap: GraphSnapshot,- filterType: string | undefined,- limit: number,- offset: number,-): GraphQueryResult {- const filteredNodes = filterType- ? snap.topNodes.filter((n) => n.type === filterType)- : snap.topNodes;- const totalRaw = filterType- ? snap.stats.nodesByType[filterType] ?? 0- : snap.stats.totalNodes;- const total = Math.max(totalRaw, filteredNodes.length);- const pageNodes = filteredNodes.slice(offset, offset + limit);- const pageIds = new Set(pageNodes.map((n) => n.id));- const pageEdges = snap.topEdges.filter(- (e) => pageIds.has(e.sourceNodeId) && pageIds.has(e.targetNodeId),- );- const degrees: Record<string, number> = {};- for (const n of pageNodes) {- const d = snap.topDegrees?.[n.id];- if (typeof d === "number") degrees[n.id] = d;- }- return {- nodes: pageNodes,- edges: pageEdges,- depth: 0,- totalNodes: total,- totalEdges: snap.stats.totalEdges,- truncated: total > pageNodes.length || snap.sampleIncomplete === true,- limit,- offset,- fromSnapshot: true,- degrees,- ...(snap.sampleIncomplete ? { warning: "Graph counts are current; the bounded snapshot sample may omit nodes or edges after replacements. Use a graph search for specific entities." } : {}),- };-}-const REBUILD_SAFE_NODE_CEILING = 25000;function nameIndexKey(type: string, name: string): string {@@ -327,62 +286,6 @@ function mergeEdge(existing: GraphEdge, incoming: GraphEdge): GraphEdge {};}-function resolvePagination(- rawLimit: number | undefined,- rawOffset: number | undefined,-): { limit: number; offset: number } {- const requested = typeof rawLimit === "number" && Number.isFinite(rawLimit)- ? Math.floor(rawLimit)- : DEFAULT_GRAPH_QUERY_LIMIT;- const limit = Math.max(1, Math.min(requested, MAX_GRAPH_QUERY_LIMIT));- const offset = Math.max(- 0,- typeof rawOffset === "number" && Number.isFinite(rawOffset)- ? Math.floor(rawOffset)- : 0,- );- return { limit, offset };-}--function paginate(- nodes: GraphNode[],- allEdges: GraphEdge[],- depth: number,- limit: number,- offset: number,-): GraphQueryResult {- const totalNodes = nodes.length;- const pageNodes = nodes.slice(offset, offset + limit);- const pageNodeIds = new Set(pageNodes.map((n) => n.id));- // Edges restricted to the page so the response payload scales with- // `limit`, not with the global edge count. An edge is included only- // when BOTH endpoints land in the page — half-edges to nodes outside- // the page would render as dangling links in the viewer.- const pageEdges = allEdges.filter(- (e) => pageNodeIds.has(e.sourceNodeId) && pageNodeIds.has(e.targetNodeId),- );- // Total edges (for the same node universe). Counted unbounded so the- // viewer can show "showing X of Y" without re-querying.- const universeIds = new Set(nodes.map((n) => n.id));- const totalEdges = allEdges.reduce(- (count, e) =>- universeIds.has(e.sourceNodeId) && universeIds.has(e.targetNodeId)- ? count + 1- : count,- 0,- );- return {- nodes: pageNodes,- edges: pageEdges,- depth,- totalNodes,- totalEdges,- truncated: totalNodes > pageNodes.length,- limit,- offset,- };-}-function parseAttrs(raw: string): Record<string, string> {const attrs: Record<string, string> = {};const attrRegex = /([A-Za-z_][\w:-]*)="([^"]*)"/g;@@ -482,7 +385,7 @@ export function extractGraphHeuristics(): GraphNode | null => {const trimmed = name.trim();if (!trimmed) return null;- const key = `${type} ${trimmed.toLowerCase()}`;+ const key = `${type}\u0000${trimmed.toLowerCase()}`;let node = nodeByKey.get(key);if (!node) {node = {@@ -911,167 +814,29 @@ export function registerGraphFunction(limit?: number;offset?: number;}): Promise<GraphQueryResult> => {- const kv = await pinGraphState(state);- const maxDepth = Math.min(data.maxDepth || 3, 5);- const { limit, offset } = resolvePagination(data.limit, data.offset);-- const noWalk = !data.query && !data.startNodeId;- if (noWalk) {- const snap = await readSnapshot(kv);- if (snap && snap.stats.totalNodes > 0) {- return paginateFromSnapshot(snap, data.nodeType, limit, offset);- }- return {- nodes: [],- edges: [],- depth: 0,- totalNodes: 0,- totalEdges: 0,- truncated: false,- limit,- offset,- warning:- "No graph snapshot available. Either no graph has been " +- "extracted yet, or you are on a legacy corpus from an older " +- "agentmemory build. Run POST /agentmemory/graph/snapshot-rebuild " +- "(safe up to ~25K nodes) or POST /agentmemory/graph/reset to " +- "wipe and let future extracts repopulate.",- };- }-- // Query / startNodeId paths still need broader access. Race the- // live enumeration against a wall-clock budget so a long- // kv.list doesn't block the worker indefinitely. On timeout the- // caller gets a snapshot-backed approximation instead of a 500.- let allNodes: GraphNode[];- let allEdges: GraphEdge[];- try {- const [rawNodes, rawEdges] = await withTimeout(- Promise.all([- kv.list<GraphNode>(KV.graphNodes),- kv.list<GraphEdge>(KV.graphEdges),- ]),- LIVE_ENUMERATION_BUDGET_MS,- "graph-query enumeration",- );- allNodes = rawNodes.filter((n) => !n.stale);- allEdges = rawEdges.filter((e) => !e.stale);- } catch (err) {- const msg = err instanceof Error ? err.message : String(err);- logger.warn("Graph query enumeration timed out, using snapshot", {- error: msg,- });- const snap = await readSnapshot(kv);- if (snap) {- return {- ...paginateFromSnapshot(snap, data.nodeType, limit, offset),- warning:- "Live graph enumeration exceeded budget. Query / " +- "startNodeId paths degrade on >25K-node corpora until a " +- "per-node edge index lands. Result reflects top-degree " +- "snapshot, not the requested walk.",- };- }- return {- nodes: [],- edges: [],- depth: 0,- totalNodes: 0,- totalEdges: 0,- truncated: false,- limit,- offset,- warning:- "Graph enumeration exceeded budget and no snapshot is available.",- };- }-- if (data.query) {- const lower = data.query.toLowerCase();- const matchingNodes = allNodes.filter(- (n) =>- n.name.toLowerCase().includes(lower) ||- Object.values(n.properties).some(- (v) => typeof v === "string" && v.toLowerCase().includes(lower),- ),- );- return paginate(matchingNodes, allEdges, 0, limit, offset);- }-- if (data.startNodeId) {- const visited = new Set<string>();- const visitedEdges = new Set<string>();- const resultNodes: GraphNode[] = [];- const resultEdges: GraphEdge[] = [];- const queue: Array<{ nodeId: string; depth: number }> = [- { nodeId: data.startNodeId, depth: 0 },- ];-- while (queue.length > 0) {- const { nodeId, depth } = queue.shift()!;- if (visited.has(nodeId) || depth > maxDepth) continue;- visited.add(nodeId);-- const node = allNodes.find((n) => n.id === nodeId);- if (node) {- if (!data.nodeType || node.type === data.nodeType) {- resultNodes.push(node);- }- }-- const neighborEdges = allEdges.filter(- (e) => e.sourceNodeId === nodeId || e.targetNodeId === nodeId,- );- for (const edge of neighborEdges) {- if (!visitedEdges.has(edge.id)) {- visitedEdges.add(edge.id);- resultEdges.push(edge);- }- const nextId =- edge.sourceNodeId === nodeId- ? edge.targetNodeId- : edge.sourceNodeId;- if (!visited.has(nextId)) {- queue.push({ nodeId: nextId, depth: depth + 1 });- }- }- }-- return paginate(resultNodes, resultEdges, maxDepth, limit, offset);- }-- // Unreachable — noWalk branch handles the rest.- return paginate([], [], 0, limit, offset);+ return queryGraphIndex(state, data);},);sdk.registerFunction("mem::graph-stats", async () => {const kv = await pinGraphState(state);- const snap = await readSnapshot(kv);- if (snap) {+ const index = await getGraphIndexStatus(kv);+ if (!index || index.status !== "ready" || index.pending || index.batchPending) {return {- ...snap.stats,- fromSnapshot: true,- updatedAt: snap.updatedAt,- ...(snap.dirty- ? {- warning:- "Snapshot is marked dirty (write was in-flight when read). " +- "Counts are eventually consistent.",- }- : {}),+ totalNodes: null, totalEdges: null, nodesByType: {}, edgesByType: {},+ fromSnapshot: false, fromIndex: false, available: false, coverage: "unknown",+ warning: "Graph index counts are unavailable. Run agentmemory graph-index status; legacy stores require an explicit migration.",};}return {- totalNodes: 0,- totalEdges: 0,- nodesByType: {},- edgesByType: {},- fromSnapshot: false,- warning:- "No graph snapshot available. Run POST /agentmemory/graph/snapshot-rebuild " +- "(safe up to ~25K nodes) or POST /agentmemory/graph/reset to wipe " +- "and let future extracts repopulate.",+ totalNodes: index.nodes, totalEdges: index.edges,+ nodesByType: index.nodesByType, edgesByType: index.edgesByType,+ fromSnapshot: false, fromIndex: true, available: true,+ coverage: index.incompleteRecords ? "partial" : "complete",+ updatedAt: index.updatedAt,+ ...(index.incompleteRecords || index.otherTypes ? {+ warning: "Counts describe indexed live records; oversized projections may be incomplete and unrecognized types are grouped under other.",+ } : {}),};});diff --git a/src/functions/reflect.ts b/src/functions/reflect.tsindex 1dfec2c1..21ebd466 100644--- a/src/functions/reflect.ts+++ b/src/functions/reflect.ts@@ -1,4 +1,5 @@-import { pinGraphState } from "../state/graph-generation.js";+import { createGraphIndexReader } from "../state/graph-retrieval-index.js";+import { lookupObservationSession } from "../state/obs-index.js";import type { IIIClient } from "iii-sdk";import type { StateKV } from "../state/kv.js";import { KV, fingerprintId } from "../state/schema.js";@@ -10,6 +11,9 @@ import type {Lesson,Crystal,MemoryProvider,+ CompressedObservation,+ Memory,+ Session,} from "../types.js";import { recordAudit } from "./audit.js";import { REFLECT_SYSTEM, buildReflectPrompt } from "../prompts/reflect.js";@@ -103,6 +107,71 @@ function reinforceInsight(insight: Insight): void {insight.updatedAt = now;}+async function reflectionGraph(state: StateKV, project?: string): Promise<{+ nodes: GraphNode[]; edges: GraphEdge[]; diagnostic: { status: string; truncated: boolean };+}> {+ const controller = new AbortController();+ const empty = (status: string) => ({ nodes: [], edges: [], diagnostic: { status, truncated: true } });+ let timer: ReturnType<typeof setTimeout> | undefined;+ const work = (async () => {+ const reader = await createGraphIndexReader(state, { signal: controller.signal, maxReads: 256, maxBytes: 512 * 1024 });+ if (!reader) return empty("unavailable");+ const read = reader.read.bind(reader);+ const sources = new Map<string, Promise<boolean>>();+ const backed = async (row: GraphNode | GraphEdge): Promise<boolean> => {+ for (const id of row.sourceObservationIds) {+ if (reader.remainingReads <= 8) return false;+ let source = sources.get(id);+ if (!source) {+ source = (async () => {+ const memory = await read<Memory>(KV.memories, id);+ if (memory) {+ if (memory.isLatest === false) return false;+ if (!project) return true;+ if (memory.project !== undefined) return memory.project === project;+ return (await read<Session>(KV.sessions, memory.sessionIds?.[0] ?? "memory"))?.project === project;+ }+ const sessionId = await lookupObservationSession({ get: read } as StateKV, id);+ if (!sessionId) return false;+ const observation = await read<CompressedObservation>(KV.observations(sessionId), id);+ if (observation?.id !== id || observation.sessionId !== sessionId) return false;+ return !project || (await read<Session>(KV.sessions, sessionId))?.project === project;+ })();+ sources.set(id, source);+ }+ if (await source) return true;+ }+ return false;+ };+ const inventory = await reader.posting("nodes", "", { limit: 64 });+ let truncated = Boolean(inventory.cursor) || reader.status.incompleteRecords > 0;+ const nodes: GraphNode[] = [];+ for (const id of inventory.ids) {+ if (reader.remainingReads <= 32) { truncated = true; break; }+ const node = await reader.getNode(id);+ if (node && node.type === "concept" && !node.stale && await backed(node)) nodes.push(node);+ }+ const selected = new Set(nodes.map((node) => node.id));+ const edges = new Map<string, GraphEdge>();+ for (const node of nodes) {+ if (reader.remainingReads <= 8) { truncated = true; break; }+ const adjacency = await reader.posting("adjacency", node.id, { limit: 16 });+ truncated ||= Boolean(adjacency.cursor);+ for (const id of adjacency.ids) {+ const edge = await reader.getEdge(id);+ if (edge && !edge.stale && edge.isLatest !== false && selected.has(edge.sourceNodeId) && selected.has(edge.targetNodeId) && await backed(edge)) edges.set(id, edge);+ }+ }+ if (!await reader.finish()) return empty("changed");+ return { nodes, edges: [...edges.values()], diagnostic: { status: "ready", truncated: truncated || reader.exhausted } };+ })().catch(() => empty("unavailable"));+ try {+ return await Promise.race([work, new Promise<ReturnType<typeof empty>>((resolve) => {+ timer = setTimeout(() => { controller.abort(); resolve(empty("timeout")); }, 500);+ })]);+ } finally { clearTimeout(timer); }+}+function buildGraphClusters(nodes: GraphNode[],edges: GraphEdge[],@@ -115,7 +184,7 @@ function buildGraphClusters(const edgeMap = new Map<string, Set<string>>();for (const edge of edges) {- if (edge.stale) continue;+ if (edge.stale || edge.isLatest === false) continue;if (!edgeMap.has(edge.sourceNodeId))edgeMap.set(edge.sourceNodeId, new Set());if (!edgeMap.has(edge.targetNodeId))@@ -238,15 +307,14 @@ export function registerReflectFunctions(const kv = state;sdk.registerFunction("mem::reflect",async (data: { maxClusters?: number; project?: string }) => {- const kv = await pinGraphState(state);+ const kv = state;const maxClusters = Math.min(data?.maxClusters ?? 10, 20);const maxInsightsPerCluster = 5;const maxTotal = 50;- const [graphNodes, graphEdges, semanticMemories, lessons, crystals] =+ const [graph, semanticMemories, lessons, crystals] =await Promise.all([- kv.list<GraphNode>(KV.graphNodes).catch(() => []),- kv.list<GraphEdge>(KV.graphEdges).catch(() => []),+ reflectionGraph(kv, data?.project),kv.list<SemanticMemory>(KV.semantic).catch(() => []),kv.list<Lesson>(KV.lessons).catch(() => []),kv.list<Crystal>(KV.crystals).catch(() => []),@@ -258,8 +326,8 @@ export function registerReflectFunctions(}let conceptClusters = buildGraphClusters(- graphNodes,- graphEdges,+ graph.nodes,+ graph.edges,maxClusters,);@@ -425,6 +493,7 @@ export function registerReflectFunctions(usedFallback,rejectedInsights,failedClusters,+ graph: { ...graph.diagnostic, nodes: graph.nodes.length, edges: graph.edges.length },};},);diff --git a/src/functions/search.ts b/src/functions/search.tsindex 7e1c4885..88a8078c 100644--- a/src/functions/search.ts+++ b/src/functions/search.ts@@ -1,4 +1,5 @@import type { IIIClient } from 'iii-sdk'+import type { GraphObservationAuthorization } from "./graph-retrieval.js";import type { IndexPersistenceStatus } from "../state/index-persistence.js";import type { CompressedObservation, Memory, SearchResult, Session } from '../types.js'import { KV } from '../state/schema.js'@@ -31,6 +32,7 @@ type HybridRanker = (query: string,limit: number,targetLayer?: SearchLayer,+ authorizeGraph?: GraphObservationAuthorization,) => Promise<Array<{ observation: CompressedObservation; sessionId: string; combinedScore: number; layer?: SearchResultLayer }>>let hybridRanker: HybridRanker | null = null@@ -816,9 +818,21 @@ export function registerSearchFunction(sdk: IIIClient, kv: StateKV): void {observation?: CompressedObservationlayer?: SearchResultLayer}>+ const authorizeGraph: GraphObservationAuthorization = async (record, sessionId, read) => {+ if (filterAgentId !== undefined && record.agentId !== filterAgentId) return false+ const memoryProject = "content" in record ? record.project : undefined+ if (projectFilter && memoryProject !== undefined && memoryProject !== projectFilter) return false+ if (cwdFilter || (projectFilter && memoryProject === undefined)) {+ const session = await read<Session>(KV.sessions, sessionId)+ if (!session) return false+ if (cwdFilter && session.cwd !== cwdFilter) return false+ if (projectFilter && memoryProject === undefined && session.project !== projectFilter) return false+ }+ return true+ }if (hybridRanker) {try {- const hybrid = await hybridRanker(query, fetchLimit, targetLayer)+ const hybrid = await hybridRanker(query, fetchLimit, targetLayer, authorizeGraph)results = hybrid.map((r) => ({obsId: r.observation.id,sessionId: r.sessionId,diff --git a/src/functions/smart-search.ts b/src/functions/smart-search.tsindex f2d4352d..19318b06 100644--- a/src/functions/smart-search.ts+++ b/src/functions/smart-search.ts@@ -1,3 +1,4 @@+import type { GraphObservationAuthorization } from "./graph-retrieval.js";import type { IIIClient } from "iii-sdk";import type {CompactLessonResult,@@ -79,7 +80,7 @@ const LESSON_CONTENT_PREVIEW_CHARS = 240;export function registerSmartSearchFunction(sdk: IIIClient,kv: StateKV,- searchFn: (query: string, limit: number, targetLayer?: SearchLayer) => Promise<HybridSearchResult[]>,+ searchFn: (query: string, limit: number, targetLayer?: SearchLayer, authorizeGraph?: GraphObservationAuthorization) => Promise<HybridSearchResult[]>,): void {sdk.registerFunction("mem::smart-search",async (data: {@@ -197,8 +198,14 @@ export function registerSmartSearchFunction(? Math.max(Math.min(limit * 10, 300), 100): limit;+ const authorizeGraph: GraphObservationAuthorization = async (record, sessionId, read) => {+ if (filterAgentId !== undefined && record.agentId !== filterAgentId) return false;+ if (!project) return true;+ if ("content" in record && record.project !== undefined) return record.project === project;+ return (await read<{ project?: string }>(KV.sessions, sessionId))?.project === project;+ };const [hybridResults, lessons] = await Promise.all([- searchFn(data.query, overFetchLimit, targetLayer),+ searchFn(data.query, overFetchLimit, targetLayer, authorizeGraph),includeLessons? recallLessons(sdk, data.query, lessonLimit, project): Promise.resolve([]),diff --git a/src/functions/status.ts b/src/functions/status.tsindex a785faa0..dfbe31b2 100644--- a/src/functions/status.ts+++ b/src/functions/status.ts@@ -2,6 +2,7 @@ import type { IndexLegStatus, IndexPersistenceStatus } from "../state/index-persimport type { VectorBackfillState } from "./search.js";import { describeGraphCompactBoot, type GraphCompactBootStatus } from "./graph-compact-boot.js";import type { CaptureStatus } from "./capture.js";+import type { GraphIndexMigrationStatus } from "../state/graph-index-migration.js";export type StatusLevel = "ok" | "info" | "warn" | "error";@@ -29,12 +30,15 @@ export interface StatusFlag {}export interface GraphStatsInput {- totalNodes?: number;- totalEdges?: number;+ totalNodes?: number | null;+ totalEdges?: number | null;+ available?: boolean;+ coverage?: "complete" | "partial" | "unknown";fromSnapshot?: boolean;updatedAt?: string;dirty?: boolean;warning?: string;+ fromIndex?: boolean;}export interface StatusInputs {@@ -75,6 +79,7 @@ export interface StatusInputs {graph: GraphStatsInput | null;graphExtractionEnabled: boolean;graphCompaction?: GraphCompactBootStatus | null;+ graphIndex?: GraphIndexMigrationStatus | null;auditLegacy: { status: string; sizeBytes?: number } | null;indexPersistence?: IndexPersistenceStatus | null;stateStore?: { ok: boolean; latencyMs?: number } | null;@@ -117,6 +122,7 @@ export interface StatusReport {indexPersistence: IndexPersistenceStatus | null;graph: (GraphStatsInput & { ageSeconds: number | null; extractionEnabled: boolean }) | null;graphCompaction: GraphCompactBootStatus | null;+ graphIndex: GraphIndexMigrationStatus | null;capture: CaptureStatus | null;observeDedup: ObserveDedupStatus | null;functions: Array<FunctionMetricInput & { failureRate: number; offWithoutLlm: boolean }>;@@ -391,14 +397,14 @@ export function evaluateStatus(input: StatusInputs): StatusReport {if (input.graph) {const ageSeconds = secondsBetween(input.now, input.graph.updatedAt);graph = { ...input.graph, ageSeconds, extractionEnabled: input.graphExtractionEnabled };- if (input.graphExtractionEnabled && !input.graph.fromSnapshot) {+ if (input.graphExtractionEnabled && !input.graph.fromSnapshot && !input.graph.fromIndex && !input.graphIndex) {problems.push({level: "warn",code: "graph-no-snapshot",- message: "Graph extraction is on but no graph snapshot exists, so graph counts read as zero.",+ message: "Graph extraction is on but no graph snapshot exists, so graph counts are unavailable.",fix: "Use Rebuild Graph in the viewer, or POST /agentmemory/graph/snapshot-rebuild.",});- } else if (input.graph.fromSnapshot && ageSeconds !== null && ageSeconds > GRAPH_SNAPSHOT_STALE_SECONDS) {+ } else if (!input.graph.fromIndex && input.graph.fromSnapshot && ageSeconds !== null && ageSeconds > GRAPH_SNAPSHOT_STALE_SECONDS) {const age = ageSeconds >= 2 * 86400 ? `${Math.floor(ageSeconds / 86400)} days` : `${Math.round(ageSeconds / 3600)} hours`;problems.push({level: "info",@@ -417,6 +423,20 @@ export function evaluateStatus(input: StatusInputs): StatusReport {}}+ const graphIndex = input.graphIndex ?? null;+ if (input.graphIndex === null || graphIndex?.state === "unavailable") {+ problems.push({ level: "info", code: "graph-index-unavailable", message: "The bounded graph index could not be checked. Keyword and vector search remain available.", fix: "Run agentmemory graph-index status and check the state store." });+ } else if (graphIndex && !graphIndex.ready) {+ problems.push({+ level: graphIndex.state === "paused" ? "warn" : "info",+ code: "graph-index-not-ready",+ message: `Graph retrieval is unavailable while its index is ${graphIndex.state}. Keyword search, vector search and capture remain available.`,+ fix: graphIndex.action,+ });+ } else if (graphIndex?.coverage === "partial") {+ problems.push({ level: "warn", code: "graph-index-partial", message: `${graphIndex.incompleteRecords ?? "Some"} graph records exceed index limits; graph retrieval has partial coverage.`, fix: graphIndex.action });+ }+const compaction = input.graphCompaction ?? null;if (compaction?.state === "running") {problems.push({@@ -502,6 +522,7 @@ export function evaluateStatus(input: StatusInputs): StatusReport {indexPersistence: persistence,graph,graphCompaction: compaction,+ graphIndex,capture: input.capture ?? null,observeDedup: input.observeDedup ?? null,functions,@@ -861,10 +882,11 @@ ${report.observeDedup ? row("Repeats skipped", escapeHtml(`${report.observeDedup<h2>Knowledge graph</h2><table>${graph? row("Extraction", graph.extractionEnabled ? "on" : "off") +- row("Nodes / edges", escapeHtml(`${graph.totalNodes ?? 0} / ${graph.totalEdges ?? 0}`)) +- row("Snapshot age", escapeHtml(graph.fromSnapshot ? formatDuration(graph.ageSeconds) : "no snapshot"))+ row("Nodes / edges", escapeHtml(`${graph.totalNodes ?? "unknown"} / ${graph.totalEdges ?? "unknown"}`)) ++ row(graph.fromIndex ? "Count source" : "Snapshot age", escapeHtml(graph.fromIndex ? "maintained index" : graph.fromSnapshot ? formatDuration(graph.ageSeconds) : "no snapshot")): row("Graph", "unavailable")}${report.graphCompaction ? row("Provenance compaction", escapeHtml(describeGraphCompactBoot(report.graphCompaction))) : ""}+${report.graphIndex ? row("Retrieval index", escapeHtml(`${report.graphIndex.state}, ${report.graphIndex.coverage} coverage`)) : ""}</table><h2>Features</h2><table class="list"><tr><th>Feature</th><th>Setting</th><th>State</th><th>How to change</th></tr>${flags}</table><h2>Process</h2><table>diff --git a/src/functions/temporal-graph.ts b/src/functions/temporal-graph.tsindex cdf27f6b..1a7a5302 100644--- a/src/functions/temporal-graph.ts+++ b/src/functions/temporal-graph.ts@@ -1,3 +1,6 @@+import { createGraphIndexReader, graphEdgeLookupKey, normalizeGraphIndexText } from "../state/graph-retrieval-index.js";+import { persistGraphRecords } from "../state/graph-ingestion.js";+import { readIndexedEntity } from "./graph-index-temporal.js";import { pinGraphState } from "../state/graph-generation.js";import type { IIIClient } from "iii-sdk";import type {@@ -191,15 +194,19 @@ export function registerTemporalGraphFunctions(const { nodes, edges } = parseTemporalGraphXml(response, obsIds);await withKeyedLock("graph:persist", async () => {- const existingNodes = await kv.list<GraphNode>(KV.graphNodes);- const existingEdges = await kv.list<GraphEdge>(KV.graphEdges);-const idRemap = new Map<string, string>();for (const node of nodes) {- const existing = existingNodes.find(- (n) =>- n.name === node.name && n.type === node.type,- );+ const reader = await createGraphIndexReader(kv);+ if (!reader) throw new Error("Graph index unavailable; run agentmemory graph-index migrate --allow-legacy-scan");+ const names = await reader.posting("name", normalizeGraphIndexText(node.name), { limit: 64 });+ let existing: GraphNode | null = null;+ for (const id of names.ids) {+ const candidate = await reader.getNode(id);+ if (candidate && !candidate.stale && candidate.name === node.name && candidate.type === node.type && (!existing || candidate.id < existing.id)) {+ existing = await kv.get<GraphNode>(KV.graphNodes, id);+ }+ }+ if (names.cursor || reader.exhausted || !await reader.finish()) throw new Error("Temporal node reconciliation exceeded the graph read budget or changed revision");if (existing) {const oldId = node.id;const merged = {@@ -218,13 +225,12 @@ export function registerTemporalGraphFunctions(],};if (merged.aliases.length === 0) delete (merged as any).aliases;- await kv.set(KV.graphNodes, existing.id, merged);+ await persistGraphRecords(kv, [merged], []);node.id = existing.id;idRemap.set(oldId, existing.id);} else {node.sourceObservationIds = boundSources([], node.sourceObservationIds ?? []);- await kv.set(KV.graphNodes, node.id, node);- existingNodes.push(node);+ await persistGraphRecords(kv, [node], []);}}@@ -235,13 +241,30 @@ export function registerTemporalGraphFunctions(if (idRemap.has(edge.targetNodeId)) {edge.targetNodeId = idRemap.get(edge.targetNodeId)!;}- const existingKey = `${edge.sourceNodeId}|${edge.targetNodeId}|${edge.type}`;- const existingEdge = existingEdges.find(- (e) =>- `${e.sourceNodeId}|${e.targetNodeId}|${e.type}` ===- existingKey,- );-+ const reader = await createGraphIndexReader(kv);+ if (!reader) throw new Error("Graph index unavailable during temporal reconciliation");+ let cursor: string | undefined;+ let existingEdge: GraphEdge | null = null;+ do {+ const page = await reader.posting("edgeKey", graphEdgeLookupKey(edge), { cursor, limit: 64 });+ for (const id of page.ids) {+ const candidate = await reader.getEdge(id);+ if (candidate && !candidate.stale && candidate.isLatest !== false &&+ candidate.sourceNodeId === edge.sourceNodeId && candidate.targetNodeId === edge.targetNodeId && candidate.type === edge.type &&+ (!existingEdge || (candidate.version ?? 1) > (existingEdge.version ?? 1) ||+ (candidate.version ?? 1) === (existingEdge.version ?? 1) && candidate.id < existingEdge.id)) {+ const canonical = await reader.read<GraphEdge>(KV.graphEdges, id);+ if (reader.exhausted) break;+ if (!canonical || canonical.id !== id || canonical.stale || canonical.isLatest === false || graphEdgeLookupKey(canonical) !== graphEdgeLookupKey(edge)) {+ throw new Error("Temporal edge reconciliation found a missing or changed canonical predecessor");+ }+ existingEdge = canonical;+ }+ }+ cursor = page.cursor ?? undefined;+ } while (cursor && !reader.exhausted);+ if (cursor || reader.exhausted || !await reader.finish()) throw new Error("Temporal edge reconciliation exceeded the graph read budget or changed revision");+ edge.sourceObservationIds = boundSources([], edge.sourceObservationIds ?? []);if (existingEdge) {const updatedOld = {...existingEdge,@@ -250,16 +273,12 @@ export function registerTemporalGraphFunctions(existingEdge.tvalidEnd || new Date().toISOString(),supersededBy: edge.id,};- await kv.set(KV.graphEdges, existingEdge.id, updatedOld);-- await kv.set(KV.graphEdgeHistory, existingEdge.id, updatedOld);-edge.version = (existingEdge.version || 1) + 1;+ await persistGraphRecords(kv, [], [updatedOld, edge]);+ await kv.set(KV.graphEdgeHistory, existingEdge.id, updatedOld);+ } else {+ await persistGraphRecords(kv, [], [edge]);}-- edge.sourceObservationIds = boundSources([], edge.sourceObservationIds ?? []);- await kv.set(KV.graphEdges, edge.id, edge);- existingEdges.push(edge);}});@@ -285,37 +304,13 @@ export function registerTemporalGraphFunctions(entityName: string;asOf?: string;includeHistory?: boolean;- }): Promise<TemporalState | { error: string }> => {+ }): Promise<(TemporalState & { truncated: boolean }) | { error: string }> => {const kv = await pinGraphState(state);- const allNodes = await kv.list<GraphNode>(KV.graphNodes);- const allEdges = await kv.list<GraphEdge>(KV.graphEdges);-- const entity = allNodes.find(- (n) =>- n.name.toLowerCase() === data.entityName.toLowerCase() ||- (n.aliases &&- n.aliases.some(- (a) =>- a.toLowerCase() === data.entityName.toLowerCase(),- )),- );-- if (!entity) {- return { error: `Entity "${data.entityName}" not found` } as any;- }-- const relatedEdges = allEdges.filter(- (e) => e.sourceNodeId === entity.id || e.targetNodeId === entity.id,- );-- const historicalEdges = await kv- .list<GraphEdge>(KV.graphEdgeHistory)- .catch(() => [] as GraphEdge[]);- const entityHistory = historicalEdges.filter(- (e) => e.sourceNodeId === entity.id || e.targetNodeId === entity.id,- );-- const allEntityEdges = [...relatedEdges, ...entityHistory];+ if (data.asOf !== undefined && !Number.isFinite(Date.parse(data.asOf))) return { error: "Invalid asOf timestamp" };+ const indexed = await readIndexedEntity(kv, data.entityName);+ if ("error" in indexed) return indexed;+ const { entity, relatedEdges, entityHistory, truncated } = indexed;+ const allEntityEdges = [...new Map([...entityHistory, ...relatedEdges].map(edge => [edge.id, edge])).values()];if (data.asOf) {const asOfTime = new Date(data.asOf).getTime();@@ -340,6 +335,7 @@ export function registerTemporalGraphFunctions(return {entity,+ truncated,currentEdges,historicalEdges: historical,timeline: buildTimeline(allEntityEdges),@@ -352,8 +348,9 @@ export function registerTemporalGraphFunctions(return {entity,+ truncated,currentEdges,- historicalEdges: data.includeHistory ? entityHistory : [],+ historicalEdges: data.includeHistory ? allEntityEdges.filter(edge => edge.isLatest === false) : [],timeline: buildTimeline(allEntityEdges),};},@@ -366,27 +363,13 @@ export function registerTemporalGraphFunctions(to?: string;}) => {const kv = await pinGraphState(state);- const allNodes = await kv.list<GraphNode>(KV.graphNodes);- const allEdges = await kv.list<GraphEdge>(KV.graphEdges);- const historicalEdges = await kv- .list<GraphEdge>(KV.graphEdgeHistory)- .catch(() => [] as GraphEdge[]);-- const entity = allNodes.find(- (n) => n.name.toLowerCase() === data.entityName.toLowerCase(),- );- if (!entity) return { error: "Entity not found" };-- const allEntityEdges = [- ...allEdges.filter(- (e) =>- e.sourceNodeId === entity.id || e.targetNodeId === entity.id,- ),- ...historicalEdges.filter(- (e) =>- e.sourceNodeId === entity.id || e.targetNodeId === entity.id,- ),- ];+ if ((data.from !== undefined && !Number.isFinite(Date.parse(data.from))) ||+ (data.to !== undefined && !Number.isFinite(Date.parse(data.to)))) return { error: "Invalid temporal range" };+ if (data.from && data.to && Date.parse(data.from) > Date.parse(data.to)) return { error: "Temporal range is reversed" };+ const indexed = await readIndexedEntity(kv, data.entityName, false);+ if ("error" in indexed) return indexed;+ const { entity, relatedEdges, entityHistory, truncated } = indexed;+ const allEntityEdges = [...new Map([...entityHistory, ...relatedEdges].map(edge => [edge.id, edge])).values()];allEntityEdges.sort((a, b) =>@@ -422,6 +405,7 @@ export function registerTemporalGraphFunctions(return {entity: entity.name,+ truncated,totalChanges: changes.length,changes,};diff --git a/src/index.ts b/src/index.tsindex eca1b13c..9b514c3a 100644--- a/src/index.ts+++ b/src/index.ts@@ -80,6 +80,7 @@ import { registerExportImportFunction } from "./functions/export-import.js";import { registerEnrichFunction } from "./functions/enrich.js";import { registerClaudeBridgeFunction } from "./functions/claude-bridge.js";import { registerGraphFunction } from "./functions/graph.js";+import { registerGraphIndexMigrationFunctions } from "./state/graph-index-migration.js";import { GRAPH_COMPACT_BOOT_DELAY_MS, runGraphCompactOnBoot, setGraphCompactBootDisabled } from "./functions/graph-compact-boot.js";import { registerGraphImportFunction } from "./functions/graph-import.js";import { registerConsolidationPipelineFunction } from "./functions/consolidation-pipeline.js";@@ -320,6 +321,7 @@ async function main() {}registerGraphFunction(sdk, kv, provider);+ const graphIndexMigration = registerGraphIndexMigrationFunctions(sdk, kv);registerGraphImportFunction(sdk, kv);bootLog(`Knowledge graph: structural extraction on (LLM relations ${isGraphExtractionEnabled() ? "enabled" : "off"})`,@@ -433,8 +435,8 @@ async function main() {graphWeight,);- const hybridRanker = (query: string, limit: number, targetLayer?: SearchLayer) =>- hybridSearch.search(query, limit, targetLayer);+ const hybridRanker = (query: string, limit: number, targetLayer?: SearchLayer, authorizeGraph?: Parameters<HybridSearch["search"]>[3]) =>+ hybridSearch.search(query, limit, targetLayer, authorizeGraph);registerSmartSearchFunction(sdk, kv, hybridRanker);setHybridRanker(hybridRanker);registerRecentSearchesSweepFunction(sdk, kv);@@ -609,7 +611,7 @@ async function main() {`Ready. ${embeddingProvider ? "Triple-stream (BM25+Vector+Graph)" : "BM25+Graph"} search active.`,);bootLog(- `REST API: 138 endpoints at http://localhost:${config.restPort}/agentmemory/*`,+ `REST API: 141 endpoints at http://localhost:${config.restPort}/agentmemory/*`,);bootLog(`MCP surface (opt-in via \`npx @agentmemory/mcp\`): ${getAllTools().length} tools · 6 resources · 3 prompts`,@@ -643,6 +645,10 @@ async function main() {config.restPort,);+ void graphIndexMigration.resumeOnBoot().catch(() => {+ bootWarn("Graph index migration could not resume. Run agentmemory graph-index status for recovery instructions.");+ });+if (isGraphCompactOnBootEnabled()) {const graphCompactTimer = setTimeout(() => {void runGraphCompactOnBoot(kv, { log: bootLog, warn: bootWarn }).catch(() => {});@@ -760,6 +766,7 @@ async function main() {healthMonitor.stop();dedupMap.stop();capture.stop();+ await settleWithin(graphIndexMigration.stop(), SHUTDOWN_FLUSH_TIMEOUT_MS);indexPersistence.stop();const viewerClosed = new Promise<void>((resolve) =>viewerServer.close(() => resolve()),diff --git a/src/state/graph-generation.ts b/src/state/graph-generation.tsindex 4b16f998..cd0e172d 100644--- a/src/state/graph-generation.ts+++ b/src/state/graph-generation.ts@@ -3,12 +3,14 @@ import { randomUUID } from "node:crypto";import type { StateKV } from "./kv.js";import { KV } from "./schema.js";import { afterKeyedLock } from "./keyed-mutex.js";+import { initializeGraphIndex } from "./graph-retrieval-index.js";const CONTROL_SCOPE = KV.graphControl;const CONTROL_KEY = "generation";const GRAPH_SCOPES = new Set<string>([KV.graphNodes, KV.graphEdges, KV.graphEdgeHistory, KV.graphSnapshot,KV.graphNameIndex, KV.graphEdgeKey, KV.graphNodeDegree, KV.graphMembership,+ KV.graphRetrievalControl, KV.graphRetrievalData, KV.graphIndexMigration, KV.graphIndexManifest,]);const pinned = new AsyncLocalStorage<{ kv: StateKV; generation: string | null }>();const views = new WeakMap<StateKV, { target: StateKV; id: string | null }>();@@ -28,6 +30,8 @@ export async function notifyGraphGenerationChange(kv: StateKV, scope: string, ke: scope.startsWith(`${KV.graphEdges}:generation:`) ? "edges" : undefined;if (!kind) return;if (afterKeyedLock("graph:persist", () => notifyGraphGenerationChange(kv, scope, key), scope)) return;+ if (kind !== "reset" && afterKeyedLock(`graph:retrieval:${scope.slice(scope.lastIndexOf(":generation:") + 12)}`,+ () => notifyGraphGenerationChange(kv, scope, key), scope)) return;if (afterKeyedLock("graph:viewer-events", () => notifyGraphGenerationChange(kv, scope, key), scope)) return;try {const current = await generation(kv);@@ -42,6 +46,17 @@ export async function isLegacyGraphState(kv: StateKV): Promise<boolean> {return await generation(kv) === null;}+export async function getGraphGeneration(kv: StateKV): Promise<string | null> {+ const view = views.get(kv);+ if (view) return view.id;+ const context = pinned.getStore();+ return context?.kv === kv ? context.generation : generation(kv);+}++export function pinGraphStateGeneration(kv: StateKV, id: string | null): StateKV {+ return graphView(views.get(kv)?.target ?? kv, id);+}+async function generation(kv: StateKV): Promise<string | null> {const value = await kv.get<unknown>(CONTROL_SCOPE, CONTROL_KEY);if (value === null || value === undefined) return null;@@ -90,6 +105,7 @@ function graphView(kv: StateKV, id: string | null): StateKV {export async function resetGraphGeneration(kv: StateKV, snapshot: unknown): Promise<void> {const id = randomUUID();await kv.set(physicalScope(KV.graphSnapshot, id), "current", snapshot);+ await initializeGraphIndex(graphView(kv, id), { status: "ready" });await kv.set(CONTROL_SCOPE, CONTROL_KEY, id);}@@ -97,7 +113,9 @@ export async function replaceGraphGeneration<T>(kv: StateKV, initialize: (fresh:const pinnedState = await pinGraphState(kv);const { target, id: previous } = views.get(pinnedState)!;const id = randomUUID();- const result = await initialize(graphView(target, id));+ const fresh = graphView(target, id);+ await initializeGraphIndex(fresh, { status: "ready" });+ const result = await initialize(fresh);// A delayed import cannot replace a reset that happened after it was pinned.if (await generation(target) === previous) {await target.set(CONTROL_SCOPE, CONTROL_KEY, id);diff --git a/src/state/graph-index-migration.ts b/src/state/graph-index-migration.tsnew file mode 100644index 00000000..51bed3ee--- /dev/null+++ b/src/state/graph-index-migration.ts@@ -0,0 +1,372 @@+import { createHash, randomUUID } from "node:crypto";+import { setImmediate as yieldToEventLoop } from "node:timers/promises";+import type { IIIClient } from "iii-sdk";+import type { GraphEdge, GraphNode } from "../types.js";+import { safeAudit } from "../functions/audit.js";+import type { StateKV } from "./kv.js";+import { KV } from "./schema.js";+import { getGraphGeneration, pinGraphState } from "./graph-generation.js";+import { withKeyedLock } from "./keyed-mutex.js";+import { withoutGraphRecovery } from "./graph-consistency.js";+import {+ GRAPH_INDEX_MAX_ID_BYTES,+ GraphIndexError,+ finishGraphIndexBuild,+ getGraphIndexStatus,+ initializeGraphIndex,+ reconcileGraphIndexRecord,+ recoverGraphIndex,+} from "./graph-retrieval-index.js";++export const GRAPH_INDEX_MIGRATION_KEY = "current";+export const GRAPH_INDEX_MANIFEST_MAX_IDS = 128;+export const GRAPH_INDEX_MANIFEST_MAX_BYTES = 32 * 1024;+const PHASES = ["nodes", "edges", "history"] as const;+type Phase = (typeof PHASES)[number];+const SCOPES = { nodes: KV.graphNodes, edges: KV.graphEdges, history: KV.graphEdgeHistory };+const KINDS = { nodes: "node", edges: "edge", history: "history" } as const;+const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;++export interface GraphIndexMigration {+ schema: 1;+ jobId: string;+ buildId: string;+ generation: string | null;+ state: "discovering" | "building" | "paused" | "ready" | "superseded";+ sealed: boolean;+ phase: Phase;+ page: number;+ offset: number;+ processed: number;+ totals: Record<Phase, number>;+ pages: Record<Phase, number>;+ startedAt: string;+ updatedAt: string;+ error?: string;+}++export interface GraphIndexManifestPage {+ schema: 1;+ jobId: string;+ phase: Phase;+ page: number;+ ids: string[];+ checksum: string;+}++export interface GraphIndexMigrationStatus {+ state: GraphIndexMigration["state"] | "needs-migration" | "unavailable";+ ready: boolean;+ coverage: "complete" | "partial" | "unknown";+ generation?: string | null;+ jobId?: string;+ buildId?: string;+ processed?: number;+ total?: number;+ sealed?: boolean;+ phase?: Phase;+ nodes?: number;+ edges?: number;+ historyEdges?: number;+ incompleteRecords?: number;+ error?: string;+ action?: string;+}++export class GraphIndexMigrationError extends Error {+ constructor(public readonly code: "consent-required" | "no-job" | "invalid-state" | "invalid-request", message: string) {+ super(message);+ }+}++function integer(value: unknown): value is number {+ return Number.isSafeInteger(value) && (value as number) >= 0;+}++function validId(value: unknown): value is string {+ return typeof value === "string" && value.length > 0 && Buffer.byteLength(value) <= GRAPH_INDEX_MAX_ID_BYTES;+}++function readJob(value: unknown): GraphIndexMigration | null {+ if (value === null || value === undefined) return null;+ const job = value as GraphIndexMigration;+ if (!job || typeof job !== "object" || Buffer.byteLength(JSON.stringify(job)) > 8192 || job.schema !== 1 ||+ typeof job.jobId !== "string" || !UUID.test(job.jobId) || typeof job.buildId !== "string" || !UUID.test(job.buildId) ||+ (job.generation !== null && (typeof job.generation !== "string" || !UUID.test(job.generation))) ||+ !["discovering", "building", "paused", "ready", "superseded"].includes(job.state) || typeof job.sealed !== "boolean" ||+ !PHASES.includes(job.phase) || !integer(job.page) || !integer(job.offset) || job.offset > GRAPH_INDEX_MANIFEST_MAX_IDS ||+ !integer(job.processed) || !job.totals || !job.pages ||+ !PHASES.every((phase) => integer(job.totals[phase]) && integer(job.pages[phase])) ||+ !Number.isFinite(Date.parse(job.startedAt)) || !Number.isFinite(Date.parse(job.updatedAt)) ||+ (job.error !== undefined && (typeof job.error !== "string" || job.error.length > 512))) {+ throw new GraphIndexMigrationError("invalid-state", "Graph index migration state is invalid; restore its saved metadata before resuming.");+ }+ const total = PHASES.reduce((sum, phase) => sum + job.totals[phase], 0);+ if (!integer(total) || job.processed > total || (job.sealed && (job.page > job.pages[job.phase] ||+ !PHASES.every((phase) => (job.totals[phase] === 0) === (job.pages[phase] === 0))))) {+ throw new GraphIndexMigrationError("invalid-state", "Graph index migration progress is inconsistent; restore its saved metadata before resuming.");+ }+ return job;+}++function checksum(ids: string[]): string {+ return createHash("sha256").update(JSON.stringify(ids)).digest("hex");+}++export function graphIndexManifestKey(jobId: string, phase: Phase, page: number): string {+ return `${jobId}:${phase}:${page}`;+}++function readPage(value: unknown, job: GraphIndexMigration): GraphIndexManifestPage {+ const page = value as GraphIndexManifestPage;+ if (!page || page.schema !== 1 || page.jobId !== job.jobId || page.phase !== job.phase || page.page !== job.page ||+ !Array.isArray(page.ids) || page.ids.length === 0 || page.ids.length > GRAPH_INDEX_MANIFEST_MAX_IDS ||+ !page.ids.every(validId) || page.checksum !== checksum(page.ids) ||+ Buffer.byteLength(JSON.stringify(page)) > GRAPH_INDEX_MANIFEST_MAX_BYTES || job.offset > page.ids.length) {+ throw new GraphIndexMigrationError("invalid-state", "Graph index migration manifest is missing or corrupt; restore the manifest before resuming.");+ }+ return page;+}++function errorMessage(error: unknown): string {+ return error instanceof GraphIndexMigrationError || error instanceof GraphIndexError+ ? error.message+ : "Graph index migration stopped after a storage failure. Check the server and resume when storage is available.";+}++export async function getGraphIndexMigrationStatus(kv: StateKV): Promise<GraphIndexMigrationStatus> {+ try {+ const state = await pinGraphState(kv);+ const generation = await getGraphGeneration(state);+ const [control, stored] = await Promise.all([+ getGraphIndexStatus(state), state.get<unknown>(KV.graphIndexMigration, GRAPH_INDEX_MIGRATION_KEY),+ ]);+ const job = readJob(stored);+ if ((await kv.get(KV.graphControl, "generation") ?? null) !== generation) {+ return { state: "unavailable", ready: false, coverage: "unknown", action: "Graph generation changed. Read status again." };+ }+ const matching = job?.generation === generation && job?.buildId === control?.buildId ? job : null;+ const blocked = control?.status === "paused" || control?.batchPending || matching?.state === "paused" || Boolean(matching?.error);+ const active = !blocked && (matching?.state === "discovering" || matching?.state === "building");+ const ready = control?.status === "ready" && !control.pending && !blocked;+ const result: GraphIndexMigrationStatus = {+ state: ready ? "ready" : blocked || (control?.pending && !active) ? "paused" : matching?.state ?? "needs-migration",+ ready,+ coverage: ready ? control.incompleteRecords > 0 ? "partial" : "complete" : "unknown",+ generation,+ ...(control ? { buildId: control.buildId, nodes: control.nodes, edges: control.edges, historyEdges: control.historyEdges, incompleteRecords: control.incompleteRecords } : {}),+ ...(matching ? { jobId: matching.jobId, processed: matching.processed, total: PHASES.reduce((sum, phase) => sum + matching.totals[phase], 0), sealed: matching.sealed, phase: matching.phase, ...(matching.error ? { error: matching.error } : {}) } : {}),+ };+ if (!ready && active) result.action = matching?.state === "discovering"+ ? "Discovery may still be in progress. If it was interrupted, run agentmemory graph-index resume --allow-legacy-scan during maintenance."+ : "Migration is in progress. Check graph-index status again for updated progress.";+ else if (!ready) result.action = matching?.sealed && control?.status !== "paused" && !matching.error?.includes("new build")+ ? "Run agentmemory graph-index resume to continue the saved migration."+ : "Run agentmemory graph-index migrate --allow-legacy-scan during maintenance; initial discovery reads complete legacy graph scopes.";+ else if (control.incompleteRecords > 0) result.action = "Some records exceed graph index limits. Canonical data is retained; graph results report partial coverage.";+ return result;+ } catch (error) {+ return { state: "unavailable", ready: false, coverage: "unknown", error: errorMessage(error) };+ }+}++export class GraphIndexMigrationController {+ private running: { jobId: string; generation: string | null; promise: Promise<void> } | null = null;+ private stopped = false;++ constructor(private readonly kv: StateKV) {}++ status(): Promise<GraphIndexMigrationStatus> { return getGraphIndexMigrationStatus(this.kv); }++ start(options: { resume?: boolean; allowLegacyScan?: boolean } = {}): Promise<GraphIndexMigrationStatus> {+ return this.accept(options, true);+ }++ private async accept(options: { resume?: boolean; allowLegacyScan?: boolean }, allowSnapshotRecovery: boolean): Promise<GraphIndexMigrationStatus> {+ if (typeof options !== "object" || options === null ||+ (options.resume !== undefined && typeof options.resume !== "boolean") ||+ (options.allowLegacyScan !== undefined && typeof options.allowLegacyScan !== "boolean")) {+ throw new GraphIndexMigrationError("invalid-request", "resume and allowLegacyScan must be booleans");+ }+ const state = await pinGraphState(this.kv);+ const generation = await getGraphGeneration(state);+ let accepted: GraphIndexMigration | null = null;+ await withKeyedLock("graph:persist", async () => {+ if ((await this.kv.get(KV.graphControl, "generation") ?? null) !== generation) {+ throw new GraphIndexMigrationError("invalid-state", "Graph generation changed. Retry the command.");+ }+ if (this.running?.generation === generation) return;+ let control: Awaited<ReturnType<typeof getGraphIndexStatus>>;+ let replaceCorruptControl = false;+ try { control = await getGraphIndexStatus(state); }+ catch (error) {+ if (!(error instanceof GraphIndexError) || options.resume || !options.allowLegacyScan) throw error;+ control = null; replaceCorruptControl = true;+ }+ let job = readJob(await state.get(KV.graphIndexMigration, GRAPH_INDEX_MIGRATION_KEY));+ if (job?.generation !== generation) job = null;+ const unfinishedCheckpoint = job?.buildId === control?.buildId && (job?.state === "paused" || Boolean(job?.error));+ if (control?.status === "ready" && !control.pending && !control.batchPending && !unfinishedCheckpoint) return;+ if (options.resume && !job) throw new GraphIndexMigrationError("no-job", "No graph index migration exists. Run graph-index migrate --allow-legacy-scan first.");+ if (options.resume && job?.sealed && !control) throw new GraphIndexMigrationError("invalid-state", "The sealed build control is missing. Start a new build with graph-index migrate --allow-legacy-scan.");+ if (options.resume && control?.status === "paused") throw new GraphIndexMigrationError("invalid-state", "The index requires a new build. Run graph-index migrate --allow-legacy-scan after fixing the reported data.");+ if (!job || replaceCorruptControl || (!control && job.sealed) || control?.status === "paused" || (control && control.buildId !== job.buildId) ||+ (!options.resume && options.allowLegacyScan && control?.batchPending)) {+ if (options.resume) throw new GraphIndexMigrationError("invalid-state", "The graph index build changed. Start an explicit migration again.");+ if (!options.allowLegacyScan) throw new GraphIndexMigrationError("consent-required", "Initial discovery requires allowLegacyScan: true; complete legacy graph scopes are read during maintenance.");+ const now = new Date().toISOString();+ job = { schema: 1, jobId: randomUUID(), buildId: randomUUID(), generation, state: "discovering", sealed: false,+ phase: "nodes", page: 0, offset: 0, processed: 0, totals: { nodes: 0, edges: 0, history: 0 }, pages: { nodes: 0, edges: 0, history: 0 }, startedAt: now, updatedAt: now };+ await state.set(KV.graphIndexMigration, GRAPH_INDEX_MIGRATION_KEY, job);+ await initializeGraphIndex(state, { buildId: job.buildId, status: "building" });+ } else {+ if (!job.sealed && !options.allowLegacyScan) throw new GraphIndexMigrationError("consent-required", "Discovery did not finish. Resume with --allow-legacy-scan to authorize another complete legacy scan.");+ if (!control) {+ // A previous control write may have been lost after other index writes succeeded.+ job = { ...job, buildId: randomUUID() };+ await this.save(state, job);+ await initializeGraphIndex(state, { buildId: job.buildId, status: "building" });+ }+ job = { ...job, state: job.sealed ? "building" : "discovering", error: undefined, updatedAt: new Date().toISOString() };+ await state.set(KV.graphIndexMigration, GRAPH_INDEX_MIGRATION_KEY, job);+ }+ accepted = job;+ this.stopped = false;+ const entry = { jobId: job.jobId, generation, promise: Promise.resolve() };+ this.running = entry;+ entry.promise = yieldToEventLoop().then(() => this.run(state, job!, allowSnapshotRecovery)).finally(() => {+ if (this.running === entry) this.running = null;+ });+ });+ if (accepted) void safeAudit(this.kv, "graph_index_migrate", "mem::graph-index-migrate", [], {+ jobId: (accepted as GraphIndexMigration).jobId, resume: options.resume === true, allowLegacyScan: options.allowLegacyScan === true,+ });+ return this.status();+ }++ async resumeOnBoot(): Promise<void> {+ const state = await pinGraphState(this.kv);+ const job = readJob(await state.get(KV.graphIndexMigration, GRAPH_INDEX_MIGRATION_KEY));+ if (!job || !job.sealed || job.state === "ready" || job.state === "superseded") return;+ await this.accept({ resume: true }, false);+ }++ async stop(): Promise<void> { this.stopped = true; await this.waitForIdle(); }+ async waitForIdle(): Promise<void> { await this.running?.promise; }++ private async assertActive(state: StateKV, job: GraphIndexMigration): Promise<void> {+ if (this.stopped) throw new GraphIndexMigrationError("invalid-state", "Migration paused during shutdown; resume continues from its saved cursor.");+ if ((await this.kv.get(KV.graphControl, "generation") ?? null) !== job.generation) {+ job.state = "superseded";+ throw new GraphIndexMigrationError("invalid-state", "Migration superseded by a graph reset or replacement.");+ }+ const control = await getGraphIndexStatus(state);+ if (control?.buildId !== job.buildId || control.status === "paused") {+ throw new GraphIndexMigrationError("invalid-state", "Graph index build is unavailable or paused; start an explicit migration after resolving its data error.");+ }+ }++ private async save(state: StateKV, job: GraphIndexMigration): Promise<void> {+ job.updatedAt = new Date().toISOString();+ await state.set(KV.graphIndexMigration, GRAPH_INDEX_MIGRATION_KEY, job);+ }++ private async discover(state: StateKV, job: GraphIndexMigration): Promise<void> {+ job.totals = { nodes: 0, edges: 0, history: 0 };+ job.pages = { nodes: 0, edges: 0, history: 0 };+ job.processed = 0; job.page = 0; job.offset = 0;+ for (const phase of PHASES) {+ job.phase = phase;+ await this.assertActive(state, job);+ await this.save(state, job);+ const rows = await state.list<{ id?: unknown }>(SCOPES[phase]);+ await this.assertActive(state, job);+ let ids: string[] = [];+ const flush = async () => {+ if (!ids.length) return;+ const page: GraphIndexManifestPage = { schema: 1, jobId: job.jobId, phase, page: job.pages[phase], ids, checksum: checksum(ids) };+ await state.set(KV.graphIndexManifest, graphIndexManifestKey(job.jobId, phase, page.page), page);+ job.pages[phase]++; job.totals[phase] += ids.length; ids = [];+ };+ for (const row of rows) {+ if (!row || !validId(row.id)) throw new GraphIndexMigrationError("invalid-state", "A legacy graph record has an invalid or oversized ID; repair it before migrating.");+ if (ids.length >= GRAPH_INDEX_MANIFEST_MAX_IDS || Buffer.byteLength(JSON.stringify(ids)) + Buffer.byteLength(JSON.stringify(row.id)) + 256 > GRAPH_INDEX_MANIFEST_MAX_BYTES) {+ await flush(); await this.assertActive(state, job); await yieldToEventLoop();+ }+ ids.push(row.id);+ }+ await flush();+ }+ await withKeyedLock("graph:persist", async () => {+ await this.assertActive(state, job);+ job.sealed = true; job.state = "building"; job.phase = "nodes"; job.page = 0; job.offset = 0;+ await this.save(state, job);+ });+ }++ private async run(state: StateKV, job: GraphIndexMigration, allowSnapshotRecovery: boolean): Promise<void> {+ try {+ if (!job.sealed) await this.discover(state, job);+ await recoverGraphIndex(state, { allowSnapshotRecovery });+ if ((await getGraphIndexStatus(state))?.batchPending) {+ throw new GraphIndexMigrationError("invalid-state", allowSnapshotRecovery+ ? "The legacy graph transaction cannot be recovered. Start a new build with graph-index migrate --allow-legacy-scan."+ : "Legacy graph transaction recovery requires an explicit graph-index resume command.");+ }+ for (const phase of PHASES) {+ if (PHASES.indexOf(phase) < PHASES.indexOf(job.phase)) continue;+ job.phase = phase;+ while (job.page < job.pages[phase]) {+ await this.assertActive(state, job);+ const page = readPage(await state.get(KV.graphIndexManifest, graphIndexManifestKey(job.jobId, phase, job.page)), job);+ while (job.offset < page.ids.length) {+ await withKeyedLock("graph:persist", async () => {+ await this.assertActive(state, job);+ const id = page.ids[job.offset]!;+ if ((await getGraphIndexStatus(state))?.batchPending) {+ throw new GraphIndexMigrationError("invalid-state", "A graph transaction interrupted migration. Run graph-index resume explicitly.");+ }+ const row = await withoutGraphRecovery(() => state.get<GraphNode | GraphEdge>(SCOPES[phase], id));+ if (row !== null && row.id !== id) throw new GraphIndexMigrationError("invalid-state", "A canonical graph record does not match its manifest ID; repair it before resuming.");+ await reconcileGraphIndexRecord(state, KINDS[phase], id, row, { readCanonical: true });+ job.offset++; job.processed++;+ await this.save(state, job);+ });+ await yieldToEventLoop();+ }+ job.page++; job.offset = 0; await this.save(state, job);+ }+ const next = PHASES[PHASES.indexOf(phase) + 1];+ if (next) { job.phase = next; job.page = 0; job.offset = 0; await this.save(state, job); }+ }+ await withKeyedLock("graph:persist", async () => {+ await this.assertActive(state, job);+ if (job.processed !== PHASES.reduce((sum, phase) => sum + job.totals[phase], 0)) throw new GraphIndexMigrationError("invalid-state", "Graph index migration counts do not match its manifest.");+ await finishGraphIndexBuild(state, job.buildId);+ job.state = "ready"; job.error = undefined; await this.save(state, job);+ });+ } catch (error) {+ if (job.state !== "superseded") job.state = "paused";+ job.error = errorMessage(error);+ await this.save(state, job).catch(() => {});+ }+ }+}++export function registerGraphIndexMigrationFunctions(sdk: IIIClient, kv: StateKV): GraphIndexMigrationController {+ const controller = new GraphIndexMigrationController(kv);+ sdk.registerFunction("mem::graph-index-status", () => controller.status());+ const mutate = (resume: boolean) => async (data: { allowLegacyScan?: unknown }) => {+ try {+ if (!data || typeof data !== "object" || (data.allowLegacyScan !== undefined && typeof data.allowLegacyScan !== "boolean")) {+ throw new GraphIndexMigrationError("invalid-request", "allowLegacyScan must be a boolean");+ }+ return { success: true, ...await controller.start({ resume, allowLegacyScan: data.allowLegacyScan as boolean | undefined }) };+ } catch (error) {+ return { success: false, code: error instanceof GraphIndexMigrationError ? error.code : "storage-unavailable", error: errorMessage(error) };+ }+ };+ sdk.registerFunction("mem::graph-index-migrate", mutate(false));+ sdk.registerFunction("mem::graph-index-resume", mutate(true));+ return controller;+}diff --git a/src/state/graph-ingestion.ts b/src/state/graph-ingestion.tsindex c6392a75..892bb627 100644--- a/src/state/graph-ingestion.ts+++ b/src/state/graph-ingestion.ts@@ -2,7 +2,7 @@ import type { GraphEdge, GraphNode, GraphSnapshot, GraphMutationWrite, GraphIndeimport type { StateKV } from "./kv.js";import { KV } from "./schema.js";import { pinGraphState, replaceGraphGeneration } from "./graph-generation.js";-import { withKeyedLock } from "./keyed-mutex.js";+import { isKeyedLockHeld, withKeyedLock } from "./keyed-mutex.js";import { withoutGraphRecovery } from "./graph-consistency.js";export interface GraphIngestionOptions {@@ -218,7 +218,8 @@ export async function persistGraphRecords(}return result;};- return withKeyedLock("graph:persist", () => options.replace+ const persist = () => options.replace? replaceGraphGeneration(kv, (fresh) => run(fresh, true))- : run(kv));+ : run(kv);+ return isKeyedLockHeld("graph:persist") ? persist() : withKeyedLock("graph:persist", persist);}diff --git a/src/state/graph-retrieval-index.ts b/src/state/graph-retrieval-index.tsnew file mode 100644index 00000000..feb71d66--- /dev/null+++ b/src/state/graph-retrieval-index.ts@@ -0,0 +1,495 @@+import { createHash, randomUUID } from "node:crypto";+import type { GraphEdge, GraphNode, GraphSnapshot, GraphRetrievalIndexControl, GraphRetrievalIndexRecord } from "../types.js";+import type { StateKV } from "./kv.js";+import { getGraphGeneration, pinGraphState, pinGraphStateGeneration } from "./graph-generation.js";+import { ensureGraphConsistency, noteGraphSnapshotWrite, withoutGraphRecovery } from "./graph-consistency.js";+import { isKeyedLockHeld, withKeyedLock } from "./keyed-mutex.js";+import { KV } from "./schema.js";++export const GRAPH_INDEX_MAX_ID_BYTES = 1024;+export const GRAPH_INDEX_MAX_RECORD_BYTES = 32 * 1024;+const MAX_CONTROL_BYTES = 8 * 1024;+const MAX_PROJECTION_BYTES = 16 * 1024;+const MAX_JOURNAL_BYTES = 8 * 1024 * 1024;+const MAX_POSTINGS = 192;+const CONTROL_KEY = "current";+export type GraphIndexRecordKind = "node" | "edge" | "history";+export type GraphPostingKind = "name" | "character" | "observation" | "edgeObservation" | "edgeKey" | "adjacency" | "history" | "nodes" | "edges" | "historyEdges";+export type GraphNodeProjection = GraphNode & { indexIncomplete?: boolean };+export type GraphEdgeProjection = GraphEdge & { indexIncomplete?: boolean };+type PostingRoot = { head: string | null; count: number };+type PostingEntry = { id: string; previous: string | null; next: string | null };+type IndexWrite = { key: string; value: unknown };+type IndexJournal = { schema: 1; buildId: string; revision: number; writes: IndexWrite[]; next: GraphRetrievalIndexControl };++export class GraphIndexError extends Error {}+const bytes = (value: unknown) => Buffer.byteLength(JSON.stringify(value) ?? "", "utf8");+const hash = (value: string) => createHash("sha256").update(JSON.stringify(value)).digest("hex");+const validId = (value: unknown): value is string => typeof value === "string" && value.length > 0 && Buffer.byteLength(value, "utf8") <= GRAPH_INDEX_MAX_ID_BYTES;+const finiteCount = (value: unknown): value is number => Number.isSafeInteger(value) && (value as number) >= 0;+const boundedOption = (value: number | undefined, fallback: number, max: number) => Number.isFinite(value) ? Math.min(max, Math.max(1, Math.floor(value!))) : fallback;+const graphLock = <T>(fn: () => Promise<T>) => isKeyedLockHeld("graph:persist") ? fn() : withKeyedLock("graph:persist", fn);+async function lock<T>(state: StateKV, fn: () => Promise<T>): Promise<T> {+ const key = `graph:retrieval:${await getGraphGeneration(state) ?? "legacy"}`;+ return isKeyedLockHeld(key) ? fn() : withKeyedLock(key, fn);+}+const WAIT_FOR_BATCH = Symbol("graph index batch");+const NODE_TYPES = new Set(["file", "function", "concept", "error", "decision", "pattern", "library", "person", "project", "preference", "location", "organization", "event"]);+const EDGE_TYPES = new Set(["uses", "imports", "modifies", "causes", "fixes", "depends_on", "related_to", "works_at", "prefers", "blocked_by", "caused_by", "optimizes_for", "rejected", "avoids", "located_in", "succeeded_by"]);+export const normalizeGraphIndexText = (value: string): string => value.normalize("NFKC").toLowerCase();+export const graphEdgeLookupKey = (edge: Pick<GraphEdge, "sourceNodeId" | "targetNodeId" | "type">): string => JSON.stringify([edge.sourceNodeId, edge.targetNodeId, edge.type]);+const token = (kind: GraphPostingKind, value: string) => hash(`${kind}\0${kind === "name" || kind === "character" ? normalizeGraphIndexText(value) : value}`);+const recordKey = (buildId: string, kind: GraphIndexRecordKind, id: string) => `${buildId}:record:${kind}:${hash(id)}`;+const rootKey = (buildId: string, posting: string) => `${buildId}:root:${posting}`;+const entryKey = (buildId: string, posting: string, id: string) => `${buildId}:entry:${posting}:${id}`;+const countKey = (kind: GraphIndexRecordKind) => kind === "node" ? "nodes" : kind === "edge" ? "edges" : "historyEdges";++function validateControl(value: GraphRetrievalIndexControl): void {+ if (bytes(value) > MAX_CONTROL_BYTES || value.schema !== 2 || typeof value.buildId !== "string" || !/^[a-zA-Z0-9_-]{1,128}$/.test(value.buildId) ||+ !["building", "ready", "paused"].includes(value.status) ||+ ![value.revision, value.nodes, value.edges, value.historyEdges, value.incompleteRecords, value.otherTypes].every(finiteCount) ||+ !value.nodesByType || !value.edgesByType ||+ Object.entries(value.nodesByType).some(([type, count]) => (!NODE_TYPES.has(type) && type !== "other") || !finiteCount(count)) ||+ Object.entries(value.edgesByType).some(([type, count]) => (!EDGE_TYPES.has(type) && type !== "other") || !finiteCount(count)) ||+ (value.batchPending !== undefined && typeof value.batchPending !== "boolean") ||+ (value.pending && (!["node", "edge", "history"].includes(value.pending.kind) || !validId(value.pending.id)))) {+ throw new GraphIndexError("Invalid graph retrieval index control; run explicit graph-index migration");+ }+}++export async function getGraphIndexStatus(state: StateKV): Promise<GraphRetrievalIndexControl | null> {+ const value = await state.get<GraphRetrievalIndexControl>(KV.graphRetrievalControl, CONTROL_KEY);+ if (value !== null && value !== undefined) validateControl(value);+ return value ?? null;+}++export async function initializeGraphIndex(state: StateKV, options: { status: "building" | "ready"; buildId?: string }): Promise<GraphRetrievalIndexControl> {+ const kv = await pinGraphState(state);+ return lock(kv, async () => {+ const value: GraphRetrievalIndexControl = { schema: 2, buildId: options.buildId ?? randomUUID(), status: options.status,+ revision: 0, nodes: 0, edges: 0, historyEdges: 0, incompleteRecords: 0, nodesByType: {}, edgesByType: {}, otherTypes: 0, updatedAt: new Date().toISOString() };+ validateControl(value);+ const previous = await kv.get<GraphRetrievalIndexControl>(KV.graphRetrievalControl, CONTROL_KEY);+ if (previous?.buildId === value.buildId) throw new GraphIndexError("A graph index rebuild requires a new build ID");+ await kv.set(KV.graphRetrievalControl, CONTROL_KEY, value);+ return value;+ });+}++function project(kind: GraphIndexRecordKind, id: string, input: unknown): GraphRetrievalIndexRecord | null {+ if (input === null || input === undefined) return null;+ const source = input as Record<string, unknown>;+ if (!source || typeof source !== "object" || Array.isArray(source) || !validId(id) || source.id !== id) {+ return { row: null, postings: [], incomplete: true };+ }+ if (source.stale === true) return { row: null, postings: [], incomplete: false };+ let incomplete = false;+ const text = (value: unknown, max = 512): string => {+ if (typeof value !== "string") { incomplete = true; return ""; }+ const result = Array.from(value.slice(0, max * 2)).slice(0, max).join("");+ if (result !== value) incomplete = true;+ return result;+ };+ const ids: string[] = [];+ if (!Array.isArray(source.sourceObservationIds)) incomplete = true;+ else for (const value of source.sourceObservationIds.slice(0, 512)) {+ if (!validId(value)) { incomplete = true; continue; }+ if (ids.includes(value)) continue;+ if (ids.length >= 128) { incomplete = true; break; }+ ids.push(value);+ }+ if (Array.isArray(source.sourceObservationIds) && source.sourceObservationIds.length > 512) incomplete = true;+ const common = { id, type: text(source.type, 64), sourceObservationIds: ids, createdAt: text(source.createdAt, 64) };+ let row: GraphNodeProjection | GraphEdgeProjection;+ if (kind === "node") {+ const properties: Record<string, unknown> = {};+ if (!source.properties || typeof source.properties !== "object" || Array.isArray(source.properties)) incomplete = true;+ else for (const key in source.properties) {+ if (!Object.hasOwn(source.properties, key)) continue;+ if (Object.keys(properties).length >= 32) { incomplete = true; break; }+ const value = (source.properties as Record<string, unknown>)[key];+ if (key.length > 128) { incomplete = true; continue; }+ if (typeof value === "string") Object.defineProperty(properties, key, { value: text(value), enumerable: true, writable: true, configurable: true });+ else if (typeof value === "boolean" || (typeof value === "number" && Number.isFinite(value))) Object.defineProperty(properties, key, { value, enumerable: true, writable: true, configurable: true });+ else incomplete = true;+ }+ const aliases = Array.isArray(source.aliases) ? source.aliases.slice(0, 16).filter((v): v is string => typeof v === "string").map((v) => text(v)) : [];+ if (Array.isArray(source.aliases) && aliases.length !== source.aliases.length) incomplete = true;+ row = { ...common, type: common.type as GraphNode["type"], name: text(source.name), properties, ...(aliases.length ? { aliases } : {}),+ ...(typeof source.updatedAt === "string" ? { updatedAt: text(source.updatedAt, 64) } : {}) };+ } else {+ if (!validId(source.sourceNodeId) || !validId(source.targetNodeId)) return { row: null, postings: [], incomplete: true };+ const edge: GraphEdgeProjection = { ...common, type: common.type as GraphEdge["type"], sourceNodeId: source.sourceNodeId,+ targetNodeId: source.targetNodeId, weight: typeof source.weight === "number" && Number.isFinite(source.weight) ? source.weight : 0 };+ if (edge.weight !== source.weight) incomplete = true;+ for (const key of ["tcommit", "tvalid", "tvalidEnd", "supersededBy"] as const) if (typeof source[key] === "string") edge[key] = text(source[key], key === "supersededBy" ? 1024 : 64);+ if (typeof source.version === "number" && Number.isFinite(source.version)) edge.version = source.version;+ if (typeof source.isLatest === "boolean") edge.isLatest = source.isLatest;+ if (source.context && typeof source.context === "object" && !Array.isArray(source.context)) {+ const context = source.context as Record<string, unknown>;+ edge.context = {};+ for (const key of ["reasoning", "sentiment"] as const) if (typeof context[key] === "string") edge.context[key] = text(context[key]);+ if (typeof context.confidence === "number" && Number.isFinite(context.confidence)) edge.context.confidence = context.confidence;+ if (context.alternatives !== undefined || context.situationalFactors !== undefined) incomplete = true;+ }+ row = edge;+ }+ while (bytes(row) > MAX_PROJECTION_BYTES - 64) {+ incomplete = true;+ if ("properties" in row && Object.keys(row.properties).length) delete row.properties[Object.keys(row.properties).at(-1)!];+ else if ("aliases" in row && row.aliases?.length) row.aliases.pop();+ else if ("context" in row && row.context) delete row.context;+ else if (row.sourceObservationIds.length) row.sourceObservationIds.pop();+ else return { row: null, postings: [], incomplete: true };+ }+ const postings = new Set<string>([token(countKey(kind), "")]);+ if (kind === "node") {+ const node = row as GraphNodeProjection;+ for (const value of [node.name, ...(node.aliases ?? [])]) postings.add(token("name", value));+ const searchable = [node.name, ...(node.aliases ?? []), ...Object.values(node.properties).filter((v): v is string => typeof v === "string")];+ const characters = new Set(searchable.flatMap((value) => Array.from(normalizeGraphIndexText(value))));+ for (const character of characters) {+ if (postings.size >= MAX_POSTINGS - row.sourceObservationIds.length) { incomplete = true; break; }+ postings.add(token("character", character));+ }+ for (const observationId of row.sourceObservationIds) postings.add(token("observation", observationId));+ } else {+ const edge = row as GraphEdgeProjection;+ for (const nodeId of [edge.sourceNodeId, edge.targetNodeId]) postings.add(token(kind === "history" ? "history" : "adjacency", nodeId));+ if (kind === "edge") {+ if (edge.isLatest !== false) postings.add(token("edgeKey", graphEdgeLookupKey(edge)));+ for (const observationId of row.sourceObservationIds) postings.add(token("edgeObservation", observationId));+ }+ }+ if (incomplete) row.indexIncomplete = true;+ return { row, postings: [...postings], incomplete };+}++function validateRecord(value: GraphRetrievalIndexRecord): void {+ if (bytes(value) > GRAPH_INDEX_MAX_RECORD_BYTES || !Array.isArray(value.postings) || value.postings.length > MAX_POSTINGS ||+ value.postings.some((p) => !/^[a-f0-9]{64}$/.test(p)) || typeof value.incomplete !== "boolean") throw new GraphIndexError("Invalid graph retrieval record");+}++async function planRecord(kv: StateKV, control: GraphRetrievalIndexControl, kind: GraphIndexRecordKind, id: string, input: unknown): Promise<IndexJournal> {+ const writes = new Map<string, unknown>();+ const read = async <T>(key: string): Promise<T | null> => writes.has(key) ? writes.get(key) as T | null : kv.get<T>(KV.graphRetrievalData, key);+ const put = (key: string, value: unknown) => { if (bytes(value) > GRAPH_INDEX_MAX_RECORD_BYTES) throw new GraphIndexError("Graph retrieval index write exceeds record budget"); writes.set(key, value); };+ const key = recordKey(control.buildId, kind, id);+ const before = await read<GraphRetrievalIndexRecord>(key);+ if (before) validateRecord(before);+ const after = project(kind, id, input);+ if (after) validateRecord(after);+ const memberId = hash(id);+ const editPosting = async (posting: string, add: boolean) => {+ const rootId = rootKey(control.buildId, posting), itemId = entryKey(control.buildId, posting, memberId);+ const root = await read<PostingRoot>(rootId) ?? { head: null, count: 0 };+ const saved = await read<PostingEntry>(itemId);+ if (!finiteCount(root.count) || (root.head !== null && !/^[a-f0-9]{64}$/.test(root.head))) throw new GraphIndexError("Invalid graph posting root");+ if (add) {+ if (saved) return;+ if (root.head) {+ const headId = entryKey(control.buildId, posting, root.head), head = await read<PostingEntry>(headId);+ if (!head) throw new GraphIndexError("Missing graph posting head");+ put(headId, { ...head, previous: memberId });+ }+ put(itemId, { id, previous: null, next: root.head });+ put(rootId, { head: memberId, count: root.count + 1 });+ } else {+ if (!saved) return;+ if (saved.previous) {+ const previousId = entryKey(control.buildId, posting, saved.previous), previous = await read<PostingEntry>(previousId);+ if (!previous) throw new GraphIndexError("Missing previous graph posting entry");+ put(previousId, { ...previous, next: saved.next });+ } else {+ if (root.head !== memberId) throw new GraphIndexError("Invalid graph posting head link");+ root.head = saved.next;+ }+ if (saved.next) {+ const nextId = entryKey(control.buildId, posting, saved.next), next = await read<PostingEntry>(nextId);+ if (!next) throw new GraphIndexError("Missing next graph posting entry");+ put(nextId, { ...next, previous: saved.previous });+ }+ put(itemId, null);+ put(rootId, { head: root.head, count: Math.max(0, root.count - 1) });+ }+ };+ const oldPostings = new Set(before?.postings ?? []), newPostings = new Set(after?.postings ?? []);+ for (const posting of oldPostings) if (!newPostings.has(posting)) await editPosting(posting, false);+ for (const posting of newPostings) if (!oldPostings.has(posting)) await editPosting(posting, true);+ put(key, after);+ const { pending: _, ...clean } = control;+ const next = { ...clean, revision: control.revision + 1, updatedAt: new Date().toISOString() };+ const counted = (row: GraphNode | GraphEdge | null | undefined) => Boolean(row && (kind !== "edge" || (row as GraphEdge).isLatest !== false));+ next[countKey(kind)] += Number(counted(after?.row)) - Number(counted(before?.row));+ next.incompleteRecords += Number(Boolean(after?.incomplete)) - Number(Boolean(before?.incomplete));+ if (kind !== "history") {+ const field = kind === "node" ? "nodesByType" : "edgesByType";+ const known = kind === "node" ? NODE_TYPES : EDGE_TYPES;+ next[field] = { ...control[field] };+ for (const [row, amount] of [[before?.row, -1], [after?.row, 1]] as const) {+ if (!row || !counted(row)) continue;+ const type = known.has(row.type) ? row.type : "other";+ next[field][type] = (next[field][type] ?? 0) + amount;+ if (!next[field][type]) delete next[field][type];+ if (type === "other") next.otherTypes += amount;+ }+ }+ validateControl(next);+ const journal: IndexJournal = { schema: 1, buildId: control.buildId, revision: control.revision, writes: [...writes].map(([key, value]) => ({ key, value })), next };+ if (bytes(journal) > MAX_JOURNAL_BYTES) throw new GraphIndexError("Graph retrieval index journal exceeds payload budget");+ return journal;+}++async function applyJournal(kv: StateKV, control: GraphRetrievalIndexControl, journal: IndexJournal): Promise<void> {+ if (journal.schema !== 1 || journal.buildId !== control.buildId || journal.revision !== control.revision ||+ !Array.isArray(journal.writes) || journal.writes.length > 2048 || bytes(journal) > MAX_JOURNAL_BYTES || journal.next.buildId !== control.buildId ||+ journal.next.revision !== control.revision + 1 || journal.next.pending) throw new GraphIndexError("Invalid graph retrieval recovery journal");+ validateControl(journal.next);+ for (const write of journal.writes) {+ if (!write || typeof write.key !== "string" || !write.key.startsWith(`${control.buildId}:`) || write.key.length > 512 || bytes(write.value) > GRAPH_INDEX_MAX_RECORD_BYTES) {+ throw new GraphIndexError("Invalid graph retrieval recovery write");+ }+ }+ for (const write of journal.writes) {+ if (write.value === null) await kv.delete(KV.graphRetrievalData, write.key);+ else await kv.set(KV.graphRetrievalData, write.key, write.value);+ }+ await kv.set(KV.graphRetrievalControl, CONTROL_KEY, journal.next);+}++async function reconcilePending(kv: StateKV, control: GraphRetrievalIndexControl, input?: unknown): Promise<void> {+ if (!control.pending) return;+ const journalKey = `journal:${control.buildId}`;+ const saved = await kv.get<IndexJournal>(KV.graphRetrievalControl, journalKey);+ if (saved?.buildId === control.buildId && saved.revision === control.revision) return applyJournal(kv, control, saved);+ const { kind, id } = control.pending;+ const scope = kind === "node" ? KV.graphNodes : kind === "edge" ? KV.graphEdges : KV.graphEdgeHistory;+ const row = input === undefined ? await withoutGraphRecovery(() => kv.get(scope, id)) : input;+ const journal = await planRecord(kv, control, kind, id, row);+ await kv.set(KV.graphRetrievalControl, journalKey, journal);+ await applyJournal(kv, control, journal);+}++export async function recoverGraphIndex(state: StateKV, options: { allowSnapshotRecovery?: boolean } = {}): Promise<void> {+ const kv = await pinGraphState(state);+ await graphLock(() => lock(kv, async () => {+ let control = await getGraphIndexStatus(kv);+ if (control?.batchPending && options.allowSnapshotRecovery) {+ const snapshot = await withoutGraphRecovery(() => kv.get<GraphSnapshot>(KV.graphSnapshot, "current"));+ if (snapshot?.pendingMutation) {+ const generation = await getGraphGeneration(kv);+ const scope = generation === null ? KV.graphSnapshot : `${KV.graphSnapshot}:generation:${generation}`;+ noteGraphSnapshotWrite(kv, scope, "current");+ await ensureGraphConsistency(kv, scope);+ } else if (snapshot && !snapshot.dirty) {+ const { batchPending: _, ...clean } = control;+ await kv.set(KV.graphRetrievalControl, CONTROL_KEY, { ...clean, revision: clean.revision + 1 });+ }+ control = await getGraphIndexStatus(kv);+ }+ if (control?.pending) await reconcilePending(kv, control);+ }));+}++export async function reconcileGraphIndexRecord(state: StateKV, kind: GraphIndexRecordKind, id: string, row: GraphNode | GraphEdge | null, options: { readCanonical?: boolean } = {}): Promise<void> {+ const kv = await pinGraphState(state);+ await lock(kv, async () => {+ let control = await getGraphIndexStatus(kv);+ if (!control) throw new GraphIndexError("Graph retrieval index must be initialized explicitly");+ if (control.status === "paused") throw new GraphIndexError("Graph retrieval index is paused; run explicit migration");+ if (control.pending) { await reconcilePending(kv, control); control = (await getGraphIndexStatus(kv))!; }+ if (!validId(id)) throw new GraphIndexError("Graph record ID exceeds index migration budget");+ control = { ...control, pending: { kind, id } };+ await kv.set(KV.graphRetrievalControl, CONTROL_KEY, control);+ await reconcilePending(kv, control, options.readCanonical ? undefined : row);+ });+}++export async function mutateIndexedGraphRecord(state: StateKV, kind: GraphIndexRecordKind, id: string, transform: (row: GraphNode | GraphEdge | null) => GraphNode | GraphEdge | null | undefined): Promise<boolean> {+ const kv = await pinGraphState(state);+ return graphLock(() => lock(kv, async () => {+ const control = await getGraphIndexStatus(kv);+ if (!control || control.status !== "ready" || control.pending || control.batchPending) throw new GraphIndexError("Graph index is unavailable; retry the graph mutation after recovery");+ const scope = kind === "node" ? KV.graphNodes : kind === "edge" ? KV.graphEdges : KV.graphEdgeHistory;+ const current = await withoutGraphRecovery(() => kv.get<GraphNode | GraphEdge>(scope, id));+ const next = transform(current);+ if (next === undefined) return false;+ if (next === null) await withoutGraphRecovery(() => kv.delete(scope, id));+ else await withoutGraphRecovery(() => kv.set(scope, id, next));+ return true;+ }));+}++export async function finishGraphIndexBuild(state: StateKV, buildId: string): Promise<GraphRetrievalIndexControl> {+ const kv = await pinGraphState(state);+ return lock(kv, async () => {+ const current = await getGraphIndexStatus(kv);+ if (!current || current.buildId !== buildId || current.status === "paused" || current.pending || current.batchPending) throw new GraphIndexError("Graph index build is incomplete or superseded");+ const next = { ...current, status: "ready" as const, revision: current.revision + 1, updatedAt: new Date().toISOString() };+ await kv.set(KV.graphRetrievalControl, CONTROL_KEY, next);+ return next;+ });+}++function canonicalScope(scope: string): { kind: GraphIndexRecordKind; generation: string | null } | null {+ for (const [base, kind] of [[KV.graphNodes, "node"], [KV.graphEdges, "edge"], [KV.graphEdgeHistory, "history"]] as const) {+ if (scope === base) return { kind, generation: null };+ if (scope.startsWith(`${base}:generation:`)) return { kind, generation: scope.slice(base.length + 12) };+ }+ return null;+}++export async function withGraphIndexMutation<T>(state: StateKV, scope: string, id: string, mutate: () => Promise<T>): Promise<T> {+ const location = canonicalScope(scope);+ if (!location) return mutate();+ const kv = pinGraphStateGeneration(state, location.generation);+ const run = () => lock(kv, async () => {+ let control = await getGraphIndexStatus(kv);+ if (!control || control.status === "paused") return mutate();+ if (control.batchPending && !isKeyedLockHeld("graph:persist")) throw WAIT_FOR_BATCH;+ if (control.pending) { await reconcilePending(kv, control); control = (await getGraphIndexStatus(kv))!; }+ if (!validId(id)) {+ await kv.set(KV.graphRetrievalControl, CONTROL_KEY, { ...control, status: "paused", error: "Graph record ID exceeds bounded recovery budget", revision: control.revision + 1 });+ return mutate();+ }+ control = { ...control, pending: { kind: location.kind, id } };+ await kv.set(KV.graphRetrievalControl, CONTROL_KEY, control);+ const result = await mutate();+ await reconcilePending(kv, control);+ return result;+ });+ try { return await run(); } catch (error) {+ if (error !== WAIT_FOR_BATCH) throw error;+ return graphLock(run);+ }+}++export async function withGraphSnapshotIndexBarrier<T>(state: StateKV, scope: string, key: string, value: unknown, mutate: () => Promise<T>): Promise<T> {+ if (key !== "current" || !(scope === KV.graphSnapshot || scope.startsWith(`${KV.graphSnapshot}:generation:`))) return mutate();+ const kv = pinGraphStateGeneration(state, scope === KV.graphSnapshot ? null : scope.slice(KV.graphSnapshot.length + 12));+ return graphLock(() => lock(kv, async () => {+ const control = await getGraphIndexStatus(kv);+ if (!control) return mutate();+ const snapshot = value as GraphSnapshot | null;+ const dirty = !snapshot || typeof snapshot !== "object" || snapshot.version !== 1 || snapshot.dirty !== false || Boolean(snapshot.pendingMutation);+ if (dirty && !control.batchPending) await kv.set(KV.graphRetrievalControl, CONTROL_KEY, { ...control, batchPending: true });+ const result = await mutate();+ if (!dirty) {+ const after = await getGraphIndexStatus(kv);+ if (after?.batchPending && !after.pending) {+ const { batchPending: _, ...clean } = after;+ await kv.set(KV.graphRetrievalControl, CONTROL_KEY, { ...clean, revision: clean.revision + 1, updatedAt: new Date().toISOString() });+ }+ }+ return result;+ }));+}++export async function withGraphSnapshotUnknownMutation<T>(state: StateKV, scope: string, key: string, mutate: () => Promise<T>): Promise<T> {+ if (key !== "current" || !(scope === KV.graphSnapshot || scope.startsWith(`${KV.graphSnapshot}:generation:`))) return mutate();+ return graphLock(async () => {+ const result = await withGraphSnapshotIndexBarrier(state, scope, key, { dirty: true }, mutate);+ const snapshot = await withoutGraphRecovery(() => state.get(scope, key));+ return withGraphSnapshotIndexBarrier(state, scope, key, snapshot, async () => result);+ });+}++export interface GraphIndexReaderOptions { signal?: AbortSignal; maxReads?: number; maxBytes?: number; maxConcurrent?: number }+export class GraphIndexReader {+ readonly status: GraphRetrievalIndexControl;+ exhausted = false;+ private reads = 2;+ private readBytes: number;+ private active = 0;+ private waiting: Array<() => void> = [];+ private cache = new Map<string, Promise<unknown>>();+ private readonly maxReads: number;+ private readonly maxBytes: number;+ private readonly maxConcurrent: number;+ constructor(private state: StateKV, private generation: string | null, status: GraphRetrievalIndexControl, private options: GraphIndexReaderOptions) {+ this.status = status;+ this.readBytes = bytes(status);+ this.maxReads = boundedOption(options.maxReads, 512, 4096);+ this.maxBytes = boundedOption(options.maxBytes, 2 * 1024 * 1024, 16 * 1024 * 1024);+ this.maxConcurrent = boundedOption(options.maxConcurrent, 4, 8);+ }+ get remainingReads(): number { return Math.max(0, this.maxReads - this.reads - 2); }+ async read<T>(scope: string, key: string): Promise<T | null> {+ if (scope !== KV.graphRetrievalData) return this.readFresh<T>(scope, key);+ const cached = this.cache.get(key);+ if (cached) return cached as Promise<T | null>;+ const pending = this.readFresh<T>(scope, key);+ if (!this.exhausted) this.cache.set(key, pending);+ return pending;+ }+ async readFresh<T>(scope: string, key: string): Promise<T | null> {+ if (this.options.signal?.aborted || this.exhausted || this.remainingReads === 0 || this.readBytes >= this.maxBytes) { this.exhausted = true; return null; }+ this.reads++;+ if (this.active >= this.maxConcurrent) await new Promise<void>((resolve) => this.waiting.push(resolve));+ else this.active++;+ try {+ if (this.options.signal?.aborted || this.exhausted) { this.exhausted = true; return null; }+ const value = await this.state.get<T>(scope, key);+ const size = bytes(value);+ this.readBytes += size;+ if (this.readBytes > this.maxBytes || this.options.signal?.aborted) { this.exhausted = true; return null; }+ if (scope === KV.graphRetrievalData && size > GRAPH_INDEX_MAX_RECORD_BYTES) throw new GraphIndexError("Graph retrieval record exceeds payload budget");+ return value;+ } finally {+ const next = this.waiting.shift();+ if (next) next(); else this.active--;+ }+ }+ private async record(kind: GraphIndexRecordKind, id: string): Promise<GraphRetrievalIndexRecord | null> {+ if (!validId(id)) { this.exhausted = true; return null; }+ const value = await this.read<GraphRetrievalIndexRecord>(KV.graphRetrievalData, recordKey(this.status.buildId, kind, id));+ if (value) validateRecord(value);+ return value;+ }+ async getNode(id: string): Promise<GraphNodeProjection | null> { return (await this.record("node", id))?.row as GraphNodeProjection | null ?? null; }+ async getEdge(id: string): Promise<GraphEdgeProjection | null> { return (await this.record("edge", id))?.row as GraphEdgeProjection | null ?? null; }+ async getHistoryEdge(id: string): Promise<GraphEdgeProjection | null> { return (await this.record("history", id))?.row as GraphEdgeProjection | null ?? null; }+ async posting(kind: GraphPostingKind, key = "", options: { cursor?: string | null; limit?: number } = {}): Promise<{ ids: string[]; cursor: string | null; total: number }> {+ const posting = token(kind, key), root = await this.read<PostingRoot>(KV.graphRetrievalData, rootKey(this.status.buildId, posting));+ const inventoryCount = key === "" && (kind === "nodes" || kind === "edges" || kind === "historyEdges") ? this.status[kind] : undefined;+ if (!root) {+ if (!this.exhausted && inventoryCount) throw new GraphIndexError("Missing graph inventory root");+ return { ids: [], cursor: null, total: 0 };+ }+ if (!finiteCount(root.count) || (root.head !== null && !/^[a-f0-9]{64}$/.test(root.head)) ||+ (root.count === 0) !== (root.head === null)) throw new GraphIndexError("Invalid graph posting root");+ if (inventoryCount !== undefined && (kind === "edges" ? root.count < inventoryCount : root.count !== inventoryCount)) {+ throw new GraphIndexError("Graph inventory count does not match index control");+ }+ const prefix = `${this.status.buildId}.${this.status.revision}.${posting}.`;+ if (options.cursor && (!options.cursor.startsWith(prefix) || !/^[a-f0-9]{64}$/.test(options.cursor.slice(prefix.length)))) throw new GraphIndexError("Invalid or expired graph posting cursor");+ let current = options.cursor ? options.cursor.slice(prefix.length) : root.head;+ const limit = boundedOption(options.limit, 64, 128), ids: string[] = [], seen = new Set<string>();+ while (current && ids.length < limit) {+ if (seen.has(current)) throw new GraphIndexError("Cyclic graph posting entry");+ seen.add(current);+ const value = await this.read<PostingEntry>(KV.graphRetrievalData, entryKey(this.status.buildId, posting, current));+ if (!value) { if (this.exhausted) break; throw new GraphIndexError("Missing graph posting entry"); }+ if (!validId(value.id) || hash(value.id) !== current || (value.next !== null && !/^[a-f0-9]{64}$/.test(value.next))) throw new GraphIndexError("Invalid graph posting entry");+ ids.push(value.id); current = value.next;+ }+ return { ids, cursor: current ? `${prefix}${current}` : null, total: root.count };+ }+ async finish(): Promise<boolean> {+ if (this.options.signal?.aborted || this.reads + 2 > this.maxReads) return false;+ this.reads += 2;+ const current = await getGraphIndexStatus(this.state);+ const generation = await this.state.get<string>(KV.graphControl, "generation");+ if ((generation ?? null) !== this.generation) return false;+ this.readBytes += bytes(generation) + bytes(current);+ if (this.readBytes > this.maxBytes) { this.exhausted = true; return false; }+ return Boolean(current && current.status === "ready" && !current.pending && !current.batchPending && current.buildId === this.status.buildId && current.revision === this.status.revision);+ }+}++export async function createGraphIndexReader(state: StateKV, options: GraphIndexReaderOptions = {}): Promise<GraphIndexReader | null> {+ if (options.signal?.aborted) return null;+ const kv = await pinGraphState(state), generation = await getGraphGeneration(kv), status = await getGraphIndexStatus(kv);+ if (!status || status.status !== "ready" || status.pending || status.batchPending) return null;+ return new GraphIndexReader(kv, generation, status, options);+}diff --git a/src/state/hybrid-search.ts b/src/state/hybrid-search.tsindex cb0a730f..8a1ed462 100644--- a/src/state/hybrid-search.ts+++ b/src/state/hybrid-search.ts@@ -15,14 +15,18 @@ import {GraphRetrieval,type GraphRetrievalResult,type GraphQuery,+ type GraphObservationAuthorization,} from "../functions/graph-retrieval.js";import { extractEntitiesFromQuery } from "../functions/query-expansion.js";import { rerank } from "./reranker.js";+import { logger } from "../logger.js";const RRF_K = 60;+const GRAPH_BUDGET_MS = 500;export class HybridSearch {private graphRetrieval: GraphRetrieval;+ private lastGraphTimeoutWarningAt = -Infinity;constructor(private bm25: SearchIndex,@@ -34,11 +38,11 @@ export class HybridSearch {private graphWeight = 0.3,private rerankEnabled = process.env.RERANK_ENABLED === "true",) {- this.graphRetrieval = new GraphRetrieval(kv, (id) => this.bm25.layerOf(id));+ this.graphRetrieval = new GraphRetrieval(kv, (id) => this.bm25.layerOf(id), (id) => this.bm25.sessionOf(id));}- async search(query: string, limit = 20, targetLayer: SearchLayer = "all"): Promise<HybridSearchResult[]> {- return this.tripleStreamSearch(query, limit, targetLayer, this.graphRetrieval.createQuery());+ async search(query: string, limit = 20, targetLayer: SearchLayer = "all", authorizeGraph?: GraphObservationAuthorization): Promise<HybridSearchResult[]> {+ return this.tripleStreamSearch(query, limit, targetLayer, this.graphRetrieval.createQuery(authorizeGraph));}async searchWithExpansion(@@ -46,6 +50,7 @@ export class HybridSearch {limit: number,expansion: QueryExpansion,targetLayer: SearchLayer = "all",+ authorizeGraph?: GraphObservationAuthorization,): Promise<HybridSearchResult[]> {const allQueries = [query,@@ -58,7 +63,7 @@ export class HybridSearch {...extractEntitiesFromQuery(query),];- const graphQuery = this.graphRetrieval.createQuery();+ const graphQuery = this.graphRetrieval.createQuery(authorizeGraph);const retrievalLimit = this.rerankEnabled ? Math.max(limit, 20) : limit;const resultSets = await Promise.all(allQueries.map((q) => this.tripleStreamSearch(q, retrievalLimit, targetLayer, graphQuery, allEntities, false)),@@ -114,33 +119,10 @@ export class HybridSearch {entityHints && entityHints.length > 0? entityHints: extractEntitiesFromQuery(query);- let graphResults: GraphRetrievalResult[] = [];- // Both traversals list every node and edge. At weight 0 their scores are- // discarded anyway, so skip the reads rather than pay for them.- const graphEnabled = this.graphWeight > 0;- if (graphEnabled && entities.length > 0) {- try {- graphResults = await graphQuery.searchByEntities(- entities,- 2,- retrievalLimit,- targetLayer,- );- } catch {- // graph search is best-effort- }- }-const topVectorObs = vectorResults.slice(0, 5).map((r) => r.obsId);- if (graphEnabled && topVectorObs.length > 0) {- try {- const expansionResults =- await graphQuery.expandFromChunks(topVectorObs, 1, 5, targetLayer);- graphResults = [...graphResults, ...expansionResults];- } catch {- // expansion is best-effort- }- }+ const graphResults = this.graphWeight > 0+ ? await this.graphResults(graphQuery, entities, topVectorObs, retrievalLimit, targetLayer)+ : [];const scores = new Map<string,@@ -259,6 +241,44 @@ export class HybridSearch {return applyReranking ? this.rerankResults(query, enriched, limit) : enriched.slice(0, limit);}+ private async graphResults(+ query: GraphQuery,+ entities: string[],+ observations: string[],+ limit: number,+ layer: SearchLayer,+ ): Promise<GraphRetrievalResult[]> {+ if (!entities.length && !observations.length) return [];+ let timer: ReturnType<typeof setTimeout> | undefined;+ try {+ const work = Promise.all([+ entities.length ? query.searchByEntities(entities, 2, limit, layer).catch(() => []) : [],+ observations.length ? query.expandFromChunks(observations, 1, 5, layer).catch(() => []) : [],+ ]).then((results) => results.flat());+ const results = await Promise.race([+ work,+ new Promise<GraphRetrievalResult[]>((resolve) => {+ timer = setTimeout(() => {+ query.cancel();+ if (Date.now() - this.lastGraphTimeoutWarningAt >= 60_000) {+ this.lastGraphTimeoutWarningAt = Date.now();+ logger.warn("Graph enrichment exceeded its budget; returning keyword and vector results", { budgetMs: GRAPH_BUDGET_MS });+ }+ resolve([]);+ }, GRAPH_BUDGET_MS);+ }),+ ]);+ const diagnostic = query.diagnostics();+ if ((diagnostic.status !== "ready" || diagnostic.truncated) && Date.now() - this.lastGraphTimeoutWarningAt >= 60_000) {+ this.lastGraphTimeoutWarningAt = Date.now();+ logger.warn("Graph enrichment is unavailable or partial", diagnostic);+ }+ return results;+ } finally {+ clearTimeout(timer);+ }+ }+private async rerankResults(query: string, results: HybridSearchResult[], limit: number): Promise<HybridSearchResult[]> {if (this.rerankEnabled && results.length > 1) {try {diff --git a/src/state/kv.ts b/src/state/kv.tsindex fe6b7e6c..7de13a22 100644--- a/src/state/kv.ts+++ b/src/state/kv.ts@@ -2,6 +2,7 @@ import type { IIIClient } from 'iii-sdk'import { withKeyedLock } from './keyed-mutex.js'import { resolveGraphScope, notifyGraphGenerationChange } from './graph-generation.js'import { ensureGraphConsistency, noteGraphSnapshotWrite } from './graph-consistency.js'+import { withGraphIndexMutation, withGraphSnapshotIndexBarrier, withGraphSnapshotUnknownMutation } from './graph-retrieval-index.js'export type StateBackend = 'file' | 'redis'@@ -126,6 +127,7 @@ export class StateKV {scope: string,key: string,operation: (resolvedScope: string) => Promise<T>,+ knownValue?: { value: unknown },): Promise<T> {scope = await resolveGraphScope(this, scope)await ensureGraphConsistency(this, scope)@@ -137,9 +139,14 @@ export class StateKV {noteGraphSnapshotWrite(this, scope, key)}}- const result = this.backend === 'redis'- ? await withKeyedLock(`state-update:${scope}\u0000${key}`, write)- : await write()+ const orderedWrite = () => this.backend === 'redis'+ ? withKeyedLock(`state-update:${scope}\u0000${key}`, write)+ : write()+ const result = await withGraphIndexMutation(this, scope, key, () =>+ knownValue+ ? withGraphSnapshotIndexBarrier(this, scope, key, knownValue.value, orderedWrite)+ : withGraphSnapshotUnknownMutation(this, scope, key, orderedWrite),+ )await notifyGraphGenerationChange(this, scope, key)return result}@@ -150,6 +157,7 @@ export class StateKV {function_id: 'state::set',payload: { scope: resolvedScope, key, value },}),+ { value },)}diff --git a/src/state/schema.ts b/src/state/schema.tsindex 924cc9cf..3485ac26 100644--- a/src/state/schema.ts+++ b/src/state/schema.ts@@ -23,6 +23,10 @@ export const KV = {graphEdgeKey: "mem:graph:edge-key",graphNodeDegree: "mem:graph:node-degree",graphMembership: "mem:graph:membership",+ graphRetrievalControl: "mem:graph:retrieval-control",+ graphRetrievalData: "mem:graph:retrieval-data",+ graphIndexMigration: "mem:graph:index-migration",+ graphIndexManifest: "mem:graph:index-manifest",semantic: "mem:semantic",procedural: "mem:procedural",teamShared: (teamId: string) => `mem:team:${teamId}:shared`,diff --git a/src/state/viewer-counts.ts b/src/state/viewer-counts.tsindex dbc66d33..e5af32df 100644--- a/src/state/viewer-counts.ts+++ b/src/state/viewer-counts.ts@@ -1,5 +1,7 @@import type { StateKV } from "./kv.js";import { KV } from "./schema.js";+import { getGraphGeneration, pinGraphStateGeneration } from "./graph-generation.js";+import { getGraphIndexStatus } from "./graph-retrieval-index.js";import { getAgentId, isAgentScopeIsolated } from "../config.js";import type {GraphSnapshot,@@ -19,14 +21,13 @@ export interface ViewerCounts {crystals: number;semantic: number;procedural: number;- graphNodes: number;- graphEdges: number;+ graphNodes: number | null;+ graphEdges: number | null;updatedAt: string;}export type CountedKind = "lessons" | "actions" | "crystals" | "semantic" | "procedural";-const GRAPH_SNAPSHOT_KEY = "current";const EMIT_DELAY_MS = 500;const MAX_LOAD_ATTEMPTS = 2;@@ -59,8 +60,31 @@ async function listOrEmpty<T>(kv: StateKV, scope: string): Promise<T[]> {return Array.isArray(rows) ? rows : [];}+export async function readViewerGraphMetadata(kv: StateKV): Promise<{+ generation: string | null;+ stats: GraphSnapshot["stats"] | null;+ updatedAt: string | null;+ graphIndex: { status: string; incomplete: boolean };+}> {+ for (let attempt = 0; attempt < 2; attempt++) {+ const generation = await getGraphGeneration(kv);+ const control = await getGraphIndexStatus(pinGraphStateGeneration(kv, generation));+ if (generation !== await getGraphGeneration(kv)) continue;+ const ready = control?.status === "ready" && !control.pending && !control.batchPending;+ return {+ generation,+ stats: ready ? { totalNodes: control.nodes, totalEdges: control.edges,+ nodesByType: control.nodesByType, edgesByType: control.edgesByType } : null,+ updatedAt: control?.updatedAt ?? null,+ graphIndex: { status: control?.pending || control?.batchPending ? "dirty" : control?.status ?? "needs-migration",+ incomplete: (control?.incompleteRecords ?? 0) > 0 },+ };+ }+ throw new Error("Graph generation changed while reading index metadata");+}+async function readCounts(kv: StateKV): Promise<ViewerCounts> {- const [sessions, memories, lessons, actions, crystals, semantic, procedural, snapshot] =+ const [sessions, memories, lessons, actions, crystals, semantic, procedural, graph] =await Promise.all([listOrEmpty<Session>(kv, KV.sessions),listOrEmpty<Memory>(kv, KV.memories),@@ -69,7 +93,7 @@ async function readCounts(kv: StateKV): Promise<ViewerCounts> {listOrEmpty<unknown>(kv, KV.crystals),listOrEmpty<unknown>(kv, KV.semantic),listOrEmpty<unknown>(kv, KV.procedural),- kv.get<GraphSnapshot>(KV.graphSnapshot, GRAPH_SNAPSHOT_KEY).catch(() => null),+ readViewerGraphMetadata(kv).catch(() => null),]);const scopedSessions = sessions.filter(inAgentScope);const scopedMemories = memories.filter(inAgentScope);@@ -84,14 +108,17 @@ async function readCounts(kv: StateKV): Promise<ViewerCounts> {crystals: crystals.length,semantic: semantic.length,procedural: procedural.length,- graphNodes: snapshot?.stats?.totalNodes ?? 0,- graphEdges: snapshot?.stats?.totalEdges ?? 0,+ graphNodes: graph?.stats?.totalNodes ?? null,+ graphEdges: graph?.stats?.totalEdges ?? null,updatedAt: new Date().toISOString(),};}export async function getViewerCounts(kv: StateKV): Promise<ViewerCounts> {- if (counts) return { ...counts };+ if (counts) {+ noteGraphStats((await readViewerGraphMetadata(kv).catch(() => null))?.stats);+ if (counts) return { ...counts };+ }if (!loading) {loading = (async () => {let result = await readCounts(kv);@@ -174,11 +201,11 @@ export function noteRowChange(kind: CountedKind, before: unknown, after: unknown}export function noteGraphStats(stats: GraphSnapshot["stats"] | null | undefined): void {- if (!stats) return;applyDelta((c) => {- if (c.graphNodes === stats.totalNodes && c.graphEdges === stats.totalEdges) return false;- c.graphNodes = stats.totalNodes;- c.graphEdges = stats.totalEdges;+ const nodes = stats?.totalNodes ?? null, edges = stats?.totalEdges ?? null;+ if (c.graphNodes === nodes && c.graphEdges === edges) return false;+ c.graphNodes = nodes;+ c.graphEdges = edges;return true;});}diff --git a/src/triggers/api.ts b/src/triggers/api.tsindex 33f751b6..fdcd003d 100644--- a/src/triggers/api.ts+++ b/src/triggers/api.ts@@ -48,6 +48,9 @@ import { MAX_FILES_UPPER_BOUND } from "../functions/replay.js";import { describeGraphNode } from "../functions/graph-node.js";import { COMPACT_SCOPES, type GraphCompactScope } from "../functions/graph.js";import { getGraphCompactBootStatus } from "../functions/graph-compact-boot.js";+import { getGraphIndexMigrationStatus } from "../state/graph-index-migration.js";+import { GraphIndexError } from "../state/graph-retrieval-index.js";+import { registerGraphIndexApi } from "./graph-index-api.js";import { LESSON_SOURCE_IDS_MAX, normalizeLessonSourceIds } from "../functions/lessons.js";import { logger } from "../logger.js";import { isValidEventId } from "../capture/event-id.js";@@ -385,7 +388,7 @@ export function createStatusReporter(sdk: IIIClient, kv: StateKV, deps: StatusReconst cached = scan.last && options.scanMaxAgeMs !== undefined && Date.now() - scan.last.at < options.scanMaxAgeMs? scan.last.value: null;- const [health, functionMetrics, graph, unindexed, auditMigrationState, stateStore, capture] = await Promise.all([+ const [health, functionMetrics, graph, unindexed, auditMigrationState, stateStore, capture, graphIndex] = await Promise.all([options.health !== undefined ? Promise.resolve(options.health) : valueWithin(getLatestHealth(kv), STATUS_CHECK_TIMEOUT_MS),deps.metricsStore ? valueWithin(deps.metricsStore.getAll(), STATUS_CHECK_TIMEOUT_MS) : Promise.resolve([]),valueWithin(@@ -399,6 +402,7 @@ export function createStatusReporter(sdk: IIIClient, kv: StateKV, deps: StatusRe),probeStateStore(kv),valueWithin(getCaptureController()?.status() ?? Promise.resolve(null), STATUS_CHECK_TIMEOUT_MS),+ valueWithin(getGraphIndexMigrationStatus(kv), STATUS_CHECK_TIMEOUT_MS),]);const observationsIndexed = [...idx.observationCountsBySession().values()].reduce((a, n) => a + n, 0);const documentKinds = idx.documentKindCounts();@@ -449,6 +453,7 @@ export function createStatusReporter(sdk: IIIClient, kv: StateKV, deps: StatusRevectorBackfillState: getVectorBackfillState(),},graph,+ graphIndex,graphCompaction: getGraphCompactBootStatus(),graphExtractionEnabled: isGraphExtractionEnabled(),auditLegacy: auditMigrationState@@ -520,6 +525,10 @@ export function registerApiTriggers(metricsStore?: MetricsStore,provider?: ResilientProvider | { circuitState?: unknown },): void {+ registerGraphIndexApi(sdk, secret, checkAuth);+ sdk.registerTrigger({ type: "http", function_id: "api::graph-index-status", config: { api_path: "/agentmemory/graph/index", http_method: "GET" } });+ sdk.registerTrigger({ type: "http", function_id: "api::graph-index-migrate", config: { api_path: "/agentmemory/graph/index/migrate", http_method: "POST" } });+ sdk.registerTrigger({ type: "http", function_id: "api::graph-index-resume", config: { api_path: "/agentmemory/graph/index/resume", http_method: "POST" } });sdk.registerFunction("middleware::api-auth",async (input: {@@ -2169,9 +2178,21 @@ export function registerApiTriggers(if (authErr) return authErr;const id = asNonEmptyString(req.query_params?.["id"]);if (!id) return { status_code: 400, body: { error: "id required" } };- const detail = await describeGraphNode(kv, id);- if (!detail) return { status_code: 404, body: { error: "graph node not found", id } };- return { status_code: 200, body: detail };+ const cursor = req.query_params?.["cursor"];+ const limit = parseOptionalPositiveInt(req.query_params?.["limit"]);+ if (cursor !== undefined && typeof cursor !== "string") return { status_code: 400, body: { error: "cursor must be a string" } };+ if (limit === null || (limit !== undefined && limit > 200)) return { status_code: 400, body: { error: "limit must be an integer between 1 and 200" } };+ try {+ const detail = await describeGraphNode(kv, id, { cursor: cursor || undefined, limit });+ if (!detail) return { status_code: 404, body: { error: "graph node not found", id } };+ if ("error" in detail) return { status_code: 503, body: detail };+ return { status_code: 200, body: detail };+ } catch (error) {+ if (error instanceof GraphIndexError && error.message === "Invalid or expired graph posting cursor") {+ return { status_code: 400, body: { error: "Invalid or expired graph cursor" } };+ }+ return { status_code: 503, body: { error: "Graph node index is unavailable" } };+ }},);sdk.registerTrigger({diff --git a/src/triggers/graph-index-api.ts b/src/triggers/graph-index-api.tsnew file mode 100644index 00000000..2a9a1f9f--- /dev/null+++ b/src/triggers/graph-index-api.ts@@ -0,0 +1,44 @@+import type { IIIClient } from "iii-sdk";+import type { HttpRequest } from "@iii-dev/helpers/http";++type Response = { status_code: number; headers?: Record<string, string>; body: unknown };++export function registerGraphIndexApi(+ sdk: IIIClient,+ secret: string | undefined,+ checkAuth: (req: HttpRequest, secret: string | undefined) => Response | null,+): void {+ sdk.registerFunction("api::graph-index-status", async (req: HttpRequest): Promise<Response> => {+ const denied = checkAuth(req, secret);+ if (denied) return denied;+ try {+ const result = await sdk.trigger({ function_id: "mem::graph-index-status", payload: {} });+ return { status_code: 200, headers: { "Cache-Control": "no-store" }, body: result };+ } catch {+ return { status_code: 503, body: { error: "Graph index status is unavailable" } };+ }+ });+ const mutate = (command: "migrate" | "resume") => async (req: HttpRequest<Record<string, unknown>>): Promise<Response> => {+ const denied = checkAuth(req, secret);+ if (denied) return denied;+ const body = req.body ?? {};+ if (!body || typeof body !== "object" || Array.isArray(body) ||+ (body.allowLegacyScan !== undefined && typeof body.allowLegacyScan !== "boolean")) {+ return { status_code: 400, body: { error: "allowLegacyScan must be a boolean" } };+ }+ try {+ const result = await sdk.trigger<unknown, { success?: boolean; code?: string; state?: string }>({+ function_id: `mem::graph-index-${command}`,+ payload: { allowLegacyScan: body.allowLegacyScan === true },+ });+ const status = result.success === false+ ? result.code === "storage-unavailable" ? 503 : result.code === "invalid-state" || result.code === "no-job" ? 409 : 400+ : result.state === "ready" ? 200 : 202;+ return { status_code: status, headers: { "Cache-Control": "no-store" }, body: result };+ } catch {+ return { status_code: 503, body: { error: "Graph index migration could not be started" } };+ }+ };+ sdk.registerFunction("api::graph-index-migrate", mutate("migrate"));+ sdk.registerFunction("api::graph-index-resume", mutate("resume"));+}diff --git a/src/triggers/viewer-streams.ts b/src/triggers/viewer-streams.tsindex a75602bf..d48f0f16 100644--- a/src/triggers/viewer-streams.ts+++ b/src/triggers/viewer-streams.ts@@ -30,6 +30,7 @@ import {getViewerCounts,invalidateViewerCounts,noteGraphStats,+ readViewerGraphMetadata,noteRowChange,setViewerCountsEmitter,type CountedKind,@@ -69,57 +70,6 @@ function eventId(prefix: string, key: string): string {return `${prefix}-${key}-${Date.now()}`;}-const GRAPH_DELTA_MAX_NODES = 200;-const GRAPH_DELTA_MAX_EDGES = 500;--type GraphSnapshotDelta = {- full: boolean;- nodes: GraphSnapshot["topNodes"];- edges: GraphSnapshot["topEdges"];- removedNodeIds: string[];- removedEdgeIds: string[];- degrees: Record<string, number>;-};--function rowsChanged<T extends { id: string }>(- before: T[] | undefined,- after: T[] | undefined,-): { upserted: T[]; removed: string[] } {- const prev = new Map<string, string>();- for (const row of before ?? []) prev.set(row.id, JSON.stringify(row));- const seen = new Set<string>();- const upserted: T[] = [];- for (const row of after ?? []) {- seen.add(row.id);- if (prev.get(row.id) !== JSON.stringify(row)) upserted.push(row);- }- const removed = [...prev.keys()].filter((id) => !seen.has(id));- return { upserted, removed };-}--export function graphSnapshotDelta(- before: GraphSnapshot | null,- after: GraphSnapshot | null,-): GraphSnapshotDelta | null {- if (!after) return null;- const nodes = rowsChanged(before?.topNodes, after.topNodes);- const edges = rowsChanged(before?.topEdges, after.topEdges);- if (nodes.upserted.length > GRAPH_DELTA_MAX_NODES || edges.upserted.length > GRAPH_DELTA_MAX_EDGES) return null;- const degrees: Record<string, number> = {};- const prevDegrees = before?.topDegrees ?? {};- for (const [id, d] of Object.entries(after.topDegrees ?? {})) {- if (prevDegrees[id] !== d) degrees[id] = d;- }- return {- full: !before,- nodes: nodes.upserted,- edges: edges.upserted,- removedNodeIds: nodes.removed,- removedEdgeIds: edges.removed,- degrees,- };-}-function indexStats() {return {bm25Documents: getSearchIndex().size,@@ -198,17 +148,33 @@ export function registerViewerStreamTriggers(});let lastGraphSnapshotEventAt = Number.NEGATIVE_INFINITY;- let lastGenerationSnapshot: { generation: string | null; snapshot: GraphSnapshot | null } | null = null;+ let lastGraphGeneration: string | null | undefined;- async function readGenerationSnapshot() {- for (let attempt = 0; attempt < 2; attempt++) {- const generation = await kv.get<string>(KV.graphControl, "generation");- const snapshot = await kv.get<GraphSnapshot>(KV.graphSnapshot, GRAPH_SNAPSHOT_KEY);- if (generation === await kv.get<string>(KV.graphControl, "generation")) {- return { generation, snapshot };- }+ async function publishGraphMetadata(reset = false): Promise<void> {+ let graph: Awaited<ReturnType<typeof readViewerGraphMetadata>>;+ try {+ graph = await readViewerGraphMetadata(kv);+ } catch (error) {+ invalidateViewerCounts();+ lastGraphSnapshotEventAt = Number.NEGATIVE_INFINITY;+ await sendViewerEvent(sdk, eventId("graph", "invalidation"), "graph.changed", {+ source: "items", coveredBySnapshot: false, ...(reset ? { reset: true } : {}),+ updatedAt: new Date().toISOString(),+ }).catch(() => {});+ throw error;+ }+ reset ||= lastGraphGeneration !== undefined && lastGraphGeneration !== graph.generation;+ noteGraphStats(graph.stats);+ try {+ await sendViewerEvent(sdk, eventId("graph", "index"), "graph.changed", {+ source: "snapshot", ...graph, reset, delta: null,+ });+ lastGraphSnapshotEventAt = Date.now();+ lastGraphGeneration = graph.generation;+ } catch (error) {+ lastGraphSnapshotEventAt = Number.NEGATIVE_INFINITY;+ throw error;}- throw new Error("Graph generation changed while reading snapshot");}sdk.registerFunction("event::viewer::graph-snapshot-changed",@@ -217,21 +183,7 @@ export function registerViewerStreamTriggers(if (!await isLegacyGraphState(kv)) return { emitted: false };const snapshot = isStateDelete(payload) ? null : payload.new_value ?? null;if (snapshot?.pendingMutation) return { emitted: false };- noteGraphStats(snapshot?.stats ?? { totalNodes: 0, totalEdges: 0, nodesByType: {}, edgesByType: {} });- try {- await sendViewerEvent(sdk, eventId("graph", "snapshot"), "graph.changed", {- source: "snapshot",- generation: null,- stats: snapshot?.stats ?? null,- updatedAt: snapshot?.updatedAt ?? new Date().toISOString(),- resetAt: snapshot?.resetAt ?? null,- delta: graphSnapshotDelta(payload.old_value ?? null, snapshot),- });- lastGraphSnapshotEventAt = Date.now();- } catch (error) {- lastGraphSnapshotEventAt = Number.NEGATIVE_INFINITY;- throw error;- }+ await publishGraphMetadata();return { emitted: true };}),);@@ -243,7 +195,7 @@ export function registerViewerStreamTriggers(const pendingGraphItems = { nodes: 0, edges: 0, since: 0 };let graphItemsTimer: ReturnType<typeof setTimeout> | null = null;- function flushGraphItems(): void {+ async function flushGraphItems(): Promise<void> {graphItemsTimer = null;const nodesChanged = pendingGraphItems.nodes;const edgesChanged = pendingGraphItems.edges;@@ -251,7 +203,8 @@ export function registerViewerStreamTriggers(pendingGraphItems.nodes = 0;pendingGraphItems.edges = 0;if (nodesChanged === 0 && edgesChanged === 0) return;- void sendViewerEvent(sdk, eventId("graph", "items"), "graph.changed", {+ noteGraphStats((await readViewerGraphMetadata(kv).catch(() => null))?.stats);+ await sendViewerEvent(sdk, eventId("graph", "items"), "graph.changed", {source: "items",nodesChanged,edgesChanged,@@ -263,7 +216,7 @@ export function registerViewerStreamTriggers(pendingGraphItems[kind] += 1;if (graphItemsTimer) return;pendingGraphItems.since = Date.now();- graphItemsTimer = setTimeout(flushGraphItems, GRAPH_ITEMS_COALESCE_MS);+ graphItemsTimer = setTimeout(() => { void flushGraphItems(); }, GRAPH_ITEMS_COALESCE_MS);if (typeof graphItemsTimer === "object" && "unref" in graphItemsTimer) graphItemsTimer.unref();}for (const [kind, scope] of [["nodes", KV.graphNodes], ["edges", KV.graphEdges]] as const) {@@ -287,41 +240,7 @@ export function registerViewerStreamTriggers(pendingGraphItems.nodes = 0;pendingGraphItems.edges = 0;}- let current: Awaited<ReturnType<typeof readGenerationSnapshot>>;- try {- current = await readGenerationSnapshot();- } catch (error) {- invalidateViewerCounts();- lastGraphSnapshotEventAt = Number.NEGATIVE_INFINITY;- await sendViewerEvent(sdk, eventId("graph", "invalidation"), "graph.changed", {- source: "items",- coveredBySnapshot: false,- ...(kind === "reset" ? { reset: true } : {}),- updatedAt: new Date().toISOString(),- }).catch(() => {});- throw error;- }- const { generation, snapshot } = current;- const reset = kind === "reset" || (lastGenerationSnapshot !== null && lastGenerationSnapshot.generation !== generation);- const before = !reset && lastGenerationSnapshot?.generation === generation ? lastGenerationSnapshot.snapshot : null;- const nextBaseline = { generation, snapshot: structuredClone(snapshot) };- noteGraphStats(snapshot?.stats ?? { totalNodes: 0, totalEdges: 0, nodesByType: {}, edgesByType: {} });- try {- await sendViewerEvent(sdk, eventId("graph", "snapshot"), "graph.changed", {- source: "snapshot",- generation,- reset,- stats: snapshot?.stats ?? null,- updatedAt: snapshot?.updatedAt ?? new Date().toISOString(),- resetAt: snapshot?.resetAt ?? null,- delta: graphSnapshotDelta(before, snapshot),- });- lastGraphSnapshotEventAt = Date.now();- lastGenerationSnapshot = nextBaseline;- } catch (error) {- lastGraphSnapshotEventAt = Number.NEGATIVE_INFINITY;- throw error;- }+ await publishGraphMetadata(kind === "reset");}));sdk.registerFunction(@@ -373,7 +292,7 @@ export function registerViewerStreamTriggers(if (authErr) return authErr;const [health, graph, lastRun] = await Promise.all([getLatestHealth(kv).catch(() => null),- readGenerationSnapshot().catch(() => null),+ readViewerGraphMetadata(kv).catch(() => null),kv.get<ConsolidationRunRecord>(KV.config, CONSOLIDATION_LAST_RUN_KEY).catch(() => null),]);return {@@ -388,9 +307,7 @@ export function registerViewerStreamTriggers(provider: detectLlmProviderKind(),embeddingProvider: getEmbeddingProvider()?.name ?? "none",flags: buildConfigFlags(),- graph: graph?.snapshot- ? { stats: graph.snapshot.stats, updatedAt: graph.snapshot.updatedAt, resetAt: graph.snapshot.resetAt ?? null, generation: graph.generation }- : null,+ graph,consolidation: {lastRun: lastRun ?? null,status: await consolidationStatus({ lastRun: lastRun ?? null }).catch(() => null),diff --git a/src/types.ts b/src/types.tsindex 87d171b1..25c5a999 100644--- a/src/types.ts+++ b/src/types.ts@@ -510,6 +510,41 @@ export interface GraphIndexMembership {ids: string[];}+export interface GraphRetrievalIndexControl {+ schema: 2;+ buildId: string;+ status: "building" | "ready" | "paused";+ revision: number;+ nodes: number;+ edges: number;+ historyEdges: number;+ incompleteRecords: number;+ nodesByType: Record<string, number>;+ edgesByType: Record<string, number>;+ otherTypes: number;+ updatedAt: string;+ pending?: { kind: "node" | "edge" | "history"; id: string };+ batchPending?: boolean;+ error?: string;+}++export interface GraphRetrievalIndexRecord {+ row: (GraphNode | GraphEdge) & { indexIncomplete?: boolean } | null;+ postings: string[];+ incomplete: boolean;+}++export interface GraphCascadeProgress {+ schema: 1;+ generation: string | null;+ sourceFingerprint: string;+ offset: number;+ reason: string;+ updatedAt: string;+}++export type { GraphIndexMigration, GraphIndexManifestPage } from "./state/graph-index-migration.js";+export interface GraphMutationWrite {scope: string;key: string;@@ -666,6 +701,7 @@ export interface AuditEntry {| "slot_delete"| "slot_reflect"| "graph_compact"+ | "graph_index_migrate"| "audit_migrate"| "session_sweep";userId?: string;diff --git a/src/viewer/index.html b/src/viewer/index.htmlindex 346be2bd..e4eb16e8 100644--- a/src/viewer/index.html+++ b/src/viewer/index.html@@ -7222,17 +7222,18 @@'New and removed vectors are written here right away and replayed after a crash, so nothing is embedded twice. Each save empties it.');}}- html += hrow('Graph', esc(fmtNum(c.graphNodes) + ' nodes · ' + fmtNum(c.graphEdges) + ' edges'), 'graph', store.graph && store.graph.updatedAt ? 'Snapshot rebuilt ' + esc(timeAgo(store.graph.updatedAt)) + '.' : '');+ html += hrow('Graph', esc(fmtNum(c.graphNodes) + ' nodes · ' + fmtNum(c.graphEdges) + ' edges'), 'graph', store.graph && store.graph.updatedAt ? 'Graph index updated ' + esc(timeAgo(store.graph.updatedAt)) + '.' : '');if (r && r.graphCompaction) html += graphCompactionRowHtml(r.graphCompaction);return html + '</div>';}function graphCompactionRowHtml(g) {var where = g.phase === 'snapshot' || g.total === null || g.total === undefined ? (g.phase || 'nodes') : g.phase + ' ' + fmtNum(g.processed || 0) + ' of ' + fmtNum(g.total);- var text = g.state === 'off' ? 'off' : g.state === 'pending' ? 'pending' : g.state === 'running' ? 'running, ' + where : g.state === 'failed' ? 'failed at ' + where : 'done';+ var text = g.state === 'off' ? 'off' : g.state === 'pending' ? 'pending' : g.state === 'deferred' ? 'deferred' : g.state === 'running' ? 'running, ' + where : g.state === 'failed' ? 'failed at ' + where : 'done';var note = 'A one-time pass that trims each graph node and edge to its newest 32 source observations, so stores from older versions stop slowing graph search.';if (g.state === 'done') note = fmtNum(g.trimmed || 0) + ' records trimmed, ' + fmtNum(g.idsRemoved || 0) + ' ids removed' + (g.completedAt ? ', finished ' + timeAgo(g.completedAt) : '') + '. It does not run again.';else if (g.state === 'running') note += ' It runs in the background, resumes after a restart and stops by itself.';+ else if (g.state === 'deferred') note = 'Compaction is deferred because it reads complete legacy graph scopes. Run it only during explicit maintenance.';else if (g.state === 'off') note = 'Turned off with AGENTMEMORY_GRAPH_COMPACT_ON_BOOT=false. Run it by hand if graph search is slow:';else if (g.state === 'failed') note = esc(g.error || 'unknown error') + '. It resumes on the next start. To run it now:';var cmd = g.state === 'off' || g.state === 'failed' ? cmdBlock(curlPost('graph/compact', {})) : '';diff --git a/test/api-graph-node.test.ts b/test/api-graph-node.test.tsindex 6ca6d9f8..51e8eb6e 100644--- a/test/api-graph-node.test.ts+++ b/test/api-graph-node.test.ts@@ -1,19 +1,33 @@import { describe, it, expect, vi } from "vitest";import { registerApiTriggers } from "../src/triggers/api.js";import { getSearchIndex } from "../src/functions/search.js";+import { initializeGraphIndex, reconcileGraphIndexRecord } from "../src/state/graph-retrieval-index.js";+import { memoryToObservation } from "../src/state/memory-utils.js";+import { indexObservationSession } from "../src/state/obs-index.js";import { KV } from "../src/state/schema.js";type Handler = (req: unknown) => Promise<{ status_code: number; body: Record<string, unknown> }>;-function setup(seed: Record<string, Record<string, unknown>>) {+async function setup(seed: Record<string, Record<string, unknown>>) {const store = new Map<string, Map<string, unknown>>();for (const [scope, rows] of Object.entries(seed)) store.set(scope, new Map(Object.entries(rows)));const kv = {get: vi.fn(async (scope: string, key: string) => store.get(scope)?.get(key) ?? null),- set: vi.fn(async (_scope: string, _key: string, value: unknown) => value),- delete: vi.fn(async () => {}),- list: vi.fn(async (scope: string) => Array.from(store.get(scope)?.values() ?? [])),+ set: vi.fn(async (scope: string, key: string, value: unknown) => {+ if (!store.has(scope)) store.set(scope, new Map());+ store.get(scope)!.set(key, value);+ return value;+ }),+ delete: vi.fn(async (scope: string, key: string) => { store.get(scope)?.delete(key); }),+ list: vi.fn(async () => { throw new Error("Unbounded list forbidden"); }),};+ await initializeGraphIndex(kv as never, { status: "ready" });+ for (const [id, row] of Object.entries(seed[KV.graphNodes] ?? {})) await reconcileGraphIndexRecord(kv as never, "node", id, row as never);+ for (const [id, row] of Object.entries(seed[KV.graphEdges] ?? {})) await reconcileGraphIndexRecord(kv as never, "edge", id, row as never);+ for (const row of Object.values(seed[KV.memories] ?? {})) {+ const memory = row as any;+ getSearchIndex().add(memoryToObservation({ content: "", strength: 5, sessionIds: [], ...memory }), "memory");+ }const handlers = new Map<string, Handler>();const sdk = {registerFunction: (id: string, fn: Handler) => handlers.set(id, fn),@@ -22,8 +36,8 @@ function setup(seed: Record<string, Record<string, unknown>>) {on: () => {},};registerApiTriggers(sdk as never, kv as never);- return (query_params: Record<string, string>) =>- handlers.get("api::graph-node")!({ headers: {}, query_params, body: {} });+ return Object.assign((query_params: Record<string, string>) =>+ handlers.get("api::graph-node")!({ headers: {}, query_params, body: {} }), { kv });}const node = (id: string, name: string, type: string, extra: Record<string, unknown> = {}) => ({@@ -31,10 +45,45 @@ const node = (id: string, name: string, type: string, extra: Record<string, unkn});describe("GET /agentmemory/graph/node", () => {+ it("resolves durable observation provenance with a cold search index", async () => {+ const call = await setup({+ [KV.graphNodes]: { cold: node("cold", "Cold source", "concept", { sourceObservationIds: ["obs_cold"] }) },+ [KV.observations("session_cold")]: { obs_cold: { id: "obs_cold", sessionId: "session_cold", title: "Durable evidence" } },+ [KV.sessions]: { session_cold: { id: "session_cold", project: "cold-project" } },+ });+ await indexObservationSession(call.kv as never, "obs_cold", "session_cold");+ getSearchIndex().clear();+ const result = await call({ id: "cold" });+ expect(result.status_code).toBe(200);+ expect(result.body).toMatchObject({ provenance: {+ observationIds: ["obs_cold"], observations: [{ id: "obs_cold", sessionId: "session_cold", title: "Durable evidence" }], projects: ["cold-project"],+ } });+ expect(call.kv.list).not.toHaveBeenCalled();+ await call.kv.delete(KV.observations("session_cold"), "obs_cold");+ expect((await call({ id: "cold" })).status_code).toBe(404);+ });++ it("resolves canonical memory provenance with a cold search index and rejects superseded sources", async () => {+ const memory = { id: "mem_cold", title: "Durable memory", type: "fact", project: "cold-project", isLatest: true, concepts: [], files: [] };+ const call = await setup({+ [KV.graphNodes]: { cold: node("cold", "Cold memory", "concept", { sourceObservationIds: [memory.id] }) },+ [KV.memories]: { [memory.id]: memory },+ });+ getSearchIndex().clear();+ const result = await call({ id: "cold" });+ expect(result.status_code).toBe(200);+ expect(result.body).toMatchObject({ provenance: {+ observationIds: [memory.id], observations: [], memories: [{ id: memory.id, title: memory.title, project: memory.project, via: "observation" }],+ } });+ expect(call.kv.list).not.toHaveBeenCalled();+ await call.kv.set(KV.memories, memory.id, { ...memory, isLatest: false });+ expect((await call({ id: "cold" })).status_code).toBe(404);+ });+it("returns relations with neighbors and resolves provenance to observations, sessions, projects and memories", async () => {const index = getSearchIndex();index.add({ id: "obs_gn_1", sessionId: "ses_gn", timestamp: "2026-09-01T00:00:00Z", type: "file_edit", title: "Edit checkout", facts: [], narrative: "edited", concepts: [], files: [], importance: 5 } as never);- const call = setup({+ const call = await setup({[KV.graphNodes]: {gn_a: node("gn_a", "/work/web-app/src/components/Checkout.tsx", "file", { sourceObservationIds: ["obs_gn_1"] }),gn_b: node("gn_b", "pricing", "concept"),@@ -76,7 +125,6 @@ describe("GET /agentmemory/graph/node", () => {]);expect(body.provenance.projects).toEqual(["web-app"]);expect(body.provenance.memories).toEqual([- { id: "mem_1", title: "Coupons first", type: "bug", project: "web-app", via: "observation" },{ id: "mem_2", title: "Checkout grid", type: "pattern", via: "name" },]);expect(body.provenance.structural).toBeNull();@@ -87,6 +135,26 @@ describe("GET /agentmemory/graph/node", () => {expect((await call({})).status_code).toBe(400);expect((await call({ id: "gn_missing" })).status_code).toBe(404);- index.remove("obs_gn_1");+ index.clear();});});+++it("paginates high-degree adjacency without full graph or memory lists", async () => {+ const graphNodes = Object.fromEntries(Array.from({ length: 8 }, (_, i) => [`n${i}`, node(`n${i}`, `Node ${i}`, "concept")]));+ const graphEdges = Object.fromEntries(Array.from({ length: 7 }, (_, i) => [`e${i}`, {+ id: `e${i}`, sourceNodeId: "n0", targetNodeId: `n${i + 1}`, type: "uses", weight: 1, sourceObservationIds: [], createdAt: "2026-01-01",+ }]));+ const call = await setup({ [KV.graphNodes]: graphNodes, [KV.graphEdges]: graphEdges });+ const first = await call({ id: "n0", limit: "2" });+ expect(first.status_code).toBe(200);+ expect(first.body.relations).toHaveLength(2);+ expect(first.body.relationsTruncated).toBe(true);+ expect(first.body.relationsTotalExact).toBe(false);+ const second = await call({ id: "n0", limit: "2", cursor: first.body.relationsCursor as string });+ expect(second.status_code).toBe(200);+ const firstIds = (first.body.relations as any[]).map((row) => row.edgeId);+ expect((second.body.relations as any[]).every((row) => !firstIds.includes(row.edgeId))).toBe(true);+ expect((await call({ id: "n0", limit: "0" })).status_code).toBe(400);+ expect((await call({ id: "n0", cursor: "not-a-cursor" })).status_code).toBe(400);+});diff --git a/test/cascade.test.ts b/test/cascade.test.tsindex 8462abc7..a46ba3bd 100644--- a/test/cascade.test.ts+++ b/test/cascade.test.ts@@ -6,15 +6,24 @@ vi.mock("../src/logger.js", () => ({import { registerCascadeFunction } from "../src/functions/cascade.js";import type { Memory, GraphNode, GraphEdge } from "../src/types.js";-import { mockKV, mockSdk } from "./helpers/mocks.js";+import { mockSdk } from "./helpers/mocks.js";+import { createGraphIndexState } from "./helpers/graph-index-state.js";+import { initializeGraphIndex } from "../src/state/graph-retrieval-index.js";+import { KV } from "../src/state/schema.js";++const memoryWithSources = (ids: string[]): Memory => ({ id: "superseded", createdAt: "now", updatedAt: "now", type: "fact", title: "old", content: "old", concepts: [], files: [], sessionIds: [], strength: 1, version: 1, isLatest: false, sourceObservationIds: ids });+const graphNode = (id: string, observationId = "source"): GraphNode => ({ id, name: id, type: "concept", createdAt: "now", properties: {}, sourceObservationIds: [observationId] });describe("Cascade Update Function", () => {let sdk: ReturnType<typeof mockSdk>;- let kv: ReturnType<typeof mockKV>;+ let kv: ReturnType<typeof createGraphIndexState>["kv"];+ let fixture: ReturnType<typeof createGraphIndexState>;- beforeEach(() => {+ beforeEach(async () => {sdk = mockSdk();- kv = mockKV();+ fixture = createGraphIndexState();+ kv = fixture.kv;+ await initializeGraphIndex(kv, { status: "ready" });vi.clearAllMocks();registerCascadeFunction(sdk as never, kv as never);});@@ -272,4 +281,58 @@ describe("Cascade Update Function", () => {expect(result.success).toBe(true);expect(result.total).toBe(0);});++ it("defers legacy graphs without implicit enumeration or snapshot reads", async () => {+ const legacy = createGraphIndexState();+ registerCascadeFunction(sdk as never, legacy.kv);+ await legacy.kv.set(KV.memories, "superseded", memoryWithSources(["source"]));+ legacy.calls.length = 0;+ const result = await sdk.trigger("mem::cascade-update", { supersededMemoryId: "superseded" }) as any;+ expect(result).toMatchObject({ success: true, graphDeferred: true, retryable: true, progressSaved: true, total: 0 });+ expect(legacy.calls.some((c) => c.id === "state::list" || c.scope === KV.graphSnapshot)).toBe(false);+ expect([...legacy.store.get(KV.graphControl)!.keys()].some((key) => key.startsWith("cascade:"))).toBe(true);+ });++ it("drains a high-degree cascade in yielding batches without whole graph reads", async () => {+ await kv.set(KV.memories, "superseded", memoryWithSources(["source"]));+ for (let i = 0; i < 80; i++) await kv.set(KV.graphNodes, `n${i}`, graphNode(`n${i}`));+ fixture.calls.length = 0;+ const result = await sdk.trigger("mem::cascade-update", { supersededMemoryId: "superseded" }) as any;+ expect(result.flagged.nodes).toBe(80); expect(result.graphDeferred).toBe(false);+ expect(fixture.calls.some((c) => c.id === "state::list" || c.scope === KV.graphSnapshot)).toBe(false);+ expect([...fixture.store.get(KV.graphControl)!.keys()].some((key) => key.startsWith("cascade:"))).toBe(false);+ });++ it("continues beyond empty early sources and resumes saved offsets after interruption", async () => {+ const sources = Array.from({ length: 80 }, (_, i) => `o${i}`);+ await kv.set(KV.memories, "superseded", memoryWithSources(sources));+ await kv.set(KV.graphNodes, "last", graphNode("last", sources.at(-1)));+ let savedOffset = false;+ fixture.intercept(async (id, payload) => {+ if (id === "state::set" && payload.scope === KV.graphControl && payload.value?.offset === 64) savedOffset = true;+ else if (savedOffset && id === "state::get" && payload.scope === KV.graphRetrievalControl) throw new Error("offline");+ });+ const first = await sdk.trigger("mem::cascade-update", { supersededMemoryId: "superseded" }) as any;+ fixture.intercept();+ expect(savedOffset).toBe(true);+ expect(first).toMatchObject({ graphDeferred: true, flagged: { nodes: 0 } });+ const restarted = createGraphIndexState(fixture.store); registerCascadeFunction(sdk as never, restarted.kv);+ const second = await sdk.trigger("mem::cascade-update", { supersededMemoryId: "superseded" }) as any;+ expect(second).toMatchObject({ graphDeferred: false, flagged: { nodes: 1 } });+ });++ it.each(["delete", "replace-provenance"])("rechecks a candidate after concurrent %s without resurrecting or staling it", async (change) => {+ await kv.set(KV.memories, "superseded", memoryWithSources(["source"]));+ await kv.set(KV.graphNodes, "candidate", graphNode("candidate"));+ fixture.intercept(async (id, payload) => {+ if (id !== "state::get" || payload.scope !== KV.graphNodes || payload.key !== "candidate") return;+ fixture.intercept();+ if (change === "delete") fixture.store.get(KV.graphNodes)!.delete("candidate");+ else fixture.store.get(KV.graphNodes)!.set("candidate", graphNode("candidate", "other"));+ });+ const result = await sdk.trigger("mem::cascade-update", { supersededMemoryId: "superseded" }) as any;+ expect(result.flagged.nodes).toBe(0);+ const current = await kv.get<GraphNode>(KV.graphNodes, "candidate");+ if (change === "delete") expect(current).toBeNull(); else expect(current?.stale).toBeUndefined();+ });});diff --git a/test/cli-engine-startup.test.ts b/test/cli-engine-startup.test.tsindex b091484d..78938bc4 100644--- a/test/cli-engine-startup.test.ts+++ b/test/cli-engine-startup.test.ts@@ -107,7 +107,7 @@ describe("fresh native engine startup", () => {});it("keeps AGENTMEMORY_URL client-only and lets local port flags win", () => {- expect(source).toContain('const URL_CLIENT_COMMANDS = new Set(["status", "doctor", "mcp", "capture"])');+ expect(source).toContain('const URL_CLIENT_COMMANDS = new Set(["status", "doctor", "mcp", "capture", "graph-index"])');expect(source).toContain("hasExplicitLocalPortOverride");expect(source).toContain("shouldUseAgentmemoryUrl()");expect(source).toContain('process.env["III_REST_PORT"] = String(base)');diff --git a/test/graph-compact-boot.test.ts b/test/graph-compact-boot.test.tsindex fa81a2ab..3f5fe79f 100644--- a/test/graph-compact-boot.test.ts+++ b/test/graph-compact-boot.test.ts@@ -129,13 +129,35 @@ async function maxIds(kv: KV): Promise<number> {return max;}-const fast = { sliceSize: 3, pauseMs: 0, retryDelayMs: 0, sleep: async () => {} };+const fast = { allowLegacyScan: true, sliceSize: 3, pauseMs: 0, retryDelayMs: 0, sleep: async () => {} };const marker = (kv: KV) => kv.get<GraphCompactBootProgress>(CONFIG, GRAPH_COMPACT_BOOT_KEY);beforeEach(() => resetGraphCompactBootStatus());afterEach(() => vi.unstubAllEnvs());describe("graph compaction on boot", () => {+ it("defers legacy enumeration without reading any graph scope during normal startup", async () => {+ const kv = mockKV();+ await seed(kv, 4, 2, 1);+ kv.calls.get.length = 0;+ kv.calls.set.length = 0;+ const status = await runGraphCompactOnBoot(kv as never);+ expect(status.state).toBe("deferred");+ expect(kv.calls).toEqual({ get: [], set: [], list: [] });+ expect(describeGraphCompactBoot(status)).toMatch(/explicit maintenance/);+ });++ it("does not resume an old bulk compaction marker implicitly after restart", async () => {+ const kv = mockKV();+ await kv.set(CONFIG, GRAPH_COMPACT_BOOT_KEY, {+ version: GRAPH_COMPACT_BOOT_VERSION, cap: MAX_GRAPH_SOURCE_OBSERVATIONS,+ status: "running", phase: "nodes", offset: 100,+ });+ kv.calls.get.length = 0;+ kv.calls.set.length = 0;+ expect((await runGraphCompactOnBoot(kv as never)).state).toBe("deferred");+ expect(kv.calls).toEqual({ get: [], set: [], list: [] });+ });it("trims nodes, edges, edge history and the snapshot to the cap and marks it done", async () => {const kv = mockKV();await seed(kv, 10, 7, 5);diff --git a/test/graph-cross-path-locks.test.ts b/test/graph-cross-path-locks.test.tsindex cbab748d..d073588f 100644--- a/test/graph-cross-path-locks.test.ts+++ b/test/graph-cross-path-locks.test.ts@@ -7,57 +7,9 @@ vi.mock("../src/logger.js", () => ({import { persistGraphDelta } from "../src/functions/graph.js";import { registerCascadeFunction } from "../src/functions/cascade.js";import { KV } from "../src/state/schema.js";+import { createGraphIndexReader, initializeGraphIndex } from "../src/state/graph-retrieval-index.js";import type { GraphNode, GraphSnapshot, Memory } from "../src/types.js";--function clone<T>(value: T): T {- return value === null || value === undefined- ? value- : (JSON.parse(JSON.stringify(value)) as T);-}--function mockKV() {- const store = new Map<string, Map<string, unknown>>();- let gate: {- scope: string;- key: string;- release: () => void;- promise: Promise<void>;- } | null = null;-- return {- store,- armGetGate: (scope: string, key: string) => {- let release!: () => void;- const promise = new Promise<void>((res) => {- release = res;- });- gate = { scope, key, release, promise };- return release;- },- get: async <T>(scope: string, key: string): Promise<T | null> => {- const value = clone((store.get(scope)?.get(key) as T) ?? null);- if (gate && gate.scope === scope && gate.key === key) {- const g = gate;- gate = null;- await g.promise;- }- return value;- },- set: async <T>(scope: string, key: string, data: T): Promise<T> => {- const stored = clone(data);- if (!store.has(scope)) store.set(scope, new Map());- store.get(scope)!.set(key, stored);- return clone(stored);- },- delete: async (scope: string, key: string) => {- store.get(scope)?.delete(key);- },- list: async <T>(scope: string): Promise<T[]> => {- const m = store.get(scope);- return m ? clone(Array.from(m.values()) as T[]) : [];- },- };-}+import { createGraphIndexState } from "./helpers/graph-index-state.js";function mockSdk() {const fns = new Map<string, (data: unknown) => Promise<unknown>>();@@ -75,7 +27,9 @@ async function flush() {describe("graph:persist serializes cross-path graph mutations", () => {it("does not lose cascade's stale flag or persistGraphDelta's merge when they race on the same node", async () => {- const kv = mockKV();+ const fixture = createGraphIndexState();+ const { kv } = fixture;+ await initializeGraphIndex(kv, { status: "ready" });const node: GraphNode = {id: "node_1",@@ -123,7 +77,16 @@ describe("graph:persist serializes cross-path graph mutations", () => {const sdk = mockSdk();registerCascadeFunction(sdk as never, kv as never);- const releaseNodeRowRead = kv.armGetGate(KV.graphNodes, "node_1");+ let releaseNodeRowRead!: () => void;+ let enteredNodeRowRead!: () => void;+ const blocked = new Promise<void>((resolve) => { releaseNodeRowRead = resolve; });+ const entered = new Promise<void>((resolve) => { enteredNodeRowRead = resolve; });+ fixture.intercept(async (id, payload) => {+ if (id !== "state::get" || payload.scope !== KV.graphNodes || payload.key !== "node_1") return;+ fixture.intercept();+ enteredNodeRowRead();+ await blocked;+ });const incomingNode: GraphNode = {id: "node_new",@@ -134,28 +97,40 @@ describe("graph:persist serializes cross-path graph mutations", () => {createdAt: new Date().toISOString(),};const persistPromise = persistGraphDelta(- kv as never,+ kv,[incomingNode],[],- ["obs_new"],);- await flush();+ await entered;+ let cascadeSettled = false;const cascadePromise = sdk.trigger("mem::cascade-update", {supersededMemoryId: "mem_old",- }) as Promise<{ success: boolean; flagged: { nodes: number } }>;-- await flush();- releaseNodeRowRead();+ }).then((value) => {+ cascadeSettled = true;+ return value as { success: boolean; graphDeferred: boolean; flagged: { nodes: number } };+ });++ try {+ await flush();+ expect(cascadeSettled).toBe(false);+ } finally {+ releaseNodeRowRead();+ }const [, cascadeResult] = await Promise.all([persistPromise, cascadePromise]);expect(cascadeResult.success).toBe(true);+ expect(cascadeResult.graphDeferred).toBe(false);expect(cascadeResult.flagged.nodes).toBe(1);const finalNode = await kv.get<GraphNode>(KV.graphNodes, "node_1");expect(finalNode!.stale).toBe(true);expect(finalNode!.sourceObservationIds.sort()).toEqual(["obs_new", "obs_old"]);+ const reader = (await createGraphIndexReader(kv))!;+ expect(await reader.getNode("node_1")).toBeNull();+ expect((await reader.posting("observation", "obs_old")).ids).toEqual([]);+ expect(await reader.finish()).toBe(true);});});diff --git a/test/graph-generation.test.ts b/test/graph-generation.test.tsindex 77ceb53c..2c39ea68 100644--- a/test/graph-generation.test.ts+++ b/test/graph-generation.test.ts@@ -1,3 +1,5 @@+import { initializeGraphIndex } from "../src/state/graph-retrieval-index.js";+import { indexObservationSession } from "../src/state/obs-index.js";import { afterEach, describe, expect, it, vi } from "vitest";import { mkdtempSync, rmSync } from "node:fs";import { join } from "node:path";@@ -37,7 +39,7 @@ function deferred<T = void>() {return { promise, resolve };}-function setup(provider = { name: "noop", compress: vi.fn(async () => xml), summarize: vi.fn(async () => "") }) {+async function setup(provider = { name: "noop", compress: vi.fn(async () => xml), summarize: vi.fn(async () => "") }) {const store = new Map<string, Map<string, unknown>>();const handlers = new Map<string, (data?: any) => Promise<any>>();const sent: any[] = [];@@ -62,6 +64,7 @@ function setup(provider = { name: "noop", compress: vi.fn(async () => xml), summ},};const kv = new StateKV(sdk as never);+ await initializeGraphIndex(kv, { status: "ready" });registerGraphFunction(sdk as never, kv, provider as never);registerTemporalGraphFunctions(sdk as never, kv, provider as never);registerExportImportFunction(sdk as never, kv);@@ -72,6 +75,10 @@ function setup(provider = { name: "noop", compress: vi.fn(async () => xml), summ}async function seed(kv: StateKV) {+ for (const id of ["old-a", "old-b", "old-edge-observation"]) {+ await kv.set(KV.observations("s"), id, { ...observation, id });+ await indexObservationSession(kv, id, "s");+ }await persistGraphDelta(kv, [node("old-a", "alpha"), node("old-b", "beta")], [edge("old-edge", "old-a", "old-b")]);await kv.set(KV.graphEdgeHistory, "old-history", edge("old-history", "old-a", "old-b"));}@@ -86,7 +93,7 @@ afterEach(() => {describe("graph generation reset", () => {it("hides legacy nodes, edges, history and indexes from every graph read path, rebuild and export", async () => {- const { kv, call, store, provider } = setup();+ const { kv, call, store, provider } = await setup();await seed(kv);await kv.set(KV.graphNodes, "undated", { ...node("undated"), createdAt: undefined });const retrieval = new GraphRetrieval(kv);@@ -125,7 +132,7 @@ describe("graph generation reset", () => {it("uses unique generations for same-millisecond resets and never merges old provenance on re-extraction", async () => {vi.useFakeTimers(); vi.setSystemTime(timestamp);- const { kv, call } = setup();+ const { kv, call } = await setup();await seed(kv);await call("mem::graph-reset");const first = await kv.get(KV.graphControl, "generation");@@ -143,12 +150,14 @@ describe("graph generation reset", () => {vi.stubEnv("GRAPH_EXTRACTION_ENABLED", "true");const entered = deferred(); const response = deferred<string>();const provider = { name: "test", compress: vi.fn(() => { entered.resolve(); return response.promise; }), summarize: vi.fn(async () => "") };- const { kv, call } = setup(provider);+ const { kv, call } = await setup(provider);const extraction = call(id, { observations: [observation] });await entered.promise;await call("mem::graph-reset");response.resolve(xml);- expect((await extraction).success).toBe(true);+ const late = await extraction;+ expect(late.success).toBe(id !== "mem::temporal-graph-extract");+ if (id === "mem::temporal-graph-extract") expect(late.error).toContain("changed revision");expect(await kv.list(KV.graphNodes)).toEqual([]);expect(await kv.list(KV.graphEdges)).toEqual([]);expect((await call("mem::graph-stats")).totalNodes).toBe(0);@@ -160,7 +169,7 @@ describe("graph generation reset", () => {});it("keeps a persist queued behind reset in its starting generation", async () => {- const { kv, call, intercept } = setup();+ const { kv, call, intercept } = await setup();const entered = deferred(); const release = deferred();intercept(async (id, p) => {if (id === "state::get" && p.scope === KV.graphNameIndex) {@@ -178,7 +187,7 @@ describe("graph generation reset", () => {});it("preserves the old active graph if snapshot initialization or pointer publication fails", async () => {- const { kv, call, intercept } = setup();+ const { kv, call, intercept } = await setup();await seed(kv);for (const failPointer of [false, true]) {intercept(async (id, p) => {@@ -194,7 +203,7 @@ describe("graph generation reset", () => {});it("retains the boundary across StateKV restart and fails closed on an unreadable or corrupt pointer", async () => {- const { kv, sdk, call, intercept } = setup();+ const { kv, sdk, call, intercept } = await setup();await seed(kv); await call("mem::graph-reset");const restarted = new StateKV(sdk as never);expect(await restarted.list(KV.graphNodes)).toEqual([]);@@ -209,7 +218,7 @@ describe("graph generation reset", () => {});it("accepts intentional imports into the active generation without timestamp-based duplication", async () => {- const { kv, call } = setup();+ const { kv, call } = await setup();await seed(kv); await call("mem::graph-reset");const imported = { ...node("imported", "alpha", "import-observation"), createdAt: "2020-01-01T00:00:00.000Z" };await call("mem::import", { exportData: { version: "0.9.30", exportedAt: timestamp, sessions: [], observations: {}, memories: [], summaries: [], graphNodes: [imported] } });@@ -225,7 +234,7 @@ describe("graph generation reset", () => {});it.each(["mem::import", "mem::mesh-receive"])("keeps delayed %s graph writes out of a concurrent reset", async (id) => {- const { kv, call, intercept } = setup();+ const { kv, call, intercept } = await setup();const entered = deferred(); const release = deferred();intercept(async (fn, p) => {if (fn === "state::set" && p.scope === KV.memories) {@@ -242,7 +251,7 @@ describe("graph generation reset", () => {});it("pins snapshot restore before filesystem and non-graph work, while allowing a later deliberate restore", async () => {- const { kv, sdk, call, intercept } = setup();+ const { kv, sdk, call, intercept } = await setup();const dir = mkdtempSync(join(process.cwd(), ".graph-generation-test-"));try {registerSnapshotFunction(sdk as never, kv, dir);@@ -271,7 +280,7 @@ describe("graph generation reset", () => {});it("never re-resolves the namespace halfway through a Redis read-modify-write", async () => {- const { kv, sdk, call, intercept, store } = setup();+ const { kv, sdk, call, intercept, store } = await setup();await seed(kv);const redis = new StateKV(sdk as never, { backend: "redis" });const entered = deferred(); const release = deferred();@@ -289,7 +298,7 @@ describe("graph generation reset", () => {});it("updates cached viewer counts and refuses late legacy snapshot events after reset", async () => {- const { kv, sdk, call, sent } = setup();+ const { kv, sdk, call, sent } = await setup();await seed(kv);const legacy = await kv.get<GraphSnapshot>(KV.graphSnapshot, "current");registerViewerStreamTriggers(sdk as never, kv);@@ -307,7 +316,7 @@ describe("graph generation reset", () => {});it("orders a delayed legacy viewer callback before reset completion", async () => {- const { kv, sdk, call, sent } = setup();+ const { kv, sdk, call, sent } = await setup();await seed(kv);const legacy = await kv.get<GraphSnapshot>(KV.graphSnapshot, "current");registerViewerStreamTriggers(sdk as never, kv);diff --git a/test/graph-index-api.test.ts b/test/graph-index-api.test.tsnew file mode 100644index 00000000..8d74fee2--- /dev/null+++ b/test/graph-index-api.test.ts@@ -0,0 +1,54 @@+import { describe, expect, it, vi } from "vitest";+import { registerGraphIndexApi } from "../src/triggers/graph-index-api.js";+import { checkAuth } from "../src/triggers/api.js";++function setup(result: unknown = { success: true, state: "building", jobId: "job" }) {+ const handlers = new Map<string, (data: any) => Promise<any>>();+ const sdk = { registerFunction: (id: string, fn: (data: any) => Promise<any>) => handlers.set(id, fn), registerTrigger: vi.fn(), trigger: vi.fn(async () => result) };+ registerGraphIndexApi(sdk as never, "profile-secret", checkAuth);+ const request = (command: string, body?: unknown, authorization = "Bearer profile-secret", method = command === "status" ? "GET" : "POST") =>+ handlers.get(`api::graph-index-${command}`)!({ method, body, headers: { authorization } });+ return { sdk, request };+}++describe("graph index API", () => {+ it.each(["status", "migrate", "resume"])("authenticates %s before reading or writing graph state", async (command) => {+ const { sdk, request } = setup();+ expect((await request(command, { allowLegacyScan: true }, "Bearer another-profile")).status_code).toBe(401);+ expect(sdk.trigger).not.toHaveBeenCalled();+ });++ it("uses a bounded status function and disables response caching", async () => {+ const { sdk, request } = setup({ state: "needs-migration" });+ expect(await request("status")).toMatchObject({ status_code: 200, headers: { "Cache-Control": "no-store" }, body: { state: "needs-migration" } });+ expect(sdk.trigger).toHaveBeenCalledWith({ function_id: "mem::graph-index-status", payload: {} });+ });++ it.each(["true", 1, null, [], {}])("rejects non-boolean discovery consent (%j)", async (value) => {+ const { sdk, request } = setup();+ expect((await request("migrate", { allowLegacyScan: value })).status_code).toBe(400);+ expect(sdk.trigger).not.toHaveBeenCalled();+ });++ it("whitelists the consent flag and returns accepted without waiting for migration", async () => {+ const { sdk, request } = setup();+ expect((await request("migrate", { allowLegacyScan: true, generation: "foreign", scope: "private", reset: true })).status_code).toBe(202);+ expect(sdk.trigger).toHaveBeenCalledWith({ function_id: "mem::graph-index-migrate", payload: { allowLegacyScan: true } });+ });++ it("resume does not silently authorize discovery", async () => {+ const { sdk, request } = setup();+ expect((await request("resume", {})).status_code).toBe(202);+ expect(sdk.trigger).toHaveBeenCalledWith({ function_id: "mem::graph-index-resume", payload: { allowLegacyScan: false } });+ });++ it.each([["consent-required", 400], ["invalid-state", 409], ["no-job", 409], ["storage-unavailable", 503]])("reports %s as HTTP%i", async (code, status) => {+ const { request } = setup({ success: false, code, error: "refused" });+ expect((await request("resume", {})).status_code).toBe(status);+ });++ it("returns 200 for an already ready index", async () => {+ const { request } = setup({ success: true, state: "ready" });+ expect((await request("migrate", { allowLegacyScan: true })).status_code).toBe(200);+ });+});diff --git a/test/graph-index-command.test.ts b/test/graph-index-command.test.tsnew file mode 100644index 00000000..d2acbda9--- /dev/null+++ b/test/graph-index-command.test.ts@@ -0,0 +1,53 @@+import { afterEach, describe, expect, it, vi } from "vitest";+import { runGraphIndexCommand } from "../src/cli/graph-index.js";++afterEach(() => vi.unstubAllGlobals());++describe("graph-index CLI", () => {+ it.each([[], ["status"], ["status", "--json"]].map((args) => ({ args })))("reads status without starting a server or migration ($args)", async ({ args }) => {+ const fetch = vi.fn(async () => new Response(JSON.stringify({ state: "needs-migration", action: "migrate" })));+ vi.stubGlobal("fetch", fetch);+ const log = vi.fn();+ expect(await runGraphIndexCommand({ base: "http://127.0.0.1:3211", secret: "private-profile", args, log })).toBe(0);+ expect(fetch).toHaveBeenCalledWith("http://127.0.0.1:3211/agentmemory/graph/index", expect.objectContaining({+ method: "GET", headers: expect.objectContaining({ Authorization: "Bearer private-profile" }),+ }));+ if (args.includes("--json")) expect(JSON.parse(log.mock.calls[0][0])).toHaveProperty("state", "needs-migration");+ });++ it.each([["migrate"], ["migrate", "--allow-legacy-scan=false"], ["status", "--allow-legacy-scan"], ["delete"], ["resume", "--wat"]].map((args) => ({ args })))("rejects unsafe or unknown arguments before any network call ($args)", async ({ args }) => {+ const fetch = vi.fn(); vi.stubGlobal("fetch", fetch);+ expect(await runGraphIndexCommand({ base: "http://localhost:3111", args, log: vi.fn() })).toBe(2);+ expect(fetch).not.toHaveBeenCalled();+ });++ it("forwards only explicit discovery consent and preserves the selected endpoint", async () => {+ const fetch = vi.fn(async () => new Response(JSON.stringify({ state: "discovering", jobId: "job" }), { status: 202 }));+ vi.stubGlobal("fetch", fetch);+ expect(await runGraphIndexCommand({ base: "https://example.test/memory", secret: "remote", args: ["migrate", "--json", "--allow-legacy-scan", "--instance", "2"], log: vi.fn() })).toBe(0);+ expect(fetch).toHaveBeenCalledWith("https://example.test/memory/agentmemory/graph/index/migrate", expect.objectContaining({+ method: "POST", body: '{"allowLegacyScan":true}',+ }));+ });++ it("resumes sealed work without authorizing a new legacy scan", async () => {+ const fetch = vi.fn(async () => new Response(JSON.stringify({ state: "building" }), { status: 202 }));+ vi.stubGlobal("fetch", fetch);+ expect(await runGraphIndexCommand({ base: "http://localhost:3111", args: ["resume"], log: vi.fn() })).toBe(0);+ expect(fetch.mock.calls[0][1]).toMatchObject({ method: "POST", body: '{"allowLegacyScan":false}' });+ });++ it.each([401, 409, 500])("returns nonzero for HTTP %i and JSON diagnostics", async (status) => {+ vi.stubGlobal("fetch", vi.fn(async () => new Response(JSON.stringify({ error: "refused" }), { status })));+ const log = vi.fn();+ expect(await runGraphIndexCommand({ base: "http://localhost:3111", args: ["status", "--json"], log })).toBe(1);+ expect(JSON.parse(log.mock.calls[0][0])).toMatchObject({ error: "refused", httpStatus: status });+ });++ it("returns machine-readable connection errors", async () => {+ vi.stubGlobal("fetch", vi.fn(async () => { throw new Error("offline"); }));+ const log = vi.fn();+ expect(await runGraphIndexCommand({ base: "http://localhost:3111", args: ["status", "--json"], log })).toBe(1);+ expect(JSON.parse(log.mock.calls[0][0])).toEqual({ error: "offline" });+ });+});diff --git a/test/graph-index-migration.test.ts b/test/graph-index-migration.test.tsnew file mode 100644index 00000000..b1d90901--- /dev/null+++ b/test/graph-index-migration.test.ts@@ -0,0 +1,357 @@+import { describe, expect, it } from "vitest";+import { StateKV } from "../src/state/kv.js";+import { KV } from "../src/state/schema.js";+import { resetGraphGeneration } from "../src/state/graph-generation.js";+import {+ GRAPH_INDEX_MANIFEST_MAX_BYTES, GRAPH_INDEX_MANIFEST_MAX_IDS, GRAPH_INDEX_MIGRATION_KEY,+ GraphIndexMigrationController, getGraphIndexMigrationStatus, graphIndexManifestKey,+ type GraphIndexManifestPage, type GraphIndexMigration,+} from "../src/state/graph-index-migration.js";+import { createGraphIndexReader, getGraphIndexStatus } from "../src/state/graph-retrieval-index.js";+import type { GraphNode } from "../src/types.js";++const now = "2026-10-14T00:00:00.000Z";+const node = (id: string): GraphNode => ({ id, name: id, type: "concept", properties: {}, sourceObservationIds: ["observation"], createdAt: now });+function deferred() {+ let resolve!: () => void;+ const promise = new Promise<void>((r) => { resolve = r; });+ return { promise, resolve };+}++function setup() {+ const rows = new Map<string, Map<string, unknown>>();+ const calls: Array<{ fn: string; scope: string; key?: string; value?: unknown }> = [];+ let intercept: ((fn: string, payload: any) => Promise<void> | void) | undefined;+ const sdk = { trigger: async ({ function_id: fn, payload }: any) => {+ calls.push({ fn, ...structuredClone(payload) });+ await intercept?.(fn, payload);+ const scope = rows.get(payload.scope) ?? new Map<string, unknown>();+ if (fn === "state::get") return structuredClone(scope.get(payload.key) ?? null);+ if (fn === "state::list") return structuredClone([...scope.values()]);+ if (fn === "state::set") { scope.set(payload.key, structuredClone(payload.value)); rows.set(payload.scope, scope); return payload.value; }+ if (fn === "state::delete") { scope.delete(payload.key); return; }+ throw new Error(`Unexpected ${fn}`);+ } };+ const kv = new StateKV(sdk as never);+ return { kv, rows, calls, sdk, controller: new GraphIndexMigrationController(kv), intercept: (f?: typeof intercept) => { intercept = f; } };+}++describe("explicit graph index migration", () => {+ it("keeps legacy status and boot read-only without a snapshot or graph listing", async () => {+ const { kv, controller, calls, rows } = setup();+ rows.set(KV.graphNodes, new Map([["old", node("old")]]));+ expect(await controller.status()).toMatchObject({ state: "needs-migration", ready: false, coverage: "unknown" });+ await controller.resumeOnBoot();+ expect(calls.some((call) => call.fn === "state::list" || call.fn === "state::set" || call.scope === KV.graphSnapshot)).toBe(false);+ await expect(controller.start()).rejects.toThrow("allowLegacyScan");+ expect(calls.some((call) => call.fn === "state::list" || call.fn === "state::set")).toBe(false);+ expect(await getGraphIndexMigrationStatus(kv)).toHaveProperty("ready", false);+ });++ it("writes intent before discovery, returns promptly and joins duplicate requests", async () => {+ const { controller, rows, calls, intercept } = setup();+ rows.set(KV.graphNodes, new Map([["old", node("old")]]));+ const entered = deferred(); const release = deferred();+ intercept(async (fn, payload) => {+ if (fn === "state::list" && payload.scope === KV.graphNodes) { entered.resolve(); await release.promise; }+ });+ const first = await controller.start({ allowLegacyScan: true });+ await entered.promise;+ expect(first).toMatchObject({ state: "discovering", sealed: false });+ expect((await controller.start({ allowLegacyScan: true })).jobId).toBe(first.jobId);+ expect((await controller.start({ resume: true })).jobId).toBe(first.jobId);+ const intent = calls.findIndex((call) => call.fn === "state::set" && call.scope === KV.graphIndexMigration);+ const scan = calls.findIndex((call) => call.fn === "state::list" && call.scope === KV.graphNodes);+ expect(intent).toBeGreaterThanOrEqual(0); expect(intent).toBeLessThan(scan);+ release.resolve(); await controller.waitForIdle();+ expect(await controller.status()).toMatchObject({ ready: true, processed: 1, total: 1, coverage: "complete" });+ expect(calls.filter((call) => call.fn === "state::list").map((call) => call.scope)).toEqual([KV.graphNodes, KV.graphEdges, KV.graphEdgeHistory]);+ });++ it("indexes empty scopes and history without changing the canonical generation or rows", async () => {+ const { controller, kv, rows } = setup();+ rows.set(KV.graphEdgeHistory, new Map([["past", { id: "past", sourceNodeId: "missing-a", targetNodeId: "missing-b", type: "uses", weight: 1, sourceObservationIds: [], createdAt: now, isLatest: false }]]));+ await controller.start({ allowLegacyScan: true }); await controller.waitForIdle();+ expect(await controller.status()).toMatchObject({ ready: true, total: 1, historyEdges: 1, nodes: 0, edges: 0 });+ expect(await kv.get(KV.graphControl, "generation")).toBeNull();+ expect(rows.get(KV.graphEdgeHistory)?.size).toBe(1);+ });++ it.each(["discovering", "building"] as const)("keeps %s status during a normal per-record pending write", async (state) => {+ const { controller, rows, calls } = setup();+ await controller.start({ allowLegacyScan: true }); await controller.waitForIdle();+ const control = rows.get(KV.graphRetrievalControl)!.get("current") as object;+ const job = rows.get(KV.graphIndexMigration)!.get(GRAPH_INDEX_MIGRATION_KEY) as GraphIndexMigration;+ rows.get(KV.graphRetrievalControl)!.set("current", { ...control, status: "building", pending: { kind: "node", id: "a" } });+ rows.get(KV.graphIndexMigration)!.set(GRAPH_INDEX_MIGRATION_KEY, { ...job, state, sealed: state === "building" });+ calls.length = 0;+ const status = await controller.status();+ expect(status).toMatchObject({ state, ready: false, coverage: "unknown" });+ expect(status.action).toContain("in progress");+ if (state === "building") expect(status.action).not.toContain("resume");+ else expect(status.action).toContain("If it was interrupted");+ expect(calls.every((call) => call.fn === "state::get")).toBe(true);+ });++ it("leaves interrupted unsealed discovery read-only at boot with an explicit consent hint", async () => {+ const { controller, rows, calls, sdk } = setup();+ await controller.start({ allowLegacyScan: true }); await controller.waitForIdle();+ const control = rows.get(KV.graphRetrievalControl)!.get("current") as object;+ const job = rows.get(KV.graphIndexMigration)!.get(GRAPH_INDEX_MIGRATION_KEY) as GraphIndexMigration;+ rows.get(KV.graphRetrievalControl)!.set("current", { ...control, status: "building" });+ rows.get(KV.graphIndexMigration)!.set(GRAPH_INDEX_MIGRATION_KEY, { ...job, state: "discovering", sealed: false });+ calls.length = 0;+ const restarted = new GraphIndexMigrationController(new StateKV(sdk as never));+ await restarted.resumeOnBoot();+ const status = await restarted.status();+ expect(status).toMatchObject({ state: "discovering", ready: false, sealed: false });+ expect(status.action).toContain("resume --allow-legacy-scan during maintenance");+ expect(calls.every((call) => call.fn === "state::get")).toBe(true);+ await expect(restarted.start({ resume: true })).rejects.toThrow("authorize another complete legacy scan");+ });++ it.each(["control-paused", "batch-pending", "job-error"])("keeps %s fail-closed even when a matching job says building", async (failure) => {+ const { controller, rows } = setup();+ await controller.start({ allowLegacyScan: true }); await controller.waitForIdle();+ const control = rows.get(KV.graphRetrievalControl)!.get("current") as object;+ const job = rows.get(KV.graphIndexMigration)!.get(GRAPH_INDEX_MIGRATION_KEY) as GraphIndexMigration;+ rows.get(KV.graphRetrievalControl)!.set("current", {+ ...control, status: failure === "control-paused" ? "paused" : "building", pending: { kind: "node", id: "a" },+ ...(failure === "batch-pending" ? { batchPending: true } : {}),+ });+ rows.get(KV.graphIndexMigration)!.set(GRAPH_INDEX_MIGRATION_KEY, { ...job, state: "building", ...(failure === "job-error" ? { error: "Storage unavailable" } : {}) });+ expect(await controller.status()).toMatchObject({ state: "paused", ready: false, coverage: "unknown" });+ expect((await controller.status()).action).not.toContain("in progress");+ });++ it("reports and resumes a failed final migration checkpoint after index publication", async () => {+ const { controller, intercept } = setup();+ intercept((fn, payload) => {+ if (fn === "state::set" && payload.scope === KV.graphIndexMigration && payload.value.state === "ready") throw new Error("final checkpoint unavailable");+ });+ await controller.start({ allowLegacyScan: true }); await controller.waitForIdle();+ expect(await controller.status()).toMatchObject({ state: "paused", ready: false, error: expect.any(String) });+ intercept();+ await controller.start({ resume: true }); await controller.waitForIdle();+ expect(await controller.status()).toMatchObject({ state: "ready", ready: true, coverage: "complete" });+ expect((await controller.status()).error).toBeUndefined();+ });++ it("bounds manifests by entry count and UTF-8 bytes", async () => {+ const { controller, rows } = setup();+ const ids = Array.from({ length: 300 }, (_, i) => `${i}:` + "界".repeat(300));+ rows.set(KV.graphNodes, new Map(ids.map((id) => [id, node(id)])));+ await controller.start({ allowLegacyScan: true }); await controller.waitForIdle();+ const pages = [...(rows.get(KV.graphIndexManifest)?.values() ?? [])] as GraphIndexManifestPage[];+ expect(pages.length).toBeGreaterThan(3);+ for (const page of pages) {+ expect(page.ids.length).toBeLessThanOrEqual(GRAPH_INDEX_MANIFEST_MAX_IDS);+ expect(Buffer.byteLength(JSON.stringify(page))).toBeLessThanOrEqual(GRAPH_INDEX_MANIFEST_MAX_BYTES);+ }+ expect(await controller.status()).toMatchObject({ ready: true, processed: 300 });+ });++ it("requires renewed consent after an interrupted discovery and never scans at boot", async () => {+ const { controller, kv, calls, rows, intercept, sdk } = setup();+ rows.set(KV.graphNodes, new Map([["old", node("old")]]));+ intercept((fn, payload) => { if (fn === "state::list" && payload.scope === KV.graphEdges) throw new Error("offline"); });+ await controller.start({ allowLegacyScan: true }); await controller.waitForIdle();+ expect(await controller.status()).toMatchObject({ state: "paused", sealed: false });+ intercept(); calls.length = 0;+ const restarted = new GraphIndexMigrationController(new StateKV(sdk as never));+ await restarted.resumeOnBoot();+ await expect(restarted.start({ resume: true })).rejects.toThrow("authorize another complete legacy scan");+ expect(calls.some((call) => call.fn === "state::list")).toBe(false);+ await restarted.start({ resume: true, allowLegacyScan: true }); await restarted.waitForIdle();+ expect(await restarted.status()).toHaveProperty("ready", true);+ expect(await kv.get(KV.graphControl, "generation")).toBeNull();+ });++ it("resumes a sealed manifest after restart without a full scan and rereads current rows", async () => {+ const { controller, rows, calls, intercept, sdk } = setup();+ rows.set(KV.graphNodes, new Map([["a", node("a")], ["b", node("b")]]));+ let pause = true;+ intercept((fn, payload) => { if (pause && fn === "state::get" && payload.scope === KV.graphNodes && payload.key === "b") throw new Error("crash"); });+ await controller.start({ allowLegacyScan: true }); await controller.waitForIdle();+ expect(await controller.status()).toMatchObject({ state: "paused", sealed: true, processed: 1 });+ rows.get(KV.graphNodes)!.delete("b");+ pause = false; calls.length = 0;+ const restarted = new GraphIndexMigrationController(new StateKV(sdk as never));+ await restarted.resumeOnBoot(); await restarted.waitForIdle();+ expect(await restarted.status()).toMatchObject({ ready: true, nodes: 1, processed: 2 });+ expect(calls.some((call) => call.fn === "state::list" || call.scope === KV.graphSnapshot)).toBe(false);+ });++ it("keeps current writer updates, deletes and late inserts while discovery is running", async () => {+ const { controller, kv, rows, intercept } = setup();+ rows.set(KV.graphNodes, new Map([["a", node("a")], ["b", node("b")]]));+ const entered = deferred(); const release = deferred();+ intercept(async (fn, payload) => {+ if (fn === "state::list" && payload.scope === KV.graphEdgeHistory) { entered.resolve(); await release.promise; }+ });+ await controller.start({ allowLegacyScan: true }); await entered.promise;+ await kv.set(KV.graphNodes, "a", { ...node("a"), name: "current" });+ await kv.delete(KV.graphNodes, "b");+ await kv.set(KV.graphNodes, "late", node("late"));+ release.resolve(); await controller.waitForIdle();+ expect(await controller.status()).toMatchObject({ ready: true, nodes: 2, total: 2 });+ const reader = (await createGraphIndexReader(kv))!;+ expect(await reader.getNode("a")).toHaveProperty("name", "current");+ expect(await reader.getNode("b")).toBeNull();+ expect(await reader.getNode("late")).toHaveProperty("id", "late");+ });++ it("retries an interrupted build initialization only with renewed discovery consent", async () => {+ const { controller, kv, rows, calls, intercept, sdk } = setup();+ rows.set(KV.graphNodes, new Map([["a", node("a")]]));+ intercept((fn, payload) => { if (fn === "state::set" && payload.scope === KV.graphRetrievalControl) throw new Error("initialize failure"); });+ await expect(controller.start({ allowLegacyScan: true })).rejects.toThrow("initialize failure");+ expect(await kv.get(KV.graphIndexMigration, GRAPH_INDEX_MIGRATION_KEY)).toMatchObject({ sealed: false });+ intercept(); calls.length = 0;+ const restarted = new GraphIndexMigrationController(new StateKV(sdk as never));+ await restarted.resumeOnBoot();+ await expect(restarted.start({ resume: true })).rejects.toThrow("authorize another complete legacy scan");+ expect(calls.some((call) => call.fn === "state::list")).toBe(false);+ await restarted.start({ resume: true, allowLegacyScan: true }); await restarted.waitForIdle();+ expect(await restarted.status()).toMatchObject({ ready: true, nodes: 1 });+ });++ it("replays an unacknowledged cursor idempotently", async () => {+ const { controller, kv, rows, intercept, sdk } = setup();+ rows.set(KV.graphNodes, new Map([["a", node("a")], ["b", node("b")]]));+ intercept((fn, payload) => {+ if (fn === "state::set" && payload.scope === KV.graphIndexMigration && payload.value.processed > 0) throw new Error("progress not persisted");+ });+ await controller.start({ allowLegacyScan: true }); await controller.waitForIdle();+ expect(await kv.get<GraphIndexMigration>(KV.graphIndexMigration, GRAPH_INDEX_MIGRATION_KEY)).toMatchObject({ sealed: true, processed: 0 });+ intercept();+ const restarted = new GraphIndexMigrationController(new StateKV(sdk as never));+ await restarted.start({ resume: true }); await restarted.waitForIdle();+ expect(await restarted.status()).toMatchObject({ ready: true, nodes: 2, processed: 2 });+ });++ it.each(["missing", "checksum", "oversized"])("pauses a %s sealed manifest instead of publishing incomplete coverage", async (corruption) => {+ const { controller, kv, rows, intercept } = setup();+ rows.set(KV.graphNodes, new Map([["a", node("a")]]));+ intercept((fn, payload) => { if (fn === "state::get" && payload.scope === KV.graphIndexManifest) throw new Error("interrupt"); });+ await controller.start({ allowLegacyScan: true }); await controller.waitForIdle();+ intercept();+ const job = (await kv.get<GraphIndexMigration>(KV.graphIndexMigration, GRAPH_INDEX_MIGRATION_KEY))!;+ const key = graphIndexManifestKey(job.jobId, "nodes", 0);+ if (corruption === "missing") rows.get(KV.graphIndexManifest)!.delete(key);+ else {+ const page = rows.get(KV.graphIndexManifest)!.get(key) as GraphIndexManifestPage;+ if (corruption === "checksum") page.ids[0] = "different";+ else page.ids = Array.from({ length: 129 }, () => "a");+ }+ await controller.start({ resume: true }); await controller.waitForIdle();+ expect(await controller.status()).toMatchObject({ ready: false, state: "paused", error: expect.stringContaining("manifest") });+ expect((await getGraphIndexStatus(kv))?.status).toBe("building");+ });++ it("keeps late discovery out of a reset generation", async () => {+ const { controller, kv, rows, intercept } = setup();+ rows.set(KV.graphNodes, new Map([["old", node("old")]]));+ const entered = deferred(); const release = deferred();+ intercept(async (fn, payload) => { if (fn === "state::list" && payload.scope === KV.graphNodes) { entered.resolve(); await release.promise; } });+ await controller.start({ allowLegacyScan: true }); await entered.promise;+ await resetGraphGeneration(kv, { version: 1, topNodes: [], topEdges: [], topDegrees: {}, stats: { totalNodes: 0, totalEdges: 0, nodesByType: {}, edgesByType: {} }, updatedAt: now, dirty: false });+ release.resolve(); await controller.waitForIdle();+ expect(rows.get(KV.graphIndexMigration)?.get(GRAPH_INDEX_MIGRATION_KEY)).toMatchObject({ state: "superseded" });+ expect((await controller.status()).jobId).toBeUndefined();+ expect((await controller.status()).nodes ?? 0).toBe(0);+ });++ it("defers sealed legacy journal recovery at boot and permits an explicit replacement build", async () => {+ const { controller, kv, rows, calls, intercept, sdk } = setup();+ rows.set(KV.graphNodes, new Map([["a", node("a")]]));+ intercept((fn, payload) => { if (fn === "state::get" && payload.scope === KV.graphIndexManifest) throw new Error("interrupt"); });+ await controller.start({ allowLegacyScan: true }); await controller.waitForIdle();+ intercept();+ const original = (await controller.status()).jobId;+ const control = (await getGraphIndexStatus(kv))!;+ rows.get(KV.graphRetrievalControl)!.set("current", { ...control, batchPending: true });+ rows.set(KV.graphSnapshot, new Map([["current", { version: 1, dirty: true }]]));+ calls.length = 0;+ const restarted = new GraphIndexMigrationController(new StateKV(sdk as never));+ await restarted.resumeOnBoot(); await restarted.waitForIdle();+ expect(await restarted.status()).toMatchObject({ state: "paused", ready: false, error: expect.stringContaining("explicit") });+ expect(calls.some((call) => call.fn === "state::list" || call.scope === KV.graphSnapshot)).toBe(false);+ await restarted.start({ resume: true }); await restarted.waitForIdle();+ expect(await restarted.status()).toMatchObject({ ready: false, action: expect.stringContaining("migrate --allow-legacy-scan"), error: expect.stringContaining("new build") });+ await restarted.start({ allowLegacyScan: true }); await restarted.waitForIdle();+ expect(await restarted.status()).toMatchObject({ ready: true, nodes: 1 });+ expect((await restarted.status()).jobId).not.toBe(original);+ });++ it("does no discovery when durable intent cannot be written", async () => {+ const { controller, calls, intercept } = setup();+ intercept((fn, payload) => { if (fn === "state::set" && payload.scope === KV.graphIndexMigration) throw new Error("disk failure"); });+ await expect(controller.start({ allowLegacyScan: true })).rejects.toThrow("disk failure");+ expect(calls.some((call) => call.fn === "state::list")).toBe(false);+ });++ it("never reuses orphaned journals when a sealed build loses its control", async () => {+ const { controller, kv, rows, calls, intercept, sdk } = setup();+ rows.set(KV.graphNodes, new Map([["a", node("a")], ["b", node("b")]]));+ intercept((fn, payload) => { if (fn === "state::get" && payload.scope === KV.graphNodes && payload.key === "b") throw new Error("interrupt"); });+ await controller.start({ allowLegacyScan: true }); await controller.waitForIdle();+ const old = await controller.status();+ expect(old).toMatchObject({ sealed: true, processed: 1 });+ rows.get(KV.graphRetrievalControl)!.delete("current");+ intercept(); calls.length = 0;+ const restarted = new GraphIndexMigrationController(new StateKV(sdk as never));+ await expect(restarted.resumeOnBoot()).rejects.toThrow("sealed build control is missing");+ await expect(restarted.start({ resume: true, allowLegacyScan: true })).rejects.toThrow("sealed build control is missing");+ await expect(restarted.start()).rejects.toThrow("allowLegacyScan");+ expect(calls.some((call) => call.fn === "state::set" || call.fn === "state::list")).toBe(false);+ await restarted.start({ allowLegacyScan: true }); await restarted.waitForIdle();+ const ready = await restarted.status();+ expect(ready).toMatchObject({ ready: true, nodes: 2, processed: 2, coverage: "complete" });+ expect(ready.buildId).not.toBe(old.buildId); expect(ready.jobId).not.toBe(old.jobId);+ const reader = (await createGraphIndexReader(kv))!;+ expect((await reader.posting("nodes", "")).ids.sort()).toEqual(["a", "b"]);+ expect(await reader.finish()).toBe(true);+ });++ it.each(["corrupt", "older-schema"])("repairs %s derived control only through an explicit new migration", async (kind) => {+ const { controller, rows, calls, intercept } = setup();+ rows.set(KV.graphNodes, new Map([["a", node("a")]]));+ await controller.start({ allowLegacyScan: true }); await controller.waitForIdle();+ const old = (await controller.status()).jobId;+ const control = rows.get(KV.graphRetrievalControl)!.get("current") as object;+ rows.get(KV.graphRetrievalControl)!.set("current", kind === "corrupt" ? { schema: 999 } : { ...control, schema: 1 });+ calls.length = 0;+ expect(await controller.status()).toMatchObject({+ state: "unavailable", ready: false,+ error: "Invalid graph retrieval index control; run explicit graph-index migration",+ });+ await expect(controller.start({ resume: true })).rejects.toThrow();+ await expect(controller.start()).rejects.toThrow();+ expect(calls.every((call) => call.fn === "state::get")).toBe(true);+ calls.length = 0;+ intercept((fn, payload) => { if (fn === "state::get" && payload.scope === KV.graphRetrievalControl) throw new Error("storage offline"); });+ expect(await controller.status()).toMatchObject({+ state: "unavailable", ready: false,+ error: "Graph index migration stopped after a storage failure. Check the server and resume when storage is available.",+ });+ await expect(controller.start({ allowLegacyScan: true })).rejects.toThrow("storage offline");+ expect(calls.some((call) => call.fn === "state::set" || call.fn === "state::list")).toBe(false);+ intercept();+ await controller.start({ allowLegacyScan: true }); await controller.waitForIdle();+ expect(await controller.status()).toMatchObject({ ready: true, nodes: 1 });+ expect((await controller.status()).jobId).not.toBe(old);+ });++ it("fails closed for corrupt descriptors and oversized legacy IDs", async () => {+ const { controller, rows, calls } = setup();+ rows.set(KV.graphIndexMigration, new Map([[GRAPH_INDEX_MIGRATION_KEY, { schema: 999 }]]));+ expect(await controller.status()).toMatchObject({ state: "unavailable", ready: false });+ await expect(controller.start({ allowLegacyScan: true })).rejects.toThrow("state is invalid");+ expect(calls.some((call) => call.fn === "state::list")).toBe(false);+ rows.delete(KV.graphIndexMigration);+ const id = "x".repeat(1025); rows.set(KV.graphNodes, new Map([[id, node(id)]]));+ await controller.start({ allowLegacyScan: true }); await controller.waitForIdle();+ expect(await controller.status()).toMatchObject({ state: "paused", sealed: false, error: expect.stringContaining("oversized ID") });+ });+});diff --git a/test/graph-index-query.test.ts b/test/graph-index-query.test.tsnew file mode 100644index 00000000..26181d0c--- /dev/null+++ b/test/graph-index-query.test.ts@@ -0,0 +1,140 @@+import { describe, expect, it, vi } from "vitest";+import { createGraphIndexState } from "./helpers/graph-index-state.js";+import { initializeGraphIndex, recoverGraphIndex } from "../src/state/graph-retrieval-index.js";+import { queryGraphIndex } from "../src/functions/graph-index-query.js";+import { readIndexedEntity } from "../src/functions/graph-index-temporal.js";+import { registerTemporalGraphFunctions } from "../src/functions/temporal-graph.js";+import { KV } from "../src/state/schema.js";+import type { GraphNode, GraphEdge } from "../src/types.js";++vi.mock("../src/logger.js", () => ({ logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn() } }));+const date = "2026-10-14T00:00:00.000Z";+const node = (id: string, name = id, more: Partial<GraphNode> = {}): GraphNode => ({+ id, name, type: "concept", properties: {}, sourceObservationIds: ["obs"], createdAt: date, ...more,+});+const edge = (id: string, sourceNodeId: string, targetNodeId: string, more: Partial<GraphEdge> = {}): GraphEdge => ({+ id, sourceNodeId, targetNodeId, type: "related_to", weight: 1, sourceObservationIds: ["obs"], createdAt: date, ...more,+});+async function setup() {+ const state = createGraphIndexState();+ await initializeGraphIndex(state.kv, { status: "ready" });+ return state;+}++describe("bounded graph query surfaces", () => {+ it("leaves unknown legacy stores untouched and does not read their snapshot or canonical scopes", async () => {+ const { kv, store, calls } = createGraphIndexState();+ store.set(KV.graphNodes, new Map([["legacy", node("legacy")]]));+ expect(await queryGraphIndex(kv, { query: "legacy" })).toMatchObject({ nodes: [], truncated: true, warning: expect.stringContaining("migration") });+ expect(await readIndexedEntity(kv, "legacy")).toHaveProperty("error");+ expect(calls.every(call => call.id === "state::get" && ![KV.graphNodes, KV.graphEdges, KV.graphSnapshot].includes(call.scope as never))).toBe(true);+ });++ it("searches Unicode primitive properties, excludes stale rows and never enumerates canonical data", async () => {+ const { kv, calls } = await setup();+ await kv.set(KV.graphNodes, "a", node("a", "first", { properties: { city: "東京大学" } }));+ await kv.set(KV.graphNodes, "b", node("b", "second", { properties: { city: "東京大学" }, stale: true }));+ calls.length = 0;+ const result = await queryGraphIndex(kv, { query: "東京" });+ expect(result.nodes.map(row => row.id)).toEqual(["a"]);+ expect(result.truncated).toBe(false);+ expect(calls.some(call => call.id === "state::list" || [KV.graphNodes, KV.graphEdges, KV.graphSnapshot].includes(call.scope as never))).toBe(false);+ });++ it("honors zero depth, terminates cycles and omits edges with missing or historical endpoints", async () => {+ const { kv } = await setup();+ for (const id of ["a", "b", "c"]) await kv.set(KV.graphNodes, id, node(id));+ for (const row of [edge("ab", "a", "b"), edge("ba", "b", "a"), edge("self", "a", "a"),+ edge("missing", "a", "gone"), edge("old", "a", "c", { isLatest: false })]) await kv.set(KV.graphEdges, row.id, row);+ expect((await queryGraphIndex(kv, { startNodeId: "a", maxDepth: 0 })).nodes.map(row => row.id)).toEqual(["a"]);+ const result = await queryGraphIndex(kv, { startNodeId: "a", maxDepth: 5 });+ expect(result.nodes.map(row => row.id).sort()).toEqual(["a", "b"]);+ expect(result.edges.map(row => row.id).sort()).toEqual(["ab", "ba", "self"]);+ });++ it("discards a response if a writer changes the revision during its point reads", async () => {+ const { kv, intercept } = await setup();+ await kv.set(KV.graphNodes, "a", node("a", "before"));+ let changed = false;+ intercept(async (id, payload) => {+ if (!changed && id === "state::get" && payload.scope === KV.graphRetrievalData && payload.key.includes(":record:node:")) {+ changed = true;+ await kv.set(KV.graphNodes, "a", node("a", "after"));+ }+ });+ expect(await queryGraphIndex(kv, {})).toMatchObject({ nodes: [], truncated: true, warning: expect.stringContaining("changed") });+ });++ it("reserves traversal capacity for unique neighbors despite parallel edges and cycles", async () => {+ const { kv, calls } = await setup();+ for (const id of ["root", "a", "b"]) await kv.set(KV.graphNodes, id, node(id));+ await kv.set(KV.graphEdges, "rb", edge("rb", "root", "b"));+ for (let i = 0; i < 512; i++) await kv.set(KV.graphEdges, `parallel-${i}`, edge(`parallel-${i}`, "root", "a"));+ await kv.set(KV.graphEdges, "self", edge("self", "root", "root"));+ calls.length = 0;+ const result = await queryGraphIndex(kv, { startNodeId: "root", maxDepth: 1 });+ expect(result.nodes.map(row => row.id).sort()).toEqual(["a", "b", "root"]);+ expect(result.edges.some(row => row.id === "rb")).toBe(true);+ expect(calls.filter(call => call.scope === KV.graphRetrievalData).length).toBeLessThanOrEqual(2048);+ });++ it("bounds impossible offsets without starting a scope listing", async () => {+ const { kv, calls } = await setup();+ for (let i = 0; i < 1100; i++) await kv.set(KV.graphNodes, `n${i}`, node(`n${i}`));+ calls.length = 0;+ expect(await queryGraphIndex(kv, { offset: Number.MAX_SAFE_INTEGER })).toMatchObject({ nodes: [], truncated: true });+ expect(calls.filter(call => call.scope === KV.graphRetrievalData).length).toBeLessThanOrEqual(2048);+ expect(calls.some(call => call.id === "state::list")).toBe(false);+ });++ it("reads aliases and history from bounded postings with canonical edge IDs deduplicated", async () => {+ const { kv, calls } = await setup();+ await kv.set(KV.graphNodes, "a", node("a", "Alice", { aliases: ["ALICE"] }));+ await kv.set(KV.graphNodes, "b", node("b"));+ const old = edge("old", "a", "b", { isLatest: false, version: 1 });+ await kv.set(KV.graphEdges, old.id, old);+ await kv.set(KV.graphEdgeHistory, old.id, old);+ await kv.set(KV.graphEdges, "new", edge("new", "a", "b", { isLatest: true, version: 2 }));+ calls.length = 0;+ const result = await readIndexedEntity(kv, "ALICE");+ expect(result).toMatchObject({ entity: { id: "a" }, truncated: false });+ if ("error" in result) throw new Error(result.error);+ expect(result.relatedEdges.map(row => row.id).sort()).toEqual(["new", "old"]);+ expect(result.entityHistory.map(row => row.id)).toEqual(["old"]);+ expect(calls.some(call => call.id === "state::list")).toBe(false);+ });++ it.each(["canonical", "history"])("recovers an interrupted temporal %s write across restart without losing history", async (failure) => {+ const { kv, intercept, store } = await setup();+ await kv.set(KV.graphNodes, "a", node("a", "Alice"));+ await kv.set(KV.graphNodes, "b", node("b", "Acme"));+ await kv.set(KV.graphEdges, "old", edge("old", "a", "b", { type: "uses", version: 1, isLatest: true }));+ const functions = new Map<string, (data: any) => Promise<any>>();+ const sdk = { registerFunction: (id: string, fn: any) => functions.set(id, fn) };+ const provider = {+ name: "fixture", summarize: async () => "", compress: async () => '<temporal_graph><entities><entity type="concept" name="Alice"></entity><entity type="concept" name="Acme"></entity></entities><relationships><relationship type="uses" source="Alice" target="Acme" weight="0.8"><reasoning>new</reasoning></relationship></relationships></temporal_graph>',+ };+ registerTemporalGraphFunctions(sdk as never, kv, provider);+ let failed = false;+ intercept(async (id, payload) => {+ if (!failed && id === "state::set" && (failure === "canonical"+ ? payload.scope === KV.graphEdges && payload.key !== "old"+ : payload.scope === KV.graphEdgeHistory)) {+ failed = true; throw new Error("interrupted replacement");+ }+ });+ expect(await functions.get("mem::temporal-graph-extract")!({ observations: [{ id: "obs", title: "test", narrative: "test", concepts: [], files: [], type: "discovery", timestamp: date }] })).toMatchObject({ success: false });+ expect((await queryGraphIndex(kv, { query: "Alice" })).nodes).toEqual([]);+ intercept();+ const restarted = createGraphIndexState(store).kv;+ await restarted.get(KV.graphNodes, "a");+ await recoverGraphIndex(restarted);+ registerTemporalGraphFunctions(sdk as never, restarted, provider);+ const state = await functions.get("mem::temporal-query")!({ entityName: "Alice", includeHistory: true });+ expect(state.currentEdges).toHaveLength(1);+ expect(state.currentEdges[0]).toMatchObject({ version: 2, isLatest: true });+ expect(state.currentEdges[0].id).not.toBe("old");+ expect(state.historicalEdges.map((edge: GraphEdge) => edge.id)).toEqual(["old"]);+ expect(state.timeline.filter((item: any) => item.edge.id === "old")).toHaveLength(1);+ });+});diff --git a/test/graph-ingestion-import-mesh.test.ts b/test/graph-ingestion-import-mesh.test.tsindex 332bfa90..2bad8d3d 100644--- a/test/graph-ingestion-import-mesh.test.ts+++ b/test/graph-ingestion-import-mesh.test.ts@@ -1,3 +1,4 @@+import { initializeGraphIndex } from "../src/state/graph-retrieval-index.js";import { afterEach, describe, expect, it, vi } from "vitest";import { StateKV, applyUpdateOps } from "../src/state/kv.js";import { KV } from "../src/state/schema.js";@@ -67,6 +68,7 @@ async function rig(rotated = true) {registerViewerStreamTriggers(sdk as never, kv);const call = (id: string, payload: any = {}) => sdk.trigger({ function_id: id, payload });if (rotated) await call("mem::graph-reset");+ else await initializeGraphIndex(kv, { status: "ready" });await getViewerCounts(kv);return { kv, call, store, intercept: (fn?: typeof intercept) => { intercept = fn; } };}diff --git a/test/graph-ingestion-snapshot.test.ts b/test/graph-ingestion-snapshot.test.tsindex 42907fe5..7ad72a88 100644--- a/test/graph-ingestion-snapshot.test.ts+++ b/test/graph-ingestion-snapshot.test.ts@@ -1,3 +1,4 @@+import { initializeGraphIndex } from "../src/state/graph-retrieval-index.js";import { execFileSync } from "node:child_process";import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";import { join } from "node:path";@@ -21,7 +22,7 @@ const memory = (id: string, marker: string, extra: Partial<Memory> = {}): Memoryconst observation = (id: string, marker: string): CompressedObservation => ({ id, sessionId: "session", timestamp, type: "discovery", title: marker, narrative: marker, concepts: [], files: [], facts: [], importance: 5 });const dirs: string[] = [];-function setup() {+async function setup() {const dir = mkdtempSync(join(process.cwd(), ".graph-ingestion-snapshot-test-"));dirs.push(dir);const store = new Map<string, Map<string, unknown>>();@@ -46,6 +47,7 @@ function setup() {},};const kv = new StateKV(sdk as never);+ await initializeGraphIndex(kv, { status: "ready" });registerGraphFunction(sdk as never, kv, { name: "noop" } as never);registerSnapshotFunction(sdk as never, kv, dir);const call = (id: string, payload: any = {}) => sdk.trigger({ function_id: id, payload });@@ -76,7 +78,7 @@ afterEach(() => {describe("Git snapshot ingestion", () => {it("round-trips graph edges and makes a restore after reset immediately queryable and deduplicated", async () => {- const r = setup();+ const r = await setup();await persistGraphDelta(r.kv, [node("a", "alpha"), node("b", "beta")], [edge("ab", "a", "b")]);const created = await r.call("mem::snapshot-create");expect(created.success).toBe(true);@@ -93,7 +95,7 @@ describe("Git snapshot ingestion", () => {});it("restores legacy nodes-only archives as an overlay without changing the Git worktree", async () => {- const r = setup();+ const r = await setup();await persistGraphDelta(r.kv, [node("existing")], []);const commitHash = r.archive({ graphNodes: [node("legacy")] });const uncommitted = '{"uncommitted":"preserve me"}';@@ -106,7 +108,7 @@ describe("Git snapshot ingestion", () => {});it("replaces warm keyword terms and stale vectors while preserving unrelated indexed rows", async () => {- const r = setup();+ const r = await setup();const oldMemory = memory("mem_restore", "olduniquememory"), oldObs = observation("obs_restore", "olduniqueobservation");const unrelated = memory("mem_keep", "unrelatedmarker");await indexRecords([oldObs], [oldMemory, unrelated]);@@ -133,7 +135,7 @@ describe("Git snapshot ingestion", () => {});it("removes old keyword and vector entries for raw observations and nonlatest memories", async () => {- const r = setup();+ const r = await setup();const oldMemory = memory("mem_raw", "oldmemory"), oldObs = observation("obs_raw", "oldobservation");await indexRecords([oldObs], [oldMemory]);const vectors = new VectorIndex();@@ -151,7 +153,7 @@ describe("Git snapshot ingestion", () => {});it("rejects corrupt graph rows before restoring any memory records", async () => {- const r = setup();+ const r = await setup();const original = memory("mem_preflight", "originalmarker");await r.kv.set(KV.memories, original.id, original);await indexRecords([], [original]);@@ -163,7 +165,7 @@ describe("Git snapshot ingestion", () => {});it("reconciles committed records when a later restore write fails", async () => {- const r = setup();+ const r = await setup();const untouched = memory("mem_failed", "untouchedmarker");await r.kv.set(KV.memories, untouched.id, untouched);await indexRecords([], [untouched]);@@ -179,7 +181,7 @@ describe("Git snapshot ingestion", () => {});it("reconciles committed search rows when graph ingestion fails", async () => {- const r = setup();+ const r = await setup();const commitHash = r.archive({ memories: [memory("mem_before_graph", "beforegraphmarker")], graphNodes: [node("graph_failure")] });r.intercept(async (id, p) => { if (id === "state::set" && p.key === "graph_failure") throw new Error("synthetic graph write failure"); });expect(await r.call("mem::snapshot-restore", { commitHash })).toMatchObject({ success: false });@@ -187,7 +189,7 @@ describe("Git snapshot ingestion", () => {});it("indexes the current committed row if another writer replaced it during restore", async () => {- const r = setup();+ const r = await setup();const commitHash = r.archive({ memories: [memory("mem_race", "snapshotmarker")], graphNodes: [node("gate")] });r.intercept(async (id, p) => {if (id === "state::set" && p.key === "gate") {@@ -201,7 +203,7 @@ describe("Git snapshot ingestion", () => {});it("preserves a newer first record while reconciliation waits for a second record", async () => {- const r = setup();+ const r = await setup();const first = memory("mem_first", "firstsnapshotmarker");const second = memory("mem_second", "secondsnapshotmarker");const commitHash = r.archive({ memories: [first, second] });@@ -235,7 +237,7 @@ describe("Git snapshot ingestion", () => {});it("reads snapshots larger than execFile's default output buffer", async () => {- const r = setup();+ const r = await setup();const large = memory("mem_large", "largemarker", { content: "largemarker " + "x".repeat(1_100_000) });const commitHash = r.archive({ memories: [large] });expect(await r.call("mem::snapshot-restore", { commitHash })).toMatchObject({ success: true });@@ -244,7 +246,7 @@ describe("Git snapshot ingestion", () => {});it("reports index reconciliation failures, flushes stale-entry removals, and permits retry", async () => {- const r = setup();+ const r = await setup();await indexRecords([], [memory("mem_retry", "obsoletekeyword")]);const save = vi.fn(async () => {});setIndexPersistence({ scheduleSave: vi.fn(), save });@@ -262,7 +264,7 @@ describe("Git snapshot ingestion", () => {});it("embeds replacement records using their restored text", async () => {- const r = setup();+ const r = await setup();const vectors = new VectorIndex();vectors.add("mem_vector", "memory", new Float32Array([1, 0]));setVectorIndex(vectors);diff --git a/test/graph-ingestion.test.ts b/test/graph-ingestion.test.tsindex 9e440194..f4d4768f 100644--- a/test/graph-ingestion.test.ts+++ b/test/graph-ingestion.test.ts@@ -4,6 +4,7 @@ import { KV } from "../src/state/schema.js";import { pinGraphState, setGraphGenerationListener } from "../src/state/graph-generation.js";import { persistGraphDelta, registerGraphFunction } from "../src/functions/graph.js";import { persistGraphRecords } from "../src/state/graph-ingestion.js";+import { initializeGraphIndex } from "../src/state/graph-retrieval-index.js";import { registerViewerStreamTriggers } from "../src/triggers/viewer-streams.js";import { resetViewerCounts } from "../src/state/viewer-counts.js";import { setAuditRecordedListener } from "../src/functions/audit.js";@@ -226,8 +227,9 @@ describe("raw graph ingestion reconciliation", () => {expect(await kv.get(KV.graphNodes, "added")).toMatchObject({ id: "added" });});- it("serves a clean committed snapshot while a larger ingestion is still planning its next batch", async () => {+ it("serves committed index counts while a larger ingestion is still planning its next batch", async () => {const { kv, call, intercept } = setup();+ await initializeGraphIndex(kv, { status: "ready" });const entered = deferred();const release = deferred();intercept(async (id, p) => {@@ -281,6 +283,7 @@ describe("raw graph ingestion reconciliation", () => {it("does not cache old clean state while an invalidated journal write is still in flight", async () => {const { kv, call, snapshot, intercept } = setup();+ await initializeGraphIndex(kv, { status: "ready" });await persistGraphRecords(kv, [node("before")], [], { mode: "merge" });const entered = deferred();const release = deferred();@@ -297,7 +300,7 @@ describe("raw graph ingestion reconciliation", () => {const reading = call("mem::graph-stats").then((result) => { observed = result; });try {await new Promise<void>((done) => setImmediate(done));- expect(observed).toMatchObject({ totalNodes: 1 });+ expect(observed).toMatchObject({ available: false, totalNodes: null });} finally {release.resolve();await Promise.all([failed, reading]);@@ -307,6 +310,7 @@ describe("raw graph ingestion reconciliation", () => {expect(recovered.pendingMutation).toBeUndefined();expect(recovered.stats.totalNodes).toBe(2);expect(await kv.get(KV.graphNodes, "after")).toMatchObject({ id: "after" });+ expect(await call("mem::graph-stats")).toMatchObject({ available: true, totalNodes: 2 });});it("matches a raw-record oracle through a deterministic sequence of overlapping mutations", async () => {diff --git a/test/graph-provenance-cap.test.ts b/test/graph-provenance-cap.test.tsindex 8def25bf..21742119 100644--- a/test/graph-provenance-cap.test.ts+++ b/test/graph-provenance-cap.test.ts@@ -1,3 +1,5 @@+import { createGraphIndexState } from "./helpers/graph-index-state.js";+import { initializeGraphIndex } from "../src/state/graph-retrieval-index.js";import { describe, it, expect, vi } from "vitest";vi.mock("../src/logger.js", () => ({@@ -10,24 +12,10 @@ import {} from "../src/functions/graph.js";import type { GraphEdge, GraphNode } from "../src/types.js";-function mockKV() {- const store = new Map<string, Map<string, unknown>>();- return {- get: async <T>(scope: string, key: string): Promise<T | null> =>- (store.get(scope)?.get(key) as T) ?? null,- set: async <T>(scope: string, key: string, data: T): Promise<T> => {- if (!store.has(scope)) store.set(scope, new Map());- store.get(scope)!.set(key, data);- return data;- },- delete: async (scope: string, key: string): Promise<void> => {- store.get(scope)?.delete(key);- },- list: async <T>(scope: string): Promise<T[]> => {- const entries = store.get(scope);- return entries ? (Array.from(entries.values()) as T[]) : [];- },- };+async function mockKV() {+ const { kv } = createGraphIndexState();+ await initializeGraphIndex(kv, { status: "ready" });+ return kv;}let seq = 0;@@ -59,14 +47,14 @@ function edge(src: string, tgt: string, sources: string[]): GraphEdge {const ids = (prefix: string, n: number) =>Array.from({ length: n }, (_, i) => `${prefix}_${i}`);-async function onlyNode(kv: ReturnType<typeof mockKV>, name: string) {+async function onlyNode(kv: Awaited<ReturnType<typeof mockKV>>, name: string) {const all = await kv.list<GraphNode>("mem:graph:nodes");const hits = all.filter((n) => n.name === name);expect(hits.length).toBe(1);return hits[0]!;}-async function onlyEdge(kv: ReturnType<typeof mockKV>) {+async function onlyEdge(kv: Awaited<ReturnType<typeof mockKV>>) {const all = await kv.list<GraphEdge>("mem:graph:edges");expect(all.length).toBe(1);return all[0]!;@@ -78,7 +66,7 @@ describe("graph provenance is bounded and attributed", () => {});it("a node merged many times never exceeds the cap", async () => {- const kv = mockKV();+ const kv = await mockKV();for (let batch = 0; batch < 20; batch++) {const batchIds = ids(`obs_b${batch}`, 10);await persistGraphDelta(kv as never, [node("CLAUDE.md", batchIds)], []);@@ -88,7 +76,7 @@ describe("graph provenance is bounded and attributed", () => {});it("the cap keeps the NEWEST ids, in arrival order", async () => {- const kv = mockKV();+ const kv = await mockKV();const all: string[] = [];for (let batch = 0; batch < 10; batch++) {const batchIds = ids(`obs_b${batch}`, 10);@@ -102,7 +90,7 @@ describe("graph provenance is bounded and attributed", () => {});it("a re-seen id moves to the newest end and survives trimming", async () => {- const kv = mockKV();+ const kv = await mockKV();await persistGraphDelta(kv as never, [node("hub", ["obs_old"])], []);const filler = ids("obs_fill", MAX_GRAPH_SOURCE_OBSERVATIONS - 1);await persistGraphDelta(kv as never, [node("hub", filler)], []);@@ -116,7 +104,7 @@ describe("graph provenance is bounded and attributed", () => {});it("ids are never duplicated", async () => {- const kv = mockKV();+ const kv = await mockKV();for (let i = 0; i < 3; i++) {await persistGraphDelta(kv as never,@@ -129,7 +117,7 @@ describe("graph provenance is bounded and attributed", () => {});it("a brand-new node carrying more than the cap is trimmed on first write", async () => {- const kv = mockKV();+ const kv = await mockKV();const many = ids("obs", MAX_GRAPH_SOURCE_OBSERVATIONS + 15);await persistGraphDelta(kv as never, [node("fresh", many)], []);const n = await onlyNode(kv, "fresh");@@ -139,7 +127,7 @@ describe("graph provenance is bounded and attributed", () => {});it("a merged node gains ITS OWN sources, not every id in the batch", async () => {- const kv = mockKV();+ const kv = await mockKV();await persistGraphDelta(kv as never, [node("a.ts", ["obs_1"])], []);await persistGraphDelta(kv as never,@@ -151,7 +139,7 @@ describe("graph provenance is bounded and attributed", () => {});it("a merged edge gains its own sources, bounded, not the batch", async () => {- const kv = mockKV();+ const kv = await mockKV();const a = node("a", ["obs_0"]);const b = node("b", ["obs_0"]);await persistGraphDelta(kv as never, [a, b], [edge(a.id, b.id, ["obs_0"])]);@@ -173,7 +161,7 @@ describe("graph provenance is bounded and attributed", () => {});it("a merged edge is capped", async () => {- const kv = mockKV();+ const kv = await mockKV();const a = node("a", ["obs_0"]);const b = node("b", ["obs_0"]);await persistGraphDelta(kv as never, [a, b], [edge(a.id, b.id, ["obs_0"])]);@@ -188,7 +176,7 @@ describe("graph provenance is bounded and attributed", () => {});it("a brand-new edge carrying more than the cap is trimmed on first write", async () => {- const kv = mockKV();+ const kv = await mockKV();const a = node("a", ["obs_0"]);const b = node("b", ["obs_0"]);const many = ids("obs", MAX_GRAPH_SOURCE_OBSERVATIONS + 5);@@ -200,7 +188,7 @@ describe("graph provenance is bounded and attributed", () => {});it("a record written before this change with an oversized list is trimmed on its next merge", async () => {- const kv = mockKV();+ const kv = await mockKV();const legacy = node("legacy", ids("obs_legacy", 500));await kv.set("mem:graph:nodes", legacy.id, legacy);await kv.set("mem:graph:name-index", "concept|legacy", legacy.id);@@ -245,7 +233,7 @@ describe("temporal graph provenance is bounded", () => {summarize: vi.fn().mockResolvedValue(response),};const sdk = mockSdk();- const kv = mockKV();+ const kv = await mockKV();registerTemporalGraphFunctions(sdk as never,kv as never,@@ -298,7 +286,7 @@ describe("temporal graph provenance is bounded", () => {summarize: vi.fn().mockResolvedValue(response),};const sdk = mockSdk();- const kv = mockKV();+ const kv = await mockKV();registerTemporalGraphFunctions(sdk as never,kv as never,diff --git a/test/graph-query.test.ts b/test/graph-query.test.tsindex c47d3532..6b482c7d 100644--- a/test/graph-query.test.ts+++ b/test/graph-query.test.ts@@ -1,7 +1,7 @@import { mkdtempSync, rmSync } from "node:fs";import { join } from "node:path";import { afterEach, describe, expect, it, vi } from "vitest";-import { GraphRetrieval } from "../src/functions/graph-retrieval.js";+import { GraphRetrieval, type GraphObservationAuthorization } from "../src/functions/graph-retrieval.js";import { registerExportImportFunction } from "../src/functions/export-import.js";import { registerMeshFunction } from "../src/functions/mesh.js";import { registerGraphFunction } from "../src/functions/graph.js";@@ -10,6 +10,8 @@ import { getSearchIndex, registerSearchFunction, setHybridRanker, setVectorIndeximport { registerSmartSearchFunction } from "../src/functions/smart-search.js";import { StateKV, applyUpdateOps } from "../src/state/kv.js";import { KV } from "../src/state/schema.js";+import { initializeGraphIndex } from "../src/state/graph-retrieval-index.js";+import { indexObservationSession } from "../src/state/obs-index.js";import { persistGraphRecords } from "../src/state/graph-ingestion.js";import { setGraphGenerationListener } from "../src/state/graph-generation.js";import { HybridSearch } from "../src/state/hybrid-search.js";@@ -25,7 +27,7 @@ const node = (id: string, name = id, sources = [`obs_${id}`]): GraphNode => ({});const edge = (id: string, from: string, to: string, weight = 0.8): GraphEdge => ({id, sourceNodeId: from, targetNodeId: to, type: "related_to", weight,- sourceObservationIds: [], createdAt: timestamp,+ sourceObservationIds: [`obs_${from}`], createdAt: timestamp,});const observation = (id: string): CompressedObservation => ({id, sessionId: "session", timestamp, type: "discovery", title: "React auth", narrative: "React authentication",@@ -38,10 +40,11 @@ function deferred() {return { promise, resolve };}-function rig() {+async function rig() {const store = new Map<string, Map<string, unknown>>();const handlers = new Map<string, (data: any) => Promise<any>>();const lists: string[] = [];+ const reads: Array<{ scope: string; key: string }> = [];let afterList: ((scope: string) => Promise<void>) | undefined;const sdk = {registerFunction: (id: string, fn: (data: any) => Promise<any>) => handlers.set(id, fn),@@ -50,7 +53,12 @@ function rig() {if (id.startsWith("stream::")) return {};if (!id.startsWith("state::")) return handlers.get(id)!(payload);const rows = store.get(payload.scope) ?? new Map<string, unknown>();- if (id === "state::get") return structuredClone(rows.get(payload.key) ?? null);+ if (id === "state::get") {+ reads.push({ scope: payload.scope, key: payload.key });+ const result = structuredClone(rows.get(payload.key) ?? null);+ await afterList?.(payload.scope);+ return result;+ }if (id === "state::list") {lists.push(payload.scope);const result = structuredClone([...rows.values()]);@@ -67,14 +75,27 @@ function rig() {},};const kv = new StateKV(sdk as never);+ await initializeGraphIndex(kv, { status: "ready" });registerGraphFunction(sdk as never, kv, { name: "noop" } as never);registerExportImportFunction(sdk as never, kv);registerMeshFunction(sdk as never, kv);return {- kv, sdk, lists,+ kv, sdk, lists, reads,call: (id: string, payload: any = {}) => sdk.trigger({ function_id: id, payload }),intercept: (fn?: typeof afterList) => { afterList = fn; },+ seedSources: async (nodes: GraphNode[], edges: GraphEdge[] = []) => {+ for (const row of [...nodes, ...edges]) for (const id of row.sourceObservationIds) {+ const sessionId = getSearchIndex().sessionOf(id) ?? "session";+ if (!await kv.get(KV.observations(sessionId), id)) await kv.set(KV.observations(sessionId), id, { ...observation(id), sessionId });+ await indexObservationSession(kv, id, sessionId);+ }+ },seed: async (nodes: GraphNode[], edges: GraphEdge[] = []) => {+ for (const row of [...nodes, ...edges]) for (const id of row.sourceObservationIds) {+ const sessionId = getSearchIndex().sessionOf(id) ?? "session";+ if (!await kv.get(KV.observations(sessionId), id)) await kv.set(KV.observations(sessionId), id, { ...observation(id), sessionId });+ await indexObservationSession(kv, id, sessionId);+ }for (const row of nodes) await kv.set(KV.graphNodes, row.id, row);for (const row of edges) await kv.set(KV.graphEdges, row.id, row);},@@ -97,9 +118,10 @@ afterEach(() => {describe("request graph views", () => {it("shares a lazy pinned view across concurrent streams, but never across requests", async () => {- const r = rig();+ const r = await rig();await r.seed([node("a", "React"), node("b", "Hooks")], [edge("ab", "a", "b")]);const retrieval = new GraphRetrieval(r.kv);+ r.reads.length = 0;const query = retrieval.createQuery();expect(graphReads(r.lists)).toEqual([]);const [entities, expanded, temporal] = await Promise.all([@@ -108,13 +130,15 @@ describe("request graph views", () => {expect(entities.map((hit) => hit.obsId)).toContain("obs_b");expect(expanded.map((hit) => hit.obsId)).toEqual(["obs_b"]);expect(temporal.entity?.id).toBe("a");- expect(graphReads(r.lists)).toEqual([KV.graphNodes, KV.graphEdges]);+ expect(graphReads(r.lists)).toEqual([]);+ const recordReads = () => r.reads.filter((read) => read.key.includes(":record:node:"));+ expect(recordReads()).toHaveLength(2);await retrieval.searchByEntities(["React"]);- expect(graphReads(r.lists)).toHaveLength(4);+ expect(recordReads()).toHaveLength(4);});it("refreshes after writes, updates, deletion and stale markers without a TTL", async () => {- const r = rig();+ const r = await rig();const retrieval = new GraphRetrieval(r.kv);await r.seed([node("a", "React"), node("b", "Hooks")], [edge("ab", "a", "b")]);expect(await retrieval.expandFromChunks(["obs_a"])).toHaveLength(1);@@ -129,11 +153,11 @@ describe("request graph views", () => {});it("does not mix generations when reset occurs between node and edge reads", async () => {- const r = rig();+ const r = await rig();await r.seed([node("a", "React"), node("b", "Hooks")], [edge("ab", "a", "b")]);const entered = deferred(); const release = deferred();r.intercept(async (scope) => {- if (scope === KV.graphNodes) { r.intercept(); entered.resolve(); await release.promise; }+ if (scope === KV.graphRetrievalData) { r.intercept(); entered.resolve(); await release.promise; }});const retrieval = new GraphRetrieval(r.kv);const pending = retrieval.searchByEntities(["React"]);@@ -141,27 +165,30 @@ describe("request graph views", () => {await r.call("mem::graph-reset");await r.seed([node("fresh", "React")]);release.resolve();- expect((await pending).map((hit) => hit.obsId).sort()).toEqual(["obs_a", "obs_b"]);+ expect(await pending).toEqual([]);expect((await retrieval.searchByEntities(["React"])).map((hit) => hit.obsId)).toEqual(["obs_fresh"]);});it("retries a failed load on the next request while sharing failure within one request", async () => {- const r = rig();+ const r = await rig();await r.seed([node("a", "React")]);- r.intercept(async () => { throw new Error("unavailable"); });+ r.reads.length = 0;+ r.intercept(async (scope) => { if (scope === KV.graphRetrievalControl) throw new Error("unavailable"); });const retrieval = new GraphRetrieval(r.kv);const query = retrieval.createQuery();await expect(query.searchByEntities(["React"])).rejects.toThrow("unavailable");await expect(query.expandFromChunks(["obs_a"])).rejects.toThrow("unavailable");- expect(graphReads(r.lists)).toHaveLength(1);+ expect(r.reads.filter((read) => read.scope === KV.graphRetrievalControl)).toHaveLength(1);+ expect(graphReads(r.lists)).toEqual([]);r.intercept();expect(await retrieval.searchByEntities(["React"])).toHaveLength(1);});it("keeps stores and layer resolvers separate and skips empty work", async () => {- const a = rig(); const b = rig();+ const a = await rig(); const b = await rig();await a.seed([node("a", "React", ["custom_memory", "mem_observation"])]);await b.seed([node("b", "React")]);+ await a.kv.set(KV.memories, "custom_memory", { id: "custom_memory", title: "Saved", content: "Saved", sessionIds: [], isLatest: true });const retrieval = new GraphRetrieval(a.kv, (id) => id === "custom_memory" ? "memory" : "observation");const query = retrieval.createQuery();expect((await query.searchByEntities(["React"], 2, 20, "memory")).map((hit) => hit.obsId)).toEqual(["custom_memory"]);@@ -177,7 +204,7 @@ describe("request graph views", () => {describe("graph traversal bounds", () => {it("caps matching starts and retains an exact match after many partial matches", async () => {- const r = rig();+ const r = await rig();await r.seed([...Array.from({ length: 100 }, (_,i) => node(`partial_${i}`, `auth-${i}`)), node("exact", "auth")]);const hits = await new GraphRetrieval(r.kv).searchByEntities(["auth"], 2, 1000);expect(hits).toHaveLength(25);@@ -185,7 +212,7 @@ describe("graph traversal bounds", () => {});it("orders exact names, partial names, and duplicate-name IDs by code units before the start cap", async () => {- const r = rig();+ const r = await rig();await r.seed([...["ä", "a", "Z", "😀"].map((id) => node(id, id === "Z" ? "AUTH" : "auth")),...["Ä", "Z", "á", "a", "😀"].map((name) => node(`partial-${name}`, `auth-${name}`)),@@ -200,7 +227,7 @@ describe("graph traversal bounds", () => {});it("orders chunk-expansion starts by code units rather than host collation", async () => {- const r = rig();+ const r = await rig();const ids = ["ä", "a", "Z", "😀"];await r.seed([...ids.map((id) => node(id, id, ["seed"])),@@ -211,7 +238,7 @@ describe("graph traversal bounds", () => {});it("breaks equal-weight adjacency ties by neighbor and edge IDs using code units", async () => {- const r = rig();+ const r = await rig();const ids = ["ä", "a", "Z", "😀"];await r.seed([node("hub", "Anchor", ["seed"]), ...ids.map((id) => node(id))], [...ids.map((id) => edge(`edge-${id}`, "hub", id)),@@ -220,11 +247,11 @@ describe("graph traversal bounds", () => {]);const hits = await new GraphRetrieval(r.kv).expandFromChunks(["seed"], 1, 10);expect(hits.map((hit) => hit.obsId)).toEqual(["obs_Z", "obs_a", "obs_ä", "obs_😀"]);- expect(hits[0].graphContext).toContain("--uses-->");+ expect(hits[0].graphContext).toBe("[graph evidence] React auth");});it("caps chunk expansion starts as well as entity starts", async () => {- const r = rig();+ const r = await rig();const starts = Array.from({ length: 100 }, (_, i) => node(`start_${i}`, `source-${i}`, ["seed"]));const ends = starts.map((_, i) => node(`end_${i}`));await r.seed([...starts, ...ends], starts.map((start, i) => edge(`edge_${i}`, start.id, ends[i].id)));@@ -234,30 +261,39 @@ describe("graph traversal bounds", () => {});it.each(["entities", "chunks"])("bounds discovered nodes on a wide hub for %s", async (mode) => {- const r = rig();+ const r = await rig();const spokes = Array.from({ length: 2500 }, (_, i) => node(`spoke_${i}`));await r.seed([node("hub", "Anchor"), ...spokes], spokes.map((spoke, i) => edge(`edge_${i}`, "hub", spoke.id)));- const retrieval = new GraphRetrieval(r.kv);- const hits = mode === "entities" ? await retrieval.searchByEntities(["Anchor"], 2, 5000) : await retrieval.expandFromChunks(["obs_hub"], 2, 5000);+ r.reads.length = 0;+ const query = new GraphRetrieval(r.kv).createQuery();+ const hits = mode === "entities" ? await query.searchByEntities(["Anchor"], 2, 5000) : await query.expandFromChunks(["obs_hub"], 2, 5000);expect(hits.length).toBeGreaterThan(0);expect(hits.length).toBeLessThanOrEqual(2000);- });+ expect(query.diagnostics().truncated).toBe(true);+ expect(r.reads.length).toBeLessThanOrEqual(512);+ expect(graphReads(r.lists)).toEqual([]);+ }, 30_000);it("bounds examined edges even when parallel edges discover very few nodes", async () => {- const r = rig();+ const r = await rig();await r.seed([node("hub", "Anchor"), node("a"), node("tail")], [...Array.from({ length: 10_001 }, (_, i) => edge(`parallel_${i}`, "hub", "a", 1)),edge("tail", "hub", "tail", 0.1),]);- const hits = await new GraphRetrieval(r.kv).searchByEntities(["Anchor"], 2, 100);+ r.reads.length = 0;+ const query = new GraphRetrieval(r.kv).createQuery();+ const hits = await query.searchByEntities(["Anchor"], 2, 100);expect(hits.map((hit) => hit.obsId)).toContain("obs_a");- expect(hits.map((hit) => hit.obsId)).not.toContain("obs_tail");- });+ expect(query.diagnostics().truncated).toBe(true);+ expect(r.reads.filter((read) => read.key.includes(":record:edge:")).length).toBeLessThanOrEqual(256);+ expect(r.reads.length).toBeLessThanOrEqual(512);+ expect(graphReads(r.lists)).toEqual([]);+ }, 30_000);});describe("hybrid graph query lifetime", () => {it.each(["mem::search", "mem::smart-search"])("retrieves graph-only observations through %s without embeddings and preserves scope", async (handler) => {- const r = rig();+ const r = await rig();const index = getSearchIndex();const captures = [{ ...observation("owned"), sessionId: "owned_session", agentId: "agent-a" },@@ -272,7 +308,7 @@ describe("hybrid graph query lifetime", () => {await r.kv.set(KV.sessions, "other_session", { id: "other_session", project: "project-b", cwd: "/other" });await r.seed([node("anchor", "GraphAnchor", captures.map((obs) => obs.id))]);const hybrid = new HybridSearch(index, null, null, r.kv, 0.4, 0.6, 0.3, false);- const rank = (query: string, limit: number, layer?: "all" | "memory" | "observation") => hybrid.search(query, limit, layer);+ const rank = (query: string, limit: number, layer?: "all" | "memory" | "observation", authorize?: GraphObservationAuthorization) => hybrid.search(query, limit, layer, authorize);setHybridRanker(rank);registerSearchFunction(r.sdk as never, r.kv);registerSmartSearchFunction(r.sdk as never, r.kv, rank);@@ -289,7 +325,7 @@ describe("hybrid graph query lifetime", () => {});async function hybridRig(weight = 0.3) {- const r = rig(); const index = new SearchIndex(); const vectors = new VectorIndex();+ const r = await rig(); const index = new SearchIndex(); const vectors = new VectorIndex();const obs = observation("obs_a"); index.add(obs); vectors.add(obs.id, obs.sessionId, new Float32Array([1, 0]));await r.kv.set(KV.observations(obs.sessionId), obs.id, obs);await r.seed([node("a", "React"), node("b", "Hooks")], [edge("ab", "a", "b")]);@@ -304,9 +340,11 @@ describe("hybrid graph query lifetime", () => {? r.hybrid.searchWithExpansion("React auth", 10, { reformulations: ["React", "authentication"], temporalConcretizations: ["recent React"], entityExtractions: ["React"] }): r.hybrid.search("React auth", 10);expect(await run()).not.toEqual([]);- expect(graphReads(r.lists)).toHaveLength(2);+ expect(graphReads(r.lists)).toEqual([]);+ const readsBefore = r.reads.filter((read) => read.scope === KV.graphRetrievalControl).length;await run();- expect(graphReads(r.lists)).toHaveLength(4);+ expect(r.reads.filter((read) => read.scope === KV.graphRetrievalControl).length).toBeGreaterThan(readsBefore);+ expect(graphReads(r.lists)).toEqual([]);});it("does not read or pin graph state when graph weight is zero, including expansion", async () => {@@ -321,12 +359,13 @@ describe("hybrid graph query lifetime", () => {describe("graph mutations between search requests", () => {it.each(["import", "mesh"])("sees %s changes without a stale warmed view", async (mode) => {- const r = rig();+ const r = await rig();const retrieval = new GraphRetrieval(r.kv);await r.call("mem::graph-reset");expect(await retrieval.searchByEntities(["React"])).toEqual([]);const graphNodes = [node("a", "React"), node("b", "Hooks")];const graphEdges = [edge("ab", "a", "b")];+ await r.seedSources(graphNodes, graphEdges);const result = mode === "import"? await r.call("mem::import", { exportData: { version: "0.9.30", exportedAt: timestamp, sessions: [], observations: {}, memories: [], summaries: [], graphNodes, graphEdges } }): await r.call("mem::mesh-receive", { graphNodes, graphEdges });@@ -337,8 +376,9 @@ describe("graph mutations between search requests", () => {});it("refreshes the same retrieval instance after snapshot restoration and reset", async () => {- const r = rig(); const dir = mkdtempSync(join(process.cwd(), ".graph-query-snapshot-")); dirs.push(dir);+ const r = await rig(); const dir = mkdtempSync(join(process.cwd(), ".graph-query-snapshot-")); dirs.push(dir);registerSnapshotFunction(r.sdk as never, r.kv, dir);+ await r.seedSources([node("a", "React"), node("b", "Hooks")], [edge("ab", "a", "b")]);await persistGraphRecords(r.kv, [node("a", "React"), node("b", "Hooks")], [edge("ab", "a", "b")]);const saved = await r.call("mem::snapshot-create"); expect(saved.success).toBe(true);const retrieval = new GraphRetrieval(r.kv);diff --git a/test/graph-retrieval-index.test.ts b/test/graph-retrieval-index.test.tsnew file mode 100644index 00000000..db414478--- /dev/null+++ b/test/graph-retrieval-index.test.ts@@ -0,0 +1,371 @@+import { afterEach, describe, expect, it } from "vitest";+import { createHash } from "node:crypto";+import { StateKV, applyUpdateOps, type StateBackend } from "../src/state/kv.js";+import { KV } from "../src/state/schema.js";+import { pinGraphState, resetGraphGeneration, setGraphGenerationListener } from "../src/state/graph-generation.js";+import { createGraphIndexReader, finishGraphIndexBuild, getGraphIndexStatus, initializeGraphIndex, reconcileGraphIndexRecord, recoverGraphIndex } from "../src/state/graph-retrieval-index.js";+import { withKeyedLock } from "../src/state/keyed-mutex.js";+import type { GraphNode } from "../src/types.js";++const node = (id: string, name = id): GraphNode => ({ id, name, type: "concept", properties: {}, sourceObservationIds: [`obs-${id}`], createdAt: "2026-10-14T00:00:00Z" });+function fixture(backend: StateBackend = "file") {+ const data = new Map<string, unknown>();+ const calls: Array<{ function_id: string; payload: any }> = [];+ let fail: ((call: typeof calls[number], committed: boolean) => void) | undefined;+ const sdk = { trigger: async (call: typeof calls[number]) => {+ calls.push(structuredClone(call)); fail?.(call, false);+ const p = call.payload, key = `${p.scope}\0${p.key}`;+ let result: unknown;+ if (call.function_id === "state::get") result = structuredClone(data.get(key) ?? null);+ else if (call.function_id === "state::set") { data.set(key, structuredClone(p.value)); result = p.value; }+ else if (call.function_id === "state::delete") data.delete(key);+ else if (call.function_id === "state::update") {+ const old_value = data.get(key) ?? null, new_value = applyUpdateOps(old_value, p.ops);+ data.set(key, structuredClone(new_value)); result = { old_value, new_value, errors: [] };+ } else throw new Error(`Unbounded or unexpected operation: ${call.function_id}`);+ fail?.(call, true); return result;+ } };+ return { data, calls, sdk, kv: new StateKV(sdk as never, { backend }), fail: (fn?: typeof fail) => { fail = fn; } };+}++afterEach(() => setGraphGenerationListener(null));++describe("bounded graph retrieval index", () => {+ it("keeps unknown legacy stores unavailable and never lists or reads their snapshot for status", async () => {+ const { kv, calls } = fixture();+ expect(await getGraphIndexStatus(kv)).toBeNull();+ expect(await createGraphIndexReader(kv)).toBeNull();+ expect(calls.every((c) => c.function_id === "state::get" && c.payload.scope !== KV.graphSnapshot)).toBe(true);+ });++ it.each(["file", "redis"] as const)("maintains duplicate names, aliases, provenance and adjacency through %s set, update, stale and delete", async (backend) => {+ const { kv } = fixture(backend); await initializeGraphIndex(kv, { status: "ready" });+ await kv.set(KV.graphNodes, "a", { ...node("a", "Alpha"), aliases: ["北京"] });+ await kv.set(KV.graphNodes, "b", node("b", "Alpha"));+ let reader = (await createGraphIndexReader(kv))!;+ expect((await reader.posting("name", "alpha")).ids.sort()).toEqual(["a", "b"]);+ expect((await reader.posting("name", "北京")).ids).toEqual(["a"]);+ expect((await reader.posting("observation", "obs-a")).ids).toEqual(["a"]);+ await kv.update(KV.graphNodes, "a", [{ type: "set", path: "name", value: "Beta" }, { type: "set", path: "aliases", value: [] }]);+ expect(await reader.finish()).toBe(false);+ reader = (await createGraphIndexReader(kv))!;+ expect((await reader.posting("name", "alpha")).ids).toEqual(["b"]);+ expect((await reader.posting("name", "北京")).ids).toEqual([]);+ const edge = { id: "e", type: "related_to", sourceNodeId: "a", targetNodeId: "b", sourceObservationIds: [], createdAt: "now", weight: 1 };+ await kv.set(KV.graphEdges, "e", edge);+ await kv.set(KV.graphEdgeHistory, "h", { ...edge, id: "h", isLatest: false });+ reader = (await createGraphIndexReader(kv))!;+ expect((await reader.posting("adjacency", "a")).ids).toEqual(["e"]);+ expect((await reader.posting("history", "a")).ids).toEqual(["h"]);+ await kv.set(KV.graphEdges, "e", { ...edge, sourceNodeId: "c", targetNodeId: "c" });+ reader = (await createGraphIndexReader(kv))!;+ expect((await reader.posting("adjacency", "a")).ids).toEqual([]);+ expect((await reader.posting("adjacency", "c")).ids).toEqual(["e"]);+ await kv.set(KV.graphNodes, "a", { ...node("a"), stale: true });+ await kv.delete(KV.graphEdges, "e");+ reader = (await createGraphIndexReader(kv))!;+ expect(await reader.getNode("a")).toBeNull();+ expect((await reader.posting("adjacency", "c")).ids).toEqual([]);+ expect(await getGraphIndexStatus(kv)).toMatchObject({ nodes: 1, edges: 0, historyEdges: 1 });+ });++ it.each([+ ["file", false], ["file", true], ["redis", false], ["redis", true],+ ] as const)("recovers an interrupted %s canonical write with committed=%s", async (backend, committed) => {+ const f = fixture(backend); await initializeGraphIndex(f.kv, { status: "ready" });+ await f.kv.set(KV.graphNodes, "a", node("a", "old"));+ f.fail((c, done) => { if (c.function_id === "state::set" && c.payload.scope === KV.graphNodes && done === committed) throw new Error("offline"); });+ await expect(f.kv.set(KV.graphNodes, "a", node("a", "new"))).rejects.toThrow("offline");+ f.fail();+ expect(await createGraphIndexReader(f.kv)).toBeNull();+ const restarted = new StateKV(f.sdk as never, { backend }); await recoverGraphIndex(restarted);+ const reader = (await createGraphIndexReader(restarted))!;+ expect((await reader.getNode("a"))?.name).toBe(committed ? "new" : "old");+ expect((await reader.posting("name", committed ? "old" : "new")).ids).toEqual([]);+ expect(await reader.finish()).toBe(true);+ });++ it("replays absolute posting edits after a lost acknowledgement without duplicates", async () => {+ const f = fixture(); await initializeGraphIndex(f.kv, { status: "ready" });+ await f.kv.set(KV.graphNodes, "a", node("a", "same"));+ let writes = 0;+ f.fail((c, done) => { if (done && c.function_id === "state::set" && c.payload.scope === KV.graphRetrievalData && ++writes === 4) throw new Error("lost ack"); });+ await expect(f.kv.set(KV.graphNodes, "b", node("b", "same"))).rejects.toThrow("lost ack");+ f.fail(); await recoverGraphIndex(new StateKV(f.sdk as never));+ const reader = (await createGraphIndexReader(f.kv))!;+ expect((await reader.posting("name", "same")).ids.sort()).toEqual(["a", "b"]);+ expect(await getGraphIndexStatus(f.kv)).toMatchObject({ nodes: 2 });+ });++ it.each(["file", "redis"] as const)("does not mutate canonical %s data when the durable barrier fails", async (backend) => {+ const f = fixture(backend); await initializeGraphIndex(f.kv, { status: "ready" });+ f.fail((c) => { if (c.function_id === "state::set" && c.payload.scope === KV.graphRetrievalControl) throw new Error("barrier"); });+ await expect(f.kv.set(KV.graphNodes, "a", node("a"))).rejects.toThrow("barrier");+ expect(f.data.has(`${KV.graphNodes}\0a`)).toBe(false);+ });++ it.each(["file", "redis"] as const)("keeps %s reset indexes isolated from a pinned old view and validates an in-flight reader", async (backend) => {+ const { kv } = fixture(backend); await initializeGraphIndex(kv, { status: "ready" });+ await kv.set(KV.graphNodes, "old", node("old"));+ const old = await pinGraphState(kv), reader = (await createGraphIndexReader(kv))!;+ await resetGraphGeneration(kv, { version: 1, dirty: false });+ await old.set(KV.graphNodes, "late", node("late"));+ expect(await reader.finish()).toBe(false);+ const fresh = (await createGraphIndexReader(kv))!;+ expect((await fresh.posting("nodes")).ids).toEqual([]);+ expect(await fresh.getNode("late")).toBeNull();+ });++ it("bounds projections and every retrieval read while reporting incomplete coverage", async () => {+ const { kv, calls } = fixture(); await initializeGraphIndex(kv, { status: "ready" });+ await kv.set(KV.graphNodes, "large", { ...node("large"), name: "界".repeat(5000), properties: { data: "x".repeat(100000) }, sourceObservationIds: Array.from({ length: 1000 }, (_, i) => `o${i}`) });+ calls.length = 0;+ const reader = (await createGraphIndexReader(kv, { maxReads: 8 }))!;+ const projection = await reader.getNode("large");+ expect(Buffer.byteLength(JSON.stringify(projection))).toBeLessThanOrEqual(16384);+ expect(projection).toMatchObject({ indexIncomplete: true });+ expect(reader.status.incompleteRecords).toBe(1);+ for (let i = 0; i < 20; i++) await reader.getNode(`absent-${i}`);+ expect(reader.exhausted).toBe(true);+ expect(calls.filter((c) => c.payload.scope === KV.graphRetrievalData)).toHaveLength(4);+ expect(await reader.finish()).toBe(true);+ expect(calls).toHaveLength(8);+ expect(calls.every((c) => c.function_id === "state::get" && ![KV.graphNodes, KV.graphEdges, KV.graphSnapshot].includes(c.payload.scope))).toBe(true);+ });++ it.each(["file", "redis"] as const)("keeps %s multirow snapshot journals unavailable and explicitly resumes them after restart", async (backend) => {+ const f = fixture(backend); await initializeGraphIndex(f.kv, { status: "ready" });+ const snapshot = { version: 1, dirty: false, topNodes: [], topEdges: [], topDegrees: {}, stats: { totalNodes: 2, totalEdges: 0, nodesByType: {}, edgesByType: {} }, updatedAt: "now" };+ const pending = { ...snapshot, dirty: true, pendingMutation: { writes: [{ scope: KV.graphNodes, key: "a", value: node("a") }, { scope: KV.graphNodes, key: "b", value: node("b") }], snapshot } };+ await f.kv.set(KV.graphSnapshot, "current", pending);+ expect(await createGraphIndexReader(f.kv)).toBeNull();+ f.calls.length = 0;+ expect(await createGraphIndexReader(new StateKV(f.sdk as never, { backend }))).toBeNull();+ expect(f.calls.every((c) => c.payload.scope !== KV.graphSnapshot)).toBe(true);+ await recoverGraphIndex(new StateKV(f.sdk as never, { backend }), { allowSnapshotRecovery: true });+ const reader = (await createGraphIndexReader(f.kv))!;+ expect((await reader.posting("nodes")).ids.sort()).toEqual(["a", "b"]);+ expect(await reader.finish()).toBe(true);+ });++ it.each(["file", "redis"] as const)("recovers a lost %s clean-snapshot acknowledgement without leaving the index stuck", async (backend) => {+ const f = fixture(backend); await initializeGraphIndex(f.kv, { status: "ready" });+ await f.kv.set(KV.graphSnapshot, "current", { version: 1, dirty: true });+ f.fail((c, done) => { if (done && c.function_id === "state::set" && c.payload.scope === KV.graphSnapshot) throw new Error("lost ack"); });+ await expect(f.kv.set(KV.graphSnapshot, "current", { version: 1, dirty: false })).rejects.toThrow("lost ack");+ f.fail(); expect(await createGraphIndexReader(f.kv)).toBeNull();+ await recoverGraphIndex(new StateKV(f.sdk as never, { backend }), { allowSnapshotRecovery: true });+ expect(await createGraphIndexReader(f.kv)).not.toBeNull();+ });++ it("maintains a building index for concurrent canonical writes and rejects stale cursors and builds", async () => {+ const { kv } = fixture();+ const build = await initializeGraphIndex(kv, { status: "building" });+ await kv.set(KV.graphNodes, "new", node("new"));+ await reconcileGraphIndexRecord(kv, "node", "legacy", node("legacy"));+ expect(await createGraphIndexReader(kv)).toBeNull();+ await expect(finishGraphIndexBuild(kv, "other")).rejects.toThrow("superseded");+ await finishGraphIndexBuild(kv, build.buildId);+ let reader = (await createGraphIndexReader(kv))!;+ const first = await reader.posting("nodes", "", { limit: 1 });+ expect(first.cursor).not.toBeNull();+ expect((await reader.posting("nodes", "", { cursor: first.cursor })).ids).toHaveLength(1);+ await kv.set(KV.graphNodes, "after", node("after"));+ reader = (await createGraphIndexReader(kv))!;+ await expect(reader.posting("nodes", "", { cursor: first.cursor })).rejects.toThrow("expired");+ await expect(initializeGraphIndex(kv, { buildId: build.buildId, status: "building" })).rejects.toThrow("new build ID");+ });++ it("detects corruption, bounds queues, caches only index data and refreshes canonical evidence", async () => {+ const f = fixture(); await initializeGraphIndex(f.kv, { status: "ready" });+ await f.kv.set(KV.graphNodes, "a", node("a"));+ const reader = (await createGraphIndexReader(f.kv, { maxReads: 12, maxConcurrent: 1 }))!;+ f.calls.length = 0;+ await Promise.all(Array.from({ length: 100 }, () => reader.getNode("a")));+ expect(f.calls).toHaveLength(1);+ await f.kv.set(KV.memories, "m", { content: "first" });+ expect(await reader.read(KV.memories, "m")).toEqual({ content: "first" });+ await f.kv.delete(KV.memories, "m");+ expect(await reader.readFresh(KV.memories, "m")).toBeNull();+ const results = await Promise.all(Array.from({ length: 100 }, (_, i) => reader.getNode(`absent${i}`)));+ expect(results.every((v) => v === null)).toBe(true);+ expect(reader.exhausted).toBe(true);+ expect(await reader.finish()).toBe(true);+ const corrupted = await getGraphIndexStatus(f.kv);+ f.data.set(`${KV.graphRetrievalControl}\0current`, { ...corrupted, revision: -1 });+ await expect(createGraphIndexReader(f.kv)).rejects.toThrow("Invalid graph retrieval index control");+ });++ it("aborts one reader without cancelling another reader's work", async () => {+ const { kv } = fixture(); await initializeGraphIndex(kv, { status: "ready" });+ await kv.set(KV.graphNodes, "a", node("a"));+ const abort = new AbortController();+ const first = (await createGraphIndexReader(kv, { signal: abort.signal }))!;+ const second = (await createGraphIndexReader(kv))!;+ abort.abort();+ expect(await first.getNode("a")).toBeNull(); expect(await first.finish()).toBe(false);+ expect(await second.getNode("a")).toMatchObject({ id: "a" }); expect(await second.finish()).toBe(true);+ });++ it.each(["nodes", "edges", "historyEdges"] as const)("rejects missing and inconsistent %s inventory roots", async (kind) => {+ const { kv, data } = fixture();+ const control = await initializeGraphIndex(kv, { status: "ready" });+ const edge = { id: "edge", type: "related_to", sourceNodeId: "a", targetNodeId: "b", weight: 1, sourceObservationIds: [], createdAt: "now" };+ if (kind === "nodes") await kv.set(KV.graphNodes, "a", node("a"));+ else await kv.set(kind === "edges" ? KV.graphEdges : KV.graphEdgeHistory, edge.id, edge);+ const posting = createHash("sha256").update(JSON.stringify(`${kind}\0`)).digest("hex");+ const key = `${KV.graphRetrievalData}\0${control.buildId}:root:${posting}`;+ const root = data.get(key) as { head: string; count: number };+ expect(root.count).toBe(1);+ data.delete(key);+ await expect((await createGraphIndexReader(kv))!.posting(kind)).rejects.toThrow("Missing graph inventory root");+ for (const invalid of [{ ...root, count: 0 }, { ...root, head: null }]) {+ data.set(key, invalid);+ await expect((await createGraphIndexReader(kv))!.posting(kind)).rejects.toThrow("Invalid graph posting root");+ }+ if (kind !== "edges") {+ data.set(key, { ...root, count: 2 });+ await expect((await createGraphIndexReader(kv))!.posting(kind)).rejects.toThrow("inventory count");+ }+ });++ it.each([+ ["file", "rename"], ["file", "delete"], ["redis", "rename"], ["redis", "delete"],+ ] as const)("recovers every durable %s %s write boundary before and after acknowledgement", async (backend, operation) => {+ const f = fixture(backend); await initializeGraphIndex(f.kv, { status: "ready" });+ for (const id of ["head", "middle", "tail"]) await f.kv.set(KV.graphNodes, id, node(id, "shared"));+ const saved = structuredClone(f.data);+ const mutate = (kv: StateKV) => operation === "rename" ? kv.set(KV.graphNodes, "middle", { ...node("middle", "renamed"), aliases: ["别名"] }) : kv.delete(KV.graphNodes, "middle");+ f.calls.length = 0; await mutate(f.kv);+ const writeCount = f.calls.filter((c) => c.function_id === "state::set" || c.function_id === "state::delete").length;+ expect(writeCount).toBeGreaterThan(10);+ for (const after of [false, true]) for (let boundary = 1; boundary <= writeCount; boundary++) {+ f.data.clear(); for (const [key, value] of saved) f.data.set(key, structuredClone(value));+ let seen = 0;+ f.fail((c, committed) => {+ if (committed === after && (c.function_id === "state::set" || c.function_id === "state::delete") && ++seen === boundary) throw new Error("interrupted");+ });+ await expect(mutate(new StateKV(f.sdk as never, { backend }))).rejects.toThrow("interrupted");+ f.fail();+ const restarted = new StateKV(f.sdk as never, { backend }); await recoverGraphIndex(restarted);+ const canonical = f.data.get(`${KV.graphNodes}\0middle`) as GraphNode | undefined;+ const reader = (await createGraphIndexReader(restarted))!;+ expect(await reader.getNode("middle")).toEqual(canonical ?? null);+ expect((await reader.posting("name", "shared")).ids.sort()).toEqual(canonical?.name === "shared" ? ["head", "middle", "tail"] : ["head", "tail"]);+ expect((await reader.posting("nodes")).ids.sort()).toEqual(canonical ? ["head", "middle", "tail"] : ["head", "tail"]);+ expect(await reader.finish()).toBe(true);+ }+ });++ it("bounds oversized identifiers by pausing before canonical writes and leaves canonical data intact", async () => {+ const { kv } = fixture(); await initializeGraphIndex(kv, { status: "ready" });+ const id = "x".repeat(1025); await kv.set(KV.graphNodes, id, node(id));+ expect(await getGraphIndexStatus(kv)).toMatchObject({ status: "paused", error: expect.stringContaining("budget") });+ expect(await createGraphIndexReader(kv)).toBeNull();+ expect(await kv.get(KV.graphNodes, id)).toEqual(node(id));+ });++ it("bounds the full record when a near-limit projection has many distinct Unicode posting keys", async () => {+ const { kv, data } = fixture(); await initializeGraphIndex(kv, { status: "ready" });+ const source = { ...node("unicode"), name: Array.from({ length: 512 }, (_, i) => String.fromCodePoint(0x4e00 + i)).join(""),+ sourceObservationIds: Array.from({ length: 128 }, (_, i) => `${i}-${"a".repeat(1000)}`), properties: { value: "p".repeat(500) } };+ await kv.set(KV.graphNodes, source.id, source);+ expect(await kv.get(KV.graphNodes, source.id)).toEqual(source);+ for (const [key, value] of data) if (key.startsWith(`${KV.graphRetrievalData}\0`)) expect(Buffer.byteLength(JSON.stringify(value))).toBeLessThanOrEqual(32768);+ const reader = (await createGraphIndexReader(kv))!;+ expect(await reader.getNode(source.id)).toMatchObject({ indexIncomplete: true });+ expect(await reader.finish()).toBe(true);+ });++ it("journals live type statistics and explicitly groups imported unknown types", async () => {+ const { kv } = fixture(); await initializeGraphIndex(kv, { status: "ready" });+ await kv.set(KV.graphNodes, "a", node("a"));+ await kv.set(KV.graphNodes, "custom", { ...node("custom"), type: "custom-type" });+ expect(await getGraphIndexStatus(kv)).toMatchObject({ nodesByType: { concept: 1, other: 1 }, otherTypes: 1 });+ await kv.set(KV.graphNodes, "a", { ...node("a"), type: "file" });+ await kv.delete(KV.graphNodes, "custom");+ expect(await getGraphIndexStatus(kv)).toMatchObject({ nodesByType: { file: 1 }, otherTypes: 0 });+ });++ it.each(["file", "redis"] as const)("guards %s snapshot updates and deletion before readers can use a ready index", async (backend) => {+ const { kv } = fixture(backend); await initializeGraphIndex(kv, { status: "ready" });+ const clean = { version: 1, dirty: false, topNodes: [], topEdges: [], topDegrees: {}, stats: { totalNodes: 0, totalEdges: 0, nodesByType: {}, edgesByType: {} }, updatedAt: "now" };+ await kv.set(KV.graphSnapshot, "current", clean);+ await kv.update(KV.graphSnapshot, "current", [{ type: "set", path: "dirty", value: true }, { type: "set", path: "pendingMutation", value: {+ writes: [{ scope: KV.graphNodes, key: "recovered", value: node("recovered") }], snapshot: { ...clean, stats: { ...clean.stats, totalNodes: 1 } },+ } }]);+ expect(await createGraphIndexReader(kv)).toBeNull();+ await recoverGraphIndex(kv);+ expect(await createGraphIndexReader(kv)).toBeNull();+ await recoverGraphIndex(kv, { allowSnapshotRecovery: true });+ expect(await (await createGraphIndexReader(kv))!.getNode("recovered")).toMatchObject({ id: "recovered" });+ await kv.delete(KV.graphSnapshot, "current");+ expect(await createGraphIndexReader(kv)).toBeNull();+ await recoverGraphIndex(kv, { allowSnapshotRecovery: true });+ expect(await createGraphIndexReader(kv)).toBeNull();+ });++ it.each(["file", "redis"] as const)("allows a %s graph listener to update the same key after its index is ready", async (backend) => {+ const { kv } = fixture(backend);+ await resetGraphGeneration(kv, { version: 1, dirty: false });+ await kv.set(KV.graphNodes, "reentrant", node("reentrant", "initial"));+ let reentered = false;+ let listenerSawReadyIndex = false;+ setGraphGenerationListener(async (kind) => {+ if (kind !== "nodes" || reentered) return;+ reentered = true;+ listenerSawReadyIndex = await createGraphIndexReader(kv) !== null;+ await kv.update(KV.graphNodes, "reentrant", [{ type: "set", path: "name", value: "listener" }]);+ });+ const updated = kv.update(KV.graphNodes, "reentrant", [{ type: "set", path: "name", value: "outer" }]);+ await Promise.race([+ updated,+ new Promise<void>((resolve) => setImmediate(resolve)).then(() => { throw new Error("Ready-index listener cannot reenter the mutation queue"); }),+ ]);+ expect(reentered).toBe(true);+ expect(listenerSawReadyIndex).toBe(true);+ expect(await kv.get(KV.graphNodes, "reentrant")).toMatchObject({ name: "listener" });+ const reader = (await createGraphIndexReader(kv))!;+ expect(await reader.getNode("reentrant")).toMatchObject({ name: "listener" });+ expect((await reader.posting("name", "outer")).ids).toEqual([]);+ expect((await reader.posting("name", "listener")).ids).toEqual(["reentrant"]);+ expect(await reader.finish()).toBe(true);+ });++ it("acquires the graph snapshot barrier before the Redis key mutex during concurrent persistence", async () => {+ const { kv } = fixture("redis");+ await initializeGraphIndex(kv, { status: "ready" });+ await kv.set(KV.graphSnapshot, "current", { version: 1, dirty: false });+ let enter!: () => void;+ let release!: () => void;+ const entered = new Promise<void>((resolve) => { enter = resolve; });+ const released = new Promise<void>((resolve) => { release = resolve; });+ const persisting = withKeyedLock("graph:persist", async () => {+ enter();+ await released;+ await kv.update(KV.graphSnapshot, "current", [{ type: "set", path: "writer", value: "persist" }]);+ });+ await entered;+ const concurrent = kv.update(KV.graphSnapshot, "current", [{ type: "set", path: "writer", value: "concurrent" }]);+ await new Promise<void>((resolve) => setImmediate(resolve));+ release();+ await Promise.race([+ Promise.all([persisting, concurrent]),+ new Promise<void>((resolve) => setImmediate(resolve)).then(() => { throw new Error("Snapshot barrier and Redis key mutex acquired in conflicting order"); }),+ ]);+ expect(await kv.get(KV.graphSnapshot, "current")).toEqual({ version: 1, dirty: false, writer: "concurrent" });+ expect(await createGraphIndexReader(kv)).not.toBeNull();+ });++ it("counts only latest live edges while retaining temporal projections and distinguishes Unicode IDs", async () => {+ const { kv } = fixture(); await initializeGraphIndex(kv, { status: "ready" });+ const edge = { id: "edge", type: "related_to", sourceNodeId: "a", targetNodeId: "b", weight: 1, sourceObservationIds: [], createdAt: "now" };+ await kv.set(KV.graphEdges, edge.id, edge);+ await kv.update(KV.graphEdges, edge.id, [{ type: "set", path: "isLatest", value: false }]);+ expect(await getGraphIndexStatus(kv)).toMatchObject({ edges: 0, edgesByType: {} });+ expect(await (await createGraphIndexReader(kv))!.getEdge(edge.id)).toMatchObject({ isLatest: false });+ for (const id of ["\ud800", "\ud801"]) await kv.set(KV.graphNodes, id, node(id, "same"));+ const reader = (await createGraphIndexReader(kv))!;+ expect((await reader.posting("name", "same")).ids.sort()).toEqual(["\ud800", "\ud801"]);+ });+});diff --git a/test/graph-retrieval.test.ts b/test/graph-retrieval.test.tsindex 9c469905..2d157022 100644--- a/test/graph-retrieval.test.ts+++ b/test/graph-retrieval.test.ts@@ -1,8 +1,11 @@-import { describe, it, expect, beforeEach } from "vitest";+import { describe, it, expect, vi } from "vitest";import { GraphRetrieval } from "../src/functions/graph-retrieval.js";+import { getGraphIndexStatus, initializeGraphIndex, reconcileGraphIndexRecord } from "../src/state/graph-retrieval-index.js";+import { indexObservationSession } from "../src/state/obs-index.js";+import { KV } from "../src/state/schema.js";import type { GraphNode, GraphEdge } from "../src/types.js";-function mockKV(+async function mockKV(nodes: GraphNode[] = [],edges: GraphEdge[] = [],) {@@ -15,7 +18,7 @@ function mockKV(for (const e of edges) edgesMap.set(e.id, e);store.set("mem:graph:edges", edgesMap);- return {+ const kv = {get: async <T>(scope: string, key: string): Promise<T | null> => {return (store.get(scope)?.get(key) as T) ?? null;},@@ -27,11 +30,16 @@ function mockKV(delete: async (scope: string, key: string): Promise<void> => {store.get(scope)?.delete(key);},- list: async <T>(scope: string): Promise<T[]> => {- const entries = store.get(scope);- return entries ? (Array.from(entries.values()) as T[]) : [];- },+ list: vi.fn(async <T>(_scope: string): Promise<T[]> => { throw new Error("Whole-store listing is forbidden during retrieval"); }),};+ await initializeGraphIndex(kv as never, { status: "ready" });+ for (const node of nodes) await reconcileGraphIndexRecord(kv as never, "node", node.id, node);+ for (const edge of edges) await reconcileGraphIndexRecord(kv as never, "edge", edge.id, edge);+ for (const row of [...nodes, ...edges]) for (const id of row.sourceObservationIds) {+ await indexObservationSession(kv as never, id, "session");+ await kv.set(KV.observations("session"), id, { id, sessionId: "session", title: `Evidence ${id}`, facts: [], narrative: "verified source", concepts: [], files: [] });+ }+ return kv;}function makeNode(@@ -76,7 +84,7 @@ describe("GraphRetrieval", () => {makeNode("n1", "React", "library", ["obs_1"]),makeNode("n2", "Vue", "library", ["obs_2"]),];- const kv = mockKV(nodes, []);+ const kv = await mockKV(nodes, []);const retrieval = new GraphRetrieval(kv as never);const results = await retrieval.searchByEntities(["React"]);@@ -86,7 +94,7 @@ describe("GraphRetrieval", () => {it("finds entities by partial name match", async () => {const nodes = [makeNode("n1", "auth-middleware", "function", ["obs_1"])];- const kv = mockKV(nodes, []);+ const kv = await mockKV(nodes, []);const retrieval = new GraphRetrieval(kv as never);const results = await retrieval.searchByEntities(["auth"]);@@ -99,7 +107,7 @@ describe("GraphRetrieval", () => {makeNode("n2", "Component", "concept", ["obs_2"]),];const edges = [makeEdge("e1", "n1", "n2", "uses")];- const kv = mockKV(nodes, edges);+ const kv = await mockKV(nodes, edges);const retrieval = new GraphRetrieval(kv as never);const results = await retrieval.searchByEntities(["React"], 2);@@ -109,7 +117,7 @@ describe("GraphRetrieval", () => {});it("returns empty for no matches", async () => {- const kv = mockKV([], []);+ const kv = await mockKV([], []);const retrieval = new GraphRetrieval(kv as never);const results = await retrieval.searchByEntities(["nonexistent"]);expect(results).toEqual([]);@@ -121,7 +129,7 @@ describe("GraphRetrieval", () => {makeNode("n2", "jwt", "concept", ["obs_2"]),];const edges = [makeEdge("e1", "n1", "n2", "uses")];- const kv = mockKV(nodes, edges);+ const kv = await mockKV(nodes, edges);const retrieval = new GraphRetrieval(kv as never);const results = await retrieval.expandFromChunks(["obs_1"]);@@ -131,7 +139,7 @@ describe("GraphRetrieval", () => {it("does not duplicate already-seen observations in expansion", async () => {const nodes = [makeNode("n1", "file.ts", "file", ["obs_1", "obs_2"])];- const kv = mockKV(nodes, []);+ const kv = await mockKV(nodes, []);const retrieval = new GraphRetrieval(kv as never);const results = await retrieval.expandFromChunks(["obs_1"]);@@ -149,7 +157,7 @@ describe("GraphRetrieval", () => {isLatest: true,},];- const kv = mockKV(nodes, edges);+ const kv = await mockKV(nodes, edges);const retrieval = new GraphRetrieval(kv as never);const result = await retrieval.temporalQuery("Alice");@@ -159,7 +167,7 @@ describe("GraphRetrieval", () => {});it("returns null entity for unknown name", async () => {- const kv = mockKV([], []);+ const kv = await mockKV([], []);const retrieval = new GraphRetrieval(kv as never);const result = await retrieval.temporalQuery("Unknown");expect(result.entity).toBeNull();@@ -175,7 +183,7 @@ describe("GraphRetrieval", () => {makeEdge("e1", "n1", "n2", "uses", 0.9),makeEdge("e2", "n2", "n3", "related_to", 0.8),];- const kv = mockKV(nodes, edges);+ const kv = await mockKV(nodes, edges);const retrieval = new GraphRetrieval(kv as never);const results = await retrieval.searchByEntities(["React"], 3);@@ -199,7 +207,7 @@ describe("GraphRetrieval", () => {makeEdge("e_strong_a", "n1", "n2", "related_to", 0.9),makeEdge("e_strong_b", "n2", "n3", "related_to", 0.9),];- const kv = mockKV(nodes, edges);+ const kv = await mockKV(nodes, edges);const retrieval = new GraphRetrieval(kv as never);const results = await retrieval.searchByEntities(["Start"], 3);@@ -208,7 +216,8 @@ describe("GraphRetrieval", () => {// Path is [Start → Mid → End] (length 3) — Dijkstra picked the// chain of two strong edges over the direct weak one.expect(endResult!.pathLength).toBe(3);- expect(endResult!.graphContext).toContain("Mid");+ expect(endResult!.graphContext).toContain("Evidence obs_end");+ expect(endResult!.graphContext).not.toContain("Mid");});it("handles disconnected nodes without crashing", async () => {@@ -219,7 +228,7 @@ describe("GraphRetrieval", () => {makeNode("n3", "Lonely", "concept", ["obs_lonely"]),];const edges = [makeEdge("e1", "n1", "n2", "related_to", 0.7)];- const kv = mockKV(nodes, edges);+ const kv = await mockKV(nodes, edges);const retrieval = new GraphRetrieval(kv as never);const results = await retrieval.searchByEntities(["A"], 5);@@ -237,7 +246,7 @@ describe("GraphRetrieval", () => {// floor at 0.01 means traversal completes with a very high cost// rather than throwing or producing Infinity.const edges = [makeEdge("e1", "n1", "n2", "related_to", 0)];- const kv = mockKV(nodes, edges);+ const kv = await mockKV(nodes, edges);const retrieval = new GraphRetrieval(kv as never);const results = await retrieval.searchByEntities(["Anchor"], 2);@@ -252,7 +261,7 @@ describe("GraphRetrieval", () => {makeNode("n2", "Hook", "concept", ["obs_neighbor"]),];const edges = [makeEdge("e1", "n1", "n2", "uses", 0.8)];- const kv = mockKV(nodes, edges);+ const kv = await mockKV(nodes, edges);const retrieval = new GraphRetrieval(kv as never);const results = await retrieval.searchByEntities(["React"], 2);@@ -276,11 +285,175 @@ describe("GraphRetrieval", () => {makeEdge("e2", "n2", "n3", "related_to", 0.8),makeEdge("e3", "n3", "n4", "related_to", 0.8),];- const kv = mockKV(nodes, edges);+ const kv = await mockKV(nodes, edges);const retrieval = new GraphRetrieval(kv as never);const results = await retrieval.searchByEntities(["Start"], 2);expect(results.find((r) => r.obsId === "obs_3")).toBeDefined();expect(results.find((r) => r.obsId === "obs_4")).toBeUndefined();});++ it("preserves exact-match observations when a neighbor has oversized legacy provenance", async () => {+ const kv = await mockKV([+ makeNode("root", "Cobalt", "concept", ["exact"]),+ makeNode("neighbor", "Neighbor", "concept", Array.from({ length: 10_001 }, (_, i) => `related_${i}`)),+ ], [makeEdge("edge", "root", "neighbor")]);+ const results = await new GraphRetrieval(kv as never).searchByEntities(["Cobalt"]);+ expect(results[0]).toMatchObject({ obsId: "exact", score: 1, pathLength: 0 });+ expect(results).toHaveLength(20);+ });++ it("bounds graph context from oversized imported properties and edge reasoning", async () => {+ const neighbor = makeNode("neighbor", "Neighbor", "concept", ["related"]);+ neighbor.properties = Object.fromEntries(Array.from({ length: 10_000 }, (_, i) => [`property_${i}`, "x".repeat(1000)]));+ const edge = { ...makeEdge("edge", "root", "neighbor"), context: { reasoning: "y".repeat(20_000) } };+ const kv = await mockKV([makeNode("root", "Cobalt", "concept", ["exact"]), neighbor], [edge]);+ const results = await new GraphRetrieval(kv as never).searchByEntities(["Cobalt"]);+ const related = results.find((result) => result.obsId === "related")!;+ expect(related.graphContext).toBe("[graph evidence] Evidence related");+ expect(related.graphContext).not.toContain("property_0=");+ expect(related.graphContext.length).toBeLessThanOrEqual(4096);+ });++ it("keeps graph hits when imported optional reasoning is not text", async () => {+ const edge = { ...makeEdge("edge", "root", "neighbor"), context: { reasoning: 42 } };+ const kv = await mockKV([+ makeNode("root", "Cobalt", "concept", ["exact"]),+ makeNode("neighbor", "Neighbor", "concept", ["related"]),+ ], [edge as unknown as GraphEdge]);+ const results = await new GraphRetrieval(kv as never).searchByEntities(["Cobalt"]);+ expect(results.map((result) => result.obsId)).toEqual(["exact", "related"]);+ });+});+++describe("bounded graph index reads", () => {+ it("reports incomplete projections when omitted searchable text produces no matches", async () => {+ const source = { ...makeNode("large", "Anchor"), properties: { description: "x".repeat(512) + "遗漏" } };+ const kv = await mockKV([source]);+ const query = new GraphRetrieval(kv as never).createQuery();+ expect(await query.searchByEntities(["遗漏"])).toEqual([]);+ expect(query.diagnostics()).toMatchObject({ status: "ready", truncated: true });+ expect(kv.list).not.toHaveBeenCalled();+ });++ it("requires explicit migration for legacy rows without enumerating them", async () => {+ const list = vi.fn(async () => { throw new Error("unbounded"); });+ const graph = new GraphRetrieval({ get: async () => null, list } as never).createQuery();+ expect(await graph.searchByEntities(["React"])).toEqual([]);+ expect(graph.diagnostics().status).not.toBe("ready");+ expect(list).not.toHaveBeenCalled();+ });++ it("matches Unicode, short names and duplicate names through bounded postings", async () => {+ const kv = await mockKV([+ makeNode("a", "数据库", "concept", ["one"]), makeNode("b", "数据库", "concept", ["two"]),+ makeNode("c", "R", "concept", ["three"]), makeNode("d", "FOO", "concept", ["four"]),+ ]);+ const retrieval = new GraphRetrieval(kv as never);+ expect((await retrieval.searchByEntities(["数据"])).map((hit) => hit.obsId)).toEqual(["one", "two"]);+ expect((await retrieval.searchByEntities(["R"])).map((hit) => hit.obsId)).toContain("three");+ expect((await retrieval.searchByEntities(["foo"])).map((hit) => hit.obsId)).toContain("four");+ expect(kv.list).not.toHaveBeenCalled();+ });++ it("excludes forgotten sources and avoids leaking properties from mixed provenance", async () => {+ const root = makeNode("root", "Cobalt", "concept", ["private", "public"]);+ root.properties = { secret: "do not disclose" };+ const kv = await mockKV([root, makeNode("neighbor", "SecretName", "concept", ["forgotten"])], [makeEdge("edge", "root", "neighbor")]);+ await kv.delete(KV.observations("session"), "forgotten");+ const query = new GraphRetrieval(kv as never).createQuery((record) => record.id !== "private");+ expect(await query.searchByEntities(["Cobalt"])).toEqual([+ expect.objectContaining({ obsId: "public", graphContext: "[graph evidence] Evidence public" }),+ ]);+ });++ it("excludes superseded edges and missing or stale neighbors", async () => {+ const kv = await mockKV([+ makeNode("root", "Cobalt", "concept", ["root"]),+ makeNode("old", "Old", "concept", ["old"]),+ { ...makeNode("stale", "Stale", "concept", ["stale"]), stale: true },+ ], [+ { ...makeEdge("old", "root", "old"), isLatest: false },+ makeEdge("stale", "root", "stale"), makeEdge("missing", "root", "missing"), makeEdge("self", "root", "root"),+ ]);+ expect((await new GraphRetrieval(kv as never).searchByEntities(["Cobalt"])).map((hit) => hit.obsId)).toEqual(["root"]);+ });++ it("retains a more expensive shallow route needed to reach a node within the depth limit", async () => {+ const kv = await mockKV([+ makeNode("a", "Start", "concept", ["a"]), makeNode("b", "Middle", "concept", ["b"]),+ makeNode("c", "Junction", "concept", ["c"]), makeNode("d", "End", "concept", ["d"]),+ ], [makeEdge("ac", "a", "c", "related_to", 0.1), makeEdge("ab", "a", "b", "related_to", 1),+ makeEdge("bc", "b", "c", "related_to", 1), makeEdge("cd", "c", "d", "related_to", 1)]);+ const results = await new GraphRetrieval(kv as never).searchByEntities(["Start"], 2);+ expect(results.find((hit) => hit.obsId === "d")?.pathLength).toBe(3);+ });++ it("shares bounded work across repeated operations and reports partial high degree", async () => {+ const nodes = [makeNode("root", "Cobalt", "concept", ["root"]), ...Array.from({ length: 180 }, (_, i) => makeNode(`n${i}`, `Other ${i}`, "concept", [`o${i}`]))];+ const kv = await mockKV(nodes, nodes.slice(1).map((node, i) => makeEdge(`e${i}`, "root", node.id)));+ const get = vi.spyOn(kv, "get");+ const query = new GraphRetrieval(kv as never).createQuery();+ const [a, b] = await Promise.all([query.searchByEntities(["Cobalt"]), query.searchByEntities(["Cobalt"])]);+ expect(a).toEqual(b);+ expect(a.length).toBeGreaterThan(0);+ expect(query.diagnostics().truncated).toBe(true);+ expect(get.mock.calls.length).toBeLessThanOrEqual(520);+ expect(kv.list).not.toHaveBeenCalled();+ });+});++it("discards results after the graph revision changes during an awaited source read", async () => {+ const root = makeNode("root", "Cobalt", "concept", ["root"]);+ const kv = await mockKV([root]);+ const get = kv.get;+ let changed = false;+ vi.spyOn(kv, "get").mockImplementation(async (scope, key) => {+ const value = await get(scope, key);+ if (!changed && scope === KV.observations("session")) {+ changed = true;+ await reconcileGraphIndexRecord(kv as never, "node", root.id, { ...root, name: "Renamed" });+ }+ return value;+ });+ const query = new GraphRetrieval(kv as never).createQuery();+ expect(await query.searchByEntities(["Cobalt"])).toEqual([]);+ expect(query.diagnostics().status).toBe("changed");+});++it("rechecks canonical evidence before exposing a result after forget", async () => {+ const kv = await mockKV([makeNode("root", "Cobalt", "concept", ["root"])]);+ const get = kv.get;+ vi.spyOn(kv, "get").mockImplementation(async (scope, key) => {+ const value = await get(scope, key);+ if (scope === KV.observations("session")) await kv.delete(scope, key);+ return value;+ });+ expect(await new GraphRetrieval(kv as never).searchByEntities(["Cobalt"])).toEqual([]);+});+++it("matches a contained node name when the query begins with a different Unicode character", async () => {+ const kv = await mockKV([makeNode("node", "React", "library", ["source"])]);+ expect((await new GraphRetrieval(kv as never).searchByEntities([" Using React "])).map((hit) => hit.obsId)).toEqual(["source"]);+});+++it("reports dirty index status without repairing or listing during retrieval", async () => {+ const kv = await mockKV([makeNode("root", "Cobalt", "concept", ["root"])]);+ const control = await getGraphIndexStatus(kv as never);+ await kv.set(KV.graphRetrievalControl, "current", { ...control, pending: { kind: "node", id: "root" } });+ const query = new GraphRetrieval(kv as never).createQuery();+ expect(await query.searchByEntities(["Cobalt"])).toEqual([]);+ expect(query.diagnostics().status).toBe("dirty");+ expect(kv.list).not.toHaveBeenCalled();+});++it("does not replay cached operations after the caller cancels its query", async () => {+ const kv = await mockKV([makeNode("root", "Cobalt", "concept", ["root"])]);+ const query = new GraphRetrieval(kv as never).createQuery();+ expect(await query.searchByEntities(["Cobalt"])).toHaveLength(1);+ query.cancel();+ await expect(query.searchByEntities(["Cobalt"])).rejects.toThrow();});diff --git a/test/graph.test.ts b/test/graph.test.tsindex 7b30e5de..1556a590 100644--- a/test/graph.test.ts+++ b/test/graph.test.ts@@ -1,3 +1,5 @@+import { createGraphIndexState } from "./helpers/graph-index-state.js";+import { initializeGraphIndex } from "../src/state/graph-retrieval-index.js";import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";vi.mock("../src/logger.js", () => ({@@ -13,26 +15,7 @@ import type {GraphQueryResult,} from "../src/types.js";-function mockKV() {- const store = new Map<string, Map<string, unknown>>();- return {- get: async <T>(scope: string, key: string): Promise<T | null> => {- return (store.get(scope)?.get(key) as T) ?? null;- },- set: async <T>(scope: string, key: string, data: T): Promise<T> => {- if (!store.has(scope)) store.set(scope, new Map());- store.get(scope)!.set(key, data);- return data;- },- delete: async (scope: string, key: string): Promise<void> => {- store.get(scope)?.delete(key);- },- list: async <T>(scope: string): Promise<T[]> => {- const entries = store.get(scope);- return entries ? (Array.from(entries.values()) as T[]) : [];- },- };-}+function mockKV() { return createGraphIndexState().kv; }function mockSdk() {const functions = new Map<string, Function>();@@ -85,9 +68,10 @@ describe("Graph Functions", () => {let kv: ReturnType<typeof mockKV>;const ORIG_GRAPH_FLAG = process.env["GRAPH_EXTRACTION_ENABLED"];- beforeEach(() => {+ beforeEach(async () => {sdk = mockSdk();kv = mockKV();+ await initializeGraphIndex(kv, { status: "ready" });vi.clearAllMocks();process.env["GRAPH_EXTRACTION_ENABLED"] = "true";registerGraphFunction(sdk as never, kv as never, mockProvider as never);@@ -235,7 +219,8 @@ describe("Graph Functions", () => {type: "concept",name: `node-${i}`,properties: {},- firstSeen: "2026-01-01T00:00:00Z",+ sourceObservationIds: [],+ createdAt: "2026-01-01T00:00:00Z",lastSeen: "2026-01-01T00:00:00Z",observationCount: 1,} as GraphNode;@@ -251,7 +236,8 @@ describe("Graph Functions", () => {targetNodeId: `n_${((i + 1) % 50).toString().padStart(4, "0")}`,weight: 1,evidence: [],- firstSeen: "2026-01-01T00:00:00Z",+ sourceObservationIds: [],+ createdAt: "2026-01-01T00:00:00Z",lastSeen: "2026-01-01T00:00:00Z",} as GraphEdge;await kv.set("mem:graph:edges", edge.id, edge);@@ -269,10 +255,7 @@ describe("Graph Functions", () => {expect(unbounded.truncated).toBe(true);expect(unbounded.limit).toBe(500);expect(unbounded.offset).toBe(0);- // The 50 connected nodes should be on the first page since the- // default ranks by degree.- const connectedOnPage = unbounded.nodes.filter((n) => /^n_00[0-4]\d$/.test(n.id));- expect(connectedOnPage.length).toBe(50);+ expect(new Set(unbounded.nodes.map(node => node.id)).size).toBe(500);});it("honors limit and offset for paged graph-query traversal", async () => {@@ -282,7 +265,8 @@ describe("Graph Functions", () => {type: "concept",name: `node-${i}`,properties: {},- firstSeen: "2026-01-01T00:00:00Z",+ sourceObservationIds: [],+ createdAt: "2026-01-01T00:00:00Z",lastSeen: "2026-01-01T00:00:00Z",observationCount: 1,} as GraphNode;@@ -319,7 +303,8 @@ describe("Graph Functions", () => {type: "concept",name: `n-${i}`,properties: {},- firstSeen: "2026-01-01T00:00:00Z",+ sourceObservationIds: [],+ createdAt: "2026-01-01T00:00:00Z",lastSeen: "2026-01-01T00:00:00Z",observationCount: 1,});@@ -330,7 +315,7 @@ describe("Graph Functions", () => {const huge = (await sdk.trigger("mem::graph-query", {limit: 999999,})) as GraphQueryResult;- expect(huge.limit).toBeLessThanOrEqual(5000);+ expect(huge.limit).toBe(500);expect(huge.nodes.length).toBe(10);expect(huge.truncated).toBe(false);});@@ -340,9 +325,10 @@ describe("Graph Functions", () => {await kv.set("mem:graph:nodes", `x_${i.toString().padStart(3, "0")}`, {id: `x_${i.toString().padStart(3, "0")}`,type: "concept",- name: `n-${i}`,+ name: i < 10 ? `cluster-${i}` : `isolated-${i}`,properties: {},- firstSeen: "2026-01-01T00:00:00Z",+ sourceObservationIds: [],+ createdAt: "2026-01-01T00:00:00Z",lastSeen: "2026-01-01T00:00:00Z",observationCount: 1,});@@ -358,7 +344,8 @@ describe("Graph Functions", () => {targetNodeId: `x_${next.toString().padStart(3, "0")}`,weight: 1,evidence: [],- firstSeen: "2026-01-01T00:00:00Z",+ sourceObservationIds: [],+ createdAt: "2026-01-01T00:00:00Z",lastSeen: "2026-01-01T00:00:00Z",});}@@ -372,13 +359,14 @@ describe("Graph Functions", () => {targetNodeId: "x_055",weight: 1,evidence: [],- firstSeen: "2026-01-01T00:00:00Z",+ createdAt: "2026-01-01T00:00:00Z",lastSeen: "2026-01-01T00:00:00Z",});await sdk.trigger("mem::graph-snapshot-rebuild", { force: true });const page = (await sdk.trigger("mem::graph-query", {+ query: "cluster",limit: 10,offset: 0,})) as GraphQueryResult;@@ -389,7 +377,7 @@ describe("Graph Functions", () => {// Cluster edges among page nodes ARE present.expect(page.edges.filter((e) => e.id.startsWith("cluster_")).length).toBe(10);// totalEdges counts every edge in the full result universe.- expect(page.totalEdges).toBe(11);+ expect(page.totalEdges).toBe(10);});describe("snapshot cache", () => {@@ -401,7 +389,7 @@ describe("Graph Functions", () => {name: `node-${i}`,properties: {},sourceObservationIds: [`obs_${i}`],- firstSeen: "2026-01-01T00:00:00Z",+ createdAt: "2026-01-01T00:00:00Z",lastSeen: "2026-01-01T00:00:00Z",observationCount: 1,stale: false,@@ -418,7 +406,7 @@ describe("Graph Functions", () => {weight: 1,evidence: [],sourceObservationIds: [`obs_${i}`],- firstSeen: "2026-01-01T00:00:00Z",+ createdAt: "2026-01-01T00:00:00Z",lastSeen: "2026-01-01T00:00:00Z",stale: false,});@@ -454,24 +442,24 @@ describe("Graph Functions", () => {expect(snap!.stats.nodesByType["function"]).toBeGreaterThan(0);});- it("graph-query empty-body branch serves from snapshot once it exists", async () => {+ it("graph-query reads indexed inventory independently of the viewer snapshot", async () => {await seed(20, 30);await sdk.trigger("mem::graph-snapshot-rebuild", { force: true });const result = (await sdk.trigger("mem::graph-query", {})) as GraphQueryResult;- expect(result.fromSnapshot).toBe(true);+ expect(result.fromSnapshot).toBe(false);expect(result.totalNodes).toBe(20);expect(result.totalEdges).toBe(30);});- it("graph-query nodeType filter respects snapshot type counts", async () => {+ it("graph-query nodeType filter counts the complete small indexed result", async () => {await seed(30, 0);await sdk.trigger("mem::graph-snapshot-rebuild", { force: true });const fileQuery = (await sdk.trigger("mem::graph-query", {nodeType: "file",})) as GraphQueryResult;- expect(fileQuery.fromSnapshot).toBe(true);+ expect(fileQuery.fromSnapshot).toBe(false);// 30 nodes, every 3rd is "file" → 10 files.expect(fileQuery.totalNodes).toBe(10);for (const n of fileQuery.nodes) {@@ -479,7 +467,7 @@ describe("Graph Functions", () => {}});- it("graph-stats returns from snapshot when not dirty", async () => {+ it("graph-stats returns indexed counts independently of the snapshot", async () => {await seed(15, 25);await sdk.trigger("mem::graph-snapshot-rebuild", { force: true });@@ -487,8 +475,10 @@ describe("Graph Functions", () => {totalNodes: number;totalEdges: number;fromSnapshot: boolean;+ fromIndex: boolean;};- expect(stats.fromSnapshot).toBe(true);+ expect(stats.fromSnapshot).toBe(false);+ expect(stats.fromIndex).toBe(true);expect(stats.totalNodes).toBe(15);expect(stats.totalEdges).toBe(25);});@@ -526,7 +516,7 @@ describe("Graph Functions", () => {expect(fileNodes.length).toBe(1);});- it("graph-stats returns empty envelope + warning when no snapshot exists", async () => {+ it("graph-stats returns current indexed counts when no snapshot exists", async () => {await seed(5, 5);const stats = (await sdk.trigger("mem::graph-stats", {})) as {@@ -536,8 +526,32 @@ describe("Graph Functions", () => {warning?: string;};expect(stats.fromSnapshot).toBe(false);- expect(stats.totalNodes).toBe(0);- expect(stats.warning).toMatch(/snapshot-rebuild|graph\/reset/);+ expect(stats.totalNodes).toBe(5);+ expect(stats.totalEdges).toBe(5);+ expect(stats.warning).toBeUndefined();+ });++ it("graph-stats reports unknown coverage for an unindexed legacy store", async () => {+ const legacy = mockKV();+ const legacySdk = mockSdk();+ await legacy.set("mem:graph:nodes", "legacy", { id: "legacy", name: "legacy", type: "concept" });+ registerGraphFunction(legacySdk as never, legacy as never, mockProvider as never);+ expect(await legacySdk.trigger("mem::graph-stats", {})).toMatchObject({+ available: false, coverage: "unknown", totalNodes: null, totalEdges: null,+ });+ });++ it("graph-stats tracks direct type updates, stale records and historical edges", async () => {+ await seed(2, 2);+ await sdk.trigger("mem::graph-snapshot-rebuild", { force: true });+ await kv.update("mem:graph:nodes", "n_0", [{ type: "set", path: "type", value: "concept" }]);+ await kv.update("mem:graph:nodes", "n_1", [{ type: "set", path: "stale", value: true }]);+ await kv.update("mem:graph:edges", "e_0", [{ type: "set", path: "isLatest", value: false }]);+ await kv.delete("mem:graph:edges", "e_1");+ expect(await sdk.trigger("mem::graph-stats", {})).toMatchObject({+ available: true, totalNodes: 1, totalEdges: 0,+ nodesByType: { concept: 1 }, edgesByType: {},+ });});it("graph-reset clears state and writes empty snapshot", async () => {@@ -667,7 +681,7 @@ describe("Graph Functions", () => {});});- describe("snapshot-reported total floor", () => {+ describe("obsolete viewer snapshots cannot repopulate the retrieval index", () => {function seedSnapshot(stats: { totalNodes: number; nodesByType: Record<string, number> },topNodeCount: number,@@ -678,7 +692,7 @@ describe("Graph Functions", () => {name: `node-${i}`,properties: {},sourceObservationIds: [`obs_${i}`],- firstSeen: "2026-01-01T00:00:00Z",+ createdAt: "2026-01-01T00:00:00Z",lastSeen: "2026-01-01T00:00:00Z",observationCount: 1,stale: false,@@ -699,35 +713,35 @@ describe("Graph Functions", () => {});}- it("does not report fewer nodes than it returns when the counter is low", async () => {+ it("ignores a legacy snapshot whose low count disagrees with its cached rows", async () => {await seedSnapshot({ totalNodes: 101, nodesByType: { file: 101 } }, 343);const result = (await sdk.trigger("mem::graph-query", {})) as GraphQueryResult;- expect(result.nodes.length).toBe(343);- expect(result.totalNodes).toBe(343);+ expect(result.nodes).toEqual([]);+ expect(result.totalNodes).toBe(0);expect(result.truncated).toBe(false);});- it("leaves a healthy counter above the top-N cap alone and raises the banner", async () => {+ it("ignores even a large legacy count when the active index is empty", async () => {await seedSnapshot({ totalNodes: 40000, nodesByType: { file: 40000 } }, 343);const result = (await sdk.trigger("mem::graph-query", {})) as GraphQueryResult;- expect(result.nodes.length).toBe(343);- expect(result.totalNodes).toBe(40000);- expect(result.truncated).toBe(true);+ expect(result.nodes).toEqual([]);+ expect(result.totalNodes).toBe(0);+ expect(result.truncated).toBe(false);});- it("applies the same floor to the type-filtered total", async () => {+ it("does not return stale cached rows through a type filter", async () => {await seedSnapshot({ totalNodes: 101, nodesByType: { file: 101 } }, 343);const result = (await sdk.trigger("mem::graph-query", {nodeType: "file",})) as GraphQueryResult;- expect(result.nodes.length).toBe(343);- expect(result.totalNodes).toBe(343);+ expect(result.nodes).toEqual([]);+ expect(result.totalNodes).toBe(0);expect(result.truncated).toBe(false);});});@@ -744,7 +758,7 @@ describe("Graph Functions", () => {};}- it("graph-query startNodeId returns warning envelope when enumeration exceeds budget", async () => {+ it("graph-query requires explicit migration instead of beginning a slow enumeration", async () => {const slow = slowKV(7000); // > LIVE_ENUMERATION_BUDGET_MS (6000ms)const localSdk = mockSdk();registerGraphFunction(localSdk as never, slow as never, mockProvider as never);@@ -754,7 +768,7 @@ describe("Graph Functions", () => {})) as GraphQueryResult;expect(result.warning).toBeTruthy();- expect(result.warning).toMatch(/budget|enumeration/i);+ expect(result.warning).toMatch(/explicit migration/i);}, 10000);// CodeRabbit raised that slowKV(setTimeout) doesn't simulate adiff --git a/test/helpers/graph-index-state.ts b/test/helpers/graph-index-state.tsnew file mode 100644index 00000000..e5037a72--- /dev/null+++ b/test/helpers/graph-index-state.ts@@ -0,0 +1,34 @@+import { StateKV, applyUpdateOps } from "../../src/state/kv.js";++export function createGraphIndexState(store = new Map<string, Map<string, unknown>>()) {+ const calls: Array<{ id: string; scope: string; key?: string }> = [];+ let intercept: ((id: string, payload: any) => Promise<void>) | undefined;+ const sdk = {+ trigger: async ({ function_id: id, payload }: { function_id: string; payload: any }): Promise<any> => {+ calls.push({ id, scope: payload.scope, key: payload.key });+ await intercept?.(id, payload);+ const rows = store.get(payload.scope) ?? new Map<string, unknown>();+ if (id === "state::get") return structuredClone(rows.get(payload.key) ?? null);+ if (id === "state::list") return structuredClone([...rows.values()]);+ if (id === "state::delete") { rows.delete(payload.key); return; }+ if (id === "state::set") {+ rows.set(payload.key, structuredClone(payload.value));+ store.set(payload.scope, rows);+ return structuredClone(payload.value);+ }+ if (id === "state::update") {+ const old_value = structuredClone(rows.get(payload.key) ?? null);+ const new_value = applyUpdateOps(old_value, payload.ops);+ rows.set(payload.key, structuredClone(new_value));+ store.set(payload.scope, rows);+ return { old_value, new_value, errors: [] };+ }+ throw new Error(`Unexpected fixture function ${id}`);+ },+ };+ const kv = new StateKV(sdk as never);+ for (const name of ["get", "set", "update", "delete", "list"] as const) {+ Object.defineProperty(kv, name, { value: kv[name].bind(kv), writable: true, enumerable: true, configurable: true });+ }+ return { kv, store, calls, intercept: (fn?: typeof intercept) => { intercept = fn; } };+}diff --git a/test/hybrid-search-graph-budget.test.ts b/test/hybrid-search-graph-budget.test.tsnew file mode 100644index 00000000..23a09a3d--- /dev/null+++ b/test/hybrid-search-graph-budget.test.ts@@ -0,0 +1,98 @@+import { afterEach, describe, expect, it, vi } from "vitest";+import { HybridSearch } from "../src/state/hybrid-search.js";+import { SearchIndex } from "../src/state/search-index.js";+import { KV } from "../src/state/schema.js";+import { initializeGraphIndex, reconcileGraphIndexRecord } from "../src/state/graph-retrieval-index.js";+import type { CompressedObservation } from "../src/types.js";++const observation: CompressedObservation = {+ id: "obs_ready", sessionId: "ses_ready", timestamp: "2026-01-01T00:00:00.000Z",+ type: "discovery", title: "Cobalt authentication", subtitle: "", facts: [],+ narrative: "Cobalt validates tokens", concepts: ["Cobalt"], files: [], importance: 5,+};++async function fixture(hold = true) {+ const index = new SearchIndex();+ index.add(observation);+ const records = new Map<string, unknown>();+ let release!: (row: unknown) => void;+ const held = new Promise<unknown>((resolve) => { release = resolve; });+ const reads: string[] = [];+ let holding = false;+ const kv = {+ get: async <T>(scope: string, key: string): Promise<T | null> => {+ reads.push(scope);+ if (holding && scope === KV.graphRetrievalControl) { await held; holding = false; }+ return (records.get(`${scope}/${key}`) as T) ?? null;+ },+ set: async <T>(scope: string, key: string, value: T) => { records.set(`${scope}/${key}`, value); return value; },+ delete: async (scope: string, key: string) => { records.delete(`${scope}/${key}`); },+ list: vi.fn(async () => { throw new Error("Unexpected full graph list"); }),+ };+ await kv.set(KV.observations(observation.sessionId), observation.id, observation);+ await initializeGraphIndex(kv as never, { status: "ready" });+ await reconcileGraphIndexRecord(kv as never, "node", "cobalt", {+ id: "cobalt", name: "Cobalt", type: "concept", properties: {},+ sourceObservationIds: [observation.id], createdAt: observation.timestamp,+ });+ reads.length = 0;+ holding = hold;+ return { hybrid: new HybridSearch(index, null, null, kv as never), release, reads, kv, index };+}++afterEach(() => vi.useRealTimers());++describe("optional graph retrieval budget", () => {+ it("returns ready keyword results while graph storage is still unresponsive", async () => {+ vi.useFakeTimers();+ const { hybrid, release } = await fixture();+ let results: Awaited<ReturnType<HybridSearch["search"]>> | undefined;+ const search = hybrid.search("Cobalt").then((value) => { results = value; });+ try {+ await vi.advanceTimersByTimeAsync(500);+ expect(results?.map((result) => result.observation.id)).toEqual([observation.id]);+ expect(results?.[0].graphScore).toBe(0);+ } finally {+ release([]);+ await search;+ }+ });++ it("does not start the edge read when a timed-out node read arrives late", async () => {+ vi.useFakeTimers();+ const { hybrid, release, reads, kv } = await fixture();+ const search = hybrid.search("Cobalt");+ await vi.advanceTimersByTimeAsync(500);+ expect((await search).map((result) => result.observation.id)).toEqual([observation.id]);+ release([]);+ await vi.advanceTimersByTimeAsync(0);+ expect(kv.list).not.toHaveBeenCalled();+ expect(reads).not.toContain(KV.graphRetrievalData);+ expect(vi.getTimerCount()).toBe(0);+ });++ it("keeps fast graph enrichment and clears its deadline", async () => {+ vi.useFakeTimers();+ const { kv, index } = await fixture(false);+ const hybrid = new HybridSearch(index, null, null, kv as never);+ const results = await hybrid.search("Cobalt");+ expect(results[0].graphScore).toBeGreaterThan(0);+ expect(results[0].graphContext).toContain(observation.title);+ expect(vi.getTimerCount()).toBe(0);+ });++ it("bounds concurrent query reformulations and can recover on the next search", async () => {+ vi.useFakeTimers();+ const { hybrid, kv, release } = await fixture();+ const search = hybrid.searchWithExpansion("Cobalt", 5, {+ reformulations: ["Cobalt tokens", "Cobalt authentication"],+ temporalConcretizations: [], entityExtractions: ["Cobalt"],+ });+ await vi.advanceTimersByTimeAsync(500);+ expect((await search).map((result) => result.observation.id)).toEqual([observation.id]);+ release([]);+ await vi.advanceTimersByTimeAsync(0);+ expect((await hybrid.search("Cobalt"))[0].graphScore).toBeGreaterThan(0);+ expect(vi.getTimerCount()).toBe(0);+ });+});diff --git a/test/hybrid-search-graph-weight.test.ts b/test/hybrid-search-graph-weight.test.tsindex 64338284..2f4e47ee 100644--- a/test/hybrid-search-graph-weight.test.ts+++ b/test/hybrid-search-graph-weight.test.ts@@ -4,10 +4,7 @@ import { SearchIndex } from "../src/state/search-index.js";import { VectorIndex } from "../src/state/vector-index.js";import type { CompressedObservation } from "../src/types.js";-// Graph retrieval lists every node and edge on each search. A graph weight-// of 0 multiplied its results by zero but still paid for both traversals,-// so AGENTMEMORY_GRAPH_WEIGHT=0 saved nothing. At weight 0 the graph must-// not be read at all.+// A disabled graph stream must not even probe index readiness.const obs: CompressedObservation = {id: "obs_1",@@ -27,10 +24,13 @@ const obs: CompressedObservation = {function mockKV() {const store = new Map<string, Map<string, unknown>>();const listed: string[] = [];+ const reads: string[] = [];return {- listed,- get: async <T>(scope: string, key: string): Promise<T | null> =>- (store.get(scope)?.get(key) as T) ?? null,+ listed, reads,+ get: async <T>(scope: string, key: string): Promise<T | null> => {+ reads.push(scope);+ return (store.get(scope)?.get(key) as T) ?? null;+ },set: async <T>(scope: string, key: string, data: T): Promise<T> => {if (!store.has(scope)) store.set(scope, new Map());store.get(scope)!.set(key, data);@@ -62,19 +62,19 @@ async function searchWith(graphWeight: number | undefined) {);return {results,- graphLists: kv.listed.filter((s) => s.startsWith("mem:graph:")),+ graphReads: kv.reads.filter((s) => s.startsWith("mem:graph:")),};}describe("graph weight 0 skips graph retrieval", () => {it("control: the default weight reads the graph", async () => {- const { graphLists } = await searchWith(undefined);- expect(graphLists.length).toBeGreaterThan(0);+ const { graphReads } = await searchWith(undefined);+ expect(graphReads.length).toBeGreaterThan(0);});- it("weight 0 never lists a graph scope", async () => {- const { graphLists } = await searchWith(0);- expect(graphLists).toEqual([]);+ it("weight 0 never reads a graph scope", async () => {+ const { graphReads } = await searchWith(0);+ expect(graphReads).toEqual([]);});it("weight 0 still returns the BM25 result", async () => {@@ -112,7 +112,7 @@ describe("graph weight 0 also skips vector-chunk graph expansion", () => {graphWeight,);await hybrid.search("validate tokens", 10);- return kv.listed.filter((sc) => sc.startsWith("mem:graph:"));+ return kv.reads.filter((sc) => sc.startsWith("mem:graph:"));}it("control: a positive weight expands through the graph", async () => {diff --git a/test/recall-layer.test.ts b/test/recall-layer.test.tsindex 98cc50d5..ae87827a 100644--- a/test/recall-layer.test.ts+++ b/test/recall-layer.test.ts@@ -1,5 +1,6 @@import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";-import { GraphRetrieval } from "../src/functions/graph-retrieval.js";+import { initializeGraphIndex, reconcileGraphIndexRecord } from "../src/state/graph-retrieval-index.js";+import { GraphRetrieval, type GraphObservationAuthorization } from "../src/functions/graph-retrieval.js";import {getSearchIndex,rankMemoryIds,@@ -42,6 +43,7 @@ function fixture() {const kv = {get: async <T>(scope: string, id: string): Promise<T | null> => (records.get(`${scope}/${id}`) as T) ?? null,set: async <T>(scope: string, id: string, value: T): Promise<T> => { records.set(`${scope}/${id}`, value); return value; },+ delete: async (scope: string, id: string): Promise<void> => { records.delete(`${scope}/${id}`); },list: async <T>(scope: string): Promise<T[]> => [...records].filter(([key]) => key.startsWith(`${scope}/`)).map(([, value]) => value as T),};const handlers = new Map<string, (data: any) => Promise<any>>();@@ -58,11 +60,16 @@ function fixture() {const vector = new VectorIndex();const embedding = { name: "fixed", dimensions: 2, embed: async () => new Float32Array([1, 0]), embedBatch: async () => [] };const hybrid = new HybridSearch(index, vector, embedding, kv as never, 0.4, 0.6, 0);- const search = vi.fn((query: string, limit: number, targetLayer?: "all" | "memory" | "observation") => hybrid.search(query, limit, targetLayer));+ const search = vi.fn((query: string, limit: number, targetLayer?: "all" | "memory" | "observation", authorizeGraph?: GraphObservationAuthorization) => hybrid.search(query, limit, targetLayer, authorizeGraph));registerSearchFunction(sdk as never, kv as never);registerSmartSearchFunction(sdk as never, kv as never, search);return {kv, sdk, index, vector, embedding, search,+ prepareGraph: async () => {+ await initializeGraphIndex(kv as never, { status: "ready" });+ for (const node of await kv.list<any>(KV.graphNodes)) await reconcileGraphIndexRecord(kv as never, "node", node.id, node);+ for (const edge of await kv.list<any>(KV.graphEdges)) await reconcileGraphIndexRecord(kv as never, "edge", edge.id, edge);+ },call: (id: string, args: Record<string, unknown>) => sdk.trigger({ function_id: id, payload: args }),addObservation: async (obs: CompressedObservation) => {index.add(obs);@@ -113,7 +120,7 @@ describe("recall layer selection", () => {setHybridRanker(f.search);const recall = await f.call("mem::search", { query: "quasar policy", limit: 1, targetLayer: "memory" });expect(recall.results[0]).toMatchObject({ layer: "memory", observation: { id: "mem_decision" } });- expect(f.search).toHaveBeenLastCalledWith("quasar policy", 1, "memory");+ expect(f.search).toHaveBeenLastCalledWith("quasar policy", 1, "memory", expect.any(Function));const smart = await f.call("mem::smart-search", { query: "quasar policy", limit: 1, targetLayer: "memory" });expect(smart.results).toEqual([expect.objectContaining({ obsId: "mem_decision", layer: "memory" })]);expect(smart).not.toHaveProperty("lessons");@@ -164,8 +171,11 @@ describe("recall layer selection", () => {id: "neighbor", name: "Rollout", type: "service", properties: {},sourceObservationIds: ["obs_3", "mem_other"],});- await f.kv.set(KV.graphEdges, "edge", { id: "edge", sourceNodeId: "node", targetNodeId: "neighbor", type: "uses", weight: 1 });- const graph = new GraphRetrieval(f.kv as never);+ await f.kv.set(KV.graphEdges, "edge", { id: "edge", sourceNodeId: "node", targetNodeId: "neighbor", type: "uses", weight: 1, sourceObservationIds: ["mem_decision"] });+ await f.addMemory(memory());+ await f.addMemory(memory({ id: "mem_other" }));+ await f.prepareGraph();+ const graph = new GraphRetrieval(f.kv as never, (id) => f.index.layerOf(id), (id) => f.index.sessionOf(id));expect((await graph.searchByEntities(["Quasar"], 2, 1, "memory")).map((hit) => hit.obsId)).toEqual(["mem_decision"]);expect((await graph.expandFromChunks(["mem_decision"], 1, 1, "memory")).map((hit) => hit.obsId)).toEqual(["mem_other"]);});@@ -267,7 +277,8 @@ describe("recall layer selection", () => {id: "node", name: "Quasar", type: "service", properties: {},sourceObservationIds: ["mem_capture", "imported-decision"],});- const graph = new GraphRetrieval(f.kv as never, (id) => f.index.layerOf(id));+ await f.prepareGraph();+ const graph = new GraphRetrieval(f.kv as never, (id) => f.index.layerOf(id), (id) => f.index.sessionOf(id));expect((await graph.searchByEntities(["Quasar"], 2, 1, "memory")).map((hit) => hit.obsId)).toEqual(["imported-decision"]);expect((await graph.searchByEntities(["Quasar"], 2, 1, "observation")).map((hit) => hit.obsId)).toEqual(["mem_capture"]);});@@ -300,3 +311,23 @@ describe("recall layer selection", () => {await expect(f.call("mem::search", { query: "quasar", targetLayer: "memory", project: "public" })).rejects.toThrow("project lookup failed");});});++it.each(["mem::search", "mem::smart-search"])("does not traverse a foreign project or agent's graph evidence in %s", async (handler) => {+ const f = fixture();+ const own = { ...observation("own"), sessionId: "own-session", agentId: "own-agent" };+ const foreign = { ...observation("foreign"), sessionId: "foreign-session", agentId: "foreign-agent" };+ await f.addObservation(own);+ await f.addObservation(foreign);+ await f.kv.set(KV.sessions, own.sessionId, { id: own.sessionId, project: "own", cwd: "/own" });+ await f.kv.set(KV.sessions, foreign.sessionId, { id: foreign.sessionId, project: "foreign", cwd: "/foreign" });+ await f.kv.set(KV.graphNodes, "seed", { id: "seed", name: "Cobalt", type: "concept", properties: {}, sourceObservationIds: [foreign.id], createdAt: timestamp });+ await f.kv.set(KV.graphNodes, "own-node", { id: "own-node", name: "Other", type: "concept", properties: {}, sourceObservationIds: [own.id], createdAt: timestamp });+ await f.kv.set(KV.graphEdges, "edge", { id: "edge", sourceNodeId: "seed", targetNodeId: "own-node", type: "uses", weight: 1, sourceObservationIds: [foreign.id], createdAt: timestamp });+ await f.prepareGraph();+ const hybrid = new HybridSearch(f.index, null, null, f.kv as never);+ const rank = (query: string, limit: number, layer?: "all" | "memory" | "observation", authorize?: GraphObservationAuthorization) => hybrid.search(query, limit, layer, authorize);+ setHybridRanker(rank);+ registerSmartSearchFunction(f.sdk as never, f.kv as never, rank);+ const result = await f.call(handler, { query: "Cobalt", project: "own", agentId: "own-agent", cwd: "/own" });+ expect(result.results).toEqual([]);+});diff --git a/test/reflect.test.ts b/test/reflect.test.tsindex 4e5f78bd..5d0f9d27 100644--- a/test/reflect.test.ts+++ b/test/reflect.test.ts@@ -5,28 +5,40 @@ vi.mock("../src/logger.js", () => ({}));import { registerReflectFunctions } from "../src/functions/reflect.js";-import { fingerprintId } from "../src/state/schema.js";+import { initializeGraphIndex, reconcileGraphIndexRecord } from "../src/state/graph-retrieval-index.js";+import { indexObservationSession } from "../src/state/obs-index.js";+import { KV, fingerprintId } from "../src/state/schema.js";import type { Insight, GraphNode, GraphEdge, SemanticMemory, Lesson, Crystal } from "../src/types.js";function mockKV() {const store = new Map<string, Map<string, unknown>>();- return {+ const kv = {get: async <T>(scope: string, key: string): Promise<T | null> => {return (store.get(scope)?.get(key) as T) ?? null;},set: async <T>(scope: string, key: string, data: T): Promise<T> => {if (!store.has(scope)) store.set(scope, new Map());store.get(scope)!.set(key, data);+ if (scope === KV.graphNodes || scope === KV.graphEdges) {+ const row = data as GraphNode | GraphEdge;+ for (const id of row.sourceObservationIds ?? []) {+ await indexObservationSession(kv as never, id, "session");+ await kv.set(KV.observations("session"), id, { id, sessionId: "session", title: "source" });+ }+ await reconcileGraphIndexRecord(kv as never, scope === KV.graphNodes ? "node" : "edge", row.id, row);+ }return data;},delete: async (scope: string, key: string): Promise<void> => {store.get(scope)?.delete(key);},list: async <T>(scope: string): Promise<T[]> => {+ if (scope === KV.graphNodes || scope === KV.graphEdges) throw new Error("Unbounded graph read forbidden");const entries = store.get(scope);return entries ? (Array.from(entries.values()) as T[]) : [];},};+ return kv;}function mockSdk() {@@ -53,7 +65,7 @@ function makeConceptNode(name: string): GraphNode {type: "concept",name,properties: {},- sourceObservationIds: [],+ sourceObservationIds: [`obs_${name}`],createdAt: "2026-04-01T00:00:00Z",};}@@ -65,7 +77,7 @@ function makeEdge(src: string, tgt: string): GraphEdge {sourceNodeId: `node_${src}`,targetNodeId: `node_${tgt}`,weight: 1,- sourceObservationIds: [],+ sourceObservationIds: [`obs_${src}`],createdAt: "2026-04-01T00:00:00Z",};}@@ -127,9 +139,10 @@ describe("Reflect", () => {let kv: ReturnType<typeof mockKV>;let provider: { name: string; compress: ReturnType<typeof vi.fn>; summarize: ReturnType<typeof vi.fn> };- beforeEach(() => {+ beforeEach(async () => {sdk = mockSdk();kv = mockKV();+ await initializeGraphIndex(kv as never, { status: "ready" });provider = {name: "test",compress: vi.fn(),@@ -627,3 +640,55 @@ describe("Reflect", () => {});});});++describe("bounded reflection graph context", () => {+ async function fixture() {+ const kv = mockKV();+ await initializeGraphIndex(kv as never, { status: "ready" });+ await kv.set(KV.graphNodes, "node_security", makeConceptNode("security"));+ await kv.set(KV.graphNodes, "node_validation", makeConceptNode("validation"));+ await kv.set(KV.graphEdges, "edge", makeEdge("security", "validation"));+ const sdk = mockSdk();+ const summarize = vi.fn(async () => "<insights></insights>");+ registerReflectFunctions(sdk as never, kv as never, { name: "fixture", summarize } as never);+ return { kv, sdk, summarize };+ }++ it("skips unmigrated graph state without enumerating canonical graph scopes", async () => {+ const f = await fixture();+ await f.kv.delete(KV.graphRetrievalControl, "current");+ const list = vi.spyOn(f.kv, "list");+ const result = await f.sdk.trigger("mem::reflect", {});+ expect(result).toMatchObject({ usedFallback: true, graph: { status: "unavailable", nodes: 0, edges: 0 } });+ expect(list.mock.calls.some(([scope]) => scope === KV.graphNodes || scope === KV.graphEdges)).toBe(false);+ });++ it("excludes foreign and forgotten graph provenance before forming project clusters", async () => {+ const f = await fixture();+ await f.kv.set(KV.sessions, "session", { id: "session", project: "private" });+ expect(await f.sdk.trigger("mem::reflect", { project: "public" })).toMatchObject({ graph: { nodes: 0, edges: 0 } });+ expect(await f.sdk.trigger("mem::reflect", { project: "private" })).toMatchObject({ graph: { nodes: 2, edges: 1 } });+ await f.kv.delete(KV.observations("session"), "obs_security");+ expect(await f.sdk.trigger("mem::reflect", { project: "private" })).toMatchObject({ graph: { nodes: 1, edges: 0 } });+ });++ it("reports partial inventory and remains within its point-read budget", async () => {+ const f = await fixture();+ for (let i = 0; i < 100; i++) await f.kv.set(KV.graphNodes, `node_${i}`, makeConceptNode(String(i)));+ const get = vi.spyOn(f.kv, "get");+ const result = await f.sdk.trigger("mem::reflect", {});+ expect(result.graph.truncated).toBe(true);+ expect(result.graph.nodes).toBeLessThanOrEqual(64);+ expect(get.mock.calls.length).toBeLessThanOrEqual(258);+ });++ it("reports incomplete projections even when the inventory fits in one page", async () => {+ const f = await fixture();+ await f.kv.set(KV.graphNodes, "node_security", {+ ...makeConceptNode("security"), properties: { description: "x".repeat(513) },+ });+ expect(await f.sdk.trigger("mem::reflect", {})).toMatchObject({+ graph: { status: "ready", truncated: true, nodes: 2, edges: 1 },+ });+ });+});diff --git a/test/skills-rest-endpoints.test.ts b/test/skills-rest-endpoints.test.tsnew file mode 100644index 00000000..68402348--- /dev/null+++ b/test/skills-rest-endpoints.test.ts@@ -0,0 +1,43 @@+import { readFileSync } from "node:fs";+import { describe, expect, it } from "vitest";+import { collectRestEndpoints } from "../scripts/skills/rest-endpoints.js";++describe("generated REST endpoint methods", () => {+ it("associates adjacent status and mutation routes with their own methods", () => {+ const source = readFileSync(new URL("../src/triggers/api.ts", import.meta.url), "utf8");+ expect(collectRestEndpoints(source).filter(({ path }) => path.startsWith("/agentmemory/graph/index"))).toEqual([+ { path: "/agentmemory/graph/index", method: "GET" },+ { path: "/agentmemory/graph/index/migrate", method: "POST" },+ { path: "/agentmemory/graph/index/resume", method: "POST" },+ ]);+ });++ it("handles either property order and ignores neighboring methods, comments and strings", () => {+ const source = `+ // api_path: "/comment", http_method: "DELETE"+ const example = 'api_path: "/string", http_method: "DELETE"';+ sdk.registerTrigger({ type: "http", config: { http_method: "GET", api_path: "/first" } });+ sdk.registerTrigger({ type: "http", config: { api_path: "/second", metadata: { http_method: "DELETE" },+ /* ${"spacing".repeat(40)} */ http_method: "POST" } });+ sdk.registerTrigger({ type: "http", config: { "api_path": '/third', "http_method": 'PATCH' } });+ sdk.registerTrigger({ type: "cron", config: { api_path: "/not-http", http_method: "GET" } });+ `;+ expect(collectRestEndpoints(source)).toEqual([+ { path: "/first", method: "GET" },+ { path: "/second", method: "POST" },+ { path: "/third", method: "PATCH" },+ ]);+ });++ it("deduplicates method/path pairs while preserving multiple methods for the same route", () => {+ const source = `+ sdk.registerTrigger({ type: "http", config: { api_path: "/shared", http_method: "POST" } });+ sdk.registerTrigger({ type: "http", config: { api_path: "/shared", http_method: "GET" } });+ sdk.registerTrigger({ type: "http", config: { api_path: "/shared", http_method: "POST" } });+ `;+ expect(collectRestEndpoints(source)).toEqual([+ { path: "/shared", method: "GET" },+ { path: "/shared", method: "POST" },+ ]);+ });+});diff --git a/test/status.test.ts b/test/status.test.tsindex eb010925..6b260ff5 100644--- a/test/status.test.ts+++ b/test/status.test.ts@@ -181,6 +181,40 @@ describe("evaluateStatus", () => {expect(report.problems[0].code).toBe("no-llm-provider");});+ it("reports graph migration separately from empty graph data and snapshot rebuilds", () => {+ const report = evaluateStatus(inputs({ graph: { fromSnapshot: false }, graphIndex: {+ state: "needs-migration", ready: false, coverage: "unknown", action: "Run graph-index migrate --allow-legacy-scan",+ } }));+ expect(codes(report)).toEqual(["graph-index-not-ready"]);+ expect(report.problems[0].fix).toContain("--allow-legacy-scan");+ expect(renderStatusHtml(report, "n")).toContain("needs-migration, unknown coverage");+ });++ it("preserves unknown graph counts instead of displaying an empty graph", () => {+ const report = evaluateStatus(inputs({ graph: { totalNodes: null, totalEdges: null, available: false, coverage: "unknown", fromIndex: false }, graphIndex: {+ state: "needs-migration", ready: false, coverage: "unknown",+ } }));+ expect(report.graph).toMatchObject({ totalNodes: null, totalEdges: null, available: false });+ const html = renderStatusHtml(report, "n");+ expect(html).toContain("<th>Nodes / edges</th><td>unknown / unknown</td>");+ expect(html).not.toContain("<th>Nodes / edges</th><td>0 / 0</td>");+ const empty = evaluateStatus(inputs({ graph: { totalNodes: 0, totalEdges: 0, available: true, fromIndex: true } }));+ expect(renderStatusHtml(empty, "n")).toContain("<th>Nodes / edges</th><td>0 / 0</td>");+ });++ it("surfaces partial and unavailable graph indexes without claiming complete coverage", () => {+ const partial = evaluateStatus(inputs({ graphIndex: { state: "ready", ready: true, coverage: "partial", incompleteRecords: 3, action: "Inspect oversized records" } }));+ expect(codes(partial)).toEqual(["graph-index-partial"]);+ expect(partial.status).toBe("warn");+ expect(partial.problems[0].message).toContain("3 graph records");+ expect(codes(evaluateStatus(inputs({ graphIndex: null })))).toEqual(["graph-index-unavailable"]);+ });++ it("does not recommend legacy snapshot scans for an index-backed graph", () => {+ const report = evaluateStatus(inputs({ graph: { fromIndex: true, totalNodes: 12, totalEdges: 2, updatedAt: "2020-01-01T00:00:00Z" }, graphIndex: { state: "ready", ready: true, coverage: "complete" } }));+ expect(report.problems).toEqual([]);+ });+it("reports a frozen legacy audit log only when the migration left it in place", () => {expect(codes(evaluateStatus(inputs({ auditLegacy: { status: "too-large", sizeBytes: 1 } })))).toContain("audit-legacy-frozen",@@ -464,7 +498,8 @@ describe("status wiring", () => {it("time-boxes every status probe so a slow store cannot hang the page", () => {const reporter = api.slice(api.indexOf("export function createStatusReporter"), api.indexOf("export function createConsolidationStatusReader"));- expect(reporter.match(/valueWithin\(/g)?.length).toBe(6);+ expect(reporter.match(/valueWithin\(/g)?.length).toBe(7);+ expect(reporter).toContain("valueWithin(getGraphIndexMigrationStatus(kv), STATUS_CHECK_TIMEOUT_MS)");expect(reporter).toMatch(/valueWithin\(scan\.run\(\), STATUS_CHECK_TIMEOUT_MS\)/);expect(api).toMatch(/run: singleFlight\(async \(\) => \{\s*const value = await findUnindexedObservations\(kv\);/);const handler = api.slice(api.indexOf('registerFunction("api::status"'), api.indexOf('function_id: "api::status"'));diff --git a/test/temporal-graph.test.ts b/test/temporal-graph.test.tsindex 55ef301d..0be4e6ce 100644--- a/test/temporal-graph.test.ts+++ b/test/temporal-graph.test.ts@@ -1,3 +1,7 @@+import { createGraphIndexState } from "./helpers/graph-index-state.js";+import { createGraphIndexReader, graphEdgeLookupKey, GraphIndexReader, initializeGraphIndex } from "../src/state/graph-retrieval-index.js";+import { GraphIndexMigrationController } from "../src/state/graph-index-migration.js";+import { KV } from "../src/state/schema.js";import { describe, it, expect, vi } from "vitest";import type { GraphNode, GraphEdge, MemoryProvider } from "../src/types.js";@@ -5,38 +9,19 @@ vi.mock("../src/logger.js", () => ({logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn() },}));-function mockKV(- nodes: GraphNode[] = [],- edges: GraphEdge[] = [],-) {- const store = new Map<string, Map<string, unknown>>();- const nodesMap = new Map<string, unknown>();- for (const n of nodes) nodesMap.set(n.id, n);- store.set("mem:graph:nodes", nodesMap);-- const edgesMap = new Map<string, unknown>();- for (const e of edges) edgesMap.set(e.id, e);- store.set("mem:graph:edges", edgesMap);-- store.set("mem:graph:edge-history", new Map());-- return {- get: async <T>(scope: string, key: string): Promise<T | null> => {- return (store.get(scope)?.get(key) as T) ?? null;- },- set: async <T>(scope: string, key: string, data: T): Promise<T> => {- if (!store.has(scope)) store.set(scope, new Map());- store.get(scope)!.set(key, data);- return data;- },- delete: async (scope: string, key: string): Promise<void> => {- store.get(scope)?.delete(key);- },- list: async <T>(scope: string): Promise<T[]> => {- const entries = store.get(scope);- return entries ? (Array.from(entries.values()) as T[]) : [];- },- };+async function mockKV(nodes: GraphNode[] = [], edges: GraphEdge[] = []) {+ const { kv } = createGraphIndexState();+ await initializeGraphIndex(kv, { status: "ready" });+ for (const node of nodes) await kv.set("mem:graph:nodes", node.id, node);+ for (const edge of edges) {+ for (const id of [edge.sourceNodeId, edge.targetNodeId]) {+ if (!await kv.get("mem:graph:nodes", id)) await kv.set("mem:graph:nodes", id, {+ id, name: id, type: "location", properties: {}, sourceObservationIds: [], createdAt: edge.createdAt,+ });+ }+ await kv.set("mem:graph:edges", edge.id, edge);+ }+ return kv;}function mockSdk() {@@ -67,7 +52,7 @@ describe("TemporalGraph", () => {"../src/functions/temporal-graph.js");const sdk = mockSdk();- const kv = mockKV();+ const kv = await mockKV();const provider: MemoryProvider = {name: "test",compress: vi.fn().mockResolvedValue(""),@@ -109,7 +94,7 @@ describe("TemporalGraph", () => {};const sdk = mockSdk();- const kv = mockKV();+ const kv = await mockKV();registerTemporalGraphFunctions(sdk as never, kv as never, provider);const result = (await sdk.trigger("mem::temporal-graph-extract", {@@ -200,7 +185,7 @@ describe("TemporalGraph", () => {};const sdk = mockSdk();- const kv = mockKV([existingNode, existingNode2], [existingEdge]);+ const kv = await mockKV([existingNode, existingNode2], [existingEdge]);registerTemporalGraphFunctions(sdk as never, kv as never, provider);const result = (await sdk.trigger("mem::temporal-graph-extract", {@@ -274,7 +259,7 @@ describe("TemporalGraph", () => {};const sdk = mockSdk();- const kv = mockKV([node], [edge1, edge2]);+ const kv = await mockKV([node], [edge1, edge2]);const provider: MemoryProvider = {name: "test",compress: vi.fn(),@@ -334,7 +319,7 @@ describe("TemporalGraph", () => {};const sdk = mockSdk();- const kv = mockKV([node], [edge1, edge2]);+ const kv = await mockKV([node], [edge1, edge2]);const provider: MemoryProvider = {name: "test",compress: vi.fn(),@@ -357,7 +342,7 @@ describe("TemporalGraph", () => {"../src/functions/temporal-graph.js");const sdk = mockSdk();- const kv = mockKV();+ const kv = await mockKV();const provider: MemoryProvider = {name: "test",compress: vi.fn(),@@ -373,3 +358,154 @@ describe("TemporalGraph", () => {expect(result.error).toBe("No observations provided");});});++describe("temporal predecessor reconciliation", () => {+ const date = "2026-10-14T00:00:00.000Z";+ const makeNode = (id: string, name = id): GraphNode => ({+ id, name, type: "concept", properties: {}, sourceObservationIds: ["obs"], createdAt: date,+ });+ const makeEdge = (id: string, more: Partial<GraphEdge> = {}): GraphEdge => ({+ id, sourceNodeId: "a", targetNodeId: "b", type: "uses", weight: 1,+ sourceObservationIds: ["obs"], createdAt: date, version: 1, isLatest: true, ...more,+ });+ const provider: MemoryProvider = {+ name: "fixture", summarize: async () => "", compress: async () => '<temporal_graph><entities><entity type="concept" name="Alice"></entity><entity type="concept" name="Acme"></entity></entities><relationships><relationship type="uses" source="Alice" target="Acme" weight="0.8"><reasoning>updated</reasoning></relationship></relationships></temporal_graph>',+ };+ const input = { observations: [{ id: "obs", title: "test", narrative: "test", concepts: [], files: [], type: "discovery", timestamp: date }] };++ it("rejects an older ready index without scans or creating another edge version", async () => {+ const { registerTemporalGraphFunctions } = await import("../src/functions/temporal-graph.js");+ const fixture = createGraphIndexState();+ await initializeGraphIndex(fixture.kv, { status: "ready" });+ for (const row of [makeNode("a", "Alice"), makeNode("b", "Acme")]) await fixture.kv.set(KV.graphNodes, row.id, row);+ await fixture.kv.set(KV.graphEdges, "old", makeEdge("old", { version: 7 }));+ const controls = fixture.store.get(KV.graphRetrievalControl)!;+ controls.set("current", { ...controls.get("current") as object, schema: 1 });+ const before = structuredClone(fixture.store);+ const restarted = createGraphIndexState(fixture.store);+ const sdk = mockSdk();+ registerTemporalGraphFunctions(sdk as never, restarted.kv, provider);+ expect(await sdk.trigger("mem::temporal-graph-extract", input)).toMatchObject({ success: false, error: expect.stringContaining("explicit graph-index migration") });+ expect(fixture.store).toEqual(before);+ expect(restarted.calls.every(call => call.id === "state::get" && ![KV.graphNodes, KV.graphEdges, KV.graphSnapshot].includes(call.scope as never))).toBe(true);+ });++ it.each(["direct", "migrated"])("uses the exact current edge key after %s writes and restart despite unrelated and historical fanout", async (mode) => {+ const { registerTemporalGraphFunctions } = await import("../src/functions/temporal-graph.js");+ const fixture = createGraphIndexState();+ const nodes = [makeNode("a", "Alice"), makeNode("b", "Acme"), makeNode("c")];+ const edges = [+ makeEdge("candidate-a", { version: 4 }), makeEdge("candidate-z", { version: 4 }),+ makeEdge("legacy-current", { version: 2, isLatest: undefined }),+ ...Array.from({ length: 300 }, (_, i) => makeEdge(`unrelated-${i}`, { targetNodeId: "c" })),+ ...Array.from({ length: 300 }, (_, i) => makeEdge(`historical-${i}`, { version: 99, isLatest: false })),+ makeEdge("stale", { version: 100, stale: true }),+ ];+ if (mode === "migrated") {+ fixture.store.set(KV.graphNodes, new Map(nodes.map(row => [row.id, row])));+ fixture.store.set(KV.graphEdges, new Map(edges.map(row => [row.id, row])));+ const migration = new GraphIndexMigrationController(fixture.kv);+ await migration.start({ allowLegacyScan: true });+ await migration.waitForIdle();+ expect(await migration.status()).toMatchObject({ ready: true });+ } else {+ await initializeGraphIndex(fixture.kv, { status: "ready" });+ for (const row of nodes) await fixture.kv.set(KV.graphNodes, row.id, row);+ for (const row of edges) await fixture.kv.set(KV.graphEdges, row.id, row);+ }+ await fixture.kv.set(KV.graphEdgeHistory, "history-only", makeEdge("history-only", { version: 1000 }));+ await fixture.kv.set(KV.graphEdgeKey, "a|b|uses", "historical-299");++ const readerCalls = vi.spyOn(GraphIndexReader.prototype, "getEdge");+ try {+ for (const version of [5, 6]) {+ const restarted = createGraphIndexState(fixture.store);+ const sdk = mockSdk();+ registerTemporalGraphFunctions(sdk as never, restarted.kv, provider);+ expect(await sdk.trigger("mem::temporal-graph-extract", input)).toMatchObject({ success: true });+ const rows = [...fixture.store.get(KV.graphEdges)!.values()] as GraphEdge[];+ const replacement = rows.find(row => row.version === version && row.sourceNodeId === "a" && row.targetNodeId === "b");+ expect(replacement).toMatchObject({ isLatest: true });+ const predecessor = version === 5 ? rows.find(row => row.id === "candidate-a") : rows.find(row => row.version === 5);+ expect(predecessor).toMatchObject({ isLatest: false, supersededBy: replacement!.id });+ expect(rows.find(row => row.id === "candidate-z")).toMatchObject({ isLatest: true, version: 4 });+ expect(rows.find(row => row.id === "historical-299")).toMatchObject({ isLatest: false, version: 99 });+ expect(restarted.calls.some(call => call.id === "state::list")).toBe(false);+ }+ expect(readerCalls.mock.calls.length).toBeLessThanOrEqual(6);+ expect(readerCalls.mock.calls.flat().some(id => /unrelated|historical|stale|history-only/.test(id))).toBe(false);+ } finally {+ readerCalls.mockRestore();+ }+ }, 30_000);++ it("keeps delimiter-safe key membership correct across direct updates, stale records, historical copies and deletion", async () => {+ const { kv, store } = createGraphIndexState();+ await initializeGraphIndex(kv, { status: "ready" });+ const first = makeEdge("first", { sourceNodeId: "a|b", targetNodeId: "c" });+ const collision = makeEdge("collision", { sourceNodeId: "a", targetNodeId: "b|c" });+ const ids = async (row: GraphEdge) => {+ const reader = (await createGraphIndexReader(createGraphIndexState(store).kv))!;+ const page = await reader.posting("edgeKey", graphEdgeLookupKey(row));+ expect(await reader.finish()).toBe(true);+ return page.ids;+ };+ await kv.set(KV.graphEdges, first.id, first);+ await kv.set(KV.graphEdges, collision.id, collision);+ expect(await ids(first)).toEqual(["first"]);+ expect(await ids(collision)).toEqual(["collision"]);+ const moved = { ...first, targetNodeId: "other" };+ await kv.set(KV.graphEdges, first.id, moved);+ expect(await ids(first)).toEqual([]);+ expect(await ids(moved)).toEqual(["first"]);+ await kv.set(KV.graphEdges, first.id, { ...moved, isLatest: false });+ await kv.set(KV.graphEdgeHistory, first.id, moved);+ expect(await ids(moved)).toEqual([]);+ await kv.set(KV.graphEdges, first.id, { ...moved, isLatest: undefined });+ expect(await ids(moved)).toEqual(["first"]);+ await kv.set(KV.graphEdges, first.id, { ...moved, stale: true });+ expect(await ids(moved)).toEqual([]);+ await kv.set(KV.graphEdges, first.id, moved);+ await kv.delete(KV.graphEdges, first.id);+ expect(await ids(moved)).toEqual([]);+ });++ it("rejects a predecessor changed during resolution and uses its new version after restart", async () => {+ const { registerTemporalGraphFunctions } = await import("../src/functions/temporal-graph.js");+ const fixture = createGraphIndexState();+ await initializeGraphIndex(fixture.kv, { status: "ready" });+ for (const row of [makeNode("a", "Alice"), makeNode("b", "Acme")]) await fixture.kv.set(KV.graphNodes, row.id, row);+ await fixture.kv.set(KV.graphEdges, "old", makeEdge("old"));+ let changed = false;+ fixture.intercept(async (id, payload) => {+ if (!changed && id === "state::get" && payload.scope === KV.graphEdges && payload.key === "old") {+ changed = true;+ await fixture.kv.set(KV.graphEdges, "old", makeEdge("old", { version: 7 }));+ }+ });+ const sdk = mockSdk();+ registerTemporalGraphFunctions(sdk as never, fixture.kv, provider);+ expect(await sdk.trigger("mem::temporal-graph-extract", input)).toMatchObject({ success: false, error: expect.stringContaining("changed revision") });+ expect(fixture.store.get(KV.graphEdges)?.size).toBe(1);+ fixture.intercept();+ registerTemporalGraphFunctions(sdk as never, createGraphIndexState(fixture.store).kv, provider);+ expect(await sdk.trigger("mem::temporal-graph-extract", input)).toMatchObject({ success: true });+ const rows = [...fixture.store.get(KV.graphEdges)!.values()] as GraphEdge[];+ expect(rows.find(row => row.id !== "old")).toMatchObject({ version: 8, isLatest: true });+ expect(rows.find(row => row.id === "old")).toMatchObject({ version: 7, isLatest: false });+ });++ it("fails closed when conflicting current predecessors exceed the existing read budget", async () => {+ const { registerTemporalGraphFunctions } = await import("../src/functions/temporal-graph.js");+ const fixture = createGraphIndexState();+ await initializeGraphIndex(fixture.kv, { status: "ready" });+ for (const row of [makeNode("a", "Alice"), makeNode("b", "Acme")]) await fixture.kv.set(KV.graphNodes, row.id, row);+ for (let i = 0; i < 300; i++) await fixture.kv.set(KV.graphEdges, `edge-${i}`, makeEdge(`edge-${i}`));+ const sdk = mockSdk();+ registerTemporalGraphFunctions(sdk as never, fixture.kv, provider);+ expect(await sdk.trigger("mem::temporal-graph-extract", input)).toMatchObject({ success: false, error: expect.stringContaining("read budget") });+ expect(fixture.store.get(KV.graphEdges)?.size).toBe(300);+ expect([...fixture.store.get(KV.graphEdges)!.values()].every(row => (row as GraphEdge).isLatest === true)).toBe(true);+ expect(fixture.store.get(KV.graphEdgeHistory)?.size ?? 0).toBe(0);+ });+});diff --git a/test/viewer-counts-reset.test.ts b/test/viewer-counts-reset.test.tsindex 64d7f1df..7ae5e9a8 100644--- a/test/viewer-counts-reset.test.ts+++ b/test/viewer-counts-reset.test.ts@@ -1,9 +1,13 @@import { afterEach, describe, expect, it, vi } from "vitest";+import { KV } from "../src/state/schema.js";import type { StateKV } from "../src/state/kv.js";import {getViewerCounts, invalidateViewerCounts, noteGraphStats,- resetViewerCounts, setViewerCountsEmitter,+ readViewerGraphMetadata, resetViewerCounts, setViewerCountsEmitter,} from "../src/state/viewer-counts.js";+import { createGraphIndexState } from "./helpers/graph-index-state.js";++const control = (nodes: number) => ({ schema: 2, buildId: "test", status: "ready", revision: 0, nodes, edges: 0, historyEdges: 0, incompleteRecords: 0, nodesByType: {}, edgesByType: {}, otherTypes: 0, updatedAt: "now" });const stats = (totalNodes: number) => ({ totalNodes, totalEdges: 0, nodesByType: {}, edgesByType: {} });@@ -16,10 +20,31 @@ function deferred() {afterEach(() => { resetViewerCounts(); vi.useRealTimers(); });describe("viewer count invalidation", () => {+ it.each([null, { status: "building" }, { status: "paused" }, { pending: { kind: "node", id: "a" } }, { batchPending: true }, { revision: -1 }])(+ "keeps unknown or unavailable graph counts unknown without touching the legacy snapshot: %j", async (overrides) => {+ const { kv, store, calls } = createGraphIndexState();+ store.set(KV.graphSnapshot, new Map([["current", { stats: stats(9000), payload: "x".repeat(512 * 1024) }]]));+ if (overrides) store.set(KV.graphRetrievalControl, new Map([["current", { ...control(5), ...overrides }]]));+ expect(await getViewerCounts(kv)).toMatchObject({ graphNodes: null, graphEdges: null });+ expect(calls.some((call) => call.scope === KV.graphSnapshot || call.scope === KV.graphNodes || call.scope === KV.graphEdges)).toBe(false);+ },+ );++ it("refreshes cached graph totals from maintained control without rescanning other scopes", async () => {+ const { kv, store, calls } = createGraphIndexState();+ store.set(KV.graphRetrievalControl, new Map([["current", control(2)]]));+ expect((await getViewerCounts(kv)).graphNodes).toBe(2);+ calls.length = 0;+ store.get(KV.graphRetrievalControl)!.set("current", control(3));+ expect((await getViewerCounts(kv)).graphNodes).toBe(3);+ expect(calls.every((call) => call.id === "state::get" && [KV.graphControl, KV.graphRetrievalControl].includes(call.scope))).toBe(true);+ expect(await readViewerGraphMetadata(kv)).toMatchObject({ stats: { totalNodes: 3 }, graphIndex: { status: "ready" } });+ });+it("reloads reset counts and preserves the live emitter", async () => {vi.useFakeTimers();let total = 2;- const kv = { list: async () => [], get: async () => ({ stats: stats(total) }) } as unknown as StateKV;+ const kv = { list: async () => [], get: async (scope: string) => scope === KV.graphRetrievalControl ? control(total) : null } as unknown as StateKV;const emit = vi.fn();setViewerCountsEmitter(emit);expect((await getViewerCounts(kv)).graphNodes).toBe(2);@@ -38,8 +63,9 @@ describe("viewer count invalidation", () => {let total = 2;const kv = {list: async () => [],- get: async () => {- const result = { stats: stats(total) };+ get: async (scope: string) => {+ if (scope !== KV.graphRetrievalControl) return null;+ const result = control(total);const index = reads++;if (index < 2) { entered[index].resolve(); await release[index].promise; }return result;diff --git a/test/viewer-dashboard-reliability.test.ts b/test/viewer-dashboard-reliability.test.tsindex d2ceafb0..07211a86 100644--- a/test/viewer-dashboard-reliability.test.ts+++ b/test/viewer-dashboard-reliability.test.ts@@ -186,6 +186,15 @@ describe("viewer live event reducer", () => {});describe("viewer navigation and shared components", () => {+ it("renders deferred graph compaction as requiring explicit maintenance, not completed", () => {+ const render = new Function("hrow", "esc", `${extractFunction("graphCompactionRowHtml")}\nreturn graphCompactionRowHtml;`)(+ (...args: unknown[]) => args, String,+ );+ expect(render({ state: "deferred" })).toEqual([+ "Provenance compaction", "deferred", null, expect.stringContaining("explicit maintenance"),+ ]);+ });+it("parses deep links into a tab and an entity id", () => {const deps = ["normalizeTab", "decodePart", "parseRoute"].map(extractFunction).join("\n");const parse = new Function(diff --git a/test/viewer-generation-deltas.test.ts b/test/viewer-generation-deltas.test.tsindex 834d2099..5ad0b439 100644--- a/test/viewer-generation-deltas.test.ts+++ b/test/viewer-generation-deltas.test.ts@@ -1,11 +1,12 @@import { afterEach, describe, expect, it, vi } from "vitest";-import { StateKV } from "../src/state/kv.js";import { KV } from "../src/state/schema.js";import { resetGraphGeneration, setGraphGenerationListener } from "../src/state/graph-generation.js";import { registerViewerStreamTriggers } from "../src/triggers/viewer-streams.js";import { getViewerCounts, resetViewerCounts } from "../src/state/viewer-counts.js";import { setAuditRecordedListener } from "../src/functions/audit.js";import type { GraphNode, GraphSnapshot } from "../src/types.js";+import { createGraphIndexState } from "./helpers/graph-index-state.js";+import { withoutGraphRecovery } from "../src/state/graph-consistency.js";function snapshot(count: number): GraphSnapshot {const topNodes: GraphNode[] = Array.from({ length: count }, (_, i) => ({@@ -20,212 +21,151 @@ function snapshot(count: number): GraphSnapshot {}async function setup() {- const store = new Map<string, Map<string, unknown>>();+ const fixture = createGraphIndexState();+ const { kv } = fixture;const handlers = new Map<string, (data: any) => Promise<any>>();- const sent: Array<{ type: string; data: { delta: any; stats: any; generation?: string; reset?: boolean } }> = [];+ const sent: Array<{ type: string; data: any }> = [];let failSend = false;const sdk = {registerFunction: vi.fn((id: string, handler: (data: any) => Promise<any>) => handlers.set(id, handler)), registerTrigger: vi.fn(),- trigger: async ({ function_id, payload: p }: { function_id: string; payload: any }): Promise<any> => {+ trigger: async ({ function_id, payload }: { function_id: string; payload: any }): Promise<any> => {if (function_id === "stream::send") {if (failSend) { failSend = false; throw new Error("stream unavailable"); }- sent.push(structuredClone(p)); return {};+ sent.push(structuredClone(payload)); return {};}- const rows = store.get(p.scope) ?? new Map<string, unknown>();- if (function_id === "state::get") return structuredClone(rows.get(p.key) ?? null);- if (function_id === "state::list") return structuredClone([...rows.values()]);- if (function_id === "state::set") {- rows.set(p.key, structuredClone(p.value)); store.set(p.scope, rows); return p.value;- }- if (function_id === "state::delete") { rows.delete(p.key); return; }throw new Error(`Unexpected call ${function_id}`);},};- const kv = new StateKV(sdk as never);await resetGraphGeneration(kv, snapshot(0));registerViewerStreamTriggers(sdk as never, kv);- const write = (value: GraphSnapshot) => kv.set(KV.graphSnapshot, "current", value);+ const write = async (count: number) => {+ for (const node of snapshot(count).topNodes) await kv.set(KV.graphNodes, node.id, node);+ await kv.set(KV.graphSnapshot, "current", snapshot(count));+ };const graphEvents = () => sent.filter((e) => e.type === "graph.changed");- return { kv, sdk, write, graphEvents, handlers, sent, failNextSend: () => { failSend = true; } };+ return { ...fixture, sdk, write, graphEvents, handlers, sent, failNextSend: () => { failSend = true; } };}afterEach(() => {- vi.restoreAllMocks();- vi.useRealTimers();- setGraphGenerationListener(null);- setAuditRecordedListener(null);- resetViewerCounts();+ vi.restoreAllMocks(); vi.useRealTimers();+ setGraphGenerationListener(null); setAuditRecordedListener(null); resetViewerCounts();});-describe("generation graph viewer deltas", () => {- it("sends a bounded single-node patch for a 250-node graph after its initial invalidation", async () => {- const { write, graphEvents } = await setup();- const initial = snapshot(250);- await write(initial);- expect(graphEvents()[0].data.delta).toBeNull();- const changed = structuredClone(initial);- changed.topNodes[125].properties = { changed: true };- await write(changed);- expect(graphEvents()[1].data.delta).toEqual({- full: false, nodes: [changed.topNodes[125]], edges: [],- removedNodeIds: [], removedEdgeIds: [], degrees: {},- });- changed.topNodes.splice(10, 1);- delete changed.topDegrees.n10;- changed.stats.totalNodes--;- await write(changed);- expect(graphEvents()[2].data.delta).toMatchObject({ full: false, nodes: [], removedNodeIds: ["n10"] });- });-- it("starts with a complete baseline after listener restart, then uses incremental patches", async () => {- const { kv, sdk, write, graphEvents } = await setup();- await write(snapshot(2));- await write(snapshot(3));- expect(graphEvents()[1].data.delta).toMatchObject({ full: false, nodes: [{ id: "n2" }] });- registerViewerStreamTriggers(sdk as never, kv);- await write(snapshot(4));- expect(graphEvents()[2].data.delta).toMatchObject({ full: true, nodes: snapshot(4).topNodes });- await write(snapshot(5));- expect(graphEvents()[3].data.delta).toMatchObject({ full: false, nodes: [{ id: "n4" }] });+describe("generation graph viewer metadata", () => {+ it("uses bounded index metadata and invalidates the viewer instead of reading snapshot payloads", async () => {+ const { kv, write, graphEvents, calls } = await setup();+ await write(3);+ calls.length = 0;+ await withoutGraphRecovery(() => kv.set(KV.graphSnapshot, "current", { ...snapshot(999), padding: "x".repeat(512 * 1024) }));+ expect(graphEvents().at(-1)?.data).toMatchObject({ source: "snapshot", stats: { totalNodes: 3 }, delta: null });+ expect(Buffer.byteLength(JSON.stringify(graphEvents().at(-1)))).toBeLessThan(4096);+ expect(calls.filter((call) => call.id === "state::get" && call.scope.startsWith(KV.graphSnapshot))).toEqual([]);});- it("does not advance its baseline when a stream send fails", async () => {- const { write, graphEvents, failNextSend } = await setup();- await write(snapshot(2));- failNextSend();- await write(snapshot(3));+ it("retains accurate metadata after listener restart and failed delivery", async () => {+ const { kv, sdk, write, graphEvents, failNextSend } = await setup();+ await write(2);+ failNextSend(); await write(3);expect(graphEvents()).toHaveLength(1);- await write(snapshot(4));- expect(graphEvents()[1].data.delta).toMatchObject({ full: false, nodes: [{ id: "n2" }, { id: "n3" }] });+ registerViewerStreamTriggers(sdk as never, kv);+ await write(4);+ expect(graphEvents().at(-1)?.data).toMatchObject({ stats: { totalNodes: 4 }, delta: null });});- it("invalidates the baseline across repeated same-timestamp resets, including a failed reset notification", async () => {+ it("detects repeated same-timestamp resets, including a lost reset notification", async () => {const { kv, write, graphEvents, failNextSend } = await setup();- await write(snapshot(2));+ await write(2);for (const lostReset of [false, true, false]) {if (lostReset) failNextSend();const before = graphEvents().length;await resetGraphGeneration(kv, snapshot(0));- if (!lostReset) expect(graphEvents()[before].data.delta).toMatchObject({ full: true, nodes: [] });- await write(snapshot(1));- const latest = graphEvents().at(-1)!.data.delta;- expect(latest).toMatchObject({ full: lostReset, nodes: [{ id: "n0" }] });- expect(graphEvents().at(-1)!.data.reset).toBe(lostReset);- expect(graphEvents().at(-1)!.data.generation).toBe(await kv.get(KV.graphControl, "generation"));- await write(snapshot(2));- expect(graphEvents().at(-1)!.data.delta).toMatchObject({ full: false, nodes: [{ id: "n1" }] });+ if (!lostReset) expect(graphEvents()[before].data).toMatchObject({ reset: true, stats: { totalNodes: 0 }, delta: null });+ await write(1);+ expect(graphEvents().at(-1)?.data).toMatchObject({ reset: lostReset, stats: { totalNodes: 1 }, delta: null,+ generation: await kv.get(KV.graphControl, "generation") });}});- it("keeps the last delivered baseline after a snapshot read failure", async () => {- const { kv, write, graphEvents } = await setup();- await write(snapshot(2));- const original = kv.get.bind(kv);- let failRead = true;- vi.spyOn(kv, "get").mockImplementation(async (scope, key) => {- if (scope === KV.graphSnapshot && failRead) { failRead = false; throw new Error("read failed"); }- return original(scope, key);- });- await write(snapshot(3));- expect(graphEvents()).toHaveLength(2);- expect(graphEvents()[1].data).toMatchObject({ source: "items", coveredBySnapshot: false });- await write(snapshot(4));- expect(graphEvents()[2].data.delta).toMatchObject({ full: false, nodes: [{ id: "n2" }, { id: "n3" }] });- });-- it("refreshes cached counts even if reading the reset snapshot fails", async () => {- const { kv, write, graphEvents } = await setup();- await write(snapshot(20));- expect((await getViewerCounts(kv)).graphNodes).toBe(20);- const original = kv.get.bind(kv);- let failRead = true;- vi.spyOn(kv, "get").mockImplementation(async (scope, key) => {- if (scope === KV.graphSnapshot && failRead) { failRead = false; throw new Error("read failed"); }- return original(scope, key);+ it("invalidates after metadata failure and refreshes counts after reset", async () => {+ const { kv, write, graphEvents, intercept } = await setup();+ await write(2);+ expect((await getViewerCounts(kv)).graphNodes).toBe(2);+ let controlReads = 0;+ intercept(async (id, payload) => {+ if (id === "state::get" && payload.scope.startsWith(KV.graphRetrievalControl) && ++controlReads === 3) {+ intercept(); throw new Error("read failed");+ }});+ await kv.set(KV.graphSnapshot, "current", snapshot(3));+ expect(graphEvents().at(-1)?.data).toMatchObject({ source: "items", coveredBySnapshot: false });await resetGraphGeneration(kv, snapshot(0));expect((await getViewerCounts(kv)).graphNodes).toBe(0);- expect(graphEvents().at(-1)!.data).toMatchObject({ source: "items", reset: true, coveredBySnapshot: false });- expect(graphEvents().at(-1)!.data).not.toHaveProperty("generation");});- it("still pushes counts immediately after a successful reset", async () => {+ it("pushes counts after reset and does not mark failed notifications as covering item changes", async () => {vi.useFakeTimers();- const { kv, write, sent } = await setup();- await write(snapshot(20));- await getViewerCounts(kv);+ const { kv, write, sent, graphEvents, failNextSend } = await setup();+ await write(2); await getViewerCounts(kv);await resetGraphGeneration(kv, snapshot(0));await vi.advanceTimersByTimeAsync(500);expect(sent.filter((e) => e.type === "counts.changed").at(-1)?.data).toMatchObject({ counts: { graphNodes: 0 } });- });-- it("does not mark coalesced item events covered by a failed snapshot send", async () => {- vi.useFakeTimers();- const { kv, write, graphEvents, failNextSend } = await setup();- await write(snapshot(2));- await kv.set(KV.graphNodes, "n2", snapshot(3).topNodes[2]);- failNextSend();- await write(snapshot(3));+ await kv.set(KV.graphNodes, "n0", snapshot(1).topNodes[0]);+ failNextSend(); await kv.set(KV.graphSnapshot, "current", snapshot(1));await vi.advanceTimersByTimeAsync(1000);expect(graphEvents().at(-1)?.data).toMatchObject({ source: "items", coveredBySnapshot: false });});- it("tags the reconnect snapshot with its generation", async () => {- const { kv, write, handlers } = await setup();- await write(snapshot(2));+ it("tags reconnect metadata with its generation without reading the snapshot", async () => {+ const { kv, write, handlers, calls } = await setup();+ await write(2); calls.length = 0;const response = await handlers.get("api::viewer-snapshot")!({ headers: {}, query_params: {} });- expect(response.body.graph).toMatchObject({- generation: await kv.get(KV.graphControl, "generation"), stats: { totalNodes: 2 },- });+ expect(response.body.graph).toMatchObject({ generation: await kv.get(KV.graphControl, "generation"), stats: { totalNodes: 2 } });+ expect(calls.filter((call) => call.id === "state::get" && call.scope.startsWith(KV.graphSnapshot))).toEqual([]);});- it("orders queued snapshot deltas from the last delivered graph", async () => {- const { kv, write, graphEvents } = await setup();- await write(snapshot(250));- const original = kv.get.bind(kv);- let entered!: () => void;- let release!: () => void;- const ready = new Promise<void>((r) => { entered = r; });- const gate = new Promise<void>((r) => { release = r; });- let pause = true;- vi.spyOn(kv, "get").mockImplementation(async (scope, key) => {- const value = await original(scope, key);- if (scope === KV.graphSnapshot && pause) { pause = false; entered(); await gate; }- return value;+ it("defers direct-write notifications until the index commit finishes, even beyond the coalescing interval", async () => {+ vi.useFakeTimers();+ const { kv, sent, graphEvents, intercept } = await setup();+ expect((await getViewerCounts(kv)).graphNodes).toBe(0);+ let entered!: () => void, release!: () => void;+ const ready = new Promise<void>((resolve) => { entered = resolve; });+ const gate = new Promise<void>((resolve) => { release = resolve; });+ intercept(async (id, payload) => {+ if (id === "state::set" && payload.scope.startsWith(KV.graphRetrievalData)) {+ intercept(); entered(); await gate;+ }});- const first = write(snapshot(251));+ const mutation = kv.set(KV.graphNodes, "n0", snapshot(1).topNodes[0]);await ready;- const second = write(snapshot(252));- release();- await Promise.all([first, second]);- expect(graphEvents()[1].data.delta).toMatchObject({ full: false, nodes: [{ id: "n250" }] });- expect(graphEvents()[2].data.delta).toMatchObject({ full: false, nodes: [{ id: "n251" }] });+ try {+ await vi.advanceTimersByTimeAsync(1500);+ expect(graphEvents()).toEqual([]);+ } finally { release(); }+ await mutation;+ await vi.advanceTimersByTimeAsync(1500);+ expect(graphEvents().at(-1)?.data).toMatchObject({ source: "items", nodesChanged: 1 });+ expect(sent.filter((event) => event.type === "counts.changed").at(-1)?.data).toMatchObject({ counts: { graphNodes: 1 } });});- it("does not associate a pre-reset snapshot with the new generation when reads overlap reset", async () => {- const { kv, write, graphEvents } = await setup();- await write(snapshot(250));- const original = kv.get.bind(kv);- let entered!: () => void;- let release!: () => void;+ it("does not associate pre-reset counts with the new generation when metadata reads overlap reset", async () => {+ const { kv, write, graphEvents, intercept } = await setup();+ await write(2);+ let entered!: () => void, release!: () => void;const ready = new Promise<void>((r) => { entered = r; });const gate = new Promise<void>((r) => { release = r; });- let pause = true;- vi.spyOn(kv, "get").mockImplementation(async (scope, key) => {- const value = await original(scope, key);- if (scope === KV.graphSnapshot && pause) { pause = false; entered(); await gate; }- return value;+ let controlReads = 0;+ intercept(async (id, payload) => {+ if (id === "state::get" && payload.scope.startsWith(KV.graphRetrievalControl) && ++controlReads === 3) {+ intercept(); entered(); await gate;+ }});- const writeBeforeReset = write(snapshot(251));+ const notification = kv.set(KV.graphSnapshot, "current", snapshot(3));await ready;- const oldGeneration = await original(KV.graphControl, "generation");+ const oldGeneration = await kv.get(KV.graphControl, "generation");const reset = resetGraphGeneration(kv, snapshot(0));- // Wait for publication, while its notification remains queued behind the old read.- await vi.waitFor(async () => expect(await original(KV.graphControl, "generation")).not.toBe(oldGeneration));- release();- await Promise.all([writeBeforeReset, reset]);- for (const event of graphEvents().slice(1)) {- expect(event.data).toMatchObject({ reset: true, stats: { totalNodes: 0 }, delta: { full: true, nodes: [] } });- }+ try { await vi.waitFor(async () => expect(await kv.get(KV.graphControl, "generation")).not.toBe(oldGeneration)); }+ finally { release(); }+ await Promise.all([notification, reset]);+ for (const event of graphEvents().slice(1)) expect(event.data).toMatchObject({ reset: true, stats: { totalNodes: 0 }, delta: null });});});diff --git a/test/viewer-streams.test.ts b/test/viewer-streams.test.tsindex 96c0b32a..8ade3c46 100644--- a/test/viewer-streams.test.ts+++ b/test/viewer-streams.test.ts@@ -45,6 +45,8 @@ function setup(kv = mockKV(), deps: Record<string, unknown> = {}) {return { handlers, triggers, sent, kv };}+const graphControl = (nodes: number, edges = 0) => ({ schema: 2, buildId: "test", status: "ready", revision: 0, nodes, edges, historyEdges: 0, incompleteRecords: 0, nodesByType: {}, edgesByType: {}, otherTypes: 0, updatedAt: "now" });+const seeded = () =>mockKV({[KV.sessions]: {@@ -60,6 +62,7 @@ const seeded = () =>l2: { id: "l2", content: "b", deleted: true },},[KV.actions]: { a1: { id: "a1" } },+ [KV.graphRetrievalControl]: { current: graphControl(12, 7) },[KV.graphSnapshot]: {current: { stats: { totalNodes: 12, totalEdges: 7, nodesByType: {}, edgesByType: {} }, updatedAt: "t" },},@@ -140,13 +143,13 @@ describe("viewer stream triggers", () => {for (let i = 0; i < 5; i++) await handlers.get("event::viewer::graph-nodes-changed")!({ key: `n${i}` });for (let i = 0; i < 3; i++) await handlers.get("event::viewer::graph-edges-changed")!({ key: `e${i}` });expect(sent.map((e) => e.type)).toEqual(["graph.changed"]);- expect(sent[0].data).toMatchObject({ source: "snapshot", stats: { totalNodes: 3, totalEdges: 2 } });+ expect(sent[0].data).toMatchObject({ source: "snapshot", stats: null, graphIndex: { status: "needs-migration" }, delta: null });await vi.advanceTimersByTimeAsync(1000);expect(sent.map((e) => e.type)).toEqual(["graph.changed", "graph.changed"]);expect(sent[1].data).toMatchObject({ source: "items", nodesChanged: 5, edgesChanged: 3 });});- it("carries the changed subgraph on graph.changed so the viewer patches without a fetch", async () => {+ it("invalidates legacy snapshot changes without forwarding unverified graph rows", async () => {vi.useFakeTimers();const { handlers, sent } = setup();const n1 = { id: "n1", type: "file", name: "a.ts", properties: {}, sourceObservationIds: ["o1"], createdAt: "t" };@@ -158,16 +161,10 @@ describe("viewer stream triggers", () => {const before = { topNodes: [n1, n2], topEdges: [e1], topDegrees: { n1: 1, n2: 1 }, stats, updatedAt: "u1" };const after = { topNodes: [n2, n3], topEdges: [e2], topDegrees: { n2: 1, n3: 1 }, stats, updatedAt: "u2" };await handlers.get("event::viewer::graph-snapshot-changed")!({ key: "current", event_type: "state:updated", old_value: before, new_value: after });- expect(sent[0].data.delta).toEqual({- full: false,- nodes: [n3],- edges: [e2],- removedNodeIds: ["n1"],- removedEdgeIds: ["e1"],- degrees: { n3: 1 },- });+ expect(sent[0].data.delta).toBeNull();+ expect(sent[0].data.stats).toBeNull();await handlers.get("event::viewer::graph-snapshot-changed")!({ key: "current", event_type: "state:created", new_value: after });- expect(sent[1].data.delta).toMatchObject({ full: true, nodes: [n2, n3], removedNodeIds: [] });+ expect(sent[1].data.delta).toBeNull();await handlers.get("event::viewer::graph-nodes-changed")!({ key: "n3" });await vi.advanceTimersByTimeAsync(1000);@@ -305,6 +302,7 @@ describe("viewer counts", () => {old_value: { id: "l1" },new_value: { id: "l1", deleted: true },});+ await kv.set(KV.graphRetrievalControl, "current", graphControl(20, 9));await handlers.get("event::viewer::graph-snapshot-changed")!({key: "current",event_type: "state:updated",