Skip to content

#1 Persistent memory for AI coding agents based on real-world benchmarks

feat(graph): compact bloated graph provenance once at boot (#1455)

Rohit Ghumare976ad3e54265
diff --git a/.env.example b/.env.example
index 607e326f..4e4e3445 100644
--- a/.env.example
+++ b/.env.example
@@ -108,6 +108,7 @@
# BM25_WEIGHT=0.4 # Hybrid search weight for BM25 leg
# VECTOR_WEIGHT=0.6 # Hybrid search weight for vector leg
# AGENTMEMORY_GRAPH_WEIGHT=0.2 # Graph traversal bonus on smart-search ranking
+# AGENTMEMORY_GRAPH_COMPACT_ON_BOOT=false # Skip the one-time background trim of oversized graph provenance at startup
# TOKEN_BUDGET=2000 # Max tokens injected via mem::context per session
# MAX_OBS_PER_SESSION=500 # Per-session observation cap before consolidation kicks in
# SUMMARIZE_CHUNK_SIZE=400 # When mem::summarize sees a session larger than this, it chunks observations and map-reduces (chunk-summarize → reduce-merge) to stay within the LLM's context window. Default 400 ≈ 50k tokens per chunk at ~110 tok/obs. Native sessions are capped by MAX_OBS_PER_SESSION; chunking primarily matters for bulk-imported jsonl sessions, which bypass that cap.
diff --git a/README.md b/README.md
index 41cbda22..4fa80f5e 100644
--- a/README.md
+++ b/README.md
@@ -1699,13 +1699,15 @@ Full endpoint list: [`src/triggers/api.ts`](src/triggers/api.ts)
</details>
-**Compacting graph provenance.** Each knowledge graph node and edge keeps the ids of the newest 32 observations it came from. Stores written before that cap can hold thousands of ids per hot node, which makes graph search and the viewer slow or drops the worker. Run `POST /agentmemory/graph/compact` once after upgrading if graph queries are slow or time out. It trims every node, edge and the cached snapshot to the cap, never lists the full graph, and is safe to re-run. When it trims ids it writes a `graph_compact` audit entry.
+**Compacting graph provenance.** Each knowledge graph node and edge keeps the ids of the newest 32 observations it came from. Stores written before that cap can hold thousands of ids per hot node, which makes graph search and the viewer slow or drops the worker. agentmemory fixes this by itself: on the first start after upgrading it trims every node, edge, superseded edge (the temporal graph history) and the cached snapshot to the cap in the background, in small slices with a pause between them, so search, capture and the viewer keep working. It saves its progress, resumes after a restart and never runs again once it has finished. `/agentmemory/status` and the viewer Health page show it as pending, running (with the current scope and position), done or failed. Set `AGENTMEMORY_GRAPH_COMPACT_ON_BOOT=false` to turn it off.
+
+To run it by hand, call `POST /agentmemory/graph/compact`. It walks the name and edge-key indexes instead of listing every node and edge, and is safe to re-run. When it trims ids it writes a `graph_compact` audit entry.
```bash
curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: application/json" -d '{}'
```
-On a large store, or when the call returns 504, run it in slices. Send `scope` (`nodes` or `edges`), `offset` and `limit`, then call again with the returned `nextOffset` until it is `null`. Do this for `nodes` and for `edges`, and finish with one `{"scope":"snapshot"}` call, because a sliced run does not touch the cached snapshot.
+On a large store, or when the call returns 504, run it in slices. Send `scope` (`nodes`, `edges` or `history`), `offset` and `limit`, then call again with the returned `nextOffset` until it is `null`. Do this for `nodes`, `edges` and `history`, and finish with one `{"scope":"snapshot"}` call, because a sliced run does not touch the cached snapshot.
```bash
curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Content-Type: application/json" -d '{"scope":"nodes","offset":0,"limit":200}'
diff --git a/plugin/skills/agentmemory-config/REFERENCE.md b/plugin/skills/agentmemory-config/REFERENCE.md
index 158b1f1e..5cc5cbfd 100644
--- a/plugin/skills/agentmemory-config/REFERENCE.md
+++ b/plugin/skills/agentmemory-config/REFERENCE.md
@@ -3,7 +3,7 @@
Generated by scanning `src/` for `AGENTMEMORY_*` usage. Do not edit the block below by hand; run `npm run skills:gen` after adding or removing a variable. Internal markers ending in two underscores are excluded.
<!-- AUTOGEN:env START - generated by scripts/skills/generate.ts, do not edit by hand -->
-Configuration is read from the environment and from `~/.agentmemory/.env` (no `export` prefix). 58 recognized variables:
+Configuration is read from the environment and from `~/.agentmemory/.env` (no `export` prefix). 59 recognized variables:
- `AGENTMEMORY_AGENT_SCOPE`
- `AGENTMEMORY_ALLOW_AGENT_SDK`
@@ -27,6 +27,7 @@ Configuration is read from the environment and from `~/.agentmemory/.env` (no `e
- `AGENTMEMORY_EXPORT_ROOT`
- `AGENTMEMORY_FOLLOWUP_WINDOW_SECONDS`
- `AGENTMEMORY_FORCE_PROXY`
+- `AGENTMEMORY_GRAPH_COMPACT_ON_BOOT`
- `AGENTMEMORY_GRAPH_WEIGHT`
- `AGENTMEMORY_III_CONFIG`
- `AGENTMEMORY_III_VERSION`
diff --git a/src/config.ts b/src/config.ts
index ef0597cc..359e7a76 100644
--- a/src/config.ts
+++ b/src/config.ts
@@ -249,6 +249,10 @@ export function getSessionSweepStaleHours(): number {
return parsed > 0 ? parsed : 24;
}
+export function isGraphCompactOnBootEnabled(): boolean {
+ return getMergedEnv()["AGENTMEMORY_GRAPH_COMPACT_ON_BOOT"] !== "false";
+}
+
export function isDropStaleIndexEnabled(): boolean {
return getMergedEnv()["AGENTMEMORY_DROP_STALE_INDEX"] === "true";
}
diff --git a/src/functions/graph-compact-boot.ts b/src/functions/graph-compact-boot.ts
new file mode 100644
index 00000000..7b5391b2
--- /dev/null
+++ b/src/functions/graph-compact-boot.ts
@@ -0,0 +1,316 @@
+import type { StateKV } from "../state/kv.js";
+import { KV } from "../state/schema.js";
+import {
+ MAX_GRAPH_SOURCE_OBSERVATIONS,
+ compactGraphProvenance,
+ type GraphCompactResult,
+ type GraphCompactScope,
+} from "./graph.js";
+import { safeAudit } from "./audit.js";
+
+export const GRAPH_COMPACT_BOOT_VERSION = 1;
+export const GRAPH_COMPACT_BOOT_KEY = `graph-compact-on-boot:v${GRAPH_COMPACT_BOOT_VERSION}`;
+export const GRAPH_COMPACT_BOOT_FUNCTION_ID = "mem::graph-compact-on-boot";
+export const GRAPH_COMPACT_BOOT_DELAY_MS = 3000;
+
+const SLICED_SCOPES = ["nodes", "edges", "history"] as const;
+type SlicedScope = (typeof SLICED_SCOPES)[number];
+type BootPhase = SlicedScope | "snapshot";
+
+export interface GraphCompactBootProgress {
+ version: number;
+ cap: number;
+ status: "running" | "done" | "failed";
+ phase: BootPhase;
+ offset: number;
+ phaseTotal: number | null;
+ scanned: number;
+ trimmed: number;
+ idsRemoved: number;
+ pending?: GraphCompactBootSlicePlan;
+ startedAt: string;
+ updatedAt: string;
+ completedAt?: string;
+ error?: string;
+}
+
+export interface GraphCompactBootSlicePlan {
+ phase: BootPhase;
+ offset: number;
+ scanned: number;
+ trimmed: number;
+ idsRemoved: number;
+ nextOffset: number | null;
+ total: number | null;
+}
+
+export type GraphCompactBootState = "off" | "pending" | "running" | "done" | "failed";
+
+export interface GraphCompactBootStatus {
+ state: GraphCompactBootState;
+ phase?: BootPhase;
+ processed?: number;
+ total?: number | null;
+ scanned?: number;
+ trimmed?: number;
+ idsRemoved?: number;
+ startedAt?: string;
+ completedAt?: string;
+ error?: string;
+}
+
+export interface GraphCompactBootOptions {
+ sliceSize?: number;
+ pauseMs?: number;
+ maxAttempts?: number;
+ retryDelayMs?: number;
+ log?: (msg: string) => void;
+ warn?: (msg: string) => void;
+ sleep?: (ms: number) => Promise<void>;
+ now?: () => Date;
+}
+
+const DEFAULT_SLICE_SIZE = 100;
+const DEFAULT_PAUSE_MS = 200;
+const DEFAULT_MAX_ATTEMPTS = 3;
+const DEFAULT_RETRY_DELAY_MS = 2000;
+
+let current: GraphCompactBootStatus = { state: "pending" };
+let running: Promise<GraphCompactBootStatus> | null = null;
+
+export function getGraphCompactBootStatus(): GraphCompactBootStatus {
+ return { ...current };
+}
+
+export function setGraphCompactBootDisabled(): void {
+ current = { state: "off" };
+}
+
+export function resetGraphCompactBootStatus(): void {
+ current = { state: "pending" };
+ running = null;
+}
+
+function statusFrom(p: GraphCompactBootProgress): GraphCompactBootStatus {
+ return {
+ state: p.status,
+ phase: p.phase,
+ processed: p.offset,
+ total: p.phaseTotal,
+ scanned: p.scanned,
+ trimmed: p.trimmed,
+ idsRemoved: p.idsRemoved,
+ startedAt: p.startedAt,
+ completedAt: p.completedAt,
+ error: p.error,
+ };
+}
+
+function isResumable(p: GraphCompactBootProgress | null): p is GraphCompactBootProgress {
+ return (
+ !!p &&
+ p.version === GRAPH_COMPACT_BOOT_VERSION &&
+ p.cap === MAX_GRAPH_SOURCE_OBSERVATIONS &&
+ (p.phase === "snapshot" || (SLICED_SCOPES as readonly string[]).includes(p.phase)) &&
+ Number.isInteger(p.offset) &&
+ p.offset >= 0
+ );
+}
+
+function storedPlan(p: GraphCompactBootProgress, phase: BootPhase): GraphCompactBootSlicePlan | null {
+ const plan = p.pending;
+ if (!plan || plan.phase !== phase || plan.offset !== p.offset) return null;
+ const counts = [plan.scanned, plan.trimmed, plan.idsRemoved];
+ return counts.every((n) => Number.isInteger(n) && n >= 0) ? plan : null;
+}
+
+function scannedOf(scope: SlicedScope, r: GraphCompactResult): [number, number] {
+ if (scope === "nodes") return [r.nodesScanned, r.nodesTrimmed];
+ if (scope === "edges") return [r.edgesScanned, r.edgesTrimmed];
+ return [r.historyScanned, r.historyTrimmed];
+}
+
+export function describeGraphCompactBoot(s: GraphCompactBootStatus): string {
+ if (s.state === "off") return "off (AGENTMEMORY_GRAPH_COMPACT_ON_BOOT=false)";
+ if (s.state === "pending") return "pending";
+ const where =
+ s.phase === "snapshot" || s.total === null || s.total === undefined
+ ? `${s.phase ?? "nodes"}`
+ : `${s.phase} ${s.processed ?? 0} of ${s.total}`;
+ if (s.state === "running") return `running, ${where}`;
+ if (s.state === "failed") return `failed at ${where}: ${s.error ?? "unknown error"}`;
+ return `done, ${s.trimmed ?? 0} records trimmed, ${s.idsRemoved ?? 0} ids removed`;
+}
+
+export function runGraphCompactOnBoot(
+ kv: StateKV,
+ opts: GraphCompactBootOptions = {},
+): Promise<GraphCompactBootStatus> {
+ if (running) return running;
+ running = runOnce(kv, opts).finally(() => {
+ running = null;
+ });
+ return running;
+}
+
+async function runOnce(kv: StateKV, opts: GraphCompactBootOptions): Promise<GraphCompactBootStatus> {
+ const sliceSize = opts.sliceSize ?? DEFAULT_SLICE_SIZE;
+ const pauseMs = opts.pauseMs ?? DEFAULT_PAUSE_MS;
+ const maxAttempts = Math.max(1, opts.maxAttempts ?? DEFAULT_MAX_ATTEMPTS);
+ const retryDelayMs = opts.retryDelayMs ?? DEFAULT_RETRY_DELAY_MS;
+ const log = opts.log ?? (() => {});
+ const warn = opts.warn ?? log;
+ const sleep = opts.sleep ?? ((ms: number) => new Promise<void>((r) => setTimeout(r, ms)));
+ const now = opts.now ?? (() => new Date());
+
+ const attempt = async <T>(fn: () => Promise<T>): Promise<T> => {
+ let lastErr: unknown;
+ for (let i = 1; i <= maxAttempts; i++) {
+ try {
+ return await fn();
+ } catch (err) {
+ lastErr = err;
+ if (i < maxAttempts) await sleep(retryDelayMs * i);
+ }
+ }
+ throw lastErr;
+ };
+
+ let progress: GraphCompactBootProgress | null = null;
+ const save = async () => {
+ if (!progress) return;
+ progress.updatedAt = now().toISOString();
+ current = statusFrom(progress);
+ await attempt(() => kv.set(KV.config, GRAPH_COMPACT_BOOT_KEY, progress));
+ };
+
+ try {
+ const stored = await attempt(() => kv.get<GraphCompactBootProgress>(KV.config, GRAPH_COMPACT_BOOT_KEY));
+ if (isResumable(stored) && stored.status === "done") {
+ current = statusFrom(stored);
+ return getGraphCompactBootStatus();
+ }
+ const startedAt = now().toISOString();
+ if (isResumable(stored)) {
+ progress = { ...stored, status: "running", error: undefined };
+ log(`Graph provenance compaction: resuming in the background at ${progress.phase} ${progress.offset}`);
+ } else {
+ progress = {
+ version: GRAPH_COMPACT_BOOT_VERSION,
+ cap: MAX_GRAPH_SOURCE_OBSERVATIONS,
+ status: "running",
+ phase: "nodes",
+ offset: 0,
+ phaseTotal: null,
+ scanned: 0,
+ trimmed: 0,
+ idsRemoved: 0,
+ startedAt,
+ updatedAt: startedAt,
+ };
+ log(`Graph provenance compaction: started in the background (cap ${MAX_GRAPH_SOURCE_OBSERVATIONS} ids per record)`);
+ }
+ await save();
+
+ const p = progress;
+ for (const scope of SLICED_SCOPES) {
+ if (p.phase === "snapshot") break;
+ if (SLICED_SCOPES.indexOf(p.phase) > SLICED_SCOPES.indexOf(scope)) continue;
+ p.phase = scope;
+ let phaseTrimmed = 0;
+ let phaseScanned = 0;
+ for (;;) {
+ const slice = { scope: scope as GraphCompactScope, offset: p.offset, limit: sliceSize };
+ let plan = storedPlan(p, scope);
+ if (!plan) {
+ const r = await attempt(() => compactGraphProvenance(kv, { ...slice, dryRun: true }));
+ const [scanned, trimmed] = scannedOf(scope, r);
+ plan = {
+ phase: scope,
+ offset: p.offset,
+ scanned,
+ trimmed,
+ idsRemoved: r.idsRemoved,
+ nextOffset: r.nextOffset,
+ total: r.total ?? null,
+ };
+ if (plan.trimmed > 0) {
+ p.pending = plan;
+ await save();
+ }
+ }
+ if (plan.trimmed > 0) await attempt(() => compactGraphProvenance(kv, slice));
+ phaseScanned += plan.scanned;
+ phaseTrimmed += plan.trimmed;
+ p.scanned += plan.scanned;
+ p.trimmed += plan.trimmed;
+ p.idsRemoved += plan.idsRemoved;
+ p.pending = undefined;
+ p.phaseTotal = plan.total;
+ p.offset = plan.nextOffset ?? p.phaseTotal ?? p.offset;
+ await save();
+ if (plan.nextOffset === null) break;
+ if (pauseMs > 0) await sleep(pauseMs);
+ }
+ log(`Graph provenance compaction: ${scope} done (${phaseScanned} checked, ${phaseTrimmed} trimmed)`);
+ const next = SLICED_SCOPES[SLICED_SCOPES.indexOf(scope) + 1];
+ p.phase = next ?? "snapshot";
+ p.offset = 0;
+ p.phaseTotal = null;
+ await save();
+ }
+
+ let snapPlan = storedPlan(p, "snapshot");
+ if (!snapPlan) {
+ const dry = await attempt(() => compactGraphProvenance(kv, { scope: "snapshot", dryRun: true }));
+ snapPlan = {
+ phase: "snapshot",
+ offset: p.offset,
+ scanned: 0,
+ trimmed: dry.snapshotTrimmed ? 1 : 0,
+ idsRemoved: 0,
+ nextOffset: null,
+ total: null,
+ };
+ if (snapPlan.trimmed > 0) {
+ p.pending = snapPlan;
+ await save();
+ }
+ }
+ if (snapPlan.trimmed > 0) await attempt(() => compactGraphProvenance(kv, { scope: "snapshot" }));
+ const snapshotTrimmed = snapPlan.trimmed > 0;
+ p.trimmed += snapPlan.trimmed;
+ p.pending = undefined;
+ p.status = "done";
+ p.completedAt = now().toISOString();
+ await save();
+ if (p.idsRemoved > 0) {
+ await safeAudit(kv, "graph_compact", GRAPH_COMPACT_BOOT_FUNCTION_ID, [], {
+ trigger: "boot",
+ recordsTrimmed: p.trimmed,
+ idsRemoved: p.idsRemoved,
+ snapshotTrimmed,
+ });
+ }
+ log(
+ `Graph provenance compaction: done (${p.scanned} records checked, ${p.trimmed} trimmed, ${p.idsRemoved} ids removed)`,
+ );
+ return getGraphCompactBootStatus();
+ } catch (err) {
+ const msg = err instanceof Error ? err.message : String(err);
+ if (progress) {
+ progress.status = "failed";
+ progress.error = msg;
+ current = statusFrom(progress);
+ try {
+ await kv.set(KV.config, GRAPH_COMPACT_BOOT_KEY, { ...progress, updatedAt: now().toISOString() });
+ } catch {}
+ } else {
+ current = { state: "failed", error: msg };
+ }
+ warn(
+ `Graph provenance compaction failed (${describeGraphCompactBoot(current)}). It resumes on the next start; to run it now, POST /agentmemory/graph/compact.`,
+ );
+ return getGraphCompactBootStatus();
+ }
+}
diff --git a/src/functions/graph.ts b/src/functions/graph.ts
index cf551727..03b840d0 100644
--- a/src/functions/graph.ts
+++ b/src/functions/graph.ts
@@ -726,21 +726,24 @@ export interface GraphCompactResult {
nodesTrimmed: number;
edgesScanned: number;
edgesTrimmed: number;
+ historyScanned: number;
+ historyTrimmed: number;
idsRemoved: number;
snapshotTrimmed: boolean;
total?: number;
nextOffset: number | null;
}
-export type GraphCompactScope = "nodes" | "edges" | "snapshot";
+export type GraphCompactScope = "nodes" | "edges" | "history" | "snapshot";
export interface GraphCompactOptions {
scope?: GraphCompactScope;
offset?: number;
limit?: number;
+ dryRun?: boolean;
}
-const COMPACT_SCOPES: readonly GraphCompactScope[] = ["nodes", "edges", "snapshot"];
+export const COMPACT_SCOPES: readonly GraphCompactScope[] = ["nodes", "edges", "history", "snapshot"];
export async function compactGraphProvenance(
kv: StateKV,
@@ -751,6 +754,7 @@ export async function compactGraphProvenance(
throw new Error(`unknown compact scope: ${String(scope)}`);
}
const offset = opts.offset ?? 0;
+ const dryRun = opts.dryRun === true;
const limit = opts.limit ?? Number.POSITIVE_INFINITY;
if (!Number.isInteger(offset) || offset < 0) {
throw new Error(`compact offset must be a non-negative integer: ${String(opts.offset)}`);
@@ -764,16 +768,18 @@ export async function compactGraphProvenance(
nodesTrimmed: 0,
edgesScanned: 0,
edgesTrimmed: 0,
+ historyScanned: 0,
+ historyTrimmed: 0,
idsRemoved: 0,
snapshotTrimmed: false,
nextOffset: null,
};
const trimScope = async <R extends { sourceObservationIds: string[] }>(
- indexScope: string,
+ listIds: () => Promise<unknown[]>,
recordScope: string,
): Promise<{ scanned: number; trimmed: number }> => {
- const allIds = [...new Set(await kv.list<string>(indexScope))]
+ const allIds = [...new Set(await listIds())]
.filter((id): id is string => typeof id === "string")
.sort();
const end = Math.min(allIds.length, offset + limit);
@@ -790,7 +796,7 @@ export async function compactGraphProvenance(
if (sources.length <= MAX_GRAPH_SOURCE_OBSERVATIONS) return;
const bounded = boundSources([], sources);
result.idsRemoved += sources.length - bounded.length;
- await kv.set(recordScope, id, { ...record, sourceObservationIds: bounded });
+ if (!dryRun) await kv.set(recordScope, id, { ...record, sourceObservationIds: bounded });
trimmed += 1;
});
}
@@ -810,21 +816,29 @@ export async function compactGraphProvenance(
});
const topNodes = trimList(snap.topNodes);
const topEdges = trimList(snap.topEdges);
- if (result.snapshotTrimmed) {
+ if (result.snapshotTrimmed && !dryRun) {
await kv.set(KV.graphSnapshot, SNAPSHOT_KEY, { ...snap, topNodes, topEdges });
}
});
if (scope === undefined || scope === "nodes") {
- const n = await trimScope<GraphNode>(KV.graphNameIndex, KV.graphNodes);
+ const n = await trimScope<GraphNode>(() => kv.list<string>(KV.graphNameIndex), KV.graphNodes);
result.nodesScanned = n.scanned;
result.nodesTrimmed = n.trimmed;
}
if (scope === undefined || scope === "edges") {
- const e = await trimScope<GraphEdge>(KV.graphEdgeKey, KV.graphEdges);
+ const e = await trimScope<GraphEdge>(() => kv.list<string>(KV.graphEdgeKey), KV.graphEdges);
result.edgesScanned = e.scanned;
result.edgesTrimmed = e.trimmed;
}
+ if (scope === undefined || scope === "history") {
+ const h = await trimScope<GraphEdge>(
+ async () => (await kv.list<GraphEdge>(KV.graphEdgeHistory)).map((r) => r?.id),
+ KV.graphEdgeHistory,
+ );
+ result.historyScanned = h.scanned;
+ result.historyTrimmed = h.trimmed;
+ }
if (scope === undefined || scope === "snapshot") {
await trimSnapshot();
}
@@ -1273,6 +1287,7 @@ export function registerGraphFunction(
limit: data?.limit,
nodesTrimmed: result.nodesTrimmed,
edgesTrimmed: result.edgesTrimmed,
+ historyTrimmed: result.historyTrimmed,
idsRemoved: result.idsRemoved,
snapshotTrimmed: result.snapshotTrimmed,
});
diff --git a/src/functions/status.ts b/src/functions/status.ts
index 3e4d8af6..5e0b3537 100644
--- a/src/functions/status.ts
+++ b/src/functions/status.ts
@@ -1,5 +1,6 @@
import type { IndexLegStatus, IndexPersistenceStatus } from "../state/index-persistence.js";
import type { VectorBackfillState } from "./search.js";
+import { describeGraphCompactBoot, type GraphCompactBootStatus } from "./graph-compact-boot.js";
export type StatusLevel = "ok" | "info" | "warn" | "error";
@@ -71,6 +72,7 @@ export interface StatusInputs {
};
graph: GraphStatsInput | null;
graphExtractionEnabled: boolean;
+ graphCompaction?: GraphCompactBootStatus | null;
auditLegacy: { status: string; sizeBytes?: number } | null;
indexPersistence?: IndexPersistenceStatus | null;
stateStore?: { ok: boolean; latencyMs?: number } | null;
@@ -104,6 +106,7 @@ export interface StatusReport {
index: StatusInputs["index"] & { breakdown: IndexBreakdown | null };
indexPersistence: IndexPersistenceStatus | null;
graph: (GraphStatsInput & { ageSeconds: number | null; extractionEnabled: boolean }) | null;
+ graphCompaction: GraphCompactBootStatus | null;
functions: Array<FunctionMetricInput & { failureRate: number; offWithoutLlm: boolean }>;
flags: Array<StatusFlag & { inactiveReason?: string }>;
problems: StatusProblem[];
@@ -385,6 +388,23 @@ export function evaluateStatus(input: StatusInputs): StatusReport {
}
}
+ const compaction = input.graphCompaction ?? null;
+ if (compaction?.state === "running") {
+ problems.push({
+ level: "info",
+ code: "graph-compaction-running",
+ message: `Oversized graph provenance is being trimmed in the background (${describeGraphCompactBoot(compaction)}). Search and capture keep working.`,
+ fix: "No action needed. It runs once, resumes after a restart and stops by itself. AGENTMEMORY_GRAPH_COMPACT_ON_BOOT=false turns it off.",
+ });
+ } else if (compaction?.state === "failed") {
+ problems.push({
+ level: "warn",
+ code: "graph-compaction-failed",
+ message: `The one-time graph provenance compaction stopped: ${compaction.error ?? "unknown error"}. Graph search can stay slow until it finishes.`,
+ fix: `It resumes on the next start. To run it now: ${graphCompactCommand(input.ports.rest)}`,
+ });
+ }
+
if (input.auditLegacy?.status === "too-large" || input.auditLegacy?.status === "unreadable") {
const size = input.auditLegacy.sizeBytes;
const sizeText = typeof size === "number" ? ` (${Math.round(size / (1024 * 1024))} MiB)` : "";
@@ -441,6 +461,7 @@ export function evaluateStatus(input: StatusInputs): StatusReport {
index: { ...input.index, breakdown: indexBreakdown(input.index) },
indexPersistence: persistence,
graph,
+ graphCompaction: compaction,
functions,
flags: input.flags.map((flag) =>
flag.enabled && flag.needsLlm && noLlm
@@ -686,6 +707,7 @@ ${graph
row("Nodes / edges", escapeHtml(`${graph.totalNodes ?? 0} / ${graph.totalEdges ?? 0}`)) +
row("Snapshot age", escapeHtml(graph.fromSnapshot ? formatDuration(graph.ageSeconds) : "no snapshot"))
: row("Graph", "unavailable")}
+${report.graphCompaction ? row("Provenance compaction", escapeHtml(describeGraphCompactBoot(report.graphCompaction))) : ""}
</table>
<h2>Features</h2><table class="list"><tr><th>Feature</th><th>Setting</th><th>State</th><th>How to change</th></tr>${flags}</table>
<h2>Process</h2><table>
@@ -696,6 +718,10 @@ ${processRows(report)}
</main></body></html>`;
}
+export function graphCompactCommand(restPort: number | null): string {
+ return `curl -X POST http://localhost:${restPort ?? 3111}/agentmemory/graph/compact -H "Content-Type: application/json" -d '{}'`;
+}
+
export const UNINDEXED_SCAN_REUSE_MS = 30_000;
export function singleFlight<T>(
diff --git a/src/index.ts b/src/index.ts
index 5786de4c..7d3726c6 100644
--- a/src/index.ts
+++ b/src/index.ts
@@ -17,6 +17,7 @@ import {
getAuditRetentionMonths,
getStateBackend,
isSessionSweepEnabled,
+ isGraphCompactOnBootEnabled,
getSessionSweepStaleHours,
} from "./config.js";
import {
@@ -72,6 +73,7 @@ import { registerExportImportFunction } from "./functions/export-import.js";
import { registerEnrichFunction } from "./functions/enrich.js";
import { registerClaudeBridgeFunction } from "./functions/claude-bridge.js";
import { registerGraphFunction } from "./functions/graph.js";
+import { GRAPH_COMPACT_BOOT_DELAY_MS, runGraphCompactOnBoot, setGraphCompactBootDisabled } from "./functions/graph-compact-boot.js";
import { registerGraphImportFunction } from "./functions/graph-import.js";
import { registerConsolidationPipelineFunction } from "./functions/consolidation-pipeline.js";
import { registerTeamFunction } from "./functions/team.js";
@@ -117,7 +119,7 @@ import { registerHealthMonitor } from "./health/monitor.js";
import { createStreamRelayProbe } from "./health/stream-relay-probe.js";
import { initMetrics, OTEL_CONFIG } from "./telemetry/setup.js";
import { VERSION } from "./version.js";
-import { bootLog } from "./logger.js";
+import { bootLog, bootWarn } from "./logger.js";
import { runtimeMetadataPath } from "./runtime-paths.js";
import { mkdirSync, writeFileSync, unlinkSync } from "node:fs";
import { dirname } from "node:path";
@@ -561,6 +563,15 @@ async function main() {
config.restPort,
);
+ if (isGraphCompactOnBootEnabled()) {
+ const graphCompactTimer = setTimeout(() => {
+ void runGraphCompactOnBoot(kv, { log: bootLog, warn: bootWarn }).catch(() => {});
+ }, GRAPH_COMPACT_BOOT_DELAY_MS);
+ graphCompactTimer.unref();
+ } else {
+ setGraphCompactBootDisabled();
+ }
+
const autoForgetIntervalMs = parseInt(process.env.AUTO_FORGET_INTERVAL_MS || "3600000", 10);
const consolidationIntervalMs = getConsolidationIntervalMs();
diff --git a/src/triggers/api.ts b/src/triggers/api.ts
index c10868aa..ff8261bd 100644
--- a/src/triggers/api.ts
+++ b/src/triggers/api.ts
@@ -42,6 +42,8 @@ import { renderViewerDocument } from "../viewer/document.js";
import { getBoundViewerPort, getViewerSkipped } from "../viewer/server.js";
import { MAX_FILES_UPPER_BOUND } from "../functions/replay.js";
import { describeGraphNode } from "../functions/graph-node.js";
+import { COMPACT_SCOPES, type GraphCompactScope } from "../functions/graph.js";
+import { getGraphCompactBootStatus } from "../functions/graph-compact-boot.js";
import { LESSON_SOURCE_IDS_MAX, normalizeLessonSourceIds } from "../functions/lessons.js";
import { logger } from "../logger.js";
import { withoutObservationSource } from "../functions/observation-source.js";
@@ -402,6 +404,7 @@ export function createStatusReporter(sdk: IIIClient, kv: StateKV, deps: StatusRe
vectorBackfillState: getVectorBackfillState(),
},
graph,
+ graphCompaction: getGraphCompactBootStatus(),
graphExtractionEnabled: isGraphExtractionEnabled(),
auditLegacy: auditMigrationState
? { status: auditMigrationState.status, sizeBytes: auditMigrationState.legacySizeBytes }
@@ -2033,8 +2036,8 @@ export function registerApiTriggers(
if (authErr) return authErr;
const body = (req.body ?? {}) as Record<string, unknown>;
const { scope, offset, limit } = body;
- if (scope !== undefined && scope !== "nodes" && scope !== "edges" && scope !== "snapshot") {
- return { status_code: 400, body: { error: "scope must be nodes, edges or snapshot" } };
+ if (scope !== undefined && !COMPACT_SCOPES.includes(scope as GraphCompactScope)) {
+ return { status_code: 400, body: { error: "scope must be nodes, edges, history or snapshot" } };
}
if (offset !== undefined && !(Number.isInteger(offset) && (offset as number) >= 0)) {
return { status_code: 400, body: { error: "offset must be a non-negative integer" } };
diff --git a/src/viewer/index.html b/src/viewer/index.html
index dd8c0949..fe6484fc 100644
--- a/src/viewer/index.html
+++ b/src/viewer/index.html
@@ -7168,9 +7168,22 @@
html += hrow('Vector save', esc(legText(p.vector)), null, 'Saved at most every ' + esc(durationText(p.saveIntervalMs || 600000)) + ' (AGENTMEMORY_INDEX_SAVE_INTERVAL_MS). ' + esc(fmtNum(p.buckets) + ' bucket' + (p.buckets === 1 ? '' : 's') + ', ' + fmtNum(p.pendingChanges) + ' unsaved change' + (p.pendingChanges === 1 ? '' : 's') + (p.saving ? ', saving now' : '') + '.'));
}
html += hrow('Graph', esc(fmtNum(c.graphNodes) + ' nodes · ' + fmtNum(c.graphEdges) + ' edges'), 'graph', store.graph && store.graph.updatedAt ? 'Snapshot rebuilt ' + esc(timeAgo(store.graph.updatedAt)) + '.' : '');
+ if (r && r.graphCompaction) html += graphCompactionRowHtml(r.graphCompaction);
return html + '</div>';
}
+ function graphCompactionRowHtml(g) {
+ var where = g.phase === 'snapshot' || g.total === null || g.total === undefined ? (g.phase || 'nodes') : g.phase + ' ' + fmtNum(g.processed || 0) + ' of ' + fmtNum(g.total);
+ var text = g.state === 'off' ? 'off' : g.state === 'pending' ? 'pending' : g.state === 'running' ? 'running, ' + where : g.state === 'failed' ? 'failed at ' + where : 'done';
+ var note = 'A one-time pass that trims each graph node and edge to its newest 32 source observations, so stores from older versions stop slowing graph search.';
+ if (g.state === 'done') note = fmtNum(g.trimmed || 0) + ' records trimmed, ' + fmtNum(g.idsRemoved || 0) + ' ids removed' + (g.completedAt ? ', finished ' + timeAgo(g.completedAt) : '') + '. It does not run again.';
+ else if (g.state === 'running') note += ' It runs in the background, resumes after a restart and stops by itself.';
+ else if (g.state === 'off') note = 'Turned off with AGENTMEMORY_GRAPH_COMPACT_ON_BOOT=false. Run it by hand if graph search is slow:';
+ else if (g.state === 'failed') note = esc(g.error || 'unknown error') + '. It resumes on the next start. To run it now:';
+ var cmd = g.state === 'off' || g.state === 'failed' ? cmdBlock(curlPost('graph/compact', {})) : '';
+ return hrow('Provenance compaction', esc(text), null, (g.state === 'failed' ? note : esc(note)) + cmd);
+ }
+
function llmCardHtml(r) {
var cb = store.circuitBreaker;
var noLlm = store.provider === 'noop' || (r && r.provider.llm === 'noop');
diff --git a/test/graph-compact-boot.test.ts b/test/graph-compact-boot.test.ts
new file mode 100644
index 00000000..bd1f88de
--- /dev/null
+++ b/test/graph-compact-boot.test.ts
@@ -0,0 +1,393 @@
+import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
+import { readFileSync } from "node:fs";
+
+vi.mock("../src/logger.js", () => ({
+ logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn() },
+}));
+
+import {
+ GRAPH_COMPACT_BOOT_KEY,
+ GRAPH_COMPACT_BOOT_VERSION,
+ describeGraphCompactBoot,
+ getGraphCompactBootStatus,
+ resetGraphCompactBootStatus,
+ runGraphCompactOnBoot,
+ setGraphCompactBootDisabled,
+ type GraphCompactBootProgress,
+} from "../src/functions/graph-compact-boot.js";
+import { MAX_GRAPH_SOURCE_OBSERVATIONS, compactGraphProvenance } from "../src/functions/graph.js";
+import { isGraphCompactOnBootEnabled } from "../src/config.js";
+import { evaluateStatus, renderStatusHtml, type StatusInputs } from "../src/functions/status.js";
+import type { GraphEdge, GraphNode } from "../src/types.js";
+
+const NODES = "mem:graph:nodes";
+const EDGES = "mem:graph:edges";
+const HISTORY = "mem:graph:edge-history";
+const CONFIG = "mem:config";
+
+function mockKV(opts: { shuffle?: boolean; store?: Map<string, Map<string, unknown>> } = {}) {
+ const store = opts.store ?? new Map<string, Map<string, unknown>>();
+ const calls = { list: [] as string[], get: [] as string[], set: [] as string[] };
+ const kv = {
+ calls,
+ store,
+ failGet: null as ((scope: string, key: string) => boolean) | null,
+ get: async <T>(scope: string, key: string): Promise<T | null> => {
+ calls.get.push(`${scope}/${key}`);
+ if (kv.failGet?.(scope, key)) throw new Error("state::get timed out");
+ const v = store.get(scope)?.get(key);
+ return (v === undefined ? null : structuredClone(v)) as T | null;
+ },
+ set: async <T>(scope: string, key: string, data: T): Promise<T> => {
+ calls.set.push(`${scope}/${key}`);
+ if (!store.has(scope)) store.set(scope, new Map());
+ store.get(scope)!.set(key, structuredClone(data));
+ return data;
+ },
+ delete: async (scope: string, key: string): Promise<void> => {
+ store.get(scope)?.delete(key);
+ },
+ list: async <T>(scope: string): Promise<T[]> => {
+ calls.list.push(scope);
+ const values = Array.from(store.get(scope)?.values() ?? []).map((v) => structuredClone(v));
+ if (opts.shuffle) {
+ for (let i = values.length - 1; i > 0; i--) {
+ const j = Math.floor(Math.random() * (i + 1));
+ [values[i], values[j]] = [values[j], values[i]];
+ }
+ }
+ return values as T[];
+ },
+ };
+ return kv;
+}
+type KV = ReturnType<typeof mockKV>;
+
+const ids = (prefix: string, n: number) => Array.from({ length: n }, (_, i) => `${prefix}_${i}`);
+
+async function seed(kv: KV, nodes: number, edges: number, history: number, bloat = 100) {
+ for (let i = 0; i < nodes; i++) {
+ const id = `gn_${String(i).padStart(4, "0")}`;
+ const n: GraphNode = {
+ id,
+ type: "concept",
+ name: `n${i}`,
+ properties: {},
+ sourceObservationIds: ids(`obs${i}`, i % 2 === 0 ? bloat : 3),
+ createdAt: "2026-09-01T00:00:00.000Z",
+ };
+ await kv.set(NODES, id, n);
+ await kv.set("mem:graph:name-index", `concept|n${i}`, id);
+ }
+ for (let i = 0; i < edges; i++) {
+ const id = `ge_${String(i).padStart(4, "0")}`;
+ const e: GraphEdge = {
+ id,
+ type: "related_to",
+ sourceNodeId: "gn_0000",
+ targetNodeId: `gn_${i}`,
+ weight: 0.5,
+ sourceObservationIds: ids(`eobs${i}`, bloat),
+ createdAt: "2026-09-01T00:00:00.000Z",
+ };
+ await kv.set(EDGES, id, e);
+ await kv.set("mem:graph:edge-key", `gn_0000|gn_${i}|related_to`, id);
+ }
+ for (let i = 0; i < history; i++) {
+ const id = `ge_h${String(i).padStart(4, "0")}`;
+ await kv.set(HISTORY, id, {
+ id,
+ type: "uses",
+ sourceNodeId: "gn_0000",
+ targetNodeId: "gn_0001",
+ weight: 0.5,
+ sourceObservationIds: ids(`hobs${i}`, bloat),
+ createdAt: "2026-09-01T00:00:00.000Z",
+ isLatest: false,
+ });
+ }
+ await kv.set("mem:graph:snapshot", "current", {
+ version: 1,
+ dirty: false,
+ topNodes: [await kv.get(NODES, "gn_0000")],
+ topEdges: [],
+ topDegrees: {},
+ stats: { totalNodes: nodes, totalEdges: edges, nodesByType: {}, edgesByType: {} },
+ updatedAt: "2026-09-01T00:00:00.000Z",
+ });
+}
+
+async function maxIds(kv: KV): Promise<number> {
+ let max = 0;
+ for (const scope of [NODES, EDGES, HISTORY]) {
+ for (const r of await kv.list<{ sourceObservationIds: string[] }>(scope)) {
+ max = Math.max(max, r.sourceObservationIds.length);
+ }
+ }
+ const snap = (await kv.get<{ topNodes: GraphNode[] }>("mem:graph:snapshot", "current"))!;
+ for (const n of snap.topNodes) max = Math.max(max, n.sourceObservationIds.length);
+ return max;
+}
+
+const fast = { sliceSize: 3, pauseMs: 0, retryDelayMs: 0, sleep: async () => {} };
+const marker = (kv: KV) => kv.get<GraphCompactBootProgress>(CONFIG, GRAPH_COMPACT_BOOT_KEY);
+
+beforeEach(() => resetGraphCompactBootStatus());
+afterEach(() => vi.unstubAllEnvs());
+
+describe("graph compaction on boot", () => {
+ it("trims nodes, edges, edge history and the snapshot to the cap and marks it done", async () => {
+ const kv = mockKV();
+ await seed(kv, 10, 7, 5);
+ const log = vi.fn();
+ const s = await runGraphCompactOnBoot(kv as never, { ...fast, log });
+ const listed = [...kv.calls.list];
+ expect(s.state).toBe("done");
+ expect(await maxIds(kv)).toBe(MAX_GRAPH_SOURCE_OBSERVATIONS);
+ expect(s.trimmed).toBe(5 + 7 + 5 + 1);
+ expect(s.idsRemoved).toBe(17 * (100 - MAX_GRAPH_SOURCE_OBSERVATIONS));
+ const m = (await marker(kv))!;
+ expect(m).toMatchObject({ status: "done", version: GRAPH_COMPACT_BOOT_VERSION, cap: MAX_GRAPH_SOURCE_OBSERVATIONS });
+ expect(log.mock.calls.map((c) => c[0]).at(-1)).toMatch(/done \(22 records checked, 18 trimmed/);
+ expect(listed).not.toContain(NODES);
+ expect(listed).not.toContain(EDGES);
+ });
+
+ it("finishes fast on a store with nothing over the cap", async () => {
+ const kv = mockKV();
+ await seed(kv, 4, 2, 1, 10);
+ kv.calls.set.length = 0;
+ const s = await runGraphCompactOnBoot(kv as never, fast);
+ expect(s).toMatchObject({ state: "done", trimmed: 0, idsRemoved: 0 });
+ expect(kv.calls.set.filter((c) => !c.startsWith(CONFIG))).toEqual([]);
+ });
+
+ it("resumes from the saved offset after the process dies mid-run", async () => {
+ const kv = mockKV();
+ await seed(kv, 12, 6, 0);
+ kv.calls.get.length = 0;
+ let pauses = 0;
+ const hang = new Promise<void>(() => {});
+ void runGraphCompactOnBoot(kv as never, {
+ ...fast,
+ pauseMs: 1,
+ sleep: async () => {
+ if (++pauses === 2) await hang;
+ },
+ });
+ await vi.waitFor(async () => expect((await marker(kv))?.offset).toBe(6));
+ const saved = (await marker(kv))!;
+ expect(saved).toMatchObject({ status: "running", phase: "nodes", offset: 6 });
+ const nodeReadsBefore = kv.calls.get.filter((c) => c.startsWith(`${NODES}/`)).length;
+ expect(nodeReadsBefore).toBe(12);
+
+ resetGraphCompactBootStatus();
+ kv.calls.get.length = 0;
+ const log = vi.fn();
+ const s = await runGraphCompactOnBoot(kv as never, { ...fast, log });
+ expect(log.mock.calls[0]![0]).toMatch(/resuming in the background at nodes 6/);
+ expect(kv.calls.get.filter((c) => c.startsWith(`${NODES}/`))).toHaveLength(12);
+ expect(s.state).toBe("done");
+ expect(s.trimmed).toBe(6 + 6 + 1);
+ expect(await maxIds(kv)).toBe(MAX_GRAPH_SOURCE_OBSERVATIONS);
+ });
+
+ it("reports the same totals as a clean run when the last progress write is lost at any point", async () => {
+ const clean = mockKV();
+ await seed(clean, 10, 7, 5);
+ const want = await runGraphCompactOnBoot(clean as never, fast);
+ const totals = (s: { scanned?: number; trimmed?: number; idsRemoved?: number }) => ({
+ scanned: s.scanned,
+ trimmed: s.trimmed,
+ idsRemoved: s.idsRemoved,
+ });
+ expect(totals(want)).toEqual({ scanned: 22, trimmed: 18, idsRemoved: 17 * (100 - MAX_GRAPH_SOURCE_OBSERVATIONS) });
+
+ const probe = mockKV();
+ await seed(probe, 10, 7, 5);
+ const cuts: Map<string, Map<string, unknown>>[] = [];
+ const set = probe.set;
+ probe.set = async <T>(scope: string, key: string, data: T): Promise<T> => {
+ if (scope === CONFIG && key === GRAPH_COMPACT_BOOT_KEY) cuts.push(structuredClone(probe.store));
+ return set(scope, key, data);
+ };
+ resetGraphCompactBootStatus();
+ await runGraphCompactOnBoot(probe as never, fast);
+ expect(cuts.length).toBeGreaterThan(20);
+
+ for (const store of cuts) {
+ const kv = mockKV({ store });
+ resetGraphCompactBootStatus();
+ const s = await runGraphCompactOnBoot(kv as never, fast);
+ expect(s.state).toBe("done");
+ expect(totals(s)).toEqual(totals(want));
+ expect(await maxIds(kv)).toBe(MAX_GRAPH_SOURCE_OBSERVATIONS);
+ }
+ });
+
+ it("is idempotent: a second full pass trims nothing", async () => {
+ const kv = mockKV();
+ await seed(kv, 6, 4, 3);
+ await runGraphCompactOnBoot(kv as never, fast);
+ const r = await compactGraphProvenance(kv as never);
+ expect(r).toMatchObject({ nodesTrimmed: 0, edgesTrimmed: 0, historyTrimmed: 0, idsRemoved: 0, snapshotTrimmed: false });
+ });
+
+ it("never runs again once the completion marker is set", async () => {
+ const kv = mockKV();
+ await seed(kv, 4, 2, 1);
+ await runGraphCompactOnBoot(kv as never, fast);
+ resetGraphCompactBootStatus();
+ kv.calls.list.length = 0;
+ kv.calls.set.length = 0;
+ const log = vi.fn();
+ const s = await runGraphCompactOnBoot(kv as never, { ...fast, log });
+ expect(s.state).toBe("done");
+ expect(kv.calls.list).toEqual([]);
+ expect(kv.calls.set).toEqual([]);
+ expect(log).not.toHaveBeenCalled();
+ });
+
+ it("runs again when the marker was written for a different cap or version", async () => {
+ const kv = mockKV();
+ await seed(kv, 4, 0, 0);
+ await kv.set(CONFIG, GRAPH_COMPACT_BOOT_KEY, {
+ version: GRAPH_COMPACT_BOOT_VERSION,
+ cap: MAX_GRAPH_SOURCE_OBSERVATIONS * 2,
+ status: "done",
+ phase: "snapshot",
+ offset: 0,
+ });
+ const s = await runGraphCompactOnBoot(kv as never, fast);
+ expect(s.state).toBe("done");
+ expect(s.trimmed).toBeGreaterThan(0);
+ expect((await marker(kv))!.cap).toBe(MAX_GRAPH_SOURCE_OBSERVATIONS);
+ });
+
+ it("covers every record when the store lists in a different order on every call", async () => {
+ const kv = mockKV({ shuffle: true });
+ await seed(kv, 40, 30, 20);
+ const s = await runGraphCompactOnBoot(kv as never, { ...fast, sliceSize: 2 });
+ expect(s.state).toBe("done");
+ expect(await maxIds(kv)).toBe(MAX_GRAPH_SOURCE_OBSERVATIONS);
+ });
+
+ it("retries a failing slice a bounded number of times, then reports and keeps the worker alive", async () => {
+ const kv = mockKV();
+ await seed(kv, 6, 0, 0);
+ kv.failGet = (scope, key) => scope === NODES && key === "gn_0004";
+ const warn = vi.fn();
+ const s = await runGraphCompactOnBoot(kv as never, { ...fast, maxAttempts: 2, warn });
+ expect(s).toMatchObject({ state: "failed", phase: "nodes", processed: 3, error: "state::get timed out" });
+ expect(kv.calls.get.filter((c) => c === `${NODES}/gn_0004`)).toHaveLength(2);
+ expect(warn).toHaveBeenCalledTimes(1);
+ expect(warn.mock.calls[0]![0]).toMatch(/failed at nodes 3 of 6: state::get timed out.*POST \/agentmemory\/graph\/compact/);
+ expect(await marker(kv)).toMatchObject({ status: "failed", offset: 3, error: "state::get timed out" });
+
+ kv.failGet = null;
+ resetGraphCompactBootStatus();
+ const again = await runGraphCompactOnBoot(kv as never, fast);
+ expect(again.state).toBe("done");
+ expect(await maxIds(kv)).toBe(MAX_GRAPH_SOURCE_OBSERVATIONS);
+ });
+
+ it("does not block the caller and runs at most once at a time", async () => {
+ const kv = mockKV();
+ await seed(kv, 6, 0, 0);
+ let release!: () => void;
+ const gate = new Promise<void>((r) => (release = r));
+ const first = runGraphCompactOnBoot(kv as never, { ...fast, pauseMs: 1, sleep: () => gate });
+ const second = runGraphCompactOnBoot(kv as never, fast);
+ expect(second).toBe(first);
+ await vi.waitFor(() => expect(getGraphCompactBootStatus().state).toBe("running"));
+ expect(describeGraphCompactBoot(getGraphCompactBootStatus())).toBe("running, nodes 3 of 6");
+ release();
+ expect((await first).state).toBe("done");
+ });
+
+ it("is scheduled after the viewer starts and never awaited during boot", () => {
+ const src = readFileSync(new URL("../src/index.ts", import.meta.url), "utf8");
+ const viewerAt = src.indexOf("startViewerServer(\n");
+ const runAt = src.indexOf("void runGraphCompactOnBoot(");
+ expect(viewerAt).toBeGreaterThan(0);
+ expect(runAt).toBeGreaterThan(viewerAt);
+ expect(src).not.toMatch(/await runGraphCompactOnBoot/);
+ expect(src).toMatch(/if \(isGraphCompactOnBootEnabled\(\)\)[\s\S]*?setGraphCompactBootDisabled\(\)/);
+ });
+});
+
+describe("AGENTMEMORY_GRAPH_COMPACT_ON_BOOT", () => {
+ it("is on by default and off only for false", () => {
+ vi.stubEnv("AGENTMEMORY_GRAPH_COMPACT_ON_BOOT", "");
+ expect(isGraphCompactOnBootEnabled()).toBe(true);
+ vi.stubEnv("AGENTMEMORY_GRAPH_COMPACT_ON_BOOT", "true");
+ expect(isGraphCompactOnBootEnabled()).toBe(true);
+ vi.stubEnv("AGENTMEMORY_GRAPH_COMPACT_ON_BOOT", "false");
+ expect(isGraphCompactOnBootEnabled()).toBe(false);
+ });
+
+ it("reports off when disabled", () => {
+ setGraphCompactBootDisabled();
+ expect(getGraphCompactBootStatus()).toEqual({ state: "off" });
+ expect(describeGraphCompactBoot(getGraphCompactBootStatus())).toMatch(/AGENTMEMORY_GRAPH_COMPACT_ON_BOOT=false/);
+ });
+});
+
+describe("graph compaction in /agentmemory/status", () => {
+ const base: StatusInputs = {
+ now: new Date("2026-10-01T12:00:00.000Z"),
+ version: "0.10.0",
+ engineVersion: "0.22.1",
+ uptimeSeconds: 10,
+ stateBackend: "file",
+ ports: { rest: 4611, streams: 4612, viewer: 4613 },
+ health: null,
+ circuitBreaker: null,
+ functionMetrics: [],
+ provider: "llm",
+ embeddingProvider: "none",
+ flags: [],
+ index: {
+ bm25Documents: 0,
+ vectorDocuments: null,
+ observationsIndexed: 0,
+ missingObservations: 0,
+ sessions: 0,
+ bm25Incomplete: false,
+ pendingVectorBackfill: 0,
+ },
+ graph: null,
+ graphExtractionEnabled: false,
+ auditLegacy: null,
+ };
+
+ it("shows a running pass as info with its position", () => {
+ const r = evaluateStatus({
+ ...base,
+ graphCompaction: { state: "running", phase: "edges", processed: 400, total: 909 },
+ });
+ expect(r.graphCompaction?.state).toBe("running");
+ const p = r.problems.find((x) => x.code === "graph-compaction-running")!;
+ expect(p.level).toBe("info");
+ expect(p.message).toMatch(/edges 400 of 909/);
+ expect(renderStatusHtml(r, "n")).toMatch(/Provenance compaction/);
+ });
+
+ it("shows a failed pass as a warning with the reason and the manual command", () => {
+ const r = evaluateStatus({
+ ...base,
+ graphCompaction: { state: "failed", phase: "nodes", processed: 3, total: 6, error: "state::get timed out" },
+ });
+ const p = r.problems.find((x) => x.code === "graph-compaction-failed")!;
+ expect(p.level).toBe("warn");
+ expect(p.message).toMatch(/state::get timed out/);
+ expect(p.fix).toContain("curl -X POST http://localhost:4611/agentmemory/graph/compact");
+ });
+
+ it("adds no problem when done or off", () => {
+ for (const state of ["done", "off", "pending"] as const) {
+ const r = evaluateStatus({ ...base, graphCompaction: { state } });
+ expect(r.problems.map((x) => x.code).filter((c) => c.startsWith("graph-compaction"))).toEqual([]);
+ }
+ });
+});