Skip to content

#1 Persistent memory for AI coding agents based on real-world benchmarks

feat: add token-efficient recall and file compression tool (#150)

Rohit Ghumare14da6c816cbb
diff --git a/AGENTS.md b/AGENTS.md
index 5aa9f562..bb0c01fd 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -111,8 +111,8 @@ Hook scripts in `src/hooks/` are standalone Node.js scripts (no iii-sdk import).
## Current Stats (v0.8.9)
-- 43 MCP tools (8 visible by default, `AGENTMEMORY_TOOLS=all` for all)
-- 103 REST endpoints
+- 44 MCP tools (8 visible by default, `AGENTMEMORY_TOOLS=all` for all)
+- 104 REST endpoints
- 6 MCP resources, 3 MCP prompts
- 12 hooks, 4 skills
- 50+ iii functions
diff --git a/README.md b/README.md
index 32e1fc24..d72ee92e 100644
--- a/README.md
+++ b/README.md
@@ -25,7 +25,7 @@
<p align="center">
<picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-recall.svg"><img src="assets/tags/stat-recall.svg" alt="95.2% retrieval R@5" height="38" /></picture>
<picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-tokens.svg"><img src="assets/tags/stat-tokens.svg" alt="92% fewer tokens" height="38" /></picture>
- <picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-tools.svg"><img src="assets/tags/stat-tools.svg" alt="43 MCP tools" height="38" /></picture>
+ <picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-tools.svg"><img src="assets/tags/stat-tools.svg" alt="44 MCP tools" height="38" /></picture>
<picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-hooks.svg"><img src="assets/tags/stat-hooks.svg" alt="12 auto hooks" height="38" /></picture>
<picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-deps.svg"><img src="assets/tags/stat-deps.svg" alt="0 external DBs" height="38" /></picture>
<picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-tests.svg"><img src="assets/tags/stat-tests.svg" alt="654 tests passing" height="38" /></picture>
@@ -133,7 +133,7 @@ agentmemory works with any agent that supports hooks, MCP, or REST API. All agen
<sub>AgentSDKProvider</sub>
</td>
<td align="center" width="12.5%">
-<img src="https://img.shields.io/badge/109-endpoints-1f6feb?style=flat-square" alt="REST API" width="48" /><br/>
+<img src="https://img.shields.io/badge/104-endpoints-1f6feb?style=flat-square" alt="REST API" width="48" /><br/>
<strong>Any agent</strong><br/>
<sub>REST API</sub>
</td>
@@ -317,7 +317,7 @@ Implementation details live in `src/cli.ts` (see `runUpgrade` around the `src/cl
### Claude Code (one block, paste it)
```
-Install agentmemory: run `npx @agentmemory/agentmemory` in a separate terminal to start the memory server. Then run `/plugin marketplace add rohitg00/agentmemory` and `/plugin install agentmemory` — the plugin registers all 12 hooks, 4 skills, AND auto-wires the `@agentmemory/mcp` stdio server via its `.mcp.json`, so you get 43 MCP tools (memory_smart_search, memory_save, memory_sessions, memory_governance_delete, etc.) without any extra config step. Verify with `curl http://localhost:3111/agentmemory/health`. The real-time viewer is at http://localhost:3113.
+Install agentmemory: run `npx @agentmemory/agentmemory` in a separate terminal to start the memory server. Then run `/plugin marketplace add rohitg00/agentmemory` and `/plugin install agentmemory` — the plugin registers all 12 hooks, 4 skills, AND auto-wires the `@agentmemory/mcp` stdio server via its `.mcp.json`, so you get 44 MCP tools (memory_smart_search, memory_save, memory_sessions, memory_governance_delete, etc.) without any extra config step. Verify with `curl http://localhost:3111/agentmemory/health`. The real-time viewer is at http://localhost:3113.
```
<details>
@@ -576,9 +576,9 @@ npm install @xenova/transformers
<h2 id="mcp-server"><picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/section-mcp.svg"><img src="assets/tags/section-mcp.svg" alt="MCP Server" height="32" /></picture></h2>
-43 tools, 6 resources, 3 prompts, and 4 skills — the most comprehensive MCP memory toolkit for any agent.
+44 tools, 6 resources, 3 prompts, and 4 skills — the most comprehensive MCP memory toolkit for any agent.
-### 43 Tools
+### 44 Tools
<details>
<summary>Core tools (always available)</summary>
@@ -586,17 +586,21 @@ npm install @xenova/transformers
| Tool | Description |
|------|-------------|
| `memory_recall` | Search past observations |
+| `memory_compress_file` | Compress markdown files while preserving structure |
| `memory_save` | Save an insight, decision, or pattern |
+| `memory_patterns` | Detect recurring patterns |
| `memory_smart_search` | Hybrid semantic + keyword search |
| `memory_file_history` | Past observations about specific files |
| `memory_sessions` | List recent sessions |
+| `memory_timeline` | Chronological observations |
| `memory_profile` | Project profile (concepts, files, patterns) |
| `memory_export` | Export all memory data |
+| `memory_relations` | Query relationship graph |
</details>
<details>
-<summary>Extended tools (43 total — set AGENTMEMORY_TOOLS=all)</summary>
+<summary>Extended tools (44 total — set AGENTMEMORY_TOOLS=all)</summary>
| Tool | Description |
|------|-------------|
@@ -772,7 +776,7 @@ Create `~/.agentmemory/.env`:
# USER_ID=
# TEAM_MODE=private
-# Tool visibility: "core" (7 tools) or "all" (43 tools)
+# Tool visibility: "core" (8 tools) or "all" (44 tools)
# AGENTMEMORY_TOOLS=core
```
@@ -780,7 +784,7 @@ Create `~/.agentmemory/.env`:
<h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/section-api.svg"><img src="assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>
-109 endpoints on port `3111`. The REST API binds to `127.0.0.1` by default. Protected endpoints require `Authorization: Bearer <secret>` when `AGENTMEMORY_SECRET` is set, and mesh sync endpoints require `AGENTMEMORY_SECRET` on both peers.
+104 endpoints on port `3111`. The REST API binds to `127.0.0.1` by default. Protected endpoints require `Authorization: Bearer <secret>` when `AGENTMEMORY_SECRET` is set, and mesh sync endpoints require `AGENTMEMORY_SECRET` on both peers.
<details>
<summary>Key endpoints</summary>
diff --git a/plugin/.claude-plugin/plugin.json b/plugin/.claude-plugin/plugin.json
index d7114962..28992a2e 100644
--- a/plugin/.claude-plugin/plugin.json
+++ b/plugin/.claude-plugin/plugin.json
@@ -1,7 +1,7 @@
{
"name": "agentmemory",
"version": "0.8.11",
- "description": "Persistent memory for AI coding agents -- captures tool usage, compresses via LLM, injects context into future sessions. 12 hooks, 43 MCP tools, 4 skills, real-time viewer.",
+ "description": "Persistent memory for AI coding agents -- captures tool usage, compresses via LLM, injects context into future sessions. 12 hooks, 44 MCP tools, 4 skills, real-time viewer.",
"author": {
"name": "Rohit Ghumare",
"url": "https://github.com/rohitg00"
diff --git a/src/functions/compress-file.ts b/src/functions/compress-file.ts
new file mode 100644
index 00000000..e7a363a6
--- /dev/null
+++ b/src/functions/compress-file.ts
@@ -0,0 +1,160 @@
+import { readFile, writeFile } from "node:fs/promises";
+import { basename, dirname, extname, join, resolve } from "node:path";
+import type { ISdk } from "iii-sdk";
+import type { MemoryProvider } from "../types.js";
+import type { StateKV } from "../state/kv.js";
+import { recordAudit } from "./audit.js";
+
+const SENSITIVE_PATH_TERMS = [
+ "secret",
+ "credential",
+ "private_key",
+ ".env",
+ "id_rsa",
+ "token",
+];
+
+const COMPRESS_FILE_SYSTEM_PROMPT = `You compress markdown while preserving structure.
+Rules:
+- Keep all headings exactly as-is.
+- Keep all URLs exactly as-is.
+- Keep all fenced code blocks exactly as-is.
+- Do not remove sections; shorten prose under each section.
+- Output only markdown, no wrappers or explanations.`;
+
+function stripMarkdownFence(text: string): string {
+ const trimmed = text.trim();
+ const match = trimmed.match(/^```(?:markdown|md)?\s*([\s\S]*?)\s*```$/i);
+ return match ? match[1].trim() : trimmed;
+}
+
+function extractUrls(text: string): string[] {
+ return Array.from(new Set(text.match(/https?:\/\/[^\s)]+/g) || []));
+}
+
+function extractHeadings(text: string): string[] {
+ return text
+ .split("\n")
+ .map((line) => line.trim())
+ .filter((line) => /^#{1,6}\s+/.test(line));
+}
+
+function extractCodeBlocks(text: string): string[] {
+ return text.match(/```[\s\S]*?```/g) || [];
+}
+
+function validateCompression(original: string, compressed: string): string[] {
+ const errors: string[] = [];
+
+ const originalHeadings = extractHeadings(original);
+ const compressedHeadings = extractHeadings(compressed);
+ for (const heading of originalHeadings) {
+ if (!compressedHeadings.includes(heading)) {
+ errors.push(`missing heading: ${heading}`);
+ }
+ }
+
+ const originalUrls = extractUrls(original).sort();
+ const compressedUrls = extractUrls(compressed).sort();
+ if (originalUrls.length !== compressedUrls.length) {
+ errors.push("url count changed");
+ } else {
+ for (let i = 0; i < originalUrls.length; i++) {
+ if (originalUrls[i] !== compressedUrls[i]) {
+ errors.push("url set changed");
+ break;
+ }
+ }
+ }
+
+ const originalBlocks = extractCodeBlocks(original);
+ const compressedBlocks = extractCodeBlocks(compressed);
+ if (originalBlocks.length !== compressedBlocks.length) {
+ errors.push("code block count changed");
+ } else {
+ for (let i = 0; i < originalBlocks.length; i++) {
+ if (originalBlocks[i] !== compressedBlocks[i]) {
+ errors.push("code block content changed");
+ break;
+ }
+ }
+ }
+
+ return errors;
+}
+
+function resolveBackupPath(filePath: string): string {
+ const base = basename(filePath, extname(filePath));
+ const name = base.endsWith(".original") ? base : `${base}.original`;
+ return join(dirname(filePath), `${name}.md`);
+}
+
+export function registerCompressFileFunction(
+ sdk: ISdk,
+ kv: StateKV,
+ provider: MemoryProvider,
+): void {
+ sdk.registerFunction(
+ "mem::compress-file",
+ async (data: { filePath: string }) => {
+ if (!data?.filePath || typeof data.filePath !== "string") {
+ return { success: false, error: "filePath is required" };
+ }
+
+ const absolutePath = resolve(data.filePath);
+ const lowerPath = absolutePath.toLowerCase();
+ if (extname(absolutePath).toLowerCase() !== ".md") {
+ return { success: false, error: "filePath must point to a .md file" };
+ }
+ if (SENSITIVE_PATH_TERMS.some((term) => lowerPath.includes(term))) {
+ return { success: false, error: "refusing to process sensitive-looking path" };
+ }
+
+ let original: string;
+ try {
+ original = await readFile(absolutePath, "utf-8");
+ } catch {
+ return { success: false, error: "failed to read file" };
+ }
+
+ if (!original.trim()) {
+ return { success: true, skipped: true, reason: "file is empty" };
+ }
+
+ const response = await provider.summarize(
+ COMPRESS_FILE_SYSTEM_PROMPT,
+ `Compress this markdown file while preserving structure and code blocks:\n\n${original}`,
+ );
+ const compressed = stripMarkdownFence(response);
+ const validationErrors = validateCompression(original, compressed);
+ if (validationErrors.length > 0) {
+ return {
+ success: false,
+ error: "compression validation failed",
+ details: validationErrors,
+ };
+ }
+
+ const backupPath = resolveBackupPath(absolutePath);
+ await writeFile(backupPath, original, "utf-8");
+ await writeFile(absolutePath, compressed, "utf-8");
+
+ try {
+ await recordAudit(kv, "compress", "mem::compress-file", [], {
+ filePath: absolutePath,
+ backupPath,
+ originalChars: original.length,
+ compressedChars: compressed.length,
+ });
+ } catch {}
+
+ return {
+ success: true,
+ filePath: absolutePath,
+ backupPath,
+ originalChars: original.length,
+ compressedChars: compressed.length,
+ };
+ },
+ );
+}
diff --git a/src/functions/search.ts b/src/functions/search.ts
index 8d296e99..62b654ab 100644
--- a/src/functions/search.ts
+++ b/src/functions/search.ts
@@ -1,5 +1,5 @@
import type { ISdk } from 'iii-sdk'
-import type { CompressedObservation, SearchResult, Session } from '../types.js'
+import type { CompactSearchResult, CompressedObservation, SearchResult, Session } from '../types.js'
import { KV } from '../state/schema.js'
import { StateKV } from '../state/kv.js'
import { SearchIndex } from '../state/search-index.js'
@@ -54,7 +54,14 @@ export async function rebuildIndex(kv: StateKV): Promise<number> {
export function registerSearchFunction(sdk: ISdk, kv: StateKV): void {
sdk.registerFunction(
'mem::search',
- async (data: { query: string; limit?: number; project?: string; cwd?: string }) => {
+ async (data: {
+ query: string
+ limit?: number
+ project?: string
+ cwd?: string
+ format?: string
+ token_budget?: number
+ }) => {
const idx = getSearchIndex()
// Input validation / normalization.
@@ -72,6 +79,17 @@ export function registerSearchFunction(sdk: ISdk, kv: StateKV): void {
}
const projectFilter = typeof data.project === 'string' && data.project.length > 0 ? data.project : undefined
const cwdFilter = typeof data.cwd === 'string' && data.cwd.length > 0 ? data.cwd : undefined
+ const format = typeof data.format === 'string' ? data.format : 'full'
+ if (!['full', 'compact', 'narrative'].includes(format)) {
+ throw new Error("mem::search: format must be one of 'full', 'compact', or 'narrative'")
+ }
+ let tokenBudget: number | undefined
+ if (data.token_budget !== undefined) {
+ if (!Number.isInteger(data.token_budget) || data.token_budget < 1) {
+ throw new Error('mem::search: token_budget must be a positive integer')
+ }
+ tokenBudget = data.token_budget
+ }
if (idx.size === 0) {
const count = await rebuildIndex(kv)
@@ -129,14 +147,86 @@ export function registerSearchFunction(sdk: ISdk, kv: StateKV): void {
enriched.map((r) => r.observation.id),
)
+ const estimateTokens = (value: unknown): number =>
+ Math.max(1, Math.ceil(JSON.stringify(value).length / 3))
+
+ const applyTokenBudget = <T>(items: T[]): {
+ items: T[]
+ used: number
+ truncated: boolean
+ } => {
+ if (!tokenBudget) return { items, used: items.reduce((sum, item) => sum + estimateTokens(item), 0), truncated: false }
+ const selected: T[] = []
+ let used = 0
+ for (const item of items) {
+ const itemTokens = estimateTokens(item)
+ if (used + itemTokens > tokenBudget) {
+ return { items: selected, used, truncated: selected.length < items.length }
+ }
+ selected.push(item)
+ used += itemTokens
+ }
+ return { items: selected, used, truncated: false }
+ }
+
+ if (format === 'compact') {
+ const compactResults: CompactSearchResult[] = enriched.map((r) => ({
+ obsId: r.observation.id,
+ sessionId: r.sessionId,
+ title: r.observation.title,
+ type: r.observation.type,
+ score: r.score,
+ timestamp: r.observation.timestamp,
+ }))
+ const packed = applyTokenBudget(compactResults)
+ return {
+ format,
+ results: packed.items,
+ tokens_used: packed.used,
+ tokens_budget: tokenBudget,
+ truncated: packed.truncated,
+ }
+ }
+
+ if (format === 'narrative') {
+ const narrativeResults = enriched.map((r) => ({
+ obsId: r.observation.id,
+ sessionId: r.sessionId,
+ title: r.observation.title,
+ narrative: r.observation.narrative,
+ score: r.score,
+ timestamp: r.observation.timestamp,
+ }))
+ const packed = applyTokenBudget(narrativeResults)
+ const text = packed.items
+ .map((r, index) => `${index + 1}. ${r.title}\n${r.narrative}`)
+ .join('\n\n')
+ return {
+ format,
+ results: packed.items,
+ text,
+ tokens_used: packed.used,
+ tokens_budget: tokenBudget,
+ truncated: packed.truncated,
+ }
+ }
+
+ const packed = applyTokenBudget(enriched)
+
// Avoid logging raw cwd/project (host paths). Log only that filters were active.
logger.info('Search completed', {
query,
- results: enriched.length,
+ results: packed.items.length,
hasProjectFilter: !!projectFilter,
hasCwdFilter: !!cwdFilter,
})
- return { results: enriched }
+ return {
+ format,
+ results: packed.items,
+ tokens_used: packed.used,
+ tokens_budget: tokenBudget,
+ truncated: packed.truncated,
+ }
}
)
}
diff --git a/src/index.ts b/src/index.ts
index 18b4a59c..0970b74f 100644
--- a/src/index.ts
+++ b/src/index.ts
@@ -75,6 +75,7 @@ import { registerSlidingWindowFunction } from "./functions/sliding-window.js";
import { registerQueryExpansionFunction } from "./functions/query-expansion.js";
import { registerTemporalGraphFunctions } from "./functions/temporal-graph.js";
import { registerRetentionFunctions } from "./functions/retention.js";
+import { registerCompressFileFunction } from "./functions/compress-file.js";
import { registerApiTriggers } from "./triggers/api.js";
import { registerEventTriggers } from "./triggers/events.js";
import { registerMcpEndpoints } from "./mcp/server.js";
@@ -239,6 +240,7 @@ async function main() {
registerQueryExpansionFunction(sdk, provider);
registerTemporalGraphFunctions(sdk, kv, provider);
registerRetentionFunctions(sdk, kv);
+ registerCompressFileFunction(sdk, kv, provider);
console.log(
`[agentmemory] v0.6 advanced retrieval: sliding-window, query-expansion, temporal-graph, retention-scoring`,
);
@@ -314,7 +316,7 @@ async function main() {
`[agentmemory] Ready. ${embeddingProvider ? "Triple-stream (BM25+Vector+Graph)" : "BM25+Graph"} search active.`,
);
console.log(
- `[agentmemory] Endpoints: 103 REST + 43 MCP tools + 6 MCP resources + 3 MCP prompts`,
+ `[agentmemory] Endpoints: 104 REST + 44 MCP tools + 6 MCP resources + 3 MCP prompts`,
);
const viewerPort = config.restPort + 2;
diff --git a/src/mcp/server.ts b/src/mcp/server.ts
index b3086c7c..82283e90 100644
--- a/src/mcp/server.ts
+++ b/src/mcp/server.ts
@@ -92,20 +92,69 @@ export function registerMcpEndpoints(
body: { error: "query is required for memory_recall" },
};
}
+ const format =
+ typeof args.format === "string" ? args.format.trim().toLowerCase() : "full";
+ if (!["full", "compact", "narrative"].includes(format)) {
+ return {
+ status_code: 400,
+ body: {
+ error: "format must be one of: full, compact, narrative",
+ },
+ };
+ }
+ const tokenBudget = asNumber(args.token_budget);
+ if (
+ args.token_budget !== undefined &&
+ (!Number.isInteger(tokenBudget) || (tokenBudget ?? 0) < 1)
+ ) {
+ return {
+ status_code: 400,
+ body: { error: "token_budget must be a positive integer" },
+ };
+ }
const result = await sdk.trigger({ function_id: "mem::search", payload: {
query: args.query,
limit: typeof args.limit === "number" ? args.limit : 10,
+ format,
+ token_budget: tokenBudget,
} });
+ const text =
+ format === "narrative" &&
+ result &&
+ typeof result === "object" &&
+ "text" in (result as Record<string, unknown>) &&
+ typeof (result as { text?: unknown }).text === "string"
+ ? (result as { text: string }).text
+ : JSON.stringify(result, null, 2);
return {
status_code: 200,
body: {
content: [
- { type: "text", text: JSON.stringify(result, null, 2) },
+ { type: "text", text },
],
},
};
}
+ case "memory_compress_file": {
+ if (typeof args.filePath !== "string" || !args.filePath.trim()) {
+ return {
+ status_code: 400,
+ body: { error: "filePath is required for memory_compress_file" },
+ };
+ }
+ const result = await sdk.trigger({
+ function_id: "mem::compress-file",
+ payload: { filePath: args.filePath.trim() },
+ });
+ return {
+ status_code: 200,
+ body: {
+ content: [{ type: "text", text: JSON.stringify(result, null, 2) }],
+ },
+ };
+ }
+
case "memory_save": {
if (typeof args.content !== "string" || !args.content.trim()) {
return {
diff --git a/src/mcp/tools-registry.ts b/src/mcp/tools-registry.ts
index 06a9466c..4297e459 100644
--- a/src/mcp/tools-registry.ts
+++ b/src/mcp/tools-registry.ts
@@ -24,10 +24,33 @@ export const CORE_TOOLS: McpToolDef[] = [
type: "number",
description: "Max results to return (default 10)",
},
+ format: {
+ type: "string",
+ description: "Result format: full, compact, or narrative (default full)",
+ },
+ token_budget: {
+ type: "number",
+ description: "Optional token budget to trim returned results",
+ },
},
required: ["query"],
},
},
+ {
+ name: "memory_compress_file",
+ description:
+ "Compress a markdown file to reduce token usage while preserving headings, URLs, and code blocks. Creates a .original.md backup before writing.",
+ inputSchema: {
+ type: "object",
+ properties: {
+ filePath: {
+ type: "string",
+ description: "Path to the markdown file to compress",
+ },
+ },
+ required: ["filePath"],
+ },
+ },
{
name: "memory_save",
description:
diff --git a/src/triggers/api.ts b/src/triggers/api.ts
index ed21141b..6691be4c 100644
--- a/src/triggers/api.ts
+++ b/src/triggers/api.ts
@@ -225,7 +225,14 @@ export function registerApiTriggers(
sdk.registerFunction("api::search",
async (
- req: ApiRequest<{ query: string; limit?: number; project?: string; cwd?: string }>,
+ req: ApiRequest<{
+ query: string;
+ limit?: number;
+ project?: string;
+ cwd?: string;
+ format?: string;
+ token_budget?: number;
+ }>,
): Promise<Response> => {
const body = (req.body ?? {}) as Record<string, unknown>;
if (typeof body.query !== "string" || !body.query.trim()) {
@@ -243,11 +250,35 @@ export function registerApiTriggers(
if (body.cwd !== undefined && typeof body.cwd !== "string") {
return { status_code: 400, body: { error: "cwd must be a string" } };
}
+ if (
+ body.format !== undefined &&
+ (typeof body.format !== "string" ||
+ !["full", "compact", "narrative"].includes(body.format.trim().toLowerCase()))
+ ) {
+ return {
+ status_code: 400,
+ body: { error: "format must be one of: full, compact, narrative" },
+ };
+ }
+ if (
+ body.token_budget !== undefined &&
+ (!Number.isInteger(body.token_budget) || (body.token_budget as number) < 1)
+ ) {
+ return {
+ status_code: 400,
+ body: { error: "token_budget must be a positive integer" },
+ };
+ }
const payload = {
query: body.query.trim(),
limit: body.limit as number | undefined,
project: body.project as string | undefined,
cwd: body.cwd as string | undefined,
+ format:
+ typeof body.format === "string"
+ ? body.format.trim().toLowerCase()
+ : undefined,
+ token_budget: body.token_budget as number | undefined,
};
const result = await sdk.trigger({ function_id: "mem::search", payload: payload });
return { status_code: 200, body: result };
@@ -263,6 +294,31 @@ export function registerApiTriggers(
},
});
+ sdk.registerFunction("api::compress-file",
+ async (req: ApiRequest<{ filePath: string }>): Promise<Response> => {
+ const authErr = checkAuth(req, secret);
+ if (authErr) return authErr;
+ const body = (req.body ?? {}) as Record<string, unknown>;
+ const filePath = asNonEmptyString(body.filePath);
+ if (!filePath) {
+ return {
+ status_code: 400,
+ body: { error: "filePath is required and must be a non-empty string" },
+ };
+ }
+ const result = await sdk.trigger({
+ function_id: "mem::compress-file",
+ payload: { filePath },
+ });
+ return { status_code: 200, body: result };
+ },
+ );
+ sdk.registerTrigger({
+ type: "http",
+ function_id: "api::compress-file",
+ config: { api_path: "/agentmemory/compress-file", http_method: "POST" },
+ });
+
sdk.registerFunction("api::session::start",
async (
req: ApiRequest<{ sessionId: string; project: string; cwd: string }>,
diff --git a/test/compress-file.test.ts b/test/compress-file.test.ts
new file mode 100644
index 00000000..8220ed17
--- /dev/null
+++ b/test/compress-file.test.ts
@@ -0,0 +1,130 @@
+import { beforeEach, describe, expect, it, vi } from "vitest";
+
+vi.mock("../src/logger.js", () => ({
+ logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn() },
+}));
+
+const fileStore = new Map<string, string>();
+
+vi.mock("node:fs/promises", () => ({
+ readFile: vi.fn(async (path: string) => {
+ const value = fileStore.get(path);
+ if (value === undefined) throw new Error("ENOENT");
+ return value;
+ }),
+ writeFile: vi.fn(async (path: string, content: string) => {
+ fileStore.set(path, content);
+ }),
+}));
+
+import { registerCompressFileFunction } from "../src/functions/compress-file.js";
+
+function mockKV() {
+ const store = new Map<string, Map<string, unknown>>();
+ return {
+ get: async <T>(scope: string, key: string): Promise<T | null> => {
+ return (store.get(scope)?.get(key) as T) ?? null;
+ },
+ set: async <T>(scope: string, key: string, data: T): Promise<T> => {
+ if (!store.has(scope)) store.set(scope, new Map());
+ store.get(scope)!.set(key, data);
+ return data;
+ },
+ delete: async (scope: string, key: string): Promise<void> => {
+ store.get(scope)?.delete(key);
+ },
+ list: async <T>(scope: string): Promise<T[]> => {
+ const entries = store.get(scope);
+ return entries ? (Array.from(entries.values()) as T[]) : [];
+ },
+ };
+}
+
+function mockSdk() {
+ const functions = new Map<string, Function>();
+ return {
+ registerFunction: (idOrOpts: string | { id: string }, handler: Function) => {
+ const id = typeof idOrOpts === "string" ? idOrOpts : idOrOpts.id;
+ functions.set(id, handler);
+ },
+ registerTrigger: () => {},
+ trigger: async (
+ idOrInput: string | { function_id: string; payload: unknown },
+ data?: unknown,
+ ) => {
+ const id = typeof idOrInput === "string" ? idOrInput : idOrInput.function_id;
+ const payload = typeof idOrInput === "string" ? data : idOrInput.payload;
+ const fn = functions.get(id);
+ if (!fn) throw new Error(`No function: ${id}`);
+ return fn(payload);
+ },
+ };
+}
+
+describe("mem::compress-file", () => {
+ let sdk: ReturnType<typeof mockSdk>;
+ let kv: ReturnType<typeof mockKV>;
+ let summarize: ReturnType<typeof vi.fn>;
+
+ beforeEach(() => {
+ fileStore.clear();
+ sdk = mockSdk();
+ kv = mockKV();
+ summarize = vi.fn();
+ registerCompressFileFunction(
+ sdk as never,
+ kv as never,
+ { name: "test-provider", summarize, compress: summarize } as never,
+ );
+ });
+
+ it("rejects non-markdown paths", async () => {
+ const result = (await sdk.trigger("mem::compress-file", {
+ filePath: "/tmp/readme.txt",
+ })) as { success: boolean; error: string };
+ expect(result.success).toBe(false);
+ expect(result.error).toContain(".md");
+ });
+
+ it("compresses markdown and writes .original.md backup", async () => {
+ const path = "/tmp/notes.md";
+ fileStore.set(
+ path,
+ "# Title\n\nVisit https://example.com\n\n```ts\nconst x = 1;\n```\n\nSome long explanation.",
+ );
+
+ summarize.mockResolvedValue(
+ "# Title\n\nVisit https://example.com\n\n```ts\nconst x = 1;\n```\n\nShort explanation.",
+ );
+
+ const result = (await sdk.trigger("mem::compress-file", {
+ filePath: path,
+ })) as {
+ success: boolean;
+ backupPath: string;
+ compressedChars: number;
+ originalChars: number;
+ };
+
+ expect(result.success).toBe(true);
+ expect(result.backupPath).toBe("/tmp/notes.original.md");
+ expect(fileStore.get("/tmp/notes.original.md")).toContain("Some long explanation.");
+ expect(fileStore.get(path)).toContain("Short explanation.");
+ expect(result.compressedChars).toBeLessThan(result.originalChars);
+ });
+
+ it("fails validation when URLs change", async () => {
+ const path = "/tmp/guide.md";
+ fileStore.set(path, "# Guide\n\nhttps://example.com\n");
+ summarize.mockResolvedValue("# Guide\n\nhttps://different.example.com\n");
+
+ const result = (await sdk.trigger("mem::compress-file", {
+ filePath: path,
+ })) as { success: boolean; error: string; details: string[] };
+
+ expect(result.success).toBe(false);
+ expect(result.error).toContain("validation");
+ expect(result.details.some((d) => d.includes("url"))).toBe(true);
+ expect(fileStore.get("/tmp/guide.original.md")).toBeUndefined();
+ });
+});
diff --git a/test/mcp-standalone.test.ts b/test/mcp-standalone.test.ts
index 81d7a7ff..d099431d 100644
--- a/test/mcp-standalone.test.ts
+++ b/test/mcp-standalone.test.ts
@@ -46,8 +46,8 @@ describe("Tools Registry", () => {
}
});
- it("CORE_TOOLS has 10 items", () => {
- expect(CORE_TOOLS.length).toBe(10);
+ it("CORE_TOOLS has 11 items", () => {
+ expect(CORE_TOOLS.length).toBe(11);
});
it("V040_TOOLS has 8 items", () => {
diff --git a/test/search.test.ts b/test/search.test.ts
new file mode 100644
index 00000000..f7e29d0d
--- /dev/null
+++ b/test/search.test.ts
@@ -0,0 +1,150 @@
+import { beforeEach, describe, expect, it, vi } from "vitest";
+
+vi.mock("../src/logger.js", () => ({
+ logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn() },
+}));
+
+import { registerSearchFunction } from "../src/functions/search.js";
+import { KV } from "../src/state/schema.js";
+import type { CompressedObservation, Session } from "../src/types.js";
+
+function mockKV() {
+ const store = new Map<string, Map<string, unknown>>();
+ return {
+ get: async <T>(scope: string, key: string): Promise<T | null> => {
+ return (store.get(scope)?.get(key) as T) ?? null;
+ },
+ set: async <T>(scope: string, key: string, data: T): Promise<T> => {
+ if (!store.has(scope)) store.set(scope, new Map());
+ store.get(scope)!.set(key, data);
+ return data;
+ },
+ delete: async (scope: string, key: string): Promise<void> => {
+ store.get(scope)?.delete(key);
+ },
+ list: async <T>(scope: string): Promise<T[]> => {
+ const entries = store.get(scope);
+ return entries ? (Array.from(entries.values()) as T[]) : [];
+ },
+ };
+}
+
+function mockSdk() {
+ const functions = new Map<string, Function>();
+ return {
+ registerFunction: (idOrOpts: string | { id: string }, handler: Function) => {
+ const id = typeof idOrOpts === "string" ? idOrOpts : idOrOpts.id;
+ functions.set(id, handler);
+ },
+ registerTrigger: () => {},
+ trigger: async (
+ idOrInput: string | { function_id: string; payload: unknown },
+ data?: unknown,
+ ) => {
+ const id = typeof idOrInput === "string" ? idOrInput : idOrInput.function_id;
+ const payload = typeof idOrInput === "string" ? data : idOrInput.payload;
+ const fn = functions.get(id);
+ if (!fn) throw new Error(`No function: ${id}`);
+ return fn(payload);
+ },
+ };
+}
+
+describe("mem::search", () => {
+ let sdk: ReturnType<typeof mockSdk>;
+ let kv: ReturnType<typeof mockKV>;
+
+ beforeEach(async () => {
+ sdk = mockSdk();
+ kv = mockKV();
+ registerSearchFunction(sdk as never, kv as never);
+
+ const session: Session = {
+ id: "ses_1",
+ project: "demo",
+ cwd: "/tmp/demo",
+ startedAt: "2026-01-01T00:00:00Z",
+ status: "completed",
+ observationCount: 2,
+ };
+ await kv.set(KV.sessions, session.id, session);
+
+ const obsA: CompressedObservation = {
+ id: "obs_a",
+ sessionId: "ses_1",
+ timestamp: "2026-01-01T00:00:00Z",
+ type: "decision",
+ title: "Auth middleware decision",
+ subtitle: "JWT strategy",
+ facts: ["Use rotating refresh tokens"],
+ narrative: "Implemented auth middleware with JWT refresh rotation.",
+ concepts: ["auth", "jwt"],
+ files: ["src/auth.ts"],
+ importance: 8,
+ };
+ const obsB: CompressedObservation = {
+ id: "obs_b",
+ sessionId: "ses_1",
+ timestamp: "2026-01-02T00:00:00Z",
+ type: "file_edit",
+ title: "UI button styling",
+ facts: ["Updated primary button color"],
+ narrative: "Adjusted button styles in the settings page.",
+ concepts: ["ui", "css"],
+ files: ["src/ui/button.tsx"],
+ importance: 4,
+ };
+
+ await kv.set(KV.observations("ses_1"), obsA.id, obsA);
+ await kv.set(KV.observations("ses_1"), obsB.id, obsB);
+ });
+
+ it("returns full format by default", async () => {
+ const result = (await sdk.trigger("mem::search", {
+ query: "auth middleware",
+ })) as { format: string; results: Array<{ observation: CompressedObservation }> };
+
+ expect(result.format).toBe("full");
+ expect(result.results).toHaveLength(1);
+ expect(result.results[0]?.observation.id).toBe("obs_a");
+ });
+
+ it("returns compact format when requested", async () => {
+ const result = (await sdk.trigger("mem::search", {
+ query: "auth",
+ format: "compact",
+ })) as { format: string; results: Array<{ obsId: string; title: string }> };
+
+ expect(result.format).toBe("compact");
+ expect(result.results[0]?.obsId).toBe("obs_a");
+ expect(result.results[0]?.title).toBe("Auth middleware decision");
+ });
+
+ it("returns narrative text and respects token budget", async () => {
+ const result = (await sdk.trigger("mem::search", {
+ query: "auth ui",
+ format: "narrative",
+ token_budget: 20,
+ })) as {
+ format: string;
+ results: Array<{ obsId: string }>;
+ text: string;
+ tokens_used: number;
+ tokens_budget: number;
+ truncated: boolean;
+ };
+
+ expect(result.format).toBe("narrative");
+ expect(result.tokens_budget).toBe(20);
+ expect(result.tokens_used).toBeLessThanOrEqual(20);
+ expect(typeof result.text).toBe("string");
+ expect(result.results.length).toBeLessThanOrEqual(2);
+ expect(result.truncated).toBe(true);
+ });
+
+ it("rejects invalid format values", async () => {
+ await expect(
+ sdk.trigger("mem::search", { query: "auth", format: "verbose" }),
+ ).rejects.toThrow("format must be one of");
+ });
+});