Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
feat(examples): add pi coding agent memory extension (#2314)
HammerHound AI26d991243fca
diff --git a/examples/pi-coding-agent-extension/DESIGN.md b/examples/pi-coding-agent-extension/DESIGN.mdnew file mode 100644index 000000000..c1708030a--- /dev/null+++ b/examples/pi-coding-agent-extension/DESIGN.md@@ -0,0 +1,901 @@+# Pi OpenViking Extension — Implementation Spec++## Design Philosophy++**Informed by all three existing OV plugins** — OpenClaw, Claude Code, and Hermes. The Claude Code plugin is the most mature and production-hardened; its patterns take precedence where they differ from OpenClaw. Key design ancestors:++- **OpenClaw**: Synchronous recall, threshold commit, memory stripping, dual-scope search+- **Claude Code plugin** (newest, most mature): Pre-compact commit, subagent isolation, comprehensive stripping, session-resume rehydration, score threshold, bypass patterns+- **Hermes** (anti-pattern): Stale prefetch, session-end-only commit, no stripping++Design comparison:++| Concern | Hermes (rejected) | OpenClaw (adopted) | This extension |+|---------|-------------------|--------------------|----|+| Recall timing | Stale prefetch (N-1 turn) | Synchronous current-turn | ✅ Synchronous via `context` event |+| Recall relevance | Wrong topic | Right topic | ✅ Right topic |+| First turn | Gets nothing | Gets relevant context | ✅ Gets relevant context |+| Injection target | User message (stale cache) | User message (fresh search) | ✅ User message via `context` event |+| Commit trigger | Session-end only | Token threshold mid-session | Token threshold mid-session + session-end + pre-compact | ✅ Threshold + session-end |+| Memory stripping | None | Strip injected blocks before sync | Strip `<relevant-memories>` + `<system-reminder>` + `<openviking-context>` + `[Subagent Context]` + null bytes | ✅ Strip all 5 + null bytes |+| History compression | None | OV archives replace transcript | Pi compaction (pre-compact commit preserves content in OV) | ❌ Pi has its own compaction |+| Tools | 5 | 8 | 9 (via MCP) | 7 (no `add_skill` — pi has its own skill system) |+| Profile injection | None | None | ✅ profile.md + preferences + entities at session start | ✅ Same |++## Architecture++```+~/.pi/agent/extensions/openviking/+├── index.ts # Entry point — registers events, tools, commands+├── client.ts # HTTP client for OV REST API (zero npm deps)+├── index_builder.ts # Build memory index (viking:// tree + archive abstracts)+├── recall.ts # Synchronous search, reranking, <relevant-memories> formatting+├── sync.ts # Turn archival, memory stripping, commit management+└── tools.ts # 7 tool schemas + handlers+```++6 files. ~1000-1200 lines total.++## Config++Inline in `index.ts`. Loaded from `~/.pi/agent/extensions/openviking/config.json`.++```typescript+interface OVConfig {+ enabled: boolean; // Master switch (default: true)+ endpoint: string; // OV server URL (default: "http://127.0.0.1:1933")+ apiKey: string; // API key (default: "" — dev mode)+ account: string; // Multi-tenant account (default: "")+ user: string; // Multi-tenant user (default: "")+ agentId: string; // Agent identity for X-OpenViking-Agent header (default: "pi")+ syncTurns: boolean; // Auto-sync conversation turns (default: true)+ recallBudget: number; // Max tokens for <relevant-memories> block (default: 2000)+ recallMaxContentChars: number; // Max chars per recall result before truncation (default: 500)+ recallPreferAbstract: boolean; // Prefer abstract/overview over full content (default: true)+ recallLimit: number; // Max results after dedup, before budget filtering (default: 6)+ recallScoreThreshold: number; // Min relevance score for recall results (default: 0.35)+ recallMinQueryLength: number; // Skip recall for queries shorter than this (default: 3)+ profileBudget: number; // Max tokens for user profile injection at session start (default: 10000)+ resumeContextBudget: number; // Max tokens for archive overview on resume/compact (default: 2000)+ indexBudget: number; // Max tokens for memory index in system prompt (default: 2000)+ captureToolResults: boolean; // Include tool result output in capture (default: false — agent inputs kept, results dropped)+ captureMode: "semantic" | "keyword"; // "semantic" = always capture, "keyword" = only when trigger phrases match (default: "semantic")+ captureMaxLength: number; // Max sanitized text length for capture decision (default: 24000)+ captureAssistantTurns: boolean; // Include assistant turns in capture (default: true — memory extraction needs both sides)+ commitTokenThreshold: number; // Commit after N pending tokens synced (default: 20000, 0 = session-end only)+ commitOnShutdown: boolean; // Commit session on session_shutdown (default: true)+ mirrorMemoryWrites: boolean; // Mirror MEMORY.md to OV at commit time (default: true)+ writeQueueFlushInterval: number; // Write queue flush interval in ms (default: 5000)+ writeQueueFlushThreshold: number; // Write queue flush after N queued turns (default: 5)+ bypassPatterns: string[]; // Glob patterns for cwd to skip (default: [])+ logLevel: "silent" | "error" | "info"; // default: "error"+}+```++Config resolution: `config.json` → env vars (`OPENVIKING_URL`, `OPENVIKING_API_KEY`, `OPENVIKING_ACCOUNT`, `OPENVIKING_USER`, `OPENVIKING_AGENT_ID`, etc.) → defaults. Follows the Claude Code plugin's priority chain.++## File Details++### client.ts (~250 lines)++HTTP client for the OpenViking REST API. Uses Node.js built-in `fetch`. Zero npm dependencies.++Wraps these endpoints:++| Method | Endpoint | Purpose |+|--------|----------|---------|+| GET | `/health` | Health check |+| POST | `/api/v1/sessions` | Create session |+| GET | `/api/v1/sessions/{id}` | Get session metadata (including pending_tokens) |+| POST | `/api/v1/sessions/{id}/messages` | Add message |+| POST | `/api/v1/sessions/{id}/commit` | Commit (triggers memory extraction) |+| POST | `/api/v1/search/find` | Quick retrieval (accepts `target_uri` for scope, `top_k`, `score_threshold`) |+| GET | `/api/v1/content/read` | Read full content (L2) |+| GET | `/api/v1/content/abstract` | Read abstract (L0) |+| GET | `/api/v1/content/overview` | Read overview (L1) |+| GET | `/api/v1/fs/ls` | List directory |+| GET | `/api/v1/fs/stat` | Stat entry |+| DELETE | `/api/v1/content` | Delete by URI |+| POST | `/api/v1/resources` | Add resource |++All methods are async. All catch errors internally and return `null`/empty on failure. Timeouts: 5s health, 10s reads, 30s writes.++Headers include `X-OpenViking-Account`, `X-OpenViking-User`, `X-OpenViking-Agent` for multi-tenant routing.++```typescript+class OVClient {+ private baseUrl: string;+ private apiKey: string;+ private account: string;+ private user: string;+ private agent: string; // always "pi"+ private connected: boolean;++ constructor(config: OVConfig);++ async health(): Promise<boolean>;+ async createSession(sessionId: string): Promise<boolean>;+ async addMessage(sessionId: string, role: string, content: string): Promise<boolean>;+ async getSession(sessionId: string): Promise<OVSessionMeta | null>;+ async commitSession(sessionId: string, wait?: boolean): Promise<string | null>;+ async find(query: string, opts?: { targetUri?: string; topK?: number; scoreThreshold?: number }): Promise<OVSearchResult[]>;+ async readContent(uri: string): Promise<string | null>;+ async abstract(uri: string): Promise<string | null>;+ async overview(uri: string): Promise<string | null>;+ async ls(uri: string): Promise<OVDirEntry[]>;+ async stat(uri: string): Promise<OVEntryInfo | null>;+ async deleteByUri(uri: string): Promise<boolean>;+ async addResource(path: string, opts?: { to?: string }): Promise<any>;++ // URI space resolution (from Claude Code plugin's resolveScopeSpace/resolveTargetUri)+ // Multi-tenant OV deployments namespace memories under viking://user/<space>/memories/.+ // The space is NOT always "default" — it's the first non-reserved, non-hidden directory+ // under viking://user/ that matches the configured user identity.+ private resolvedSpaces: Map<string, string>; // cache: scope → space name++ // Discover the actual namespace for a given scope ("user" or "agent").+ // Probes /api/v1/system/status for the user identity, then ls viking://<scope>+ // to find a matching space. Falls back to "default".+ async resolveScopeSpace(scope: "user" | "agent"): Promise<string>;++ // Expand a bare viking:// URI (e.g., viking://user/memories) to its fully-qualified+ // form with the resolved space inserted (e.g., viking://user/alice/memories).+ // Reserved directory names (memories, skills, instructions, workspaces) trigger space+ // insertion; non-reserved paths pass through unchanged.+ async resolveTargetUri(targetUri: string): Promise<string>;+}+```++### index_builder.ts (~80 lines)++**The table of contents.** Builds a browsable memory index that tells the model *what OV knows*, so it can make informed decisions about when to search deeper. Inspired by OpenClaw's preflight `assemble()` which provides `latest_archive_overview` and `pre_archive_abstracts` to the model.++Without this index, the model is flying blind — it can only retrieve what it thinks to ask for, with no topical overview to guide its queries. The index is the map; recall is the flashlight.++#### What goes into the index++The index has two parts, both built from OV's filesystem API:++1. **Directory listing**: `client.ls("viking://")` → top-level children (user/, agent/, resources/, session/), then one level deeper for each. Shows *what categories of knowledge exist*.+2. **Abstract summaries**: For each leaf memory in `viking://user/memories/`, fetch L0 abstracts. These are ~100 tokens each and give the model a one-line summary of each stored memory.+3. **Archive overview**: If the current session has a previous archive, its L1 overview (~2k tokens) is included as `[Session History Summary]`.++#### When the index is built++- **Session start** (`session_start`): build once, cache in memory.+- **After commit** (threshold or shutdown): rebuild if new memories were extracted. The commit callback triggers a rebuild.++The index is NOT rebuilt on every prompt — that would be wasteful. It's a relatively stable snapshot that refreshes only when the knowledge base actually changes (after commits).++#### Index format++The index is injected into the system prompt (via `before_agent_start`'s `systemPrompt` return). Capped at `indexBudget` tokens (~2000 default).++```+## OpenViking Knowledge Index+[Showing what's in your long-term memory]++### viking://user/memories/ (12 memories)+- Prefers local/self-hosted solutions over cloud services+- Project X uses SQLite, not PostgreSQL, pool size 5+- Chrome DevTools MCP gets stuck on closed tabs; pkill to fix+- pip "Successfully installed" can lie — verify with import+- (8 more — use viking_search to find specific memories)++### viking://resources/ (3 resources)+- OpenViking reference doc (viking://resources/openviking-reference)+- Project X architecture diagram (viking://resources/projx-arch)+- (1 more — use viking_browse to explore)++### viking://session/archives/ (2 archives)+- Archive 2026-05-25: 15-turn session about pi extension design+- (1 more — use viking_archive_expand for detail)++Tools: viking_search | viking_read | viking_browse | viking_remember | viking_forget | viking_add_resource | viking_archive_expand+```++#### Why not include full memory content++The index is intentionally a *table of contents*, not the full encyclopedia. Reasons:+- Token budget: full content of all memories would blow past system prompt limits.+- Relevance: most memories are irrelevant to the current task — that's what recall is for.+- Freshness: the index refreshes after commits, but recall is always current-turn.++The model sees the index and knows "OV knows about X, Y, Z." When a task touches those topics, it uses `viking_search` for depth or relies on automatic `<relevant-memories>` injection.++```typescript+class IndexBuilder {+ private client: OVClient;+ private cachedIndex: string | null;++ constructor(client: OVClient);++ // Build index from scratch — called at session_start and after commits+ async buildIndex(): Promise<string>;++ // Get cached index (returns empty string if not built or OV is down)+ getIndex(): string;+}+```++### recall.ts (~150 lines)++Synchronous recall that runs on every user prompt, injecting relevant OV context into the user message before the LLM sees it. This is the *flashlight* — targeted retrieval for the current query. The *index* (from `index_builder.ts`) is the *map* that helps the model know when to use it.++#### How it works++1. Extract query text from the user's prompt (plain text, no tool calls/images)+2. **Short-circuit**: if query length < `recallMinQueryLength` (default 3), skip recall — queries like "y", "ok", "go" don't carry enough signal for useful retrieval (from Claude Code plugin)+3. **Triple-scope parallel search**: three concurrent `client.find()` calls via `Promise.all` (from Claude Code plugin). All URIs are dynamically resolved via `client.resolveTargetUri()` to insert the correct user/agent namespace (see client.ts → URI space resolution):+ - `viking://user/memories` → resolved to `viking://user/<space>/memories` → user's personal memories+ - `viking://agent/memories` → resolved to `viking://agent/<space>/memories` → agent's operational memories+ - `viking://agent/skills` → resolved to `viking://agent/<space>/skills` → stored skills and procedures++ Resources are explicitly excluded from automatic recall (cross-namespace leakage prevention — resources can be searched on demand via `viking_search` with `scope`). Per-source limit: `max(recallLimit * 2, 8)` = up to 12 candidates per source (36 total before filtering).+4. **Query profiling**: analyze the query for intent signals before ranking (from Claude Code plugin):+ ```typescript+ function buildQueryProfile(query: string): QueryProfile {+ return {+ tokens: extractContentTokens(query), // content words minus stopwords+ wantsPreference: /prefer|favorite|like|want|usually|always|never/i.test(query),+ wantsTemporal: /when|yesterday|last |recent|ago|last week/i.test(query),+ };+ }+ ```+ This gates the category boosts — preference memories only get boosted when the query has preference intent, event memories only when the query has temporal intent. Without this, boosting everything on every query dilutes the signal.+5. **Score filter**: discard any results below `recallScoreThreshold` (default 0.35) — irrelevant results are worse than no results (from Claude Code plugin). Filtering is client-side, not server-side.+6. **Deduplication** with category-specific strategies (from Claude Code plugin):+ - Events/cases → dedupe by URI (same event can appear with different abstracts)+ - Everything else → dedupe by abstract text (lowercased), fall back to URI++ Without this, vector search can return near-duplicate results that waste the token budget.+7. **Rerank** beyond pure vector score using the query profile:+ - Leaf preference boost (+0.12): item level == 2 or URI ends in `.md`+ - Event boost (+0.10): query has temporal intent AND item is in events/cases category+ - Preference boost (+0.08): query has preference intent AND item is in preferences category+ - Lexical overlap boost (up to +0.20): query tokens found in item URI + abstract, normalized by min(tokens.length, 4)+8. **Content resolution** for each ranked item (from Claude Code plugin):+ - If `recallPreferAbstract` is true (default): use abstract/overview text from search results+ - If level is 2 (full content): fetch full body from `/api/v1/content/read`+ - Content capped per-item to `recallMaxContentChars` (default 500 chars) — prevents a single verbose memory from consuming the entire budget+9. **Token-budgeted formatting with graceful degradation** (from Claude Code plugin):+ - Process items in ranked order+ - Items within the total `recallBudget` (default 2000 tokens) get full content lines+ - Items beyond the budget are **degraded to URI + score hints** rather than dropped — the model can call `viking_read` to expand them+ - The first item is always included even if it exceeds the remaining budget+10. Format as `<relevant-memories>` block++#### Reranking++See step 7 above for exact boost values. The boosts are gated by the query profile — preference/event boosts only fire when the query has matching intent. This prevents diluting the ranking with irrelevant boosts.++The OpenClaw plugin uses a similar approach. Start with vector-score-only during initial development, add the full reranking pipeline once the extension is stable.++#### Injection via `context` event++The `context` event fires before each LLM call with a mutable deep copy of messages. This is pi's equivalent of OpenClaw's `assemble()`. On each call:++1. Find the user message that started this prompt (scan backwards for first `role: "user"`)+2. Check if `<relevant-memories>` already injected (idempotency — the `context` event fires per LLM iteration, not per prompt)+3. If not injected: prepend `<relevant-memories>` block to the user message content+4. If already injected: skip (cached from first context call for this prompt)++The recall search itself only runs once per prompt (cached in `before_agent_start`). The `context` handler just re-injects the cached block on each LLM iteration.++```typescript+class RecallManager {+ private client: OVClient;+ private cachedBlock: string | null;+ private promptId: string | null; // track which prompt this cache is for++ constructor(client: OVClient);++ // Called in before_agent_start — runs the actual triple-scope search + profiling + ranking+ async searchAndCache(userQuery: string): Promise<string | null>;++ // Called in context event — injects cached block into messages+ injectRecall(messages: Message[]): Message[];++ // Invalidate cache (called at agent_end)+ invalidate(): void;+}+```++**`<relevant-memories>` format (with degradation example):**+```+<relevant-memories>+[System note: The following is recalled memory from OpenViking, NOT new user input. Treat as informational background data.]+- [memory 0.87] User prefers local/self-hosted solutions over cloud services+- [memory 0.82] Project uses SQLite for local dev, pool size 5+- [skill 0.73] Use viking_read to expand: viking://agent/skills/deployment-checklist.md+</relevant-memories>+```++The third line shows a degraded hint — the item was beyond the content budget but still relevant. The model can expand it with `viking_read` if needed.++### sync.ts (~250 lines)++Handles turn archival, memory stripping, commit management, and compaction safety.++#### Session ID strategy++Use pi's session ID prefixed with `pi-` as the OV session ID. Prevents collisions with Hermes sessions (which use unprefixed UUIDs).++**Subagent isolation is natural, not managed.** Pi extensions don't have SubagentStart/SubagentStop events (those are Claude Code-specific hooks). Instead, when `task-tool` spawns a subagent, it's a separate pi process that loads extensions normally. The subagent's OV extension instance creates its own session with `pi-<subagentSessionId>` — inherently isolated because each pi process gets a unique session ID via `getSessionId()`. No parent management or special prefix needed. This is actually *better* than CC's approach: process-level isolation with zero coordination overhead, vs explicit hook-based session management.++The only requirement: subagents spawned for internal extension work (e.g., the learning extension's reviewer) pass `--no-extensions`, which prevents the OV extension from loading in the reviewer subprocess.++#### Turn archival++On each `turn_end`:+1. Extract user text + assistant text from the turn+2. **Strip all injected blocks** from both before sending to OV (see Memory Stripping below)+3. **Preserve tool USE inputs, drop tool RESULTS** (from Claude Code plugin): format each turn's tool interactions as `[tool: <name>]\n<input>`. Tool use inputs are agent-authored and carry signal ("the agent chose to read file X, run command Y"). Tool results are typically noise for memory extraction (file contents, command stdout). If `captureToolResults` is true, include tool results up to a reasonable cap.+4. **Add tool summary line** to assistant turn: `[assistant used tools: read, edit, bash]`. This gives OV's memory extractor context about what the agent *did* beyond prose — "ran bash, edited a file, then read another file" is more signal than just the assistant's text response (from Claude Code plugin).+5. Estimate token count for the stripped content (using CJK-aware estimator — see Token Estimation below)+6. Fire-and-forget batch add to OV session (non-blocking)+7. Track cumulative `pendingTokens`; check against threshold++#### Capture filtering (from Claude Code plugin)++Not every turn should be archived. One-word acknowledgments, slash commands, pure questions without substance, and punctuation-only turns carry zero signal for memory extraction and pollute the OV session. The Claude Code plugin's `shouldCapture()` filter pipeline prevents this noise from reaching OV (ported from `auto-capture.mjs`).++**Filter pipeline** (applied to each user turn before syncing):++1. **Empty check**: strip + trim → skip if empty.+2. **Length bounds**: skip if compact text < 4 chars (CJK) / 10 chars (Latin) or > `captureMaxLength` (default 24000). CJK uses a higher min-density because CJK chars carry more meaning per character.+3. **Command detection**: skip if text starts with `/` followed by a command name (e.g., `/help`, `/compact`). These are framework directives, not conversational content.+4. **Non-content detection**: skip if text is entirely punctuation/symbols/whitespace (no semantic content).+5. **Question-only detection**: skip if text matches the pattern `/^(who|what|when|where|why|how|...)...?[??]$/i` — pure interrogatives with no substance beyond the question itself.+6. **Keyword/semantic mode gate**: in `"keyword"` mode, skip unless at least one user turn matches a `MEMORY_TRIGGERS` regex. In `"semantic"` mode (default), skip this gate — always capture.++```typescript+const MEMORY_TRIGGERS = [+ /remember|preference|prefer|important|decision|decided|always|never/i,+ /[\w.-]+@[\w.-]+\.\w+/, // email patterns+ /(?:my)\s*(?:name|live|from|birthday|phone|email)/i, // identity signals+ /(?:i)\s*(?:like|hate|love|want|need|think|believe)/i, // preference signals+ /(?:favorite|favourite|love|hate|enjoy|dislike)/i,+];++function shouldCapture(text: string, mode: "semantic" | "keyword"): { capture: boolean; reason: string } {+ const normalized = stripAndTrim(text);+ if (!normalized) return { capture: false, reason: "empty" };++ const compact = normalized.replace(/\s+/g, "");+ const isCJK = /[-ヿ㐀-鿿豈-가-]/.test(compact);+ const minLen = isCJK ? 4 : 10;+ if (compact.length < minLen || normalized.length > config.captureMaxLength)+ return { capture: false, reason: "length_out_of_range" };++ if (/^\/[a-z0-9_-]{1,64}\b/i.test(normalized))+ return { capture: false, reason: "command" };++ if (/^[\p{P}\p{S}\s]+$/u.test(normalized))+ return { capture: false, reason: "non_content" };++ if (/^(who|what|when|where|why|how|is|are|does|did|can|could|would|should)\b.{0,200}[??]$/i.test(normalized))+ return { capture: false, reason: "question_only" };++ if (mode === "keyword") {+ const hasTrigger = MEMORY_TRIGGERS.some(re => re.test(normalized));+ return { capture: hasTrigger, reason: hasTrigger ? "trigger_matched" : "no_trigger" };+ }++ // semantic mode — always capture (default)+ return { capture: true, reason: "semantic" };+}+```++**Batch-level caveat** (from CC plugin): `shouldCapture()` is designed for single user messages. When applied to a concatenated multi-turn batch, it misfires (combined text exceeds max length → entire batch dropped, or a leading `/cmd` flips the whole batch to "command"). For pi's `turn_end` event, each turn is evaluated individually, so this problem doesn't arise. The filter runs per-turn before the dedup guard.++**Integration into turn archival**: `shouldCapture()` runs on the user's stripped text after step 2 (strip injected blocks). If the decision is `capture: false`, the turn is skipped entirely — no OV message push, no token count. The `syncedTurnCount` still advances so the dedup guard stays correct.++#### Memory stripping (critical)++Before syncing any content to OV, strip **all** injected/synthetic blocks — not just `<relevant-memories>`. The Claude Code plugin strips `<openviking-context>`, `<system-reminder>`, `<relevant-memories>`, and `[Subagent Context]` blocks. Without comprehensive stripping, OV indexes injected context as conversation, creating a feedback loop that pollutes future recall quality.++```typescript+function stripInjectedBlocks(text: string): string {+ // Strip all blocks that OV or the agent framework injects+ text = text.replace(/<relevant-memories>[\s\S]*?<\/relevant-memories>/g, "");+ text = text.replace(/<system-reminder>[\s\S]*?<\/system-reminder>/g, "");+ text = text.replace(/<openviking-context>[\s\S]*?<\/openviking-context>/g, "");+ text = text.replace(/\[Subagent Context\][\s\S]*?(?=\n\n|$)/g, "");+ text = text.replace(/\x00/g, ""); // null bytes from encoding issues+ return text.trim();+}+```++#### Token estimation (CJK-aware)++All token budgets in the spec use a CJK-aware estimator, not flat chars/4. The Claude Code plugin discovered that chars/4 silently undercounts CJK content by 4-6× — a "5000 token budget" with chars/4 becomes ~500 real tokens for Chinese text (from `profile-inject.mjs`).++```typescript+function estimateTokens(text: string): number {+ if (!text) return 0;+ let cjk = 0;+ for (let i = 0; i < text.length; i++) {+ if (text.charCodeAt(i) >= 0x3000) cjk++;+ }+ const other = text.length - cjk;+ return Math.ceil(cjk * 1.5 + other / 4);+}+```++Rule: codepoint >= 0x3000 (CJK / Hiragana / Katakana / Hangul / fullwidth) counts at 1.5 tokens/char. Everything else at chars/4. Errs on the side of overcounting CJK by ~10-20% — safe direction for budget enforcement.++This affects: recall budget (`recallBudget`), profile budget (`profileBudget`), index budget (`indexBudget`), and per-item content cap (`recallMaxContentChars`). The per-item cap is in chars but should be validated against the CJK-aware estimator for content known to be CJK-heavy.++#### Commit management++Three commit triggers (matching Claude Code plugin's three-way approach):++1. **Threshold commit**: When cumulative `pendingTokens` crosses `commitTokenThreshold` (default 20000), trigger `commit(wait=false)`. Token-based is more accurate than turn-based — a 1-line ack and a 10-tool-call turn are very different content volumes. Archive generation and memory extraction happen asynchronously on the OV server. If the session crashes at turn 80, memories from turns 1-70 are already committed.++2. **Pre-compact commit**: When pi fires `compaction` event (before it rewrites the transcript), trigger `commit(wait=true)`. This is critical — without it, content that gets compacted away is lost to OV forever. The Claude Code plugin's `PreCompact` hook does the same thing.++3. **Shutdown commit**: On `session_shutdown`, trigger `commit(wait=true)`. Blocks until extraction completes (with timeout). The safety net for the final turns.++```typescript+class SyncManager {+ private client: OVClient;+ private ovSessionId: string | null;+ private pendingTokens: number;+ private commitTokenThreshold: number;+ private syncedTurnCount: number; // incremental counter — prevents duplicate pushes+ private writeQueue: WriteQueue; // batches turns for efficient OV delivery+ private initialized: boolean;++ constructor(client: OVConfig, piSessionId: string, commitTokenThreshold: number);++ async ensureSession(): Promise<boolean>;+ async syncTurn(userMsg: string, assistantMsg: string, turnIndex: number): Promise<void>;+ // syncTurn runs shouldCapture() on userMsg, enqueues to writeQueue if passing,+ // checks turnIndex > syncedTurnCount before enqueuing (dedup guard),+ // estimates tokens (CJK-aware), adds to pendingTokens, checks threshold+ async commit(wait?: boolean): Promise<string | null>; // returns archive ID if committed+ async getPendingTokens(): Promise<number>; // fetches pending_tokens from session metadata+ async flushQueue(): Promise<void>; // flush write queue + check commit threshold+}++The `syncedTurnCount` counter prevents duplicate pushes if `turn_end` fires multiple times for the same turn (retries, errors). Each call checks `turnIndex > syncedTurnCount` before enqueuing; after successful enqueue, advances the counter. Persisted to a state file alongside the OV session so it survives across compactions within a session.+```++#### Write queue (async batching)++**Why pi doesn't need CC's detached-worker pattern.** CC hooks have strict timeouts (Stop = 45s, SessionEnd = 30s). If an HTTP call takes 20s, the user waits 20s. CC's `async-writer.mjs` solves this by draining stdin, approving immediately, and spawning a detached child process to do the HTTP work.++Pi extensions don't have this problem — event handlers are naturally async. `turn_end` handlers return promises; pi's event loop doesn't block the user on them. The spec already says "fire-and-forget" for turn additions.++**But pi benefits from batching.** Instead of one HTTP call per turn (CC's approach — `addMessage()` in a loop), a write queue accumulates turns locally and flushes them in a single batch. This reduces HTTP overhead from N round-trips to 1 per flush.++```typescript+class WriteQueue {+ private client: OVClient;+ private ovSessionId: string;+ private queue: { role: string; content: string }[];+ private flushTimer: NodeJS.Timeout | null;+ private flushIntervalMs: number; // default: 5000 (5 seconds)+ private flushThreshold: number; // default: 5 turns+ private flushing: boolean; // guard against concurrent flushes++ constructor(client: OVClient, ovSessionId: string);++ // Add a turn to the queue. Triggers flush if threshold reached.+ enqueue(role: string, content: string): void;++ // Flush all queued turns to OV in a single batch. Called automatically+ // at threshold or interval, and manually at pre-compact/shutdown.+ async flush(): Promise<void>;++ // Cancel any pending timer (called at shutdown).+ cancelPending(): void;+}+```++**Flush triggers:**+1. **Threshold**: when `queue.length >= flushThreshold` (default 5 turns), flush immediately.+2. **Interval**: a `setInterval` timer flushes every `flushIntervalMs` (default 5000ms). Catches the case where the user sends a few turns then pauses.+3. **Pre-compact**: `session_before_compact` calls `queue.flush()` synchronously before commit.+4. **Shutdown**: `session_shutdown` calls `queue.flush()` before final commit.++**Error handling**: if a flush fails (OV unreachable), the turns stay in the queue. The next flush attempt will retry them. This is a deliberate improvement over CC's approach, which advances the turn counter regardless of per-turn failures.++**Config additions for write queue:**+```typescript+writeQueueFlushInterval: number; // Flush interval in ms (default: 5000)+writeQueueFlushThreshold: number; // Flush after N queued turns (default: 5)+```++### tools.ts (~200 lines)++7 tools for agent-initiated OV operations. All tools use the shared `OVClient` instance.++#### `viking_search`+```typescript+{+ name: "viking_search",+ description: "Semantic search over the OpenViking knowledge base. Returns ranked results with viking:// URIs and abstracts. Use when you need to recall past decisions, user preferences, or project-specific knowledge not in current context.",+ promptSnippet: "Search OpenViking knowledge base for past decisions, preferences, and project knowledge",+ promptGuidelines: [+ "Use viking_search when you need information from previous sessions that may not be in MEMORY.md.",+ "Use viking_search before making decisions that might conflict with established patterns or past decisions.",+ ],+ parameters: Type.Object({+ query: Type.String({ description: "Search query" }),+ scope: Type.Optional(Type.String({ description: "Viking URI prefix to scope search (e.g., 'viking://resources/')" })),+ limit: Type.Optional(Type.Number({ description: "Max results (default: 10)" })),+ }),+}+```++#### `viking_read`+```typescript+{+ name: "viking_read",+ description: "Read content at a viking:// URI. Three detail levels: 'abstract' (~100 tokens), 'overview' (~2k tokens), 'full' (complete). Start with abstract, escalate to overview/full when needed.",+ promptSnippet: "Read OpenViking content at a viking:// URI with tiered detail levels",+ parameters: Type.Object({+ uri: Type.String({ description: "viking:// URI to read" }),+ level: StringEnum(["abstract", "overview", "full"] as const),+ }),+}+```++#### `viking_browse`+```typescript+{+ name: "viking_browse",+ description: "Browse the OpenViking knowledge store like a filesystem. List directory contents, get metadata, or view the hierarchy tree.",+ promptSnippet: "Browse the viking:// directory tree in OpenViking",+ parameters: Type.Object({+ action: StringEnum(["list", "stat"] as const),+ uri: Type.Optional(Type.String({ description: "viking:// URI (default: 'viking://')" })),+ }),+}+```++#### `viking_remember`+```typescript+{+ name: "viking_remember",+ description: "Store a fact or memory in OpenViking. Stored as a session message and extracted into long-term memory on commit. Use for important information the agent should remember: preferences, decisions, gotchas, lessons learned.",+ promptSnippet: "Store a fact in OpenViking for cross-session persistence",+ promptGuidelines: [+ "Use viking_remember for facts that should survive across sessions but don't belong in MEMORY.md.",+ "Good for: user preferences, architectural decisions, gotchas, environment details.",+ ],+ parameters: Type.Object({+ content: Type.String({ description: "The fact or observation to store" }),+ category: Type.Optional(Type.String({ description: "Category hint: 'preference', 'entity', 'event', 'case', 'pattern'" })),+ }),+}+```++#### `viking_forget`+```typescript+{+ name: "viking_forget",+ description: "Delete a memory by URI or search for a specific memory and remove it. Use to correct outdated or wrong information in the knowledge base.",+ promptSnippet: "Delete a memory from OpenViking by URI or query",+ parameters: Type.Object({+ uri: Type.Optional(Type.String({ description: "Exact viking:// URI to delete" })),+ query: Type.Optional(Type.String({ description: "Search query — deletes the strongest match if score > 0.8" })),+ }),+}+```++#### `viking_add_resource`+```typescript+{+ name: "viking_add_resource",+ description: "Ingest a URL, file path, or document into the OpenViking knowledge base. OV auto-processes it into L0/L1/L2 tiers and indexes it for semantic search. Use for bootstrapping knowledge or adding reference documentation.",+ promptSnippet: "Ingest a URL or document into OpenViking for indexed retrieval",+ parameters: Type.Object({+ url: Type.String({ description: "URL or file path to ingest" }),+ reason: Type.Optional(Type.String({ description: "Why this resource is relevant (improves indexing)" })),+ }),+}+```++#### `viking_archive_expand`+```typescript+{+ name: "viking_archive_expand",+ description: "Expand an archived session back into raw messages. Use when the archive summary is too coarse and you need the detailed conversation history. Returns the full message transcript for that archive.",+ promptSnippet: "Expand an archived session to see raw conversation messages",+ parameters: Type.Object({+ archive_id: Type.Optional(Type.String({ description: "Archive ID to expand (from session context)" })),+ session_id: Type.Optional(Type.String({ description: "OV session ID to expand" })),+ }),+}+```++### index.ts (~200 lines)++Main entry point. Wires everything together.++#### Event registrations++| Event | Handler | What it does |+|-------|---------|-------------|+| `session_start` | Init + Resume + Profile | Health check OV, check bypass, create/reuse session, **inject user profile** (profile.md + preferences/ + entities/ listing, capped at `profileBudget`), on resume: fetch archive overview, build memory index, register tools |+| `before_agent_start` | Recall + System prompt | Synchronous recall search, inject memory index + tool ad into system prompt |+| `context` | Recall injection | Prepend `<relevant-memories>` to user message (re-inject cached block on each LLM iteration) |+| `turn_end` | Sync | Strip all injected blocks, **capture filter (shouldCapture)**, **preserve tool USE inputs + tool summary line**, drop tool RESULTS, **enqueue to write queue** (auto-flushes at threshold/interval), track pending tokens, check commit threshold |+| `session_before_compact` | Pre-compact commit + rehydration | Synchronous `commit(wait=true)` before pi rewrites the transcript, then fetch new archive overview and cache for next `before_agent_start` injection — content about to be compacted is preserved in OV and rehydrated after compaction |+| `session_shutdown` | Final commit | Commit OV session (blocking), rebuild index, optionally mirror MEMORY.md |+| `agent_end` | Cleanup | Invalidate recall cache |++#### Guard pattern++Two-level guard:++1. **Health check**: At `session_start`, ping OV health. If unreachable: set `connected = false`, log once, all subsequent operations become no-ops. No retrying, no spamming. Tools return "OpenViking server is not reachable."++2. **Bypass check**: Before any OV operation, check `config.bypassPatterns` against `process.cwd()`. If the cwd matches any pattern (e.g., `/tmp/**`, `**/scratch/**`), skip all OV operations for this session. This prevents throwaway experiments from polluting long-term memory (from Claude Code plugin's `OPENVIKING_BYPASS_SESSION_PATTERNS`).++#### Session resume rehydration++When `session_start` fires with `reason: "resume"`, the session may have previous OV archives from a prior run. Fetch the latest archive overview (L1, ~2k tokens) and inject it alongside the memory index. This rehydrates the model's context with "what happened in the previous session" (from Claude Code plugin's SessionStart resume behavior).++#### System prompt injection++Via `before_agent_start`'s `systemPrompt` return field. Composes up to four things:++1. **Profile block** (from session_start cache) — user identity + preferences + entities. Capped at `profileBudget`. Only present if OV has a user profile.+2. **Archive overview** (from session_start resume OR pre-compact rehydration) — "what happened in previous sessions" or "what happened before compaction". Capped at `resumeContextBudget` tokens.+3. **Memory index** (from `index_builder.ts`) — a browsable table of contents showing what OV knows. Refreshed at session start and after commits.+4. **Tool advertisement** — the standard tool usage instructions.++```+## OpenViking Context+<openviking-context source="session-start">+<user-profile uri="viking://user/default/memories/profile.md">+User prefers local/self-hosted solutions...+</user-profile>+<available-memories>+ viking://user/default/memories/preferences/+ - dark_mode.md — prefers dark mode in all editors+ viking://user/default/memories/entities/+ - project_x.md — Project X uses SQLite+</available-memories>+</openviking-context>++[Session History Summary]+Archive 2026-05-27: 15-turn session about pi extension design...++## OpenViking Knowledge Index+[Showing what's in your long-term memory]++### viking://user/memories/ (12 memories)+- Prefers local/self-hosted solutions over cloud services+- ...++### viking://resources/ (3 resources)+- ...++Tools: viking_search | viking_read | viking_browse | viking_remember | viking_forget | viking_add_resource | viking_archive_expand+```++This is a key difference from Hermes (tool ad only, model is blind) and closer to OpenClaw's preflight `assemble()` (model sees archive overview + abstract index before deciding to search).++#### Memory mirroring at commit time++Don't intercept individual `write`/`edit` tool calls (fragile, complex). Instead, at `session_shutdown` commit time:+1. Read `.memory/MEMORY.md` if it exists+2. Write it to OV as `viking://user/memories/memory-md` (or append as a session message tagged `[Memory mirror]`)+3. OV's extraction picks it up during commit++Simple, correct, handles all edge cases (external edits, multiple writes, etc.).++#### Manual commit command++A `/viking commit` command (or a `viking_commit` tool) triggers a synchronous `commit(wait=true)`. This is the equivalent of OpenClaw's `compact()` — the user or agent can force a memory extraction mid-session without waiting for the token threshold. Useful when the user says "remember this" and wants immediate assurance that the memory was archived.++## Event Flow (Detailed)++### Session Start+```+1. session_start fires+2. Load config+3. Check bypassPatterns against cwd+ └── MATCH → set bypassed = true, skip all OV ops, return+4. client.health()+ ├── OK → connected = true, continue+ └── FAIL → connected = false, log once, return+5. sync.ensureSession() → create or reuse OV session "pi-{sessionId}"+6. **Profile injection** (all sessions, from Claude Code plugin):+ a. Resolve user space: `client.resolveScopeSpace("user")` → discover namespace via /api/v1/system/status + fs/ls+ b. Read profile.md from viking://user/<space>/memories/profile.md+ c. List preferences/ and entities/ directories with abstracts+ d. **Profile elision** (from Claude Code plugin): if profile exceeds `profileBudget` tokens, keep head (identity block, first 8 lines) + tail (most-recent events, fits remaining budget), drop noisy middle. Preserves both stable identity facts (top of file) and recent activity (bottom of file) — only the noisy middle timeline is sacrificed. Falls back to head-only truncate when file is too short to elide.+ e. Compose <openviking-context> block with user-profile + available-memories+ f. Capped at profileBudget tokens (default 10000) using CJK-aware estimator+ g. Cached for system prompt injection in before_agent_start+7. If event.reason == "resume":+ a. Fetch latest archive overview from OV (L1)+ b. Inject as [Session History Summary] alongside memory index+8. index_builder.buildIndex() → build memory index (viking:// tree + abstracts)+9. Register 7 tools+```++### Per Prompt (User sends message)+```+1. before_agent_start fires+ a. Extract user prompt text+ b. recall.searchAndCache(prompt) ← SYNCHRONOUS OV search (~50-200ms)+ c. Compose system prompt: event.systemPrompt + profileBlock + archiveOverview + indexBuilder.getIndex() + toolAdBlock+ - Profile block: cached from session_start (or empty if OV has no profile)+ - Archive overview: cached from session_start resume, or from pre-compact rehydration+ d. Return { systemPrompt: composed }++2. [For each LLM iteration within this prompt:]+ a. context event fires+ b. recall.injectRecall(event.messages) ← prepend cached <relevant-memories> to user message+ c. Return { messages: modified }++3. [Turns execute — LLM may call viking_search, etc.]++4. agent_end fires+ a. recall.invalidate() ← clear cached block+```++### Per Turn+```+1. turn_end fires+2. Extract user text + assistant text from event+3. Strip ALL injected blocks (<relevant-memories>, <system-reminder>, <openviking-context>, [Subagent Context], null bytes) from both+4. Preserve tool USE inputs as [tool: <name>]\n<input> — drop tool RESULTS (unless captureToolResults is true)+5. Add tool summary line to assistant turn: `[assistant used tools: read, edit, bash]`+6. **Capture filter**: shouldCapture(strippedUserText, captureMode)+ └── SKIP → advance syncedTurnCount, return (no OV push)+7. If captureAssistantTurns is false: only push user message, skip assistant+8. sync.syncTurn(strippedUser, strippedAssistant, turnIndex)+ a. Dedup guard: if turnIndex <= syncedTurnCount, skip (prevents duplicate pushes on retries)+ b. Enqueue turn to write queue (queue auto-flushes at threshold or interval)+ c. Estimate token count for stripped content (CJK-aware)+ d. pendingTokens += estimatedTokens+ e. If pendingTokens >= commitTokenThreshold: writeQueue.flush() then sync.commit(wait=false)+ f. syncedTurnCount = turnIndex + 1+```++### Pre-Compact+```+1. session_before_compact event fires (pi is about to rewrite the transcript)+2. writeQueue.flush() ← flush any queued turns before committing+3. sync.commit(wait=true) ← BLOCKING: archive all pending content before pi mutates it+3. Content about to be compacted away is now preserved in OV as an archive+4. **Post-compact rehydration**: fetch the newly-committed archive overview (L1) and cache it+5. On the next before_agent_start, inject the cached archive overview alongside the memory index+ → The model gets rehydrated with "what happened before compaction" from OV's long-term record+ → This mirrors CC's SessionStart(source="compact") dual injection pattern+```++### Session Shutdown+```+1. session_shutdown fires+2. writeQueue.cancelPending() ← cancel any pending flush timer+3. writeQueue.flush() ← flush remaining queued turns+4. If mirrorMemoryWrites: read .memory/MEMORY.md → send to OV as session message+5. sync.commit(wait=true) ← blocking, with timeout+6. index_builder.buildIndex() ← refresh index after commit (new memories extracted)+7. Cleanup+```++## Comparison: Hermes vs OpenClaw vs Claude Code vs This Extension++| Aspect | Hermes | OpenClaw | Claude Code | Pi Extension |+|--------|--------|----------|-------------|------------- |+| Plugin type | Built-in memory provider | Context engine plugin | CC hooks + MCP | pi extension |+| Recall mechanism | Stale background prefetch | Synchronous `assemble()` | Synchronous `UserPromptSubmit` hook | Synchronous `context` event |+| Recall timing | N-1 turn (wrong topic) | Current turn | Current turn | Current turn |+| First turn recall | Nothing | Relevant context | Relevant context | Relevant context + user profile |+| Search scope | Unscoped | Dual (user + agent) | Triple (user + agent + skills) | ✅ Triple (user + agent + skills) |+| Query profiling | None | None | ✅ Intent detection (preference/temporal) | ✅ Intent detection (preference/temporal) |+| Score threshold | None | None | 0.35 | ✅ 0.35 |+| Min query filter | None | None | 3 chars | ✅ 3 chars |+| Deduplication | None | None | ✅ By URI (events) + by abstract (others) | ✅ By URI (events) + by abstract (others) |+| Content resolution | None (abstracts only) | None | ✅ Tiered (abstract → overview → full) | ✅ Tiered (abstract preferred, full on demand) |+| Result degradation | None | None | ✅ URI hints beyond budget | ✅ URI hints beyond budget |+| Per-item content cap | None | None | ✅ 500 chars | ✅ 500 chars |+| Memory index in prompt | None (model blind) | Archive overview + abstracts | Archive overview on resume | ✅ viking:// tree + memory abstracts |+| Profile injection | None | None | ✅ profile.md + preferences + entities | ✅ profile.md + preferences + entities |+| Commit trigger | Session-end only | Token threshold | Token threshold + pre-compact + session-end | ✅ Token threshold + pre-compact + session-end |+| Memory stripping | None | Strip `<relevant-memories>` | Strip all injected blocks (5+ tag types + null bytes) | ✅ Strip all injected blocks (5+ tag types + null bytes) |+| Capture: tool use inputs | Not captured | Not captured | ✅ Preserved verbatim | ✅ Preserved verbatim |+| Capture: tool results | Not captured | Not captured | ✅ Dropped by default | ✅ Dropped by default (configurable) |+| Capture dedup | None | None | ✅ Incremental turn counter | ✅ Incremental turn counter |+| Pre-compact commit | N/A | N/A | ✅ `PreCompact` hook | ✅ `session_before_compact` event |+| Post-compact rehydration | N/A | N/A | ✅ SessionStart(source="compact") | ✅ Archive overview cached at pre-compact, injected at next before_agent_start |+| Token estimation | N/A | N/A | ✅ CJK-aware (1.5 tokens/char for CJK, chars/4 otherwise) | ✅ CJK-aware |+| Subagent isolation | None | None | ✅ Isolated OV sessions via hooks | ✅ Natural process-level isolation (separate pi process = separate session) |+| Session resume | N/A | Archive overview | ✅ Archive overview rehydration | ✅ Archive overview rehydration |+| Bypass patterns | None | None | ✅ Glob on cwd/session | ✅ Glob on cwd |+| Capture filtering | None | None | ✅ shouldCapture (length, command, question-only, keyword/semantic modes) | ✅ shouldCapture (length, command, question-only, keyword/semantic modes) |+| URI space resolution | None | None | ✅ resolveScopeSpace + resolveTargetUri | ✅ resolveScopeSpace + resolveTargetUri |+| Async write path | None | None | ✅ Detached-worker (hook timeout avoidance) | ✅ Write queue (batching, not timeout avoidance — pi events are async) |+| Tools | 5 | 8 | 9 (via MCP) | 7 |+| Dependencies | httpx (Python) | Pure HTTP client | Plain .mjs scripts, no deps | Zero npm deps (built-in `fetch`) |++## Why a Memory Index?++The spec has two complementary context mechanisms:++1. **Memory index** (from `index_builder.ts`) — a *map*. "Here's what OV knows about." Injected into the system prompt. Rebuilt at session start and after commits. Always visible to the model. ~2000 tokens.++2. **Recall** (from `recall.ts`) — a *flashlight*. "Here's what's relevant to the current query." Injected into the user message per-turn. Always current. ~2000 tokens.++Without the index, the model has no idea what categories of knowledge exist in OV. It can only retrieve what it thinks to ask for — the Hermes problem. With the index, the model sees "OV knows about my preferences, project X architecture, and debugging gotchas" and can proactively decide to search deeper when a task touches those topics.++This mirrors OpenClaw's preflight `assemble()` which provides `latest_archive_overview` (archive summary) and `pre_archive_abstracts` (memory abstracts) to the model. The index is the pi equivalent.++## Dependencies++**Zero npm dependencies.** Uses:+- Node.js built-in `fetch` (Node 18+, pi requires 18+)+- `@mariozechner/pi-coding-agent` (types, `isToolCallEventType`, `StringEnum`, `truncateHead`)+- `typebox` (tool parameter schemas)+- `@mariozechner/pi-ai` (`StringEnum` for Google-compatible enums)++## Implementation Order++1. **client.ts** — HTTP wrapper, testable independently against running OV+2. **sync.ts** — depends on client+3. **index_builder.ts** — depends on client+4. **recall.ts** — depends on client+5. **tools.ts** — depends on client+6. **index.ts** — wires everything, registers tools and events++## Testing Strategy++- **Unit test `client.ts`**: Run against live OV server at `127.0.0.1:1933`+- **Integration**: Start pi with extension, have a conversation, verify in OV studio that messages synced, recall blocks stripped, commit triggered+- **Recall accuracy**: Ask a question about a topic from a previous session, verify `<relevant-memories>` appears in context+- **Index visibility**: Start a new session, verify the system prompt contains the memory index with correct counts and abstracts+- **Tool test**: Call each of the 7 tools manually in a pi session+- **Pre-compact commit**: Have a 20+ turn conversation (triggering threshold commits), then trigger compaction. Verify that pre-compact commit fires and content is archived before compaction mutates the transcript.+- **Bypass**: Start pi in `/tmp`, verify no OV operations fire.+- **Session resume**: End a session with committed content, start new session with resume, verify archive overview appears in context.+- **Post-compact rehydration**: Trigger compaction in a long session (after threshold commit fires), verify that pre-compact commit archives content, and the next before_agent_start includes the new archive overview.+- **Capture filtering**: Send one-word turns ("ok", "y", "/help"), pure questions ("what is X?"), and substantive turns. Verify that noise turns are skipped and only substantive turns reach OV.+- **Keyword mode**: Set captureMode to "keyword", send turns with and without trigger phrases, verify only triggered turns are captured.+- **URI space resolution**: On a multi-user OV setup, verify that viking://user/memories resolves to viking://user/<alice>/memories correctly.+- **Write queue batching**: Send 3 turns quickly, verify they're queued and flushed as a batch at the threshold. Verify the flush timer fires after the interval if threshold isn't reached.++## Config File++Default: `~/.pi/agent/extensions/openviking/config.json`++```json+{+ "enabled": true,+ "endpoint": "http://127.0.0.1:1933",+ "apiKey": "",+ "account": "",+ "user": "",+ "agentId": "pi",+ "syncTurns": true,+ "recallBudget": 2000,+ "recallMaxContentChars": 500,+ "recallPreferAbstract": true,+ "recallLimit": 6,+ "recallScoreThreshold": 0.35,+ "recallMinQueryLength": 3,+ "profileBudget": 10000,+ "resumeContextBudget": 2000,+ "indexBudget": 2000,+ "commitTokenThreshold": 20000,+ "commitOnShutdown": true,+ "captureToolResults": false,+ "captureMode": "semantic",+ "captureMaxLength": 24000,+ "captureAssistantTurns": true,+ "mirrorMemoryWrites": true,+ "writeQueueFlushInterval": 5000,+ "writeQueueFlushThreshold": 5,+ "bypassPatterns": [],+ "logLevel": "error"+}+```diff --git a/examples/pi-coding-agent-extension/README.md b/examples/pi-coding-agent-extension/README.mdnew file mode 100644index 000000000..745e352d2--- /dev/null+++ b/examples/pi-coding-agent-extension/README.md@@ -0,0 +1,251 @@+# OpenViking Memory Extension for Pi Coding Agent++Long-term semantic memory for [Pi](https://github.com/mariozechner/pi-coding-agent) sessions, powered by [OpenViking](https://github.com/volcengine/OpenViking). Recall happens automatically before every prompt, capture happens after every turn, and sessions are committed for persistent memory extraction — all via pi's native extension API.++> Design informed by lessons from all three OpenViking agent plugins: synchronous recall from OpenClaw, production-hardened capture/ranking from Claude Code, and anti-patterns dodged from Hermes's stale prefetch approach. See [DESIGN.md](./DESIGN.md) for the full design spec with comparison tables and event flow diagrams.++## Quick Start++### Prerequisites++- **Pi coding agent** installed (`npm i -g @mariozechner/pi-coding-agent`)+- **Node.js 18+** (for the extension's TypeScript runtime)+- **An OpenViking server** reachable — local or remote++### 1. Have an OpenViking server reachable++Either run one locally or point at a remote one. The [quickstart guide](../../docs/en/getting-started/02-quickstart.md) walks through both options. Default port is `1933`; local mode runs without authentication.++Verify it's up:++```bash+curl http://localhost:1933/health # or your remote URL+```++### 2. Install the extension++Copy the extension directory into pi's extension folder:++```bash+mkdir -p ~/.pi/agent/extensions+cp -r examples/pi-coding-agent-extension ~/.pi/agent/extensions/openviking+```++Pi auto-discovers extensions in `~/.pi/agent/extensions/` — no explicit registration needed. The extension loads on next `pi` invocation.++### 3. Configure (optional)++The extension ships with defaults that work out-of-the-box for a local OpenViking server (`http://127.0.0.1:1933`, no auth). To connect to a remote server or tune behavior, edit `~/.pi/agent/extensions/openviking/config.json`:++```json+{+ "enabled": true,+ "endpoint": "https://your-openviking-server.example.com",+ "apiKey": "<your-api-key>",+ "account": "my-team",+ "user": "alice",+ "agentId": "pi"+}+```++All config fields can also be overridden via environment variables:++| Env Var | Config field | Default |+|--------------------------|------------------|-------------------------------|+| `OPENVIKING_URL` | `endpoint` | `http://127.0.0.1:1933` |+| `OPENVIKING_API_KEY` | `apiKey` | `""` (no auth) |+| `OPENVIKING_ACCOUNT` | `account` | `""` |+| `OPENVIKING_USER` | `user` | `""` |+| `OPENVIKING_AGENT_ID` | `agentId` | `"pi"` |++Env vars take priority over `config.json`.++### 4. Start Pi++```bash+pi+```++The extension shows an `[OpenViking]` status line on startup. Tools (`viking_search`, `viking_remember`, etc.) are registered automatically. Memories persist across sessions — no additional setup.++## Configuration Reference++### Tuning fields++All fields below live in `config.json`. Defaults are shown.++| Field | Default | Description |+|--------------------------|------------|--------------------------------------------------------------------------|+| `enabled` | `true` | Set `false` to disable the extension entirely |+| `endpoint` | (local) | OpenViking server URL |+| `apiKey` | `""` | API key for remote servers |+| `account` | `""` | Multi-tenant account (`X-OpenViking-Account`) |+| `user` | `""` | Multi-tenant user (`X-OpenViking-User`) |+| `agentId` | `"pi"` | Agent identity (`X-OpenViking-Agent`) |++### Recall tuning++| Field | Default | Description |+|--------------------------|------------|--------------------------------------------------------------------------|+| `recallBudget` | `2000` | Token budget for inline recall content |+| `recallMaxContentChars` | `500` | Per-item content cap for search results |+| `recallPreferAbstract` | `true` | Prefer L0 abstract over L2 full body when available |+| `recallLimit` | `6` | Max memories to inject per prompt |+| `recallScoreThreshold` | `0.35` | Min relevance score (0–1) |+| `recallMinQueryLength` | `3` | Skip recall for queries shorter than N characters |++### Capture tuning++| Field | Default | Description |+|--------------------------|------------|--------------------------------------------------------------------------|+| `syncTurns` | `true` | Enable auto-capture of conversation turns |+| `captureMode` | `"semantic"` | `"semantic"` (always capture) or `"keyword"` (trigger-based) |+| `captureMaxLength` | `24000` | Max sanitized text length for the capture decision |+| `captureAssistantTurns` | `true` | Include assistant turns (text + tool USE inputs) |+| `captureToolResults` | `false` | Include tool result output (noisy — off by default) |+| `commitTokenThreshold` | `20000` | Pending-token threshold for client-driven commit |+| `commitOnShutdown` | `true` | Auto-commit session on exit |++### Injection tuning++| Field | Default | Description |+|--------------------------|------------|--------------------------------------------------------------------------|+| `profileBudget` | `10000` | Token budget for user profile block |+| `resumeContextBudget` | `2000` | Token budget for archive overview on session resume |+| `indexBudget` | `2000` | Token budget for the knowledge index (map of what OV knows) |++### Misc++| Field | Default | Description |+|--------------------------|------------|--------------------------------------------------------------------------|+| `mirrorMemoryWrites` | `true` | Mirror built-in memory writes into the OV session |+| `writeQueueFlushInterval`| `5000` | Write queue flush interval (ms) |+| `writeQueueFlushThreshold`| `5` | Write queue flush after N turns |+| `bypassPatterns` | `[]` | Glob patterns to skip extension processing |+| `logLevel` | `"error"` | `"silent"`, `"error"`, or `"info"` |++## Architecture++```+┌──────────────────────────────────────────────────────┐+│ Pi Coding Agent │+│ │+│ session_start before_agent_start context turn_end│+│ session_before_compact session_shutdown │+└────────┬──────────────────┬───────────┬──────────────┘+ │ │ │+ │ ┌───────────────▼───────────▼────────┐+ │ │ extension modules (.ts) │+ │ │ client / sync / recall / tools │──────► OpenViking+ │ └─────────────────────────────────────┘ Server+ │ (HTTP API)+ │ ┌──────────────────────────────────────┐+ └──► 7 registered LLM tools │+ │ viking_search / viking_read / … │+ └──────────────────────────────────────┘+```++The extension is a single directory of TypeScript files loaded by pi's `jiti` transpiler — no build step, no npm dependencies, no MCP server. All communication goes over HTTP to the OpenViking REST API.++### Event Flow++| Pi Event | Extension Action |+|------------------------|----------------------------------------------------------------------------------|+| `session_start` | Health check → create/find OV session → build profile block → build memory index → register tools |+| `before_agent_start` | Fallback tool registration (for `pi -c` resume) + inject archive overview |+| `context` | Search OV with current prompt → inject `<relevant-memories>` block |+| `turn_end` | Extract user + assistant turns → queue writes to OV session |+| `session_before_compact`| Commit pending messages before pi rewrites the transcript |+| `session_shutdown` | Final commit so the last window is archived |++### Recall: Synchronous, Not Stale++Unlike Hermes's stale prefetch (recall from previous turn's query, injected one turn late), this extension searches OpenViking with the **current** user prompt via pi's `context` event. Results are injected into the same turn as `<relevant-memories>` blocks. This means:++- **First turn** of a session gets relevant context immediately+- **Topic switches** within a session get correct recall+- No waiting for the next turn to see relevant memories++### Memory Pollution Prevention++Before pushing turns to OpenViking, the extension strips injected context blocks (`<openviking-context>`, `<relevant-memories>`, `<system-reminder>`) to prevent a self-referential pollution loop where recall context is captured back as user messages.++### Tool Use Preservation++Tool capture preserves agent-authored inputs (`[tool: read]\n<path>`) and drops raw tool results by default. The memory extractor sees what the agent *did*, not megabytes of raw output.++## LLM Tools++The extension registers 7 tools that pi's model can invoke on demand:++| Tool | Description |+|--------------------------|------------------------------------------------------------|+| `viking_search` | Semantic search across memories, resources, and skills |+| `viking_read` | Read a `viking://` URI at abstract / overview / full level |+| `viking_browse` | List directory contents or stat a `viking://` URI |+| `viking_remember` | Store a fact or preference into long-term memory |+| `viking_forget` | Delete a memory by URI or search query |+| `viking_add_resource` | Ingest a URL into OpenViking for indexed retrieval |+| `viking_archive_expand` | Expand an archived session back into raw conversation |++The canonical `/viking` command (type `/viking` in pi's chat) displays connection status, session info, and accepts `commit` for manual synchronous commit.++## Compared to Pi's Built-in Memory++Pi has a built-in `MEMORY.md` file system. This extension **complements** it:++| Feature | Built-in `MEMORY.md` | OpenViking extension |+|--------------|-----------------------------------|---------------------------------------------------|+| Storage | Flat markdown | Vector DB + structured extraction |+| Search | Loaded into context wholesale | Semantic similarity + ranking + token budget |+| Scope | Per-project | Cross-project, cross-session, cross-agent |+| Capacity | Context-limited | Unlimited (server-side storage) |+| Extraction | Manual rules | LLM-powered entity / preference / event extraction|+| Subagents | Same as parent | Isolated session + typed agent namespace |++## Compared to Claude Code Plugin++Both plugins share the same core design (informed by each other):++| Feature | Claude Code Plugin | Pi Extension |+|---------------------|----------------------------------------|----------------------------------------|+| Architecture | Hook scripts (.mjs) + MCP delegation | Native TypeScript extension |+| Recall timing | Synchronous (UserPromptSubmit hook) | Synchronous (context event) |+| Tool delivery | OV server's MCP endpoint (9 tools) | pi.registerTool() (7 tools) |+| Write path | Detached worker (async) | Async promise (pi's event loop) |+| Installation | `claude plugin install` + setup script | Copy directory → auto-discovered |+| Memory index | None (flashlight search model) | Built (map model — model sees what OV knows) |+| Subagent isolation | Explicit hook management | Natural process-level isolation |++## Extension Structure++See [DESIGN.md](./DESIGN.md) for the full design specification — comparison of all three OV plugins, detailed event flow, design rationale, and implementation guidance useful for building OV extensions for any agent harness.++```+pi-coding-agent-extension/+├── config.json # Default configuration (edit to customize)+├── config.ts # Config loader (defaults + config.json merge)+├── client.ts # OpenViking HTTP client (fetch + response envelope)+├── sync.ts # Turn capture, write queue, session lifecycle+├── recall.ts # Synchronous recall with ranking + budget+├── tools.ts # 7 registered LLM tools + /viking command+├── index-builder.ts # Memory index builder (knowledge map)+├── index.ts # Extension entry point (event handlers)+└── README.md+```++All TypeScript files are loaded directly by pi's built-in `jiti` transpiler — zero dependencies beyond Node.js.++## Troubleshooting++| Symptom | Cause | Fix |+|-----------------------------------------|------------------------------------------------------|-------------------------------------------------------------|+| Extension not loading | `enabled: false` in config.json | Set `"enabled": true` |+| No recall on first prompt | OpenViking server not running or wrong URL | `curl http://localhost:1933/health` |+| Tools not showing after `pi -c` resume | Known pi issue (tools not re-registered on resume) | Workaround built in — tools register in `before_agent_start`|+| Extension crashes on load | Wrong OV server URL or network issue | Check `logLevel` and server accessibility |+| No memories extracted | Wrong embedding/extraction model in OV config | Check OV's `embedding` / `vlm` configuration |++## License++Apache-2.0 — same as [OpenViking](https://github.com/volcengine/OpenViking).diff --git a/examples/pi-coding-agent-extension/client.ts b/examples/pi-coding-agent-extension/client.tsnew file mode 100644index 000000000..158fa8387--- /dev/null+++ b/examples/pi-coding-agent-extension/client.ts@@ -0,0 +1,375 @@+import type { OVConfig } from "./config.js";++// --- OV API Response Shapes ---+// All OV responses wrap in: { status: "ok"|"error", result: T, error?: {...}, ... }+// This client normalizes to { ok, result } internally.++export interface OVSearchResult {+ uri: string;+ context_type: string; // "memory" | "resource" | "skill"+ score: number;+ abstract: string;+ overview: string | null;+ level: number; // 0=L0, 1=L1, 2=L2+ category: string;+ match_reason: string;+}++export interface OVDirEntry {+ uri: string;+ name: string;+ isDir: boolean;+ size: number;+ mode: number;+ modTime: string;+ abstract: string;+}++export interface OVStatInfo {+ name: string;+ size: number;+ mode: number;+ modTime: string;+ isDir: boolean;+ isLocked: boolean;+ uri?: string;+ count?: number; // directories only+}++export interface OVSessionMeta {+ session_id: string;+ message_count: number;+ total_message_count?: number;+ commit_count: number;+ pending_tokens?: number;+ memories_extracted?: Record<string, number>;+ last_commit_at?: string;+}++export interface OVSessionContext {+ latest_archive_overview: string | null;+ pre_archive_abstracts: any[];+ messages: any[];+ estimatedTokens: number;+ stats: {+ totalArchives: number;+ includedArchives: number;+ droppedArchives: number;+ failedArchives: number;+ activeTokens: number;+ archiveTokens: number;+ };+}++export class OVClient {+ private baseUrl: string;+ private apiKey: string;+ private account: string;+ private user: string;+ private agent: string;+ connected: boolean = false;++ private resolvedSpaces: Map<string, string> = new Map();++ private static RESERVED_USER = new Set(["memories"]);+ private static RESERVED_AGENT = new Set(["memories", "skills", "instructions", "workspaces"]);++ /** Read-only access to config (for value access across modules). */+ readonly cfg: OVConfig;++ constructor(config: OVConfig) {+ this.cfg = config;+ this.baseUrl = config.endpoint.replace(/\/+$/, "");+ this.apiKey = config.apiKey;+ this.account = config.account;+ this.user = config.user;+ this.agent = config.agentId;+ }++ private headers(): Record<string, string> {+ const h: Record<string, string> = { "Content-Type": "application/json" };+ if (this.apiKey) h["Authorization"] = `Bearer ${this.apiKey}`;+ if (this.account) h["X-OpenViking-Account"] = this.account;+ if (this.user) h["X-OpenViking-User"] = this.user;+ if (this.agent) h["X-OpenViking-Agent"] = this.agent;+ return h;+ }++ /** Core fetch wrapper. Returns { ok, result } after parsing OV's { status, result } envelope. */+ private async fetchJSON<T>(path: string, init?: RequestInit, timeoutMs = 10000): Promise<{ ok: boolean; result: T | null; error?: any }> {+ try {+ const controller = new AbortController();+ const timer = setTimeout(() => controller.abort(), timeoutMs);+ const resp = await fetch(`${this.baseUrl}${path}`, {+ ...init,+ headers: { ...this.headers(), ...(init?.headers as Record<string, string> || {}) },+ signal: controller.signal,+ });+ clearTimeout(timer);+ const body = await resp.json().catch(() => ({}));+ if (!resp.ok || body.status === "error") {+ return { ok: false, result: null, error: body.error || { message: `HTTP ${resp.status}` } };+ }+ return { ok: true, result: (body.result ?? body) as T };+ } catch (err: any) {+ return { ok: false, result: null, error: { message: err?.message || String(err) } };+ }+ }++ // ========== Health ==========++ async health(): Promise<boolean> {+ const res = await this.fetchJSON<any>("/health", undefined, 5000);+ this.connected = res.ok;+ return res.ok;+ }++ // ========== Sessions ==========++ /** POST /api/v1/sessions — create or reuse session */+ async createSession(sessionId: string): Promise<boolean> {+ const res = await this.fetchJSON<any>("/api/v1/sessions", {+ method: "POST",+ body: JSON.stringify({ session_id: sessionId }),+ });+ return res.ok;+ }++ /** GET /api/v1/sessions/{id} — session metadata */+ async getSession(sessionId: string, autoCreate = false): Promise<OVSessionMeta | null> {+ const q = autoCreate ? "?auto_create=true" : "";+ const res = await this.fetchJSON<OVSessionMeta>(+ `/api/v1/sessions/${encodeURIComponent(sessionId)}${q}`,+ undefined, 5000,+ );+ return res.ok ? res.result : null;+ }++ /** GET /api/v1/sessions/{id}/context — assembled context with archive overview */+ async getSessionContext(sessionId: string, tokenBudget = 128000): Promise<OVSessionContext | null> {+ const res = await this.fetchJSON<OVSessionContext>(+ `/api/v1/sessions/${encodeURIComponent(sessionId)}/context?token_budget=${tokenBudget}`,+ undefined, 10000,+ );+ return res.ok ? res.result : null;+ }++ /** POST /api/v1/sessions/{id}/messages — add a message (simple text mode) */+ async addMessage(sessionId: string, role: string, content: string): Promise<boolean> {+ const res = await this.fetchJSON<any>(+ `/api/v1/sessions/${encodeURIComponent(sessionId)}/messages`,+ { method: "POST", body: JSON.stringify({ role, content }) },+ 10000,+ );+ return res.ok;+ }++ /** POST /api/v1/sessions/{id}/messages — add a message with parts */+ async addMessageParts(sessionId: string, role: string, parts: any[]): Promise<boolean> {+ const res = await this.fetchJSON<any>(+ `/api/v1/sessions/${encodeURIComponent(sessionId)}/messages`,+ { method: "POST", body: JSON.stringify({ role, parts }) },+ 10000,+ );+ return res.ok;+ }++ /** POST /api/v1/sessions/{id}/commit — commit session for archiving + extraction */+ async commitSession(sessionId: string): Promise<{ task_id: string; archive_uri: string } | null> {+ const res = await this.fetchJSON<{ task_id: string; archive_uri: string }>(+ `/api/v1/sessions/${encodeURIComponent(sessionId)}/commit`,+ { method: "POST", body: JSON.stringify({}) },+ 30000,+ );+ return res.ok ? res.result : null;+ }++ /** DELETE /api/v1/sessions/{id} */+ async deleteSession(sessionId: string): Promise<boolean> {+ const res = await this.fetchJSON<any>(+ `/api/v1/sessions/${encodeURIComponent(sessionId)}`,+ { method: "DELETE" },+ 10000,+ );+ return res.ok;+ }++ // ========== Search ==========++ /** POST /api/v1/search/find — basic vector search */+ async find(+ query: string,+ opts?: { targetUri?: string; topK?: number; scoreThreshold?: number },+ ): Promise<OVSearchResult[]> {+ const body: Record<string, unknown> = { query };+ if (opts?.targetUri) body.target_uri = opts.targetUri;+ if (opts?.topK) body.limit = opts.topK;+ if (opts?.scoreThreshold) body.score_threshold = opts.scoreThreshold;++ const res = await this.fetchJSON<any>("/api/v1/search/find", {+ method: "POST", body: JSON.stringify(body),+ }, 10000);+ if (!res.ok || !res.result) return [];++ // OV returns { memories: [...], resources: [...], skills: [...], total }+ const all: OVSearchResult[] = [];+ for (const bucket of ["memories", "resources", "skills"]) {+ const items = res.result[bucket];+ if (Array.isArray(items)) {+ for (const m of items) {+ all.push({+ uri: m.uri ?? "",+ context_type: m.context_type ?? bucket === "memories" ? "memory" : bucket === "skills" ? "skill" : "resource",+ score: m.score ?? 0,+ abstract: m.abstract ?? "",+ overview: m.overview ?? null,+ level: m.level ?? 0,+ category: m.category ?? "",+ match_reason: m.match_reason ?? "",+ });+ }+ }+ }+ return all;+ }++ // ========== Content ==========++ /** GET /api/v1/content/abstract — L0 summary */+ async abstract(uri: string): Promise<string | null> {+ const res = await this.fetchJSON<string>(+ `/api/v1/content/abstract?uri=${encodeURIComponent(uri)}`,+ undefined, 10000,+ );+ return res.ok ? res.result : null;+ }++ /** GET /api/v1/content/overview — L1 overview (directories only) */+ async overview(uri: string): Promise<string | null> {+ const res = await this.fetchJSON<string>(+ `/api/v1/content/overview?uri=${encodeURIComponent(uri)}`,+ undefined, 10000,+ );+ return res.ok ? res.result : null;+ }++ /** GET /api/v1/content/read — L2 full content (files only) */+ async readContent(uri: string): Promise<string | null> {+ const res = await this.fetchJSON<string>(+ `/api/v1/content/read?uri=${encodeURIComponent(uri)}`,+ undefined, 10000,+ );+ return res.ok ? res.result : null;+ }++ // ========== Filesystem ==========++ /** GET /api/v1/fs/ls — list directory */+ async ls(uri: string): Promise<OVDirEntry[]> {+ const res = await this.fetchJSON<any[]>(+ `/api/v1/fs/ls?uri=${encodeURIComponent(uri)}`,+ undefined, 10000,+ );+ if (!res.ok || !Array.isArray(res.result)) return [];+ return res.result.map(e => ({+ uri: e.uri ?? "",+ name: e.name ?? uriBasename(e.uri ?? ""),+ isDir: e.isDir ?? false,+ size: e.size ?? 0,+ mode: e.mode ?? 0,+ modTime: e.modTime ?? "",+ abstract: e.abstract ?? "",+ }));+ }++ /** GET /api/v1/fs/stat — file/directory metadata */+ async stat(uri: string): Promise<OVStatInfo | null> {+ const res = await this.fetchJSON<OVStatInfo>(+ `/api/v1/fs/stat?uri=${encodeURIComponent(uri)}`,+ undefined, 10000,+ );+ return res.ok ? res.result : null;+ }++ /** DELETE /api/v1/fs — remove file or directory */+ async delete(uri: string, recursive = false): Promise<boolean> {+ const res = await this.fetchJSON<any>(+ `/api/v1/fs?uri=${encodeURIComponent(uri)}&recursive=${recursive}`,+ { method: "DELETE" },+ 10000,+ );+ return res.ok;+ }++ // ========== Resources ==========++ /** POST /api/v1/resources — ingest a URL or file path */+ async addResource(+ path: string, opts?: { to?: string },+ ): Promise<{ root_uri: string } | null> {+ const body: Record<string, unknown> = { path };+ if (opts?.to) body.to = opts.to;+ const res = await this.fetchJSON<{ root_uri: string }>(+ "/api/v1/resources",+ { method: "POST", body: JSON.stringify(body) },+ 30000,+ );+ return res.ok ? res.result : null;+ }++ // ========== URI Space Resolution ==========++ async resolveScopeSpace(scope: "user" | "agent"): Promise<string> {+ const cached = this.resolvedSpaces.get(scope);+ if (cached) return cached;++ // Probe system status for user identity fallback+ let fallbackSpace = "default";+ const statusRes = await this.fetchJSON<any>("/api/v1/system/status", undefined, 5000);+ if (statusRes.ok && typeof statusRes.result?.user === "string" && statusRes.result.user.trim()) {+ fallbackSpace = statusRes.result.user.trim();+ }++ // List scope root for actual namespaces+ const reserved = scope === "user" ? OVClient.RESERVED_USER : OVClient.RESERVED_AGENT;+ const entries = await this.ls(`viking://${scope}/`);+ const spaces = entries+ .filter(e => e.isDir && !e.name.startsWith(".") && !reserved.has(e.name))+ .map(e => e.name);++ if (spaces.length > 0) {+ // Prefer the fallback space if it exists, then "default", then first available+ let chosen = spaces[0];+ if (spaces.includes(fallbackSpace)) chosen = fallbackSpace;+ else if (spaces.includes("default")) chosen = "default";+ this.resolvedSpaces.set(scope, chosen);+ return chosen;+ }++ this.resolvedSpaces.set(scope, fallbackSpace);+ return fallbackSpace;+ }++ async resolveTargetUri(targetUri: string): Promise<string> {+ const trimmed = targetUri.trim().replace(/\/+$/, "");+ const m = trimmed.match(/^viking:\/\/(user|agent)(?:\/(.*))?$/);+ if (!m) return trimmed;+ const scope = m[1] as "user" | "agent";+ const rawRest = (m[2] ?? "").trim();+ if (!rawRest) return trimmed;+ const parts = rawRest.split("/").filter(Boolean);+ if (parts.length === 0) return trimmed;++ const reserved = scope === "user" ? OVClient.RESERVED_USER : OVClient.RESERVED_AGENT;+ if (!reserved.has(parts[0])) return trimmed; // already has space++ const space = await this.resolveScopeSpace(scope);+ return `viking://${scope}/${space}/${parts.join("/")}`;+ }+}++function uriBasename(uri: string): string {+ const cleaned = uri.replace(/\/+$/, "");+ const last = cleaned.lastIndexOf("/");+ return last >= 0 ? cleaned.slice(last + 1) : cleaned;+}diff --git a/examples/pi-coding-agent-extension/config.json b/examples/pi-coding-agent-extension/config.jsonnew file mode 100644index 000000000..7c388eed3--- /dev/null+++ b/examples/pi-coding-agent-extension/config.json@@ -0,0 +1,29 @@+{+ "enabled": true,+ "endpoint": "http://127.0.0.1:1933",+ "apiKey": "",+ "account": "",+ "user": "",+ "agentId": "pi",+ "syncTurns": true,+ "recallBudget": 2000,+ "recallMaxContentChars": 500,+ "recallPreferAbstract": true,+ "recallLimit": 6,+ "recallScoreThreshold": 0.35,+ "recallMinQueryLength": 3,+ "profileBudget": 10000,+ "resumeContextBudget": 2000,+ "indexBudget": 2000,+ "commitTokenThreshold": 20000,+ "commitOnShutdown": true,+ "captureToolResults": false,+ "captureMode": "semantic",+ "captureMaxLength": 24000,+ "captureAssistantTurns": true,+ "mirrorMemoryWrites": true,+ "writeQueueFlushInterval": 5000,+ "writeQueueFlushThreshold": 5,+ "bypassPatterns": [],+ "logLevel": "error"+}diff --git a/examples/pi-coding-agent-extension/config.ts b/examples/pi-coding-agent-extension/config.tsnew file mode 100644index 000000000..ec55de51a--- /dev/null+++ b/examples/pi-coding-agent-extension/config.ts@@ -0,0 +1,74 @@+import { readFileSync, existsSync } from "node:fs";+import { join, dirname } from "node:path";++export interface OVConfig {+ enabled: boolean;+ endpoint: string;+ apiKey: string;+ account: string;+ user: string;+ agentId: string;+ syncTurns: boolean;+ recallBudget: number;+ recallMaxContentChars: number;+ recallPreferAbstract: boolean;+ recallLimit: number;+ recallScoreThreshold: number;+ recallMinQueryLength: number;+ profileBudget: number;+ resumeContextBudget: number;+ indexBudget: number;+ commitTokenThreshold: number;+ commitOnShutdown: boolean;+ captureToolResults: boolean;+ captureMode: "semantic" | "keyword";+ captureMaxLength: number;+ captureAssistantTurns: boolean;+ mirrorMemoryWrites: boolean;+ writeQueueFlushInterval: number;+ writeQueueFlushThreshold: number;+ bypassPatterns: string[];+ logLevel: "silent" | "error" | "info";+}++const DEFAULT_CONFIG: OVConfig = {+ enabled: true,+ endpoint: "http://127.0.0.1:1933",+ apiKey: "",+ account: "",+ user: "",+ agentId: "pi",+ syncTurns: true,+ recallBudget: 2000,+ recallMaxContentChars: 500,+ recallPreferAbstract: true,+ recallLimit: 6,+ recallScoreThreshold: 0.35,+ recallMinQueryLength: 3,+ profileBudget: 10000,+ resumeContextBudget: 2000,+ indexBudget: 2000,+ commitTokenThreshold: 20000,+ commitOnShutdown: true,+ captureToolResults: false,+ captureMode: "semantic",+ captureMaxLength: 24000,+ captureAssistantTurns: true,+ mirrorMemoryWrites: true,+ writeQueueFlushInterval: 5000,+ writeQueueFlushThreshold: 5,+ bypassPatterns: [],+ logLevel: "error",+};++export function loadConfig(extensionDir: string): OVConfig {+ const configPath = join(extensionDir, "config.json");+ if (!existsSync(configPath)) return { ...DEFAULT_CONFIG };++ try {+ const file = JSON.parse(readFileSync(configPath, "utf8"));+ return { ...DEFAULT_CONFIG, ...file };+ } catch {+ return { ...DEFAULT_CONFIG };+ }+}diff --git a/examples/pi-coding-agent-extension/index-builder.ts b/examples/pi-coding-agent-extension/index-builder.tsnew file mode 100644index 000000000..f18d028bf--- /dev/null+++ b/examples/pi-coding-agent-extension/index-builder.ts@@ -0,0 +1,80 @@+import type { OVClient } from "./client.js";+import type { OVConfig } from "./config.js";++export class IndexBuilder {+ private client: OVClient;+ private config: OVConfig;+ private cachedIndex = "";++ constructor(client: OVClient, config: OVConfig) {+ this.client = client;+ this.config = config;+ }++ async buildIndex(): Promise<string> {+ if (!this.client.connected) return "";++ const sections: string[] = ["## OpenViking Knowledge Index"];++ try {+ // 1. User memories+ const memUri = await this.client.resolveTargetUri("viking://user/memories");+ const memEntries = await this.client.ls(memUri);+ const memAbstracts: string[] = [];+ for (const entry of memEntries.slice(0, 20)) {+ if (!entry.isDir) {+ const abs = await this.client.abstract(+ `${memUri}/${entry.name}`,+ );+ if (abs) memAbstracts.push(abs);+ }+ }+ sections.push(`### ${memUri} (${memEntries.length} entries)`);+ for (const a of memAbstracts.slice(0, 10)) {+ sections.push(`- ${a}`);+ }+ if (memAbstracts.length > 10) {+ sections.push(`- (${memAbstracts.length - 10} more — use viking_search)`);+ }++ // 2. Resources+ const resEntries = await this.client.ls("viking://resources/");+ if (resEntries.length > 0) {+ sections.push(`### viking://resources/ (${resEntries.length} resources)`);+ for (const e of resEntries.slice(0, 5)) {+ sections.push(`- ${e.name}`);+ }+ }+ } catch {+ // Graceful degradation — return what we have+ }++ // 3. Tools advertisement+ sections.push(+ "Tools: viking_search | viking_read | viking_browse | viking_remember | viking_forget | viking_add_resource | viking_archive_expand",+ );++ const index = sections.join("\n");++ // Token budget enforcement+ let tokens = 0;+ let cjk = 0;+ for (let i = 0; i < index.length; i++) {+ if (index.charCodeAt(i) >= 0x3000) cjk++;+ }+ tokens = Math.ceil(cjk * 1.5 + (index.length - cjk) / 4);++ if (tokens > this.config.indexBudget) {+ // Truncate to fit+ this.cachedIndex = index.slice(0, this.config.indexBudget * 3);+ } else {+ this.cachedIndex = index;+ }++ return this.cachedIndex;+ }++ getIndex(): string {+ return this.cachedIndex;+ }+}diff --git a/examples/pi-coding-agent-extension/index.ts b/examples/pi-coding-agent-extension/index.tsnew file mode 100644index 000000000..a438210f7--- /dev/null+++ b/examples/pi-coding-agent-extension/index.ts@@ -0,0 +1,370 @@+/**+ * Pi OpenViking Extension+ *+ * Integrates pi with an OpenViking context database for persistent,+ * cross-session memory. Syncs conversation turns to OV, recalls+ * relevant memories on each prompt, and commits sessions for long-term+ * memory extraction.+ *+ * Design informed by: OpenClaw (synchronous recall), Claude Code plugin+ * (most mature, production-hardened), Hermes (anti-pattern: stale prefetch).+ */+import type { ExtensionAPI } from "@mariozechner/pi-coding-agent";+import { dirname } from "node:path";+import { readFileSync, existsSync } from "node:fs";+import { loadConfig, type OVConfig } from "./config.js";+import { OVClient } from "./client.js";+import { RecallManager } from "./recall.js";+import { SyncManager, estimateTokens } from "./sync.js";+import { IndexBuilder } from "./index-builder.js";+import { registerTools } from "./tools.js";++export default async function (pi: ExtensionAPI) {+ // --- Load config ---+ const config = loadConfig(dirname(new URL(import.meta.url).pathname));+ if (!config.enabled) return;++ // Env overrides+ if (process.env.OPENVIKING_URL) config.endpoint = process.env.OPENVIKING_URL;+ if (process.env.OPENVIKING_API_KEY) config.apiKey = process.env.OPENVIKING_API_KEY;+ if (process.env.OPENVIKING_ACCOUNT) config.account = process.env.OPENVIKING_ACCOUNT;+ if (process.env.OPENVIKING_USER) config.user = process.env.OPENVIKING_USER;+ if (process.env.OPENVIKING_AGENT_ID) config.agentId = process.env.OPENVIKING_AGENT_ID;++ // --- Initialize modules ---+ const client = new OVClient(config);+ const recall = new RecallManager(client, config);+ const sync = new SyncManager(client, config);+ const indexBuilder = new IndexBuilder(client, config);++ // Session state+ let connected = false;+ let bypassed = false;+ let profileBlock = "";+ let archiveOverview = "";+ let toolsRegistered = false;++ // ================================================================+ // Event Handlers+ // ================================================================++ // --- session_start ---+ pi.on("session_start", async (event, ctx) => {+ // Bypass check+ const cwd = process.cwd();+ for (const pattern of config.bypassPatterns) {+ if (matchBypass(cwd, pattern)) {+ bypassed = true;+ return;+ }+ }++ // Health check+ connected = await client.health();+ if (!connected) {+ if (config.logLevel === "info") {+ ctx.ui.notify("OpenViking: server not reachable", "warning");+ }+ return;+ }++ // Ensure OV session+ const piSessionId = ctx.sessionManager.getSessionId();+ const ok = await sync.ensureSession(piSessionId);+ if (!ok) {+ if (config.logLevel !== "silent") {+ ctx.ui.notify("OpenViking: failed to create session", "error");+ }+ return;+ }++ // Profile injection+ profileBlock = await buildProfileBlock(client, config);++ // Resume rehydration — fetch archive overview if session was previously committed+ if (sync.sessionId) {+ archiveOverview = await fetchArchiveOverview(client, sync.sessionId, config);+ }++ // Build memory index+ await indexBuilder.buildIndex();++ // Register tools (also re-registered in before_agent_start for pi -c continuations)+ registerTools(pi, client, sync);+ toolsRegistered = true;++ if (config.logLevel === "info") {+ ctx.ui.notify(`OpenViking connected (${piSessionId.slice(0, 8)}...)`, "info");+ }+ });++ // --- before_agent_start ---+ pi.on("before_agent_start", async (event, _ctx) => {+ // Re-register tools on resume — session_start doesn't fire for pi -c continuations+ if (!toolsRegistered) {+ registerTools(pi, client, sync);+ toolsRegistered = true;+ }++ if (!connected || bypassed) return;++ // Synchronous recall+ await recall.searchAndCache(event.prompt);++ // Compose system prompt additions+ const parts: string[] = [];+ if (profileBlock) parts.push(profileBlock);+ if (archiveOverview) parts.push(archiveOverview);++ const idx = indexBuilder.getIndex();+ if (idx) parts.push(idx);++ const additions = parts.join("\n\n");+ if (!additions) return;++ return {+ systemPrompt: event.systemPrompt + "\n\n" + additions,+ };+ });++ // --- context ---+ pi.on("context", async (event, _ctx) => {+ if (!connected || bypassed) return;+ const messages = recall.injectRecall(event.messages);+ return { messages };+ });++ // --- turn_end ---+ pi.on("turn_end", async (event, ctx) => {+ if (!connected || bypassed || !config.syncTurns) return;++ // Extract user message from session entries+ const branch = ctx.sessionManager.getBranch();+ let userText = "";+ for (let i = branch.length - 1; i >= 0; i--) {+ const entry = branch[i];+ if (entry.type === "message" && (entry as any).message?.role === "user") {+ const msg = (entry as any).message;+ userText = typeof msg.content === "string"+ ? msg.content+ : Array.isArray(msg.content)+ ? msg.content+ .filter((b: any) => b.type === "text")+ .map((b: any) => b.text)+ .join("")+ : "";+ break;+ }+ }++ // Extract assistant text+ const assistantMsg = event.message as any;+ let assistantText = "";+ const toolLines: string[] = [];+ const toolNames: string[] = [];++ if (assistantMsg?.content && Array.isArray(assistantMsg.content)) {+ for (const block of assistantMsg.content) {+ if (block.type === "text") {+ assistantText += block.text + "\n";+ } else if (block.type === "toolCall") {+ toolNames.push(block.name);+ toolLines.push(+ `[tool: ${block.name}]\n${JSON.stringify(block.arguments)}`,+ );+ }+ }+ }++ // Add tool summary line+ if (toolNames.length > 0) {+ assistantText = `[assistant used tools: ${toolNames.join(", ")}]\n` + assistantText;+ }++ await sync.syncTurn(+ userText, assistantText, toolLines, event.turnIndex,+ );+ });++ // --- session_before_compact ---+ pi.on("session_before_compact", async (_event, _ctx) => {+ if (!connected || bypassed) return;++ // Flush write queue + synchronous commit+ await sync.shutdown();+ const archiveId = await sync.commit(true);++ // Cache archive overview for rehydration after compaction+ if (archiveId && sync.sessionId) {+ archiveOverview = await fetchArchiveOverview(+ client, sync.sessionId, config,+ );+ }+ // Return nothing → pi proceeds with default compaction+ });++ // --- session_shutdown ---+ pi.on("session_shutdown", async (_event, ctx) => {+ if (!connected || bypassed) return;++ await sync.shutdown();++ // Mirror MEMORY.md+ if (config.mirrorMemoryWrites && sync.sessionId) {+ const memoryPath = `${ctx.cwd}/.memory/MEMORY.md`;+ if (existsSync(memoryPath)) {+ try {+ const content = readFileSync(memoryPath, "utf8");+ if (content.trim()) {+ await client.addMessage(+ sync.sessionId, "user",+ `[Memory mirror]\n${content.slice(0, 50000)}`,+ );+ }+ } catch {+ // Best effort+ }+ }+ }++ // Final commit+ if (config.commitOnShutdown) {+ await sync.commit(true);+ }+ });++ // --- agent_end ---+ pi.on("agent_end", async (_event, _ctx) => {+ recall.invalidate();+ });++ // ================================================================+ // Commands+ // ================================================================++ pi.registerCommand("viking", {+ description: "OpenViking status and manual operations. Use 'commit' to force a sync.",+ handler: async (args, ctx) => {+ if (!connected) {+ ctx.ui.notify("OpenViking: not connected", "warning");+ return;+ }++ if (args?.trim() === "commit") {+ await sync.shutdown();+ const result = await sync.commit(true);+ if (result) {+ await indexBuilder.buildIndex();+ ctx.ui.notify("OpenViking: committed successfully", "info");+ } else {+ ctx.ui.notify("OpenViking: commit failed", "error");+ }+ return;+ }++ // Status+ const sid = sync.sessionId ?? "none";+ ctx.ui.notify(+ `OpenViking: ${connected ? "connected" : "disconnected"} | session: ${sid.slice(0, 12)}...`,+ "info",+ );+ },+ });+}++// ================================================================+// Helper Functions+// ================================================================++/** Simple bypass pattern matching (prefix and glob). */+function matchBypass(cwd: string, pattern: string): boolean {+ if (pattern.startsWith("*")) {+ return cwd.endsWith(pattern.slice(1));+ }+ if (pattern.endsWith("*")) {+ return cwd.startsWith(pattern.slice(0, -1));+ }+ return cwd === pattern || cwd.startsWith(pattern + "/");+}++/** Build the <openviking-context> profile block. */+async function buildProfileBlock(+ client: OVClient, config: OVConfig,+): Promise<string> {+ try {+ const memUri = await client.resolveTargetUri("viking://user/memories");+ const entries = await client.ls(memUri);++ // Look for profile.md+ const profileEntry = entries.find(e => e.name === "profile.md");+ let profileText = "";+ if (profileEntry) {+ const content = await client.readContent(`${memUri}/profile.md`);+ if (content) {+ // Profile elision: keep head (8 lines) + tail (fits budget)+ const lines = content.split("\n");+ if (lines.length > 20) {+ const head = lines.slice(0, 8).join("\n");+ const tailBudget = config.profileBudget - 200;+ const tail = lines.slice(-Math.max(10, tailBudget)).join("\n");+ profileText = head + "\n...\n" + tail;+ } else {+ profileText = content;+ }+ }+ }++ // List preferences and entities directories+ const prefUri = `${memUri}/preferences`;+ const entUri = `${memUri}/entities`;+ const [prefs, ents] = await Promise.all([+ client.ls(prefUri),+ client.ls(entUri),+ ]);++ const sections: string[] = ["<openviking-context>"];+ if (profileText) {+ sections.push(`<user-profile>${profileText}</user-profile>`);+ }+ if (prefs.length > 0 || ents.length > 0) {+ sections.push("<available-memories>");+ if (prefs.length > 0) {+ sections.push(` ${prefUri}/ (${prefs.length} entries)`);+ }+ if (ents.length > 0) {+ sections.push(` ${entUri}/ (${ents.length} entries)`);+ }+ sections.push("</available-memories>");+ }+ sections.push("</openviking-context>");++ const block = sections.join("\n");+ // Budget check+ const tokens = estimateTokens(block);+ if (tokens > config.profileBudget) {+ return block.slice(0, config.profileBudget * 3);+ }+ return block;+ } catch {+ return "";+ }+}++/** Fetch archive overview for rehydration using the session context API. */+async function fetchArchiveOverview(+ client: OVClient, sessionId: string, config: OVConfig,+): Promise<string> {+ try {+ const ctx = await client.getSessionContext(sessionId, config.resumeContextBudget);+ if (!ctx || !ctx.latest_archive_overview) return "";++ const result = `[Session History Summary]\n${ctx.latest_archive_overview}`;+ const tokens = estimateTokens(result);+ if (tokens > config.resumeContextBudget) {+ return result.slice(0, config.resumeContextBudget * 3);+ }+ return result;+ } catch {+ return "";+ }+}diff --git a/examples/pi-coding-agent-extension/recall.ts b/examples/pi-coding-agent-extension/recall.tsnew file mode 100644index 000000000..b7ff185bf--- /dev/null+++ b/examples/pi-coding-agent-extension/recall.ts@@ -0,0 +1,295 @@+import type { OVClient, OVSearchResult } from "./client.js";+import type { OVConfig } from "./config.js";++export interface QueryProfile {+ tokens: string[]; // content words minus stopwords+ wantsPreference: boolean;+ wantsTemporal: boolean;+}++export interface RankedResult {+ uri: string;+ score: number; // base + boosts+ abstract: string;+ content: string | null; // resolved content, capped+ category: string;+ level: number;+}++export interface RecallCache {+ block: string | null; // formatted <relevant-memories> text+ promptText: string; // the query this cache is for+}++// Stopwords for query profiling+const STOPWORDS = new Set([+ "the", "a", "an", "is", "are", "was", "were", "be", "been", "being",+ "have", "has", "had", "do", "does", "did", "will", "would", "could",+ "should", "may", "might", "can", "shall", "to", "of", "in", "for",+ "on", "with", "at", "by", "from", "as", "into", "through", "during",+ "before", "after", "above", "below", "between", "out", "off", "over",+ "under", "again", "further", "then", "once", "and", "but", "or",+ "nor", "not", "so", "yet", "both", "either", "neither", "each",+ "every", "all", "any", "few", "more", "most", "other", "some",+ "such", "no", "only", "own", "same", "than", "too", "very",+ "just", "because", "if", "when", "where", "how", "what", "which",+ "who", "whom", "this", "that", "these", "those", "i", "me", "my",+ "it", "its", "we", "you", "they", "them", "there", "here",+]);++export class RecallManager {+ private client: OVClient;+ private config: OVConfig;+ private cache: RecallCache = { block: null, promptText: "" };++ constructor(client: OVClient, config: OVConfig) {+ this.client = client;+ this.config = config;+ }++ // --- CJK-aware Token Estimation ---++ estimateTokens(text: string): number {+ if (!text) return 0;+ let cjk = 0;+ for (let i = 0; i < text.length; i++) {+ if (text.charCodeAt(i) >= 0x3000) cjk++;+ }+ const other = text.length - cjk;+ return Math.ceil(cjk * 1.5 + other / 4);+ }++ // --- Query Profiling ---++ private profileQuery(query: string): QueryProfile {+ const words = query.toLowerCase()+ .replace(/[^\w\s]/g, " ")+ .split(/\s+/)+ .filter(w => w.length > 1 && !STOPWORDS.has(w));++ return {+ tokens: words,+ wantsPreference: /prefer|favorite|like|want|usually|always|never|hate|love/i.test(query),+ wantsTemporal: /when|yesterday|last |recent|ago|last week|earlier|before/i.test(query),+ };+ }++ // --- Triple-Scope Search ---++ async searchAndCache(userQuery: string): Promise<string | null> {+ // Short-circuit on short queries+ if (userQuery.trim().length < this.config.recallMinQueryLength) {+ return null;+ }++ const profile = this.profileQuery(userQuery);+ const perSourceLimit = Math.max(this.config.recallLimit * 2, 8);++ // Resolve URIs dynamically+ const [userMemUri, agentMemUri, agentSkillsUri] = await Promise.all([+ this.client.resolveTargetUri("viking://user/memories"),+ this.client.resolveTargetUri("viking://agent/memories"),+ this.client.resolveTargetUri("viking://agent/skills"),+ ]);++ // Triple-scope parallel search+ const [userResults, agentResults, skillResults] = await Promise.all([+ this.client.find(userQuery, {+ targetUri: userMemUri, topK: perSourceLimit,+ scoreThreshold: this.config.recallScoreThreshold,+ }),+ this.client.find(userQuery, {+ targetUri: agentMemUri, topK: perSourceLimit,+ scoreThreshold: this.config.recallScoreThreshold,+ }),+ this.client.find(userQuery, {+ targetUri: agentSkillsUri, topK: perSourceLimit,+ scoreThreshold: this.config.recallScoreThreshold,+ }),+ ]);++ // Merge all results+ const all = [...userResults, ...agentResults, ...skillResults];++ // Score filter (client-side, in case server doesn't enforce)+ const filtered = all.filter(+ r => r.score >= this.config.recallScoreThreshold+ );+ if (filtered.length === 0) return null;++ // Dedup+ const deduped = this.dedup(filtered);++ // Rerank with query profile+ const ranked = this.rerank(deduped, profile);++ // Content resolution with per-item cap+ const resolved = await this.resolveContent(ranked);++ // Token-budgeted formatting with graceful degradation+ const block = this.formatResults(resolved);+ this.cache = { block, promptText: userQuery };+ return block;+ }++ // --- Injection ---++ injectRecall(messages: any[]): any[] {+ if (!this.cache.block) return messages;++ // Find the user message (scan backwards)+ for (let i = messages.length - 1; i >= 0; i--) {+ const msg = messages[i];+ if (msg.role === "user") {+ // Idempotency check+ const content = typeof msg.content === "string"+ ? msg.content+ : Array.isArray(msg.content)+ ? msg.content.filter((b: any) => b.type === "text").map((b: any) => b.text).join("")+ : "";++ if (content.includes("<relevant-memories>")) break;++ // Prepend block to user message+ const block = this.cache.block;+ if (typeof msg.content === "string") {+ msg.content = block + "\n" + msg.content;+ } else if (Array.isArray(msg.content)) {+ const textBlocks = msg.content.filter((b: any) => b.type === "text");+ if (textBlocks.length > 0) {+ (textBlocks[0] as any).text = block + "\n" + (textBlocks[0] as any).text;+ }+ }+ break;+ }+ }+ return messages;+ }++ invalidate(): void {+ this.cache = { block: null, promptText: "" };+ }++ // --- Dedup ---++ private dedup(results: OVSearchResult[]): OVSearchResult[] {+ const seen = new Map<string, OVSearchResult>();++ for (const r of results) {+ // Events/cases dedupe by URI; others by abstract text+ const isEvent = r.category === "event" || r.category === "case";+ const key = isEvent+ ? r.uri+ : (r.abstract?.toLowerCase().trim() || r.uri);++ if (!seen.has(key)) {+ seen.set(key, r);+ } else {+ // Keep higher score+ const existing = seen.get(key)!;+ if (r.score > existing.score) seen.set(key, r);+ }+ }+ return [...seen.values()];+ }++ // --- Reranking ---++ private rerank(+ results: OVSearchResult[], profile: QueryProfile,+ ): OVSearchResult[] {+ return results.map(r => {+ let score = r.score;++ // Leaf preference boost+ if ((r.level === 2) || r.uri.endsWith(".md")) {+ score += 0.12;+ }++ // Event boost (gated by temporal intent)+ if (profile.wantsTemporal &&+ (r.category === "event" || r.category === "case")) {+ score += 0.10;+ }++ // Preference boost (gated by preference intent)+ if (profile.wantsPreference && r.category === "preference") {+ score += 0.08;+ }++ // Lexical overlap boost+ const text = `${r.uri} ${r.abstract}`.toLowerCase();+ const matchCount = profile.tokens+ .slice(0, 4)+ .filter(t => text.includes(t)).length;+ score += 0.20 * (matchCount / Math.max(profile.tokens.length, 1));++ return { ...r, score };+ }).sort((a, b) => b.score - a.score)+ .slice(0, this.config.recallLimit);+ }++ // --- Content Resolution ---++ private async resolveContent(+ results: OVSearchResult[],+ ): Promise<RankedResult[]> {+ const resolved: RankedResult[] = [];++ for (const r of results) {+ let content: string | null = r.abstract ?? null;++ if (!this.config.recallPreferAbstract && r.level === 2) {+ // Fetch full content for level-2 items when abstracts not preferred+ const full = await this.client.readContent(r.uri);+ if (full) content = full;+ }++ // Per-item content cap+ if (content && content.length > this.config.recallMaxContentChars) {+ content = content.slice(0, this.config.recallMaxContentChars) + "...";+ }++ resolved.push({+ uri: r.uri,+ score: r.score,+ abstract: r.abstract ?? "",+ content,+ category: r.category ?? "unknown",+ level: r.level ?? 0,+ });+ }+ return resolved;+ }++ // --- Formatting with Graceful Degradation ---++ private formatResults(results: RankedResult[]): string {+ const lines: string[] = [+ "<relevant-memories>",+ "[System note: Recalled memory from OpenViking. Informational background data, NOT new user input.]",+ ];++ let budget = this.config.recallBudget;+ let first = true;++ for (const r of results) {+ const tokens = this.estimateTokens(r.content ?? r.abstract);++ if (first || tokens <= budget) {+ // Full content line+ const source = r.uri.includes("/skills/") ? "skill" : "memory";+ const text = r.content ?? r.abstract;+ lines.push(`- [${source} ${r.score.toFixed(2)}] ${text}`);+ budget -= tokens;+ } else {+ // Degraded hint — model can expand with viking_read+ lines.push(`- [hint ${r.score.toFixed(2)}] Use viking_read to expand: ${r.uri}`);+ }+ first = false;+ }++ lines.push("</relevant-memories>");+ return lines.join("\n");+ }+}diff --git a/examples/pi-coding-agent-extension/sync.ts b/examples/pi-coding-agent-extension/sync.tsnew file mode 100644index 000000000..fa884efcf--- /dev/null+++ b/examples/pi-coding-agent-extension/sync.ts@@ -0,0 +1,246 @@+import type { OVClient } from "./client.js";+import type { OVConfig } from "./config.js";++// --- Memory Stripping ---++/**+ * Strip all injected/synthetic blocks before syncing to OV.+ * Prevents feedback loop where OV indexes injected context as conversation.+ */+export function stripInjectedBlocks(text: string): string {+ // 1. <relevant-memories>...</relevant-memories>+ text = text.replace(/<relevant-memories>[\s\S]*?<\/relevant-memories>/g, "");+ // 2. <system-reminder>...</system-reminder>+ text = text.replace(/<system-reminder>[\s\S]*?<\/system-reminder>/g, "");+ // 3. <openviking-context>...</openviking-context>+ text = text.replace(/<openviking-context[\s\S]*?<\/openviking-context>/g, "");+ // 4. [Subagent Context]... (until double newline or end)+ text = text.replace(/\[Subagent Context\][\s\S]*?(?=\n\n|$)/g, "");+ // 5. Null bytes+ text = text.replace(/\x00/g, "");+ return text.trim();+}++// --- CJK-aware Token Estimation ---++export function estimateTokens(text: string): number {+ if (!text) return 0;+ let cjk = 0;+ for (let i = 0; i < text.length; i++) {+ if (text.charCodeAt(i) >= 0x3000) cjk++;+ }+ const other = text.length - cjk;+ return Math.ceil(cjk * 1.5 + other / 4);+}++// --- Capture Filtering ---++const MEMORY_TRIGGERS = [+ /remember|preference|prefer|important|decision|decided|always|never/i,+ /[\w.-]+@[\w.-]+\.\w+/, // email+ /(?:my)\s*(?:name|live|from|birthday|phone|email)/i, // identity+ /(?:i)\s*(?:like|hate|love|want|need|think|believe)/i, // preference+ /(?:favorite|favourite|love|hate|enjoy|dislike)/i,+];++export function shouldCapture(+ text: string, mode: "semantic" | "keyword",+): { capture: boolean; reason: string } {+ const normalized = text.trim();+ if (!normalized) return { capture: false, reason: "empty" };++ const compact = normalized.replace(/\s+/g, "");+ const isCJK = /[-ヿ㐀-鿿豈-가-]/.test(compact);++ // Length bounds+ const minLen = isCJK ? 4 : 10;+ if (compact.length < minLen) return { capture: false, reason: "too_short" };+ if (normalized.length > 24000) return { capture: false, reason: "too_long" };++ // Command detection+ if (/^\/[a-z0-9_-]{1,64}\b/i.test(normalized)) {+ return { capture: false, reason: "command" };+ }++ // Non-content (punctuation/symbols only)+ if (/^[\p{P}\p{S}\s]+$/u.test(normalized)) {+ return { capture: false, reason: "non_content" };+ }++ // Question-only+ if (/^(who|what|when|where|why|how|is|are|does|did|can|could|would|should)\b.{0,200}[??]$/i.test(normalized)) {+ return { capture: false, reason: "question_only" };+ }++ // Keyword mode gate+ if (mode === "keyword") {+ const hasTrigger = MEMORY_TRIGGERS.some(re => re.test(normalized));+ return { capture: hasTrigger, reason: hasTrigger ? "trigger_matched" : "no_trigger" };+ }++ // Semantic mode — always capture+ return { capture: true, reason: "semantic" };+}++// --- Write Queue ---++interface QueuedTurn {+ role: string;+ content: string;+}++export class WriteQueue {+ private client: OVClient;+ private sessionId: string;+ private queue: QueuedTurn[] = [];+ private flushTimer: ReturnType<typeof setInterval> | null = null;+ private flushing = false;+ private intervalMs: number;+ private threshold: number;++ constructor(+ client: OVClient, sessionId: string,+ intervalMs: number, threshold: number,+ ) {+ this.client = client;+ this.sessionId = sessionId;+ this.intervalMs = intervalMs;+ this.threshold = threshold;+ }++ start(): void {+ if (this.intervalMs > 0) {+ this.flushTimer = setInterval(() => this.flush(), this.intervalMs);+ }+ }++ enqueue(role: string, content: string): void {+ this.queue.push({ role, content });+ if (this.queue.length >= this.threshold) {+ this.flush(); // fire-and-forget+ }+ }++ async flush(): Promise<void> {+ if (this.flushing || this.queue.length === 0) return;+ this.flushing = true;++ const batch = this.queue.splice(0);+ for (const turn of batch) {+ const ok = await this.client.addMessage(+ this.sessionId, turn.role, turn.content,+ );+ if (!ok) {+ // Re-queue failed turns at the front for retry+ this.queue.unshift(turn);+ break;+ }+ }+ this.flushing = false;+ }++ cancelPending(): void {+ if (this.flushTimer) {+ clearInterval(this.flushTimer);+ this.flushTimer = null;+ }+ }+}++// --- SyncManager ---++export class SyncManager {+ private client: OVClient;+ private config: OVConfig;+ private ovSessionId: string | null = null;+ private pendingTokens = 0;+ private syncedTurnCount = 0;+ private writeQueue: WriteQueue | null = null;++ constructor(client: OVClient, config: OVConfig) {+ this.client = client;+ this.config = config;+ }++ get sessionId(): string | null { return this.ovSessionId; }++ async ensureSession(piSessionId: string): Promise<boolean> {+ if (this.ovSessionId) return true;++ const id = `pi-${piSessionId}`;+ const created = await this.client.createSession(id);+ if (!created) return false;++ this.ovSessionId = id;+ this.writeQueue = new WriteQueue(+ this.client, id,+ this.config.writeQueueFlushInterval,+ this.config.writeQueueFlushThreshold,+ );+ this.writeQueue.start();+ return true;+ }++ async syncTurn(+ userText: string, assistantText: string, toolLines: string[],+ turnIndex: number,+ ): Promise<void> {+ if (!this.ovSessionId || !this.writeQueue) return;++ // Dedup guard+ if (turnIndex <= this.syncedTurnCount) return;++ // Capture filter on user text+ const filterResult = shouldCapture(userText, this.config.captureMode);+ if (!filterResult.capture) {+ this.syncedTurnCount = turnIndex;+ return;+ }++ // Strip injected blocks+ const cleanUser = stripInjectedBlocks(userText);+ if (!cleanUser) {+ this.syncedTurnCount = turnIndex;+ return;+ }++ // Enqueue user message+ this.writeQueue.enqueue("user", cleanUser);++ // Enqueue assistant message (if configured)+ if (this.config.captureAssistantTurns) {+ const cleanAssistant = stripInjectedBlocks(assistantText);+ const combined = toolLines.length > 0+ ? `${toolLines.join("\n")}\n${cleanAssistant}`+ : cleanAssistant;+ if (combined.trim()) {+ this.writeQueue.enqueue("assistant", combined);+ }+ }++ // Track tokens+ const totalText = cleanUser + assistantText + toolLines.join("");+ this.pendingTokens += estimateTokens(totalText);+ this.syncedTurnCount = turnIndex;++ // Check commit threshold+ if (this.config.commitTokenThreshold > 0 &&+ this.pendingTokens >= this.config.commitTokenThreshold) {+ await this.writeQueue.flush();+ await this.commit(false);+ }+ }++ async commit(wait: boolean = false): Promise<string | null> {+ if (!this.ovSessionId) return null;+ const result = await this.client.commitSession(this.ovSessionId, wait);+ if (result) this.pendingTokens = 0;+ return result;+ }++ async shutdown(): Promise<void> {+ if (!this.writeQueue) return;+ this.writeQueue.cancelPending();+ await this.writeQueue.flush();+ }+}diff --git a/examples/pi-coding-agent-extension/tools.ts b/examples/pi-coding-agent-extension/tools.tsnew file mode 100644index 000000000..bf7f6f828--- /dev/null+++ b/examples/pi-coding-agent-extension/tools.ts@@ -0,0 +1,252 @@+import { Type } from "typebox";+import { StringEnum } from "@mariozechner/pi-ai";+import type { OVClient } from "./client.js";+import type { SyncManager } from "./sync.js";++export function registerTools(pi: any, client: OVClient, sync?: SyncManager): void {++ // --- viking_search ---+ pi.registerTool({+ name: "viking_search",+ label: "Viking Search",+ description: "Semantic search over the OpenViking knowledge base. Returns ranked results with viking:// URIs and abstracts. Use to recall past decisions, user preferences, or project-specific knowledge not in current context.",+ promptSnippet: "Search OpenViking for past decisions, preferences, and project knowledge",+ promptGuidelines: [+ "Use viking_search when you need information from previous sessions not in MEMORY.md.",+ "Use viking_search before making decisions that might conflict with past decisions.",+ ],+ parameters: Type.Object({+ query: Type.String({ description: "Search query" }),+ scope: Type.Optional(Type.String({ description: "Viking URI prefix to scope search (e.g., 'viking://user/memories/')" })),+ limit: Type.Optional(Type.Number({ description: "Max results (default: 10)" })),+ }),+ async execute(+ _id: string, params: any, _signal: AbortSignal,+ _onUpdate: any, _ctx: any,+ ) {+ if (!client.connected) {+ return { content: [{ type: "text", text: "OpenViking server is not reachable." }] };+ }+ const results = await client.find(params.query, {+ targetUri: params.scope,+ topK: params.limit ?? 10,+ });+ if (results.length === 0) {+ return { content: [{ type: "text", text: "No results found." }] };+ }+ const maxChars = client.cfg.recallMaxContentChars;+ const lines = results.map(r => {+ const abs = r.abstract.length > maxChars+ ? r.abstract.slice(0, maxChars) + "..."+ : r.abstract;+ return `[${r.score.toFixed(2)}] ${r.uri}\n ${abs}`; }+ );+ return {+ content: [{ type: "text", text: lines.join("\n\n") }],+ details: { results },+ };+ },+ });++ // --- viking_read ---+ pi.registerTool({+ name: "viking_read",+ label: "Viking Read",+ description: "Read content at a viking:// URI. Three detail levels: 'abstract' (~100 tokens), 'overview' (~2k tokens), 'full' (complete). Start with abstract, escalate when needed.",+ promptSnippet: "Read OpenViking content at a viking:// URI with tiered detail levels",+ parameters: Type.Object({+ uri: Type.String({ description: "viking:// URI to read" }),+ level: StringEnum(["abstract", "overview", "full"] as const),+ }),+ async execute(+ _id: string, params: any, _signal: AbortSignal,+ _onUpdate: any, _ctx: any,+ ) {+ if (!client.connected) {+ return { content: [{ type: "text", text: "OpenViking server is not reachable." }] };+ }+ let content: string | null = null;+ switch (params.level) {+ case "abstract": content = await client.abstract(params.uri); break;+ case "overview": content = await client.overview(params.uri); break;+ case "full": content = await client.readContent(params.uri); break;+ }+ if (!content) {+ return { content: [{ type: "text", text: `No content at ${params.uri}` }] };+ }+ return { content: [{ type: "text", text: content }] };+ },+ });++ // --- viking_browse ---+ pi.registerTool({+ name: "viking_browse",+ label: "Viking Browse",+ description: "Browse the OpenViking knowledge store like a filesystem. List directory contents or get metadata.",+ promptSnippet: "Browse the viking:// directory tree in OpenViking",+ parameters: Type.Object({+ action: StringEnum(["list", "stat"] as const),+ uri: Type.Optional(Type.String({ description: "viking:// URI (default: 'viking://')" })),+ }),+ async execute(+ _id: string, params: any, _signal: AbortSignal,+ _onUpdate: any, _ctx: any,+ ) {+ if (!client.connected) {+ return { content: [{ type: "text", text: "OpenViking server is not reachable." }] };+ }+ const uri = params.uri ?? "viking://";+ if (params.action === "stat") {+ const info = await client.stat(uri);+ if (!info) return { content: [{ type: "text", text: `Not found: ${uri}` }] };+ return { content: [{ type: "text", text: JSON.stringify(info, null, 2) }] };+ }+ // list+ const entries = await client.ls(uri);+ if (entries.length === 0) {+ return { content: [{ type: "text", text: `Empty directory: ${uri}` }] };+ }+ const lines = entries.map(e => `${e.isDir ? "📁" : "📄"} ${e.name}`);+ return { content: [{ type: "text", text: lines.join("\n") }] };+ },+ });++ // --- viking_remember ---+ pi.registerTool({+ name: "viking_remember",+ label: "Viking Remember",+ description: "Store a fact or memory in OpenViking. Stored as a session message and extracted into long-term memory on commit. Use for important information the agent should remember: preferences, decisions, gotchas, lessons learned.",+ promptSnippet: "Store a fact in OpenViking for cross-session persistence",+ promptGuidelines: [+ "Use viking_remember for facts that should survive across sessions but don't belong in MEMORY.md.",+ "Good for: user preferences, architectural decisions, gotchas, environment details.",+ ],+ parameters: Type.Object({+ content: Type.String({ description: "The fact or observation to store" }),+ category: Type.Optional(Type.String({ description: "Category hint: 'preference', 'entity', 'event', 'case', 'pattern'" })),+ }),+ async execute(+ _id: string, params: any, _signal: AbortSignal,+ _onUpdate: any, _ctx: any,+ ) {+ if (!client.connected) {+ return { content: [{ type: "text", text: "OpenViking server is not reachable." }] };+ }+ // Store as a tagged message directly in OV — the extractor picks up [Remember — ...] prefix+ const category = params.category ?? "general";+ const tagged = `[Remember — ${category}] ${params.content}`;++ // Directly add to OV session if available+ let stored = false;+ if (sync?.sessionId) {+ stored = await client.addMessage(sync.sessionId, "user", tagged);+ }++ return {+ content: [{ type: "text", text: stored ? `Remembered in OpenViking: "${params.content}" (${category})` : `Queued for OpenViking: "${params.content}" (${category})` }],+ details: { stored, category, tagged },+ };+ },+ });++ // --- viking_forget ---+ pi.registerTool({+ name: "viking_forget",+ label: "Viking Forget",+ description: "Delete a memory by URI, or search for a specific memory and remove it. Use to correct outdated or wrong information.",+ promptSnippet: "Delete a memory from OpenViking by URI or query",+ parameters: Type.Object({+ uri: Type.Optional(Type.String({ description: "Exact viking:// URI to delete" })),+ query: Type.Optional(Type.String({ description: "Search query — deletes the strongest match if score > 0.8" })),+ }),+ async execute(+ _id: string, params: any, _signal: AbortSignal,+ _onUpdate: any, _ctx: any,+ ) {+ if (!client.connected) {+ return { content: [{ type: "text", text: "OpenViking server is not reachable." }] };+ }+ if (params.uri) {+ const ok = await client.delete(params.uri);+ return {+ content: [{ type: "text", text: ok ? `Deleted: ${params.uri}` : `Failed to delete: ${params.uri}` }],+ };+ }+ if (params.query) {+ const results = await client.find(params.query, { topK: 1 });+ if (results.length > 0 && results[0].score > 0.8) {+ const ok = await client.delete(results[0].uri);+ return {+ content: [{ type: "text", text: ok ? `Deleted: ${results[0].uri}` : `Failed: ${results[0].uri}` }],+ };+ }+ return { content: [{ type: "text", text: "No strong match found (score > 0.8 required)." }] };+ }+ return { content: [{ type: "text", text: "Provide either 'uri' or 'query'." }] };+ },+ });++ // --- viking_add_resource ---+ pi.registerTool({+ name: "viking_add_resource",+ label: "Viking Add Resource",+ description: "Ingest a URL into OpenViking. The page is auto-processed into L0/L1/L2 tiers and indexed for semantic search. HTTP only — local file paths are not supported by the OV server.",+ promptSnippet: "Ingest a URL into OpenViking for indexed retrieval",+ parameters: Type.Object({+ url: Type.String({ description: "URL to ingest (HTTP only, no file paths)" }),+ reason: Type.Optional(Type.String({ description: "Why this resource is relevant (improves indexing)" })),+ }),+ async execute(+ _id: string, params: any, _signal: AbortSignal,+ _onUpdate: any, _ctx: any,+ ) {+ if (!client.connected) {+ return { content: [{ type: "text", text: "OpenViking server is not reachable." }] };+ }+ const result = await client.addResource(params.url);+ if (!result) {+ return { content: [{ type: "text", text: `Failed to ingest: ${params.url}` }] };+ }+ return {+ content: [{ type: "text", text: `Ingested: ${result.root_uri}` }],+ details: result,+ };+ },+ });++ // --- viking_archive_expand ---+ pi.registerTool({+ name: "viking_archive_expand",+ label: "Viking Archive Expand",+ description: "Expand an archived session back into raw messages. Use when the archive summary is too coarse and you need detailed conversation history.",+ promptSnippet: "Expand an archived session to see raw conversation messages",+ parameters: Type.Object({+ archive_id: Type.Optional(Type.String({ description: "Archive ID to expand" })),+ session_id: Type.Optional(Type.String({ description: "OV session ID to expand" })),+ }),+ async execute(+ _id: string, params: any, _signal: AbortSignal,+ _onUpdate: any, _ctx: any,+ ) {+ if (!client.connected) {+ return { content: [{ type: "text", text: "OpenViking server is not reachable." }] };+ }+ const sid = params.session_id ?? params.archive_id;+ if (!sid) {+ return { content: [{ type: "text", text: "Provide session_id or archive_id." }] };+ }+ // Read the session's overview — sessions are at viking://session/{sid}+ const uri = `viking://session/${sid}`;+ const content = await client.overview(uri);+ if (!content) {+ // Try reading the history subdirectory+ const history = await client.overview(`${uri}/history`);+ if (!history) {+ return { content: [{ type: "text", text: `Archive not found: ${sid}` }] };+ }+ return { content: [{ type: "text", text: history }] };+ }+ return { content: [{ type: "text", text: content }] };+ },+ });+}