Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

Merge branch 'main' into audit/pr-300

AkitaOnRails180837f30e9b
diff --cc CHANGELOG.md
index a86bef82,00fa8779..4e4e64f0
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@@ -7,20 -7,269 +7,284 @@@ and this project adheres to [Semantic V
## [Unreleased]
+### Added
+- New `openai-compat` embedding provider for self-hosted engines
+ (Ollama, LM Studio, vLLM). Set
+ `AI_MEMORY_EMBEDDING_PROVIDER=openai-compat` together with explicit
+ `AI_MEMORY_EMBEDDING_BASE_URL`, `AI_MEMORY_EMBEDDING_MODEL`, and
+ `AI_MEMORY_EMBEDDING_DIM` — there is no safe default model or
+ dimensionality for a self-hosted engine, so each is required rather
+ than guessed. Unlike the other providers it is keyless: a bearer
+ token is sent only when `LLM_API_KEY` is present, for gateways that
+ want one. Embeddings are stored under their own
+ `provider="openai-compat"` identity, so switching an existing
+ `openai`+base-URL setup over changes the stored
+ `{provider, model, dim}` triple — run `ai-memory embed --force` to
- re-embed.
++ re-embed. (#300)
++
+ ### Fixed
+ - Antigravity CLI's `PreInvocation` hook now maps only its documented
+ `invocationNum = 0` call to ai-memory's synthetic `SessionStart`. Later model
+ invocations perform no capture or destructive handoff fetch, so a manual
+ handoff created while the conversation winds down remains open for the next
+ session. Native Windows hooks also emit Antigravity's required `injectSteps`
+ envelope instead of Claude Code's `hookSpecificOutput` shape. (#298)
+ - Automatic handoff selection now prefers the newest cwd-eligible session over
+ a stale, more-specific ancestor. A new automatic handoff expires prior open
+ automatic handoffs from the exact cwd, and accepting the winner atomically
+ expires older eligible automatic handoffs. Manual and sibling-directory
+ handoffs remain open, preventing stale delivery and inflated pending counts.
+ (#293)
+ - OpenAI-compatible providers now send each structured operation's JSON Schema
+ through `response_format=json_schema` by default, so local models cannot
+ replace consolidation JSON with prose or omit required fields. Explicit
+ structured-output capability rejections fall back to the tolerant parser;
+ other HTTP failures still propagate, and
+ `AI_MEMORY_LLM_COMPAT_STRICT=false` remains the compatibility opt-out. (#292)
+ - Recognized Antigravity CLI's native file/edit and search tools, applied path
+ exclusions to its `TargetFile` operations, and captured bounded successful
+ edit content from `toolCall.args` when the hook omits an output field. Generic
+ MCP/resource tools remain fail-closed until their path schemas are proven,
+ while failed edits retain their error instead of attempted content. (#294)
+
+ ## [1.19.2] - 2026-07-28
+
+ ### Fixed
+ - Source installation no longer fails when `cargo install --path` resolves
+ rmcp 1.8, whose `peer_info()` return type differs from rmcp 1.7. CI now checks
+ the unlocked source-install resolution separately from the workspace's
+ lock-aware gates, while the documented persistent Windows install uses
+ `--locked` for reproducibility. (#285)
+ - Antigravity CLI hook installation and documentation now expose the existing
+ agent-aware manual finalizer:
+ `ai-memory finalize-session --agent antigravity-cli`. Antigravity's `Stop`
+ event ends one execution loop rather than the conversation, so it remains a
+ normal observation; the explicit command closes the latest scoped session
+ through the canonical SessionEnd path, producing its summary and automatic
+ handoff and queueing opt-in consolidation. The docs also clarify that
+ `memory_handoff_begin` deliberately creates a session-neutral, project-wide
+ manual handoff for every MCP client; attributed handoffs come from canonical
+ SessionEnd processing. (#284)
+
+ ## [1.19.1] - 2026-07-27
+
+ ### Changed
+ - Wiki search now applies a bounded source-authority adjustment after FTS5,
+ graph, and optional vector candidate generation. Canonical rules, decisions,
+ procedures, gotchas, semantic/procedural tiers, `pinned` pages, and
+ `canonical` / `active` / `source-of-truth` tags win close relevance contests;
+ episodic sessions, `_lint/` output, investigations, and pages tagged
+ `superseded`, `historical`, `test-fixture`, or `do-not-answer-from` are
+ downgraded but remain searchable. Exact session-only queries still retrieve
+ their evidence, and the returned `rank` includes the bounded adjustment so
+ multi-scope merging preserves the same order. (#269)
+ - Client CLI commands now resolve their `(workspace, project)` from the
+ nearest `.ai-memory.toml` marker, not just the lifecycle hooks. Previously
+ only the hook path read the marker, so a checkout declaring
+ `workspace = "acme"` had its captures land in `acme` while `run`,
+ `bootstrap`, `search`, `write-page` and every other scope-taking command
+ resolved into `default` — the same repository split across two scopes, with
+ `ai-memory run`'s managed workstream stranded on the wrong side. Each field
+ still prefers an explicit flag; when the marker decides one, the command
+ announces the resolved scope on stderr, naming which half the marker
+ decided. `AI_MEMORY_IGNORE_MARKER=1` restores the previous resolution for
+ one invocation (client commands only — the hooks keep reading the marker).
+ `embed --force` without `--project` still fans out across the workspace and
+ no longer needs a derivable project name. `ai-memory serve` is unchanged:
+ it has no caller cwd, and its `--workspace` / `--project` remain the baked
+ fallback for hook events without a usable one. (#259)
+ - Marker discovery now stays inside its trust boundary when the caller's cwd
+ is outside `$HOME`: it walks no higher than the nearest checkout root, or
+ checks only cwd for a non-git directory. Workspace-only markers also keep
+ the hooks' documented `project = basename(cwd)` behavior for CLI commands,
+ including subdirectories and linked worktrees. (#259)
+
+ ### Fixed
+ - Scheduled hollow-project cleanup now treats managed workstreams as project
+ data. Older projects whose only history is a managed workstream, including
+ those with a live run, are no longer cascade-deleted out from under the
+ workstream heartbeat or left with orphaned transcript segments. (#279)
+ - Hybrid search now gives its FTS, vector, and graph streams the same bounded
+ candidate window used by authority-aware FTS search. Small result limits no
+ longer exclude a canonical page before post-fusion authority ranking can
+ promote it, and candidate-limit arithmetic is saturating throughout. (#277)
+ - Forced workspace deletion now removes the immutable managed-workstream
+ segment directories whose database rows are removed by the workspace
+ cascade. Its admin report includes workstream/run counts and IDs, and raw
+ segment cleanup participates in the existing filesystem partial-failure
+ reporting instead of leaving transcript data orphaned. (#275)
+ - Lossless `move-project` true moves now re-stamp managed workstreams into the
+ destination workspace in the same transaction as the project and its other
+ denormalized child rows. Previously the project moved while its managed
+ workstreams retained the source `workspace_id`, hiding portable history from
+ destination-scope lookup and violating the project/workspace pairing
+ invariant. The admin response now reports `workstreams_moved`. (#273)
+ - SessionEnd recovery now commits the ended generation and automatic handoff in
+ one SQLite transaction, then lets an already-ended native replay converge the
+ remaining wiki commit, durable consolidation enqueue, and ingest-key
+ completion. An interruption after `ended_at` can no longer strand a missing
+ handoff or permanently pending spool key, and missing or scope/agent-
+ mismatched SessionEnd events no longer attempt consolidation recovery against
+ an unrelated session. (#271)
+ - Bare `install-hooks --apply` re-runs, including the Docker wrapper's
+ post-upgrade refresh, now preserve an install's baked `repo-root` project
+ strategy for every supported hook integration. An explicit
+ `--project-strategy basename` still removes the install-wide default. (#267)
+ - Installer `--apply` modes now write through symlinked agent configuration
+ files instead of atomically replacing the symlink itself. Symlink chains and
+ dangling final targets are preserved, while backups remain next to the
+ user-facing configuration path. (#264)
+ - SessionEnd re-consolidation now converges by comparing the current
+ observation count with a persisted count stamped by the latest completed
+ end, instead of comparing independently generated wall-clock timestamps.
+ Clock skew could otherwise leave an old observation permanently "new" and
+ repeatedly rewrite the same session page, handoff, and opt-in LLM job with no
+ agent activity. Existing ended sessions are baselined during migration so an
+ upgrade does not enqueue historical catch-up work. (#268)
+ - Capture exclusions now canonicalize an existing hook working directory
+ before matching paths, so filesystem aliases such as macOS `/var` versus
+ `/private/var` cannot turn an excluded file event into a spooled event.
+ Marker discovery tests likewise accept the canonical path they request.
+ (#265)
+ - Opt-in SessionEnd LLM consolidation now runs from a durable, generation-
+ idempotent queue instead of inside the hook batch request. The hook commits
+ its deterministic session page and handoff, persists the provider job, and
+ returns without waiting for LLM latency; a single bounded worker recovers
+ queued or expired-lease work after restart and makes at most five provider
+ attempts with backoff. A stale SessionEnd redelivery also repairs the
+ enqueue when the original request was cancelled just after `ended_at`, so
+ the default hook drain timeout can no longer silently strand the heuristic
+ page as the final result. (#265)
+ - `purge-project` no longer deletes a project out from under a running agent.
+ `workstreams` cascades from `projects` and `managed_runs` cascades from
+ `workstreams`, so purging a scope that still held a live managed run tore
+ out its lease row: the wrapper then failed every heartbeat with
+ `409 managed run lease is not active` and the session's transcript never
+ reached the ledger. The purge now refuses with a `409` naming the offending
+ workstreams unless `--force` is passed, and its report counts the
+ `workstreams` and `managed_runs` the cascade removes; their
+ `raw/workstreams/<id>/` directories are now removed server-side and included
+ in the same filesystem success/failure report instead of being orphaned.
+ Those counters previously showed `0 pages, 0 sessions, …` and made such a
+ scope look safe to delete. Liveness is the lease, not the row state: a
+ crashed wrapper leaves `state = 'active'` behind until the next
+ `ai-memory run` sweeps it, so only a lease that has not yet expired blocks
+ the purge. `move-project`'s
+ copy-purge merge surfaces the same conflict as a `409` naming how many
+ pages were already copied, instead of a `500`; its `--force` flag only
+ overrides the active-project guard and never destroys a live managed-run
+ lease. (#259)
+
+ ## [1.19.0] - 2026-07-25
+
+ ### Fixed
+ - Claude Desktop's rendered Windows MCP instructions now distinguish the
+ unpackaged `%APPDATA%` config from the detected MSIX `LocalCache` config, and
+ contributor, managed-workstream, and CLI reference docs no longer omit
+ recently shipped harnesses or commands. (#256)
+ - The opt-in managed-workstream real-harness acceptance runner now verifies
+ context delivery from the managed-run cursor/acknowledgement state and a new
+ persisted assistant event instead of requiring the model to quote a prior
+ sentinel. Large Claude Code hook packets can be file-backed, so acceptance no
+ longer passes or fails based on whether the model chooses to use `Read`.
+ The deterministic fake Grok leg covers the same assertion path. (#242)
+
+ ### Added
+ - `install-mcp --client claude-code --session-aware` now registers an
+ ai-memory-owned stdio bridge that forwards Claude Code's
+ `CLAUDE_CODE_SESSION_ID` as `X-Memory-Actor-Session-Id` on every upstream
+ HTTP MCP request. This makes `[auto_scope] mode = "per_session"` effective
+ for concurrent Claude Code sessions against local or remote servers while
+ leaving the existing static HTTP registration as the default. The bridge
+ preserves bearer auth, stateless/stateful HTTP compatibility, and uninstall
+ ownership; both Docker wrappers forward the Claude session variable into
+ the helper container. (#244)
+ - `GET /admin/open-sessions` lists open (not yet ended) sessions for one
+ workspace/project/agent, newest first (`all=true` returns every match).
+ `ai-memory finalize-session` now uses this endpoint instead of opening
+ the local SQLite index directly, so every CLI command is a thin HTTP
+ client of the running server; the command now requires a reachable
+ server and no longer works against an offline data directory. (#236)
+ - Managed workstream support for Grok Build CLI: `ai-memory run grok` (alias
+ `grok-build`) creates fresh sessions with a wrapper-generated `--session-id`,
+ resumes linked sessions with `--resume`, maps wrapper `--yolo` onto Grok's
+ native `--yolo`/`--always-approve`, and delivers the bounded workstream
+ context packet through Grok's native `--rules` flag (system-prompt append,
+ acknowledged only after the child spawns). Transcript import reads
+ `$GROK_HOME/sessions/*/*/chat_history.jsonl` read-only with a
+ prefix-validated cursor and content-hash event ids so rewind-driven journal
+ rewrites cannot duplicate history; system prompts and encrypted reasoning
+ are excluded as loss annotations, as are the harness-injected `<user_info>`
+ and `<system-reminder>` blocks Grok stores inside `user` records (project
+ instructions, the skills catalogue, and connected MCP servers), which would
+ otherwise leak harness internals into the portable ledger and evict real
+ conversation from the startup packet budget. Discovery
+ matches checkouts through `summary.json`'s recorded `info.cwd` and honors
+ `GROK_HOME`. Grok stays out of the bare-mode automatic pool. Verified
+ against Grok Build CLI v0.2.111 ([#237]).
+
+ ### Changed
+ - Single-page, batch, and bootstrap consolidation prompts now ask the model
+ to connect related wiki pages with path-based wikilinks and to mirror the
+ dominant natural language of the source material while preserving code,
+ identifiers, paths, commands, error strings, and JSON field names. (#238)
+ - The M8 access-counter reinforcement now bumps a page's `access_count` and
+ `last_accessed_at` at most once per minute instead of on every search that
+ returns it. A first sighting still bumps immediately, and a continuously hot
+ page remains eligible once per window, while the cooldown map self-prunes to
+ the pages searched within the window. This reduces redundant single-writer
+ work under bursty or overlapping searches while intentionally making
+ `access_count` a coarser retention signal. (#239)
+
+ ### Fixed
+ - Managed workstream packets now carry a versioned origin marker, and Claude
+ Code transcript import excludes a tool result only when its content begins
+ with that marker (or the legacy rendered packet header). This prevents a
+ large SessionStart packet that Claude persists and later reads from
+ `tool-results/` from re-entering the ledger and recursively consuming future
+ packet budgets, while ordinary tool results that merely mention the marker
+ remain visible. (#241)
+ - Lifecycle `user-prompt` and `post-compaction` bodies are now truncated
+ UTF-8-safely at 16 KiB, while notification and tool excerpts remain capped at
+ 2 KB. Native hook commands apply the event cap before local spooling or
+ transport, the server repeats it for direct and older clients, and the
+ sanitized observation boundary independently caps every durable body at
+ 16 KiB so neither SQLite nor observation FTS can grow to the 10 MiB HTTP
+ transport limit. (#249)
+ - `ai-memory run <harness>` now verifies an ai-memory-injected native resume
+ target still exists in that harness's read-only session store. A confirmed
+ orphan starts a fresh native session and repoints the same workstream instead
+ of retrying the dead id forever. The new wrapper-owned `--fresh` flag forces
+ the same per-workstream recovery without a resume attempt or adoption prompt;
+ explicit native resume/session/fork selectors remain authoritative and
+ cannot be combined with `--fresh`. (#240)
+ - `install-mcp --client claude-desktop` now detects an MSIX-packaged
+ Claude Desktop on Windows and writes to its virtualized
+ `AppData\Local\Packages\Claude_<id>\LocalCache\Roaming\Claude\claude_desktop_config.json`
+ instead of the plain `%APPDATA%\Claude\` path. Previously this
+ silently wrote a config file the running app ignored, so the MCP
+ server never appeared after restart. The detector uses Windows'
+ resolved local and roaming app-data roots, prefers an existing config
+ when multiple package directories exist, and fails with an explicit
+ `--config-file` recovery instruction when the active package is
+ ambiguous. Unpackaged installs keep resolving to the plain path.
+ (#250)
+
+ - The Docker wrappers (`bin/ai-memory`, `bin/ai-memory.ps1`) kept stdin
+ attached only on a real terminal, so every piped or redirected
+ invocation reached the container with a closed stdin. `ai-memory
+ write-page --body -` therefore stored a page with frontmatter and an
+ empty body while still reporting a successful write, and the same
+ applied to any other stdin reader (hooks fed by a pipe). The wrappers
+ now always pass `-i`, while `-t` is added only when stdin and stdout are
+ both terminals. `AI_MEMORY_NO_TTY=1` disables only TTY allocation and
+ no longer disconnects stdin. (#243)
+ - Managed `SessionStart` delivery now includes a pending single-use
+ handoff before the portable workstream ledger and optional project
+ brief. Handoff and ledger acknowledgements are claimed together only
+ after the complete response has been assembled; a failed or racing
+ ledger claim cannot consume the handoff or suppress retry delivery.
+ (#235)
## [1.18.0] - 2026-07-23
diff --cc README.md
index 4b134f49,d34bbecb..b07b01bb
--- a/README.md
+++ b/README.md
@@@ -41,7 -41,7 +41,7 @@@
| VS Code Copilot | MCP-only | `.vscode/mcp.json` for Copilot agent mode; no lifecycle hooks (Copilot does not expose them yet). |
| Hermes Agent | Community | A community-maintained [`ai-memory-hermes-plugin`](https://github.com/MrLuciano/ai-memory-hermes-plugin) is available. It is not part of ai-memory's first-party install surface; review its compatibility matrix, install/uninstall scripts, and secret handling before using it. |
| LLM/auth providers | Supported | Anthropic, OpenAI, OpenAI OAuth/Codex, GitHub Copilot, Gemini, OpenCode Zen/Go, OpenAI-compatible endpoints, and generic OIDC device auth for native hooks. |
--| Embedding providers | Supported | OpenAI, Voyage, and Google Gemini. |
++| Embedding providers | Supported | OpenAI, Voyage, Google Gemini, and keyless OpenAI-compatible endpoints such as Ollama, LM Studio, and vLLM. |
## What it is
@@@ -662,21 -719,19 +719,22 @@@ also set `COPILOT_GITHUB_TOKEN`, `GH_TO
> high-effort thinking models for your coding agent.
> [!TIP]
- > **On a local engine (Ollama, vLLM, LM Studio, llama.cpp) with
- > `openai-compat`, if consolidation fails on large sessions** with
- > `did not contain a JSON object` or `serde: unknown variant`, set
- > `AI_MEMORY_LLM_COMPAT_STRICT=true`. It sends `response_format=json_schema`
- > (strict) so capable engines constrain output to the schema. If the strict
- > raw call fails, ai-memory falls back to the default tolerant parser. Off by
- > default.
+ > **OpenAI-compatible structured output is schema-constrained by default.**
+ > ai-memory sends each operation's JSON Schema through
+ > `response_format=json_schema`, which recent Ollama, vLLM, LM Studio, and
+ > llama.cpp releases honour. It falls back to the tolerant parser when an
+ > endpoint explicitly rejects that field or returns a malformed shape. Set
+ > `AI_MEMORY_LLM_COMPAT_STRICT=false` only for an incompatible endpoint.
Embeddings are optional and separate from the LLM provider. Set
-`AI_MEMORY_EMBEDDING_PROVIDER=openai`, `voyage`, `google`, or `gemini` when
-you want vector reranking in addition to FTS5 + graph-neighbor retrieval.
-Both the FTS-only and hybrid paths apply the same bounded page-authority
-adjustment after candidate generation; embeddings improve relevance recall but
-do not decide which source is canonical.
+`AI_MEMORY_EMBEDDING_PROVIDER=openai`, `voyage`, `google`/`gemini`, or
+`openai-compat` when you want vector reranking in addition to FTS5 +
+graph-neighbor retrieval. `openai-compat` targets self-hosted engines
+(Ollama, LM Studio, vLLM): it needs no API key and requires explicit
+`AI_MEMORY_EMBEDDING_BASE_URL`, `AI_MEMORY_EMBEDDING_MODEL`, and
- `AI_MEMORY_EMBEDDING_DIM`.
++`AI_MEMORY_EMBEDDING_DIM`. Both the FTS-only and hybrid paths apply the same
++bounded page-authority adjustment after candidate generation; embeddings
++improve relevance recall but do not decide which source is canonical.
See [`docs/install.md#llm-provider-tiers`](docs/install.md#llm-provider-tiers)
for env vars and Ollama/OpenRouter/Atlas Cloud examples, and