Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
Merge main into release/2.1: pick up the 2.0.4 fix batch
AkitaOnRails97b8e4cbe1da
diff --cc CHANGELOG.mdindex 67e8465f,a050f732..2a8800ce--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -7,75 -7,74 +7,142 @@@ and this project adheres to [Semantic V## [Unreleased]+### Added+- `bootstrap --resume` recovers an interrupted run from durable per-chunk+ progress instead of re-paying for every LLM call (#621). Each chunk's pages+ are recorded (keyed by a fingerprint of the pruned sources + chunk budget, so+ a re-run after new commits re-chunks and starts fresh) and `--resume` seeds+ them and skips those chunks; the wiki write stays atomic (nothing lands until+ the whole run completes) and progress is cleared on success. It adopts only+ the **contiguous** prefix of completed chunks and re-runs from the first gap+ or unreadable row (#635), so a resumed run can never diverge from a clean one+ by seeding a later chunk with context an earlier, still-missing chunk never+ produced. Complements the transient-retry from 2.0.3 (#617): retry reduces+ how often you need resume; resume covers the crash / restart case.+- Added first-party OpenCode 2.0 beta (`opencode2`) support.+ `install-mcp --client opencode2` merges the V2 `mcp.servers` remote entry+ (no `enabled` field, `oauth: false` for header credentials);+ `install-hooks --agent opencode2` writes an `ai-memory-opencode2.ts`+ plugin in the beta `{ id, setup }` shape (no runtime dependency:+ `Plugin.define` is identity), with handoff injection through the+ `context` hook and the same `session.*` lifecycle coverage as v1;+ `ai-memory run opencode2` launches through the `opencode2` binary and+ imports the beta `session_v2`/`session_message` transcript. The beta+ shares v1's config dir, session store, and agent kind, so no store+ migration ships. Verified live against beta-18999 (hook capture,+ handoff loop, managed launch/resume/import, `opencode↔opencode2`+ session resume). Two beta caveats are documented: back up+ `~/.local/share/opencode` first (the beta has migrated the shared+ database in place before), and move its background service off port+ 49374 when ai-memory serves there. Managed ledger-delta+ acknowledgement does not fire through the shared background service+ (redelivery, never loss); see the managed-workstreams notes (#622).+- `install-hooks --agent codex` now wires Codex's `SessionEnd` hook, so a+ finished Codex session gets the same automatic end-of-session summary+ and cross-agent handoff Claude Code already gets. Codex shipped a+ first-class `SessionEnd` event in Codex CLI 0.145.0 (openai/codex#33895);+ the `hooks/codex/session-end.{sh,ps1}` scripts already existed and only+ needed adding to the Codex hook profile. On older Codex the event key is+ inert; `ai-memory finalize-session --agent codex` remains the fallback.+ `merge_codex_payload` no longer strips a `SessionEnd` key on every apply+ — a third-party `SessionEnd` hook is preserved alongside ours. ([#604])+- `AI_MEMORY_LLM_HEADERS` (`llm_headers` in `config.toml`) attaches extra+ HTTP headers to every LLM chat request, for gateways that require a+ caller-identifying header. Entries are `Name=Value` or `Name: Value`,+ comma separated in the env var. Parsed and validated once at the+ configuration boundary, so a malformed entry fails at startup rather than+ on the first consolidation pass, and providers consume typed header+ material instead of operator strings. Headers ai-memory sets itself+ (`authorization`, `content-type`, `x-api-key`, `x-goog-api-key`,+ `anthropic-version`, `anthropic-beta`, `openai-beta`, `host`,+ `content-length`) are refused: `reqwest` appends rather than replaces, so a+ duplicate would break the request instead of overriding it. Values are+ marked sensitive on the wire and never logged — the `Debug` output carries+ header names only. ([#606])++ ### Changed+ - The build is self-contained on every platform: the vendored web stylesheet+ is now the default and `TAILWIND_BUILD=1 cargo build -p ai-memory-web`+ regenerates it, so no command needs `TAILWIND_SKIP=1` any more. CI+ regenerates the bundle on Linux and fails if the committed+ `static/tailwind.css` is stale, a check that did not exist before.+ - Developer loop: `cargo t` (everyday, skips `slow`/`stress` modules) and+ `cargo tf` (everything) aliases over cargo-nextest, integration tests that+ compile into each crate's own test harness from `tests/suite/` (78 test+ binaries down to 11 in the everyday loop; only the CLI keeps a separate one,+ and the evals harness builds only under `--workspace`), a dev profile that+ keeps only line tables, and an opt-in pre-push hook that runs the full tier.+ `bin/release` and the documented gate also run `git diff --check`. Measured:+ workspace edit-to-result ~380s to ~150s on macOS; the warm everyday test run+ 28s to 19s on a 32-thread Windows box.++ ### Fixed+ - Authentication-disabled HTTP servers now ignore stale or unexpected Bearer+ headers and preserve anonymous access. Previously, a client retaining an old+ `AI_MEMORY_AUTH_TOKEN` received `401 Unauthorized` even though the server+ reported `auth=false`; invalid Bearers remain rejected whenever static or+ human authentication is enabled. (#639)+ - Cursor sessions no longer land in the default `default/scratch` bucket.+ Cursor sends the workspace directory only as `workspace_roots` — its+ `sessionStart` / `sessionEnd` payloads carry no `cwd` key at all, and its+ tool events send `cwd: ""` — so cwd resolution produced nothing and the+ server fell back to its default project for every Cursor event. Both the+ native `ai-memory hook` path and the POSIX/PowerShell hook scripts now read+ `workspace_roots` (alongside Antigravity's `workspacePaths`) and treat an+ empty `cwd` as absent rather than as an answer.+ - Cursor sessions are no longer attributed to `claude-code`. The Cursor CLI+ also runs the hook commands declared in Claude Code's+ `~/.claude/settings.json`, which `install-hooks --agent claude-code`+ hardcoded to `--agent claude-code`, so a Cursor-driven session was stored+ with `agent_kind = claude-code`. Hook payloads carrying Cursor's+ `cursor_version` marker are now attributed to `cursor` regardless of the+ `?agent=` the hook command declared.+ - A project that first materializes while the server is running is now+ self-describing immediately, instead of only after the next startup+ backfill (#643). Scope manifests (`_meta.md`) were written at startup and on+ the rename/move admin paths, so a session in a checkout the server had not+ seen before produced a scope directory with pages but no manifest. Stop the+ server in that window and `reindex` could not rebuild that tree — the one+ situation where an operator most needs the rebuild to work. The manifest is+ now written with the scope's first page, one store lookup per scope per+ process, byte-identical to what the backfill writes so restarts still do not+ churn the wiki's git history. The startup backfill is unchanged and remains+ the repair path for trees written by older releases.+ - `ai-memory reindex` now names the exact missing or unreadable scope+ `_meta.md` path instead of collapsing the filesystem error to a bare `No such+ file or directory (os error 2)`. This makes the existing startup-backfill+ workaround discoverable when a scope was first created during the server's+ last run. (#643)+ - Generated routing instructions and all project-scoped managed Agent Skills now+ distinguish session-aware MCP clients from static clients. Static clients are+ told to pass exact `workspace` + `project` values from `.ai-memory.toml` or+ operator configuration on every project-scoped call, preventing another+ session's last active project from capturing reads or writes; global searches+ and global preference writes retain their scope-free argument rules (#372).+ - Consolidation prompt assembly no longer re-renders the whole observation+ projection and re-scores every observation after each pruned one. With a few+ hundred long observations the quadratic loop cost ~14s per prompt; pruning+ now works from per-observation scores and block sizes computed once, with+ byte-identical output.++ ## [2.0.3] - 2026-09-04+### Changed+- Every LLM chat request now sends `User-Agent: ai-memory/<version>`.+ `reqwest` sends no user agent unless one is configured, so provider+ requests previously arrived anonymous, and gateways that require callers to+ identify themselves reported ai-memory as an unknown client. An+ `AI_MEMORY_LLM_HEADERS` entry for `user-agent` overrides it. The Copilot+ provider is unchanged: it keeps `GitHubCopilotChat/<version>`, the+ editor-plugin agent GitHub's Copilot API expects. ([#606])+- `x-opencode-session` and the `opencode` user agent, both shipped in 2.0.2,+ are now operator-overridable like any other header: an+ `AI_MEMORY_LLM_HEADERS` entry for either name takes precedence over the+ provider's default. The default remains 2.0.2's — one `LlmOperationId` per+ logical operation, stable across retries and the strict/tolerant fallback —+ so nothing changes unless an operator asks for it. Override the session+ header to tell several ai-memory instances apart in OpenCode's metrics.+ ([#606])- Release, Docker, and the `cargo install --git` snippet now build with`--locked` (#628). Without it `cargo install` re-resolves dependencies andignores the committed `Cargo.lock`, so a source install could silently pulldiff --cc crates/ai-memory-llm/src/lib.rsindex bc63caea,1d0db028..5e4d4f32--- a/crates/ai-memory-llm/src/lib.rs+++ b/crates/ai-memory-llm/src/lib.rs@@@ -122,6 -100,15 +122,16 @@@ pub use provider::{LlmProvider, completpub use reranker::{LlmReranker, RerankCandidate, RerankScore, Reranker};pub use stored_token::StoredOAuthToken;pub use types::{- ChatMessage, ChatRequest, ChatResponse, LlmOperationId, ReasoningEffort, Role, Usage,+ ChatMessage, ChatRequest, ChatResponse, ExtraHeaders, LlmOperationId, ReasoningEffort, Role,+ Usage,};++ // Integration tests compile into this crate's test harness instead of a+ // separate binary: every test binary is another link and, on macOS and+ // Windows, another first-run malware scan. They still exercise only the+ // public API; `extern crate self` lets them keep addressing it by crate name.+ #[cfg(test)]+ extern crate self as ai_memory_llm;+ #[cfg(test)]+ #[path = "../tests/suite/mod.rs"]+ mod integration;diff --cc crates/ai-memory-llm/tests/suite/extra_headers_on_the_wire.rsindex f93cd577,00000000..f93cd577mode 100644,000000..100644--- a/crates/ai-memory-llm/tests/suite/extra_headers_on_the_wire.rs+++ b/crates/ai-memory-llm/tests/suite/extra_headers_on_the_wire.rsdiff --cc crates/ai-memory-llm/tests/suite/mod.rsindex 00000000,a1e390b6..0337bf86mode 000000,100644..100644--- a/crates/ai-memory-llm/tests/suite/mod.rs+++ b/crates/ai-memory-llm/tests/suite/mod.rs@@@ -1,0 -1,6 +1,7 @@@+ //! This crate's integration tests. Every file here is a module of the lib's+ //! test harness (see the `integration` module in `src/lib.rs`), so they cost+ //! no extra binary; a new file must be declared below.+++mod extra_headers_on_the_wire;+ mod openai_compat_embedder;+ mod openai_compat_strict;diff --cc crates/ai-memory-wiki/src/wiki.rsindex 0510b11e,624e7ca0..9d06a3d7--- a/crates/ai-memory-wiki/src/wiki.rs+++ b/crates/ai-memory-wiki/src/wiki.rs@@@ -1,8 -1,8 +1,8 @@@//! [`Wiki`] — the only correct write path for the markdown source-of-truth.- use std::collections::HashMap;-use std::collections::HashSet;++use std::collections::{HashMap, HashSet};use std::path::{Path, PathBuf};- use std::sync::Arc;+ use std::sync::{Arc, Mutex};use ai_memory_core::{ActorContext, AutoImproveProposalId, NewPage, PageId, PagePath, ProjectId, Sanitizer,@@@ -115,20 -111,13 +115,25 @@@ pub struct Wiki/// the directory rename and SQLite re-stamp so stale writes cannot land/// files under the old workspace while the project is in flight.mutation_lock: Arc<RwLock<()>>,+ /// Per-page write serialization (#607, item 3). Two concurrent writes to+ /// the *same* `(ws, proj, path)` would otherwise interleave their+ /// file-rename and DB-upsert under the shared `mutation_lock` read guard,+ /// transiently leaving on-disk markdown disagreeing with the DB `latest`+ /// row. Each path gets its own async mutex, so same-path writers serialize+ /// while different paths still run concurrently. The map is GC'd+ /// opportunistically — entries no writer currently holds are dropped on the+ /// next acquisition — so it stays bounded to currently-contended paths.+ page_locks: Arc<std::sync::Mutex<HashMap<PageKey, Arc<tokio::sync::Mutex<()>>>>>,+ /// Scopes this process has already materialized `_meta.md` manifests for.+ /// Keeps [`Wiki::ensure_scope_manifests`] to one hash lookup per page+ /// write after the scope's first — the store query and the two manifest+ /// reads happen once per scope per process, never per page.+ manifested_scopes: Arc<Mutex<HashSet<(WorkspaceId, ProjectId)>>>,}+/// Key uniquely identifying a page for per-path write serialization.+type PageKey = (WorkspaceId, ProjectId, PagePath);+impl Wiki {/// Construct a wiki handle rooted at `<data_dir>/wiki/`. Creates the/// directory if absent and initialises a git repo inside it.@@@ -149,7 -138,7 +154,8 @@@admission_chain: None,store_reader: None,mutation_lock: Arc::new(RwLock::new(())),+ page_locks: Arc::new(std::sync::Mutex::new(HashMap::new())),+ manifested_scopes: Arc::new(Mutex::new(HashSet::new())),})}