Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
Merge remote-tracking branch 'origin/pr/1807' into release/3.2.10
AkitaOnRails5f003062438c
diff --cc CHANGELOG.mdindex 374d5534,6eeb5bb1..d5a98342--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -13,64 -13,11 +13,69 @@@ version in [`docs/changelog/`](docs/cha### Fixed+- Sorted recent page lists by timestamp rather than RFC3339 text, so updates+ with different fractional-second precision appear in chronological order+ in the legacy recent API, project activity, folder previews and namespace+ listings. Kept title-sort and equal-timestamp path tie-breaks (#1247).++- Fixed `restore` replacing live data when a backup without `config.toml`+ contains API keys that the configured pepper cannot verify. It now refuses+ before the swap, preserving the live wiki, database and config, and names+ the original configuration as the remedy (#1495).++- Preserved explicit DOCX/PPTX line breaks and DOCX tabs in document imports+ when XML uses paired tags, including formatted PowerPoint breaks in slides+ and speaker notes. DOCX carriage returns now also keep adjacent text runs+ on separate lines instead of joining their words. Paragraph tab-stop+ definitions no longer insert spurious tabs into the extracted text (#1786).++- Kept a wiki-sync server page when its repository file was recreated after+ the sync planned a deletion. `sync --propagate-deletes --apply` now rechecks+ local absence before sending the delete, preserves the page's baseline and+ reports the changed file for the next sync, including with `--prefer repo`+ (#1787).+- Transcript imports (`backfill`, the `run` ledger, `recover`) now drop Codex+ memory-writing agent runs by the same signals the live hook uses: the+ checkout cwd, an event's recorded cwd, or a Codex user prompt that opens+ with `## Memory Writing Agent`. A custom `CODEX_HOME` whose last path+ component is not `.codex` previously reimported those sessions (#1790).+- Admission webhook HTTP requests now follow redirects only within the+ configured `scheme://host:port` origin (up to 10 hops) and fail on+ cross-origin redirects without forwarding page payloads or applying foreign+ response mutations, surfacing the redirect refusal cause in logs and+ `reject`-policy errors (#1794).+- `ai-memory project access` and `ai-memory project grants` now refuse empty+ or whitespace-only `--workspace`, `--project`, and (for access) `--mode`+ values before sending any request to the server. These commands take+ required names and never call `resolve_scope`, so the #1774 choke point+ did not cover them (#1791).+- `ai-memory user grant` and `ai-memory user revoke` now refuse empty or+ whitespace-only `--user`, `--workspace`, `--project`, and (for grant)+ `--level` values before sending any request to the server. These commands+ take required names and never call `resolve_scope`, so the #1774 choke+ point did not cover them (#1792).+- Stopped the lifecycle relay from treating an oversized HTTP response as a+ transport failure and immediately resending its batch. A response beyond the+ 64 KiB acknowledgement cap now retains its HTTP status, stops the flush, and+ keeps unacknowledged events pending, including on HTTP 429 backpressure (#1799).+- Decoded WebVTT cue character references before document-import sanitization,+ preserving literal escaped tags and applying the existing text-size limit+ to the decoded text. SRT extraction is unchanged (#1800).+- Preserved native OIDC hook-spool entries when the POSIX or PowerShell script+ bundle drained the shared queue. Scripts now leave those entries for native+ credential resolution instead of posting them anonymously and retiring them+ on a 2xx or 401 response; static, anonymous and legacy entries still drain+ normally (#1801).+- Refused backup restores with a missing or invalid gzip trailer before+ replacing live data, even when all tar entries were readable (#1805).+- Kept separate pre-migration archives when installations share a backup+ destination and start in the same second, instead of overwriting an+ existing installation's snapshot (#1806).+ - Restored the previous wiki file when `restore-page` could not commit its+ SQLite update, including a rejected case-colliding historical path. Restores+ now share the page writer's lock through conformance, file replacement and+ indexing so a concurrent write to that same path cannot leave disk and+ SQLite disagreeing (#1807).- Refused empty and whitespace-only `--workspace` and `--project` flags at theshared CLI scope-resolution choke point (`resolve_scope`,`resolve_scope_noting_marker_project`, `resolve_scope_with_marker`, anddiff --cc docs/security-boundaries.mdindex 72ee710e,13cccdfd..c6babc91--- a/docs/security-boundaries.md+++ b/docs/security-boundaries.md@@@ -68,9 -68,9 +68,9 @@@ the legitimate restricted-project contr| 4o | An OMP subagent session never claims a handoff or leaves one behind | `ai-memory-hooks/src/router.rs` SessionEnd `child_session`: an OpenCode or OMP end whose body carries a subagent marker (`payload.rs` `body_is_subagent`) closes and summarizes the session without a baton. The generated OMP extension (`ai-memory-cli/src/commands/install_hooks.rs` `build_omp_extension`) records a subagent session from `ctx.agent.kind` on the session's own `session_start`, `before_agent_start` and `session_shutdown`, never on a tool event (an advisor's tool calls reach the session they advise as `kind: "sub"` under that session's id), skips `fetchHandoff` for it and posts its events with `agent_id`. Pi reports no `ctx.agent`, so every Pi session stays top-level | router `omp_subagent_session_end_closes_and_summarizes_without_baton` (a marked OMP end leaves no baton; control: a top-level OMP session in the same project still leaves one), beside `opencode_child_session_end_closes_and_summarizes_without_baton`, and `drop_subagent_captures_drops_an_omp_subagent_session` (an opted-in project drops the marked session through its end; control: the top-level session is stored); `install_hooks.rs` `omp_extension_marks_subagent_sessions_and_skips_their_handoff` and, running the generated OMP extension in Node, `slow_generated_handoff_runtime::omp_extension_keeps_subagent_sessions_off_the_baton` (a task subagent, a `/tan` clone and a revived subagent never request `/handoff` and carry `agent_id` on every event, their ends included; controls: the top-level session and a session without `ctx.agent` claim, and the advisor's tool calls leave the top-level session unmarked). Each fails with its guard removed | STRONG || 5a | Pages shared: `author_id` is never a read filter (invariant #16) | `ai-memory-store/src/reader.rs` `search_pages`/`page_body_by_ids` — `author_id` is an attribution JOIN only, never a WHERE term; `observation_stream_votes` lets every session vote whatever its owner (the owner filter only withholds the observation excerpt and ids, never the vote or the page) | `multi_session.rs` — a page with a **non-null** `author_id` (operator A) is readable by operator B in the same project; `multi_session.rs` `another_operators_observations_lift_their_session_page_in_a_shared_project` (operator B's owned session, authored page and observations lift that page for operator A's query); `ai-memory-mcp` `server.rs` `observation_stream_evidence_stays_inside_a_restricted_project` (a second operator with only a read grant gets the session page lifted by the first operator's observation); `multi_session.rs` `observation_excerpts_follow_the_session_owner_while_the_vote_stays_shared` (B still gets A's session page ranked by A's observation, with the page's own snippet) | STRONG || 5b | Page supersession (loser stays reachable) | `ai-memory-store/src/ops.rs` `upsert_page_in_tx` — demote `is_latest=0` (never delete) + `supersedes` chain; read side: `reader.rs` `page_versions` lists every row at the path, with no `is_latest` filter, for `/web/w/…/h/*path` and `/api/v1/…/history/*path`. It does apply TTL expiry (`reader.rs` `history_visible`): a version whose own `expires_at` passed is neither listed nor readable by id, a path whose live version expired shows no history at all (the forget sweep hard-deletes every row there), and a `supersedes` link to a hidden version reads as `null`. Expiry is the only thing that hides a superseded version; a concurrent write never does | `multi_session.rs` `concurrent_writes_to_one_path_supersede_rather_than_destroy`; `retrieval_superseded.rs`; `multi_session.rs` `page_history_keeps_the_overwritten_version_and_stays_in_scope`; `ai-memory-web` `history_replay.rs` `history_keeps_both_sides_of_a_concurrent_divergent_write` (two attributed writers on one base: all three versions and both authors listed, one `is_latest`, the two writers' versions diff by id) and `history_lists_k_plus_one_versions_newest_first_and_diffs_the_latest`; bite: an `is_latest = 1` filter in `page_versions` fails both; `history_hides_expired_versions_and_expired_pages` (an expired superseded version and every version of a page whose live version expired answer 404 by path and by id on HTML and JSON, and the live successor's `supersedes` is `null`; control: a TTL in the future is listed and diffable; bites with the per-row predicate removed, with the live-version clause disabled, and with the `supersedes` hiding removed) | STRONG |- | 5c | Watcher reindex and Wiki writes share per-page serialization (invariant #16) | `ai-memory-wiki/src/wiki.rs` `reindex_page` takes `mutation_lock.read` before `reindex_page_locked` takes `lock_page`, covering disk read through the single-writer upsert; callers already holding the exclusive global guard do not reacquire it. `write_page` uses the same page mutex and `apply_batch` retains sorted, deduplicated locks; embedding runs after both guards are released | `wiki.rs` `reindex_page_lock_waits_before_reading_and_allows_another_path` (transient snapshot refused while locked, another path progresses, release indexes the committed bytes without rewriting disk), `reindex_page_lock_serializes_write_through_the_upsert` and `reindex_page_lock_and_reversed_duplicate_batches_do_not_deadlock` (real concurrent operations, final disk/SQL winner and supersession ancestry preserved); all three fail with the reindex page guard removed. `reindex_page_lock_releases_both_guards_before_embedding` checks both body and abstract embedding; `decay_cleanup_reindex_does_not_embed_under_the_exclusive_lock` covers the exclusive caller | STRONG for a shared Wiki handle and its clones. External editors and independently constructed Wiki handles do not participate in these locks |+ | 5c | Watcher reindex and Wiki writes share per-page serialization (invariant #16) | `ai-memory-wiki/src/wiki.rs` `reindex_page` takes `mutation_lock.read` before `reindex_page_locked` takes `lock_page`, covering disk read through the single-writer upsert; callers already holding the exclusive global guard do not reacquire it. `write_page` and `restore_page_from_checkpoint` use the same page mutex and `apply_batch` retains sorted, deduplicated locks; restore reads current frontmatter under the guard and rolls back its installed file if the store refuses the version; embedding runs after both guards are released | `wiki.rs` `reindex_page_lock_waits_before_reading_and_allows_another_path` (transient snapshot refused while locked, another path progresses, release indexes the committed bytes without rewriting disk), `reindex_page_lock_serializes_write_through_the_upsert` and `reindex_page_lock_and_reversed_duplicate_batches_do_not_deadlock` (real concurrent operations, final disk/SQL winner and supersession ancestry preserved); all three fail with the reindex page guard removed. `restore_page_lock_waits_before_conformance_and_allows_another_path` (a queued writer wins first, its timestamp is preserved, another path progresses; fails without the restore page guard or with conformance before it), `restore_page_collision_keeps_disk_and_store_unchanged` and `restore_page_store_failure_rolls_back_replacement_and_new_file` (rejected historical alias and injected upsert failures preserve disk and SQL, ordinary restore succeeds after removing the fault; fail without restore rollback). `reindex_page_lock_releases_both_guards_before_embedding` checks both body and abstract embedding; `decay_cleanup_reindex_does_not_embed_under_the_exclusive_lock` covers the exclusive caller | STRONG for a shared Wiki handle and its clones. External editors and independently constructed Wiki handles do not participate in these locks || 5d | Wiki project-tree filesystem confinement | `ai-memory-wiki/src/confinement.rs` checks the wiki root, workspace/project roots, every existing ancestor, the final component, and the in-root ordinary `.git` directory with link-aware metadata; repository workdir/gitdir/commondir are containment-checked after opening, and write parents are created one component at a time and rechecked. `Wiki` checks before admission and again under mutation/page locks, while `GitAdapter` rechecks immediately before writes, appends, renames, and removals. Direct reindex, checkpoint restore, manifests, auto-improvement artifacts, ledger append, startup repair, purge/session moves, rollback, migrations, and git tree staging use the same boundary. Confinement is a distinct `WikiError` and therefore cannot trigger missing-file DB fallback | `wiki.rs` `page_reads_refuse_leaf_and_ancestor_symlinks_without_fallback_classification`, `ai-memory-mcp` `admin_read_page.rs` `read_page_does_not_db_fallback_through_a_symlink`, `page_mutations_refuse_ancestor_symlink_and_leave_outside_and_store_unchanged`, `confinement_refusal_happens_before_blocking_and_observer_admission`, `restore_and_reindex_refuse_ancestor_symlink_without_touching_canary_or_db`; `git.rs` `git_metadata_link_is_refused_without_touching_external_repository`, `gitfile_redirect_outside_is_refused_without_touching_target`, `git_metadata_redirect_files_are_refused_before_repository_open`, `configured_worktree_outside_the_wiki_root_is_refused`, `linked_git_metadata_never_reaches_commit_fallback`, `gitfile_metadata_never_reaches_commit_fallback`, `out_of_root_worktree_never_reaches_commit_fallback`, `fallback_preflight_refuses_invalid_git_metadata`, `windows_commit_fallback_accepts_established_eligible_errors`, `eligible_windows_open_error_dispatches_the_fallback_after_validation`, `windows_init_error_dispatches_fallback_after_root_preflight`, `invalid_root_never_reaches_init_fallback`, `checkpoint_and_revision_reads_refuse_a_replaced_git_directory`, and ordinary-repository/API controls `init_is_idempotent_and_creates_dotgit`; `linked_wiki_root_is_refused_during_construction`, `linked_or_dangling_scope_roots_are_refused_but_real_roots_work`, existing `decay_cleanup_refuses_to_reindex_a_symlinked_recreation` and `reindex_rejects_symlinked_scope_manifest`; `ai-memory-mcp` `admin_move.rs` `true_move_refuses_an_invalid_destination_namespace_without_db_changes` and `delete_workspace_refuses_invalid_namespace_before_db_or_notification`; `admin_backup.rs` `backup_refuses_redirected_git_metadata_before_returning_an_archive`, `backup_refuses_wiki_leaf_links_before_returning_an_archive`, `backup_refuses_linked_wiki_directories_before_returning_an_archive`, and ordinary archive controls; `watcher.rs` `direct_file_event_skips_symlink`, `walk_markdown_skips_linked_directories_and_their_ledgers`, and `shared_link_like_helper_accepts_regular_files_and_directories` (Windows directory-link coverage exercises reparse metadata when privilege is available); `backup.rs` `a_pre_migration_backup_refuses_a_linked_wiki_tree_without_an_archive`; `confinement.rs` metadata-helper controls run on every platform | STRONG for imported/static links and same-process mutations. Component checks do not claim descriptor-relative protection against a concurrently malicious same-user process; Windows reparse behavior requires the `windows` PR label/full CI |-| 5e | Conditional page mutation (#986, invariant #16) | `ai-memory-store/src/ops.rs` `upsert_page_if` reads the latest version and writes in one transaction, the authority; `ai-memory-wiki/src/wiki.rs` `write_page_if` pre-checks under the per-page lock before touching disk and restores the file when the store check fails; conditional deletes reuse `delete_page_if_latest` (exclusive mutation lock plus an in-transaction check). `ai-memory-mcp/src/server.rs` checks the precondition only after scope resolution and the write grant, against the resolved scope's own latest row, and returns `precondition_failed` with no write, checkpoint or webhook dispatch. The agentmemory importer (`agentmemory/mod.rs` `apply_page`) resumes from a conflicting destination only when its complete body equals a planned version; a retained attribution key alone cannot supply the next write's expected version. `companions/ai-memory-wikisync/src/bidi.rs` `build_plan` reads the full page for an edited repository file, keeping classification and `expected_page_id` on the same server version | `multi_session.rs` `conditional_writers_on_one_base_exactly_one_wins_and_nothing_is_destroyed` (two writers from one version: one wins, the base stays reachable, no loser row; fails with the store check removed); `server.rs` `conditional_page_writes::*` (stale id, `create_only` on an existing page, a version id from another project, and conditional deletes are refused and change nothing, with and without the wiki pre-check; the store check proven to bite with the reader pre-check off); `a_reader_may_read_everything_and_change_nothing` (a read-only user's conditional write or delete, with the right id or a wrong one, gets the authorization error and no version data; the page is unchanged); importer `resume_refuses_an_edited_body_even_with_the_original_ingest_key` and `e2e_export_import_rebuilds_chains_and_honours_ttl` (an edited destination body keeps its attribution key but cannot be superseded by a resumed import; both fail with marker-only matching); `resume_after_an_injected_failure_writes_no_duplicates` and `a_foreign_page_at_the_destination_is_never_replaced` are the legitimate-resume and foreign-page controls; wikisync `tests/api_server.rs` `sync_does_not_rebase_an_import_onto_a_concurrent_server_edit` (an edit after the read snapshot is refused by the original version precondition, preserving server, repository and baseline; fails without the conditional-read guard), `sync_imports_a_repo_edit_with_its_metadata` and `sync_exports_metadata_and_round_trips_without_writes` (ordinary import and unchanged-file ETag controls) | STRONG. Importer resumption compares bodies, not frontmatter-only edits; custom server sanitation that changes the planned body fails closed |+| 5e | Conditional page mutation (#986, invariant #16) | `ai-memory-store/src/ops.rs` `upsert_page_if` reads the latest version and writes in one transaction, the authority; `ai-memory-wiki/src/wiki.rs` `write_page_if` pre-checks under the per-page lock before touching disk and restores the file when the store check fails; conditional deletes reuse `delete_page_if_latest` (exclusive mutation lock plus an in-transaction check). `ai-memory-mcp/src/server.rs` checks the precondition only after scope resolution and the write grant, against the resolved scope's own latest row, and returns `precondition_failed` with no write, checkpoint or webhook dispatch. The agentmemory importer (`agentmemory/mod.rs` `apply_page`) resumes from a conflicting destination only when its complete body equals a planned version; a retained attribution key alone cannot supply the next write's expected version. `companions/ai-memory-wikisync/src/bidi.rs` `build_plan` reads the full page for an edited repository file, keeping classification and `expected_page_id` on the same server version; `delete_server_page` rechecks the repository file's absence before sending a conditional server delete | `multi_session.rs` `conditional_writers_on_one_base_exactly_one_wins_and_nothing_is_destroyed` (two writers from one version: one wins, the base stays reachable, no loser row; fails with the store check removed); `server.rs` `conditional_page_writes::*` (stale id, `create_only` on an existing page, a version id from another project, and conditional deletes are refused and change nothing, with and without the wiki pre-check; the store check proven to bite with the reader pre-check off); `a_reader_may_read_everything_and_change_nothing` (a read-only user's conditional write or delete, with the right id or a wrong one, gets the authorization error and no version data; the page is unchanged); importer `resume_refuses_an_edited_body_even_with_the_original_ingest_key` and `e2e_export_import_rebuilds_chains_and_honours_ttl` (an edited destination body keeps its attribution key but cannot be superseded by a resumed import; both fail with marker-only matching); `resume_after_an_injected_failure_writes_no_duplicates` and `a_foreign_page_at_the_destination_is_never_replaced` are the legitimate-resume and foreign-page controls; wikisync `tests/api_server.rs` `sync_does_not_rebase_an_import_onto_a_concurrent_server_edit` (an edit after the read snapshot is refused by the original version precondition, preserving server, repository and baseline; fails without the conditional-read guard), `sync_imports_a_repo_edit_with_its_metadata` and `sync_exports_metadata_and_round_trips_without_writes` (ordinary import and unchanged-file ETag controls); `sync_keeps_a_server_page_when_its_file_reappears_during_planning` (restored file blocks server deletion and keeps the baseline, with and without `--prefer repo`; another absent file is deleted and a later sync imports the restored edit; fails without the local-absence recheck) | STRONG. Importer resumption compares bodies, not frontmatter-only edits; custom server sanitation that changes the planned body fails closed. Wiki-sync rechecks local absence immediately before the request; this is not atomic across the filesystem and server || 5f | Wiki-sync companion destination confinement | `companions/ai-memory-wikisync/src/paths.rs` `prepare_dest` compares canonical and lexical destination components without resolving the lexical path; Windows disk and UNC verbatim prefixes compare as equivalent spellings, while changed directory components are refused. `secure_join` keeps validated page paths below that root and refuses existing symlink components | `paths.rs` `prepare_dest_refuses_windows_junctions_without_touching_the_target` refuses a junction and a destination reached through it, leaves an outside canary unchanged, and accepts the real directory; `windows_destination_prefixes_do_not_hide_different_paths` refuses different drives, shares and directory components with ordinary/verbatim controls; `prepare_dest_refuses_symlinks_and_files` and `secure_join_stays_under_root_and_refuses_symlink_components` retain the Unix link controls | Static path checks; no descriptor-relative protection against a concurrently malicious same-user process. The dedicated `windows.yml` wiki sync companion job runs the prefix and junction tests on companion/workflow changes, `windows` or `full-ci` labels, nightly runs and dispatch || 6 | Active-project pointer (PerActor, no clobber) | `ai-memory-core/src/active_project.rs` `set_for`/`lookup_for` (fail-closed on `Mismatch`); `ai-memory-cli/src/commands/mcp_bridge.rs` `bridge_session_id`: the session-less bridge (`--no-session-id`, every non-Claude stdio client on a token-file install) never forwards an inherited `CLAUDE_CODE_SESSION_ID`, so a client launched from a Claude Code terminal cannot steer that session's per-session pointer; the session-aware bridge reads only the variable its `--agent` names (`CLAUDE_CODE_SESSION_ID`, or `MUSE_SESSION_ID` for Meta Muse Code), so neither harness's id stands in for the other's | `active_project.rs` `parallel_harnesses_of_one_user_keep_separate_pointers`, `two_operators_never_read_each_others_pointer`, `a_session_mismatch_fails_closed_once_anything_has_been_keyed`; `multi_session.rs` `same_operator_multiple_sessions_isolate_active_project_pointers_and_resolution` (concurrent sessions of one operator under `AccessMode::Restricted` resolve unscoped reads/writes to their own active project without cross-session clobber, while an unbound session of the same operator fails closed on unscoped write with `AmbiguousUnscopedWrite` and falls back to `DefaultAfterMismatch` on read); `mcp_bridge.rs` `only_the_session_aware_mode_forwards_and_requires_a_session_id` (an inherited id is dropped in session-less mode; the session-aware control forwards it and still fails closed without one), `each_harness_forwards_only_its_own_session_id` (a Muse bridge never forwards an inherited Claude id and fails closed without its own, a Claude bridge never forwards an inherited Muse id; controls forward each harness's own id) and `a_session_less_upstream_config_carries_no_session_header`; `auth_default_e2e.rs` `a_session_less_bridge_serves_any_client_without_holding_the_token` (the built `mcp-bridge --agent muse` refuses to start with only `CLAUDE_CODE_SESSION_ID` set) | STRONG || 6b | Native `_meta` routing coordinate is routing-only (never grants identity) | `ai-memory-mcp/src/server.rs` `attach_native_session` reads only `_meta["ai.opencode/sessionID"]` into `ActorKey.session_id` (trimmed, non-empty-string-validated); `user` still comes from the authenticated `ActorContext`, so a forged `_meta` cannot cross a user/scope boundary — it only selects a session slot behind the existing `active_project` guard | `server.rs` `native_session_metadata_is_routing_only_and_preserves_precedence`, `native_session_metadata_validates_strings_without_granting_identity`; `autoscope_multiuser.rs` `native_metadata_routes_concurrent_reads_and_writes_to_hook_workspaces` (#864) | STRONG |