Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
Merge branch 'main' into PR #1134
AkitaOnRails5d350c574ad1
diff --cc CHANGELOG.mdindex ef56d745,cb07897e..e9538963--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -8,6 -8,6 +8,10 @@@ and this project adheres to [Semantic V## [Unreleased]### Fixed++- Fixed `ai-memory backfill` failing on every Antigravity CLI session: it and++ `ai-memory run` now import the session's user prompts from `agy`'s++ `history.jsonl`, keeping only lines whose conversation and workspace match++ the checkout. (#1134)- Fixed the scheduled auto-improve tests' intermittent empty log captures(the same latent flaw #1116 fixed for the hooks checkpoint test): a shared`warn!` callsite's first-in-process execution on a bare thread cachesdiff --cc docs/security-boundaries.mdindex f80f8b74,1a5f5739..44fe8127--- a/docs/security-boundaries.md+++ b/docs/security-boundaries.md@@@ -75,10 -75,9 +75,10 @@@ boundary not yet built| 11i | OpenCode executable probing and generated-config cleanup (#1097) | `ai-memory-cli/src/config.rs` captures the full process environment once in typed `RuntimeEnv`; `run.rs` overlays explicit `run --env` values with platform key semantics and uses that same effective environment for executable resolution, probing, and launch without rereading process environment; `opencode_dialect.rs` invokes only direct argv `--version`, caps output and time, and rejects malformed/future majors. Automatic launch skips only OpenCode on probe failure; explicit OpenCode fails. `install_hooks.rs` quarantines the incompatible sibling before reading its exact ownership markers, restores unowned content with an atomic no-replace link, and never deletes or overwrites a concurrent replacement; `install_mcp.rs` owns the shared dual-schema endpoint/token inference used by both installers, rejects conflicting generated V1/V2 entries before mutation, and migrates an alternate-major entry only when endpoint, generated shape, and absent-or-Authorization-only headers match, preserving custom headers and ambiguous entries; auto-wire migrates MCP before hooks infer from that exact config path and skips hook mutation when transition preflight fails | `run.rs` `automatic_opencode_probe_failure_keeps_another_harness_available`, `opencode_executable_probe_selects_v2_for_launch_and_autowire`, effective-child-environment PATH casing controls, fake V1/V2/env-derived-major/malformed/future/timeout executable tests, `setup-agent` V1/V2 no-probe integration controls; `install_hooks.rs` `opencode_plugin_cleanup_removes_only_exact_owned_incompatible_sibling`, `opencode_plugin_cleanup_preserves_a_concurrent_user_replacement`, `opencode_plugin_restore_moves_quarantine_when_destination_is_free`, `opencode_plugin_restore_preserves_an_existing_replacement`, `opencode_plugin_restore_does_not_replace_a_file_created_at_restore_window`, `opencode_plugin_cleanup_does_not_escape_custom_target_directory`; `install_mcp.rs` `opencode_install_reuses_owned_endpoint_and_token_across_major_transitions`, `opencode_install_refuses_conflicting_owned_transition_entries`, `opencode_install_does_not_infer_from_a_user_owned_transition_entry`, `opencode_mcp_migration_moves_only_owned_alternate_shape`, `opencode_mcp_migration_preserves_ambiguous_user_entry`, `opencode_mcp_migration_preserves_custom_headers`, `opencode_mcp_migration_accepts_only_historical_generated_headers` | STRONG for Unix process probes and platform-neutral ownership gates; exact Windows executable resolution remains covered by `run.rs` resolver tests and requires the PR `windows` label || 12 | Network/auth posture | `config.rs` loopback `DEFAULT_BIND`; `serve.rs` `validate_http_exposure`, `require_allowed_host`; `auth.rs` `require_bearer`; `serve.rs` mounts `/identity` inside the host and machine-auth gates even with web disabled; `nix/nixos-module.nix` treats only literal `127.0.0.1`/`::1` as loopback, requires exactly one secret environment source for other binds, keeps secret auth and `llm_headers` values out of the Nix store, and brackets IPv6 in `ExecStart` | `serve.rs` host-guard (missing→400 / forged→403), non-loopback-requires-token; `auth.rs` wrong-token 401; `serve.rs` `machine_identity_remains_authenticated_with_web_disabled_and_expired_keys` (wrong and expired keys refused, valid machine key accepted, web remains disabled); `flake.nix` `nixos-module-eval` refuses `localhost` and `0.0.0.0` without a secret source, refuses two secret sources, `settings.auth` secrets, and `settings.llm_headers`, with literal IPv4/IPv6 loopback and one-source controls | STRONG || 12b | Managed-workstream provenance privacy, stable replay identity and server event namespace | `ai-memory-core/src/workstream.rs` `scrub_workstream_provenance`; `ai-memory-hooks/src/workstream.rs` `finish_run` reserved client-id check and `sanitize_events` before immutable segments, with `sanitize_events` and `append_boundary_events` scrubbing event/checkpoint/losses content *before* the 64 KiB cap so a cap-straddling secret is never cut into an unmatchable prefix (#1113); `ai-memory-store/src/workstream.rs` `finish_run` stable identity check and `stored_event`, scrubbing content with the explicitly supplied configured sanitizer before its 16 KiB store-boundary bound; no changes to the typed observation boundary | `store/tests/suite/workstream_provenance.rs` `workstream_provenance_is_bounded_scrubbed_and_idempotent_before_storage` (hostile and oversized source ids, unknown metadata, legitimate correlation, retry and distinct event ids), `workstream_provenance_legacy_reads_strip_secrets_and_unknown_metadata` (search and tail, malformed/oversized legacy JSON), `workstream_provenance_reused_event_id_retains_data_and_refuses_identity_change` (first indexed version retained, identity refusal and batch rollback), `workstream_provenance_replay_survives_sanitizer_changes` (new extra patterns and removal of a built-in-pattern allowlist exemption, with identity-refusal control), `workstream_provenance_replay_preserves_legacy_rows_above_bounds`, `workstream_provenance_finished_run_ignores_late_events`, `workstream_provenance_retry_with_marker_matching_pattern_is_idempotent`, `workstream_content_is_scrubbed_before_the_store_cap` (a credential straddling the 16 KiB store cap is redacted, not persisted as an unmatchable prefix; benign control verbatim; #1113); `hooks` `workstream_provenance_finish_scrubs_before_raw_segment_and_search` (custom sanitizer, immutable segment, SQL recovery and finished-run no-op), `workstream_provenance_client_cannot_use_server_event_ids` (reserved ids refused before raw writes, normal client event and server-generated checkpoint/loss controls), `finish_run_scrubs_event_content_before_the_byte_cap` and `finish_run_scrubs_boundary_events_before_the_byte_cap` (a credential straddling the 64 KiB event cap is redacted in the raw segment and never reaches the index; checkpoint/losses and benign controls; #1113); `cli/tests/suite/workstream_provenance.rs` (text/JSON and old server DTOs) | STRONG: privacy, stable-identity and reserved-namespace regressions fail with their guards bypassed; an isolated allowlist bypass fails on the unknown key, and bypassing the raw-id search authorization returns 200 where its test requires 403. No provenance is emitted in startup packets, and old raw segments are not rewritten |- | 13 | Managed-run transcript attribution (concurrent launches in one checkout, invariant #16) | `ai-memory-store/src/workstream.rs` `link_native_session` stamps `native_session_linked_at` on its own run only, both `finish` updates drop the stamp when the session changes, `run_status` reports it; `ai-memory-cli/src/commands/run.rs` `resolve_native_session_after_run` takes a linked session only when `ai-memory-workstream` `native_session_in_checkout` holds it for this checkout (OpenCode by recorded directory), never falls back to a link it set aside, and turns an `AmbiguousNativeSession` into a warning with nothing imported; for Claude it moves the run's own session to a background session only when `ai-memory-workstream` `claude_attached_background_session` finds a `sessionKind: "bg"` record that names that exact session as its attached client and this checkout as `cwd` (#1050); `ai-memory-workstream/src/transcript.rs` `discover_crush` claims only the one top-level session created (or, with `--continue`, touched) during the run, and in a data directory outside the project only one that edited a file in it | `multi_session.rs` `a_session_linked_by_one_managed_run_is_not_another_runs`; `run.rs` `a_session_linked_during_the_run_wins_over_discovery` (concurrent newer session, another checkout's link refused, no fallback to it, unlinked control); `transcript.rs` `claude_attached_background_session_follows_only_this_checkouts_bg_records` (a concurrent launch's background session, another checkout's, and a non-background record refused; own attachment control); `run.rs` `a_claude_run_that_attached_to_a_background_session_follows_it`; `transcript.rs` `native_session_in_checkout_checks_the_opencode_directory`; `store/src/lib.rs` `managed_run_status_reports_a_link_made_during_the_run`; `run.rs` `ambiguous_crush_discovery_keeps_the_run`; `transcript.rs` `crush_discovery_claims_only_the_session_the_run_created`, `crush_discovery_in_a_shared_store_claims_only_an_edit_here` | PARTIAL: a run whose child links nothing still falls back to discovering the newest session in the checkout; Crush, which has no hooks, relies on that discovery alone and claims nothing when it is ambiguous |+ | 13 | Managed-run transcript attribution (concurrent launches in one checkout, invariant #16) | `ai-memory-store/src/workstream.rs` `link_native_session` stamps `native_session_linked_at` on its own run only, both `finish` updates drop the stamp when the session changes, `run_status` reports it; `ai-memory-cli/src/commands/run.rs` `resolve_native_session_after_run` takes a linked session only when `ai-memory-workstream` `native_session_in_checkout` holds it for this checkout (OpenCode by recorded directory), never falls back to a link it set aside, and turns an `AmbiguousNativeSession` into a warning with nothing imported; for Claude it moves the run's own session to a background session only when `ai-memory-workstream` `claude_continued_session` finds a `sessionKind: "bg"` record that names that exact session as its attached client and this checkout as `cwd` (#1050), and to a `/clear`ed session only when that session's own transcript records the `/clear` command and names the run's session as its launch `session_id` from this checkout or below it; `ai-memory-workstream/src/transcript.rs` `discover_crush` claims only the one top-level session created (or, with `--continue`, touched) during the run, and in a data directory outside the project only one that edited a file in it | `multi_session.rs` `a_session_linked_by_one_managed_run_is_not_another_runs`; `run.rs` `a_session_linked_during_the_run_wins_over_discovery` (concurrent newer session, another checkout's link refused, no fallback to it, unlinked control); `transcript.rs` `claude_continued_session_follows_only_this_checkouts_bg_records` (a concurrent launch's background session, another checkout's, and a non-background record refused; own attachment control); `transcript.rs` `claude_continued_session_follows_only_this_launchs_clear` (an uncleared transcript, a quoted `/clear`, another checkout's clear, a concurrent launch's clear, a subagent file, and a pre-run clear refused; own clear and the newest of two as controls); `run.rs` `a_claude_run_that_attached_to_a_background_session_follows_it`, `a_claude_run_that_was_cleared_follows_the_new_session`; `transcript.rs` `native_session_in_checkout_checks_the_opencode_directory`; `store/src/lib.rs` `managed_run_status_reports_a_link_made_during_the_run`; `run.rs` `ambiguous_crush_discovery_keeps_the_run`; `transcript.rs` `crush_discovery_claims_only_the_session_the_run_created`, `crush_discovery_in_a_shared_store_claims_only_an_edit_here` | PARTIAL: a run whose child links nothing still falls back to discovering the newest session in the checkout; Crush, which has no hooks, relies on that discovery alone and claims nothing when it is ambiguous || 13a | Managed Codex stale-daemon run recovery (#987; concurrent launches, project/worktree isolation, operator isolation) | `managed_runs.owner_user` (V71) uses the topology-aware `IdentityKey` owner stamp; `router.rs` resolves and authorizes the SessionStart repository before handling the untrusted run id; store `link_or_adopt_native_session` selects and links in one writer transaction and admits only the sole live, undelivered, never-linked Codex run in the exact workspace/project/checkout-cwd/owner bucket. An active supplied id must itself match every boundary; another harness, an ambiguity, or a second native session is refused | `multi_session.rs` `stale_codex_run_recovery_cannot_cross_project_owner_or_session_boundaries` (foreign owner, foreign checkout, and cross-project candidates refused; project-local controls and racing rebind), `stale_codex_run_recovery_fails_closed_on_ambiguity_and_active_mismatch` (two candidates and active foreign id refused, exact control); `workstream.rs` `prepare_stamps_the_operator_bucket_used_by_session_start_recovery` (HTTP actor stamp with foreign and legitimate controls); `router.rs` `managed_session_start_delivers_ledger_and_pending_handoff` supplies a finished stale id and proves the new run is linked and delivered | STRONG for SessionStart recovery. No candidate is selected from a run id's stale scope, another checkout, or another operator bucket; ambiguity deliberately falls back to no managed packet || 13b | Managed-run lease exclusivity and explicit recovery (one active run per workstream, invariant #16) | `ai-memory-store/src/workstream.rs` `prepare_run` expires lapsed leases and otherwise refuses an `active` run inside the writer transaction (`StoreError::WorkstreamBusy`). The explicit `force_unlock` path may expire and replace that row in the same transaction only when its durable `owner_user` equals the incoming actor's attribution; different operators remain isolated. `heartbeat` renews only `active` rows. `ai-memory-cli/src/commands/run.rs` normally uses `wait_out_held_lease`; `--force-unlock` instead asks once and fails immediately when refused. Neither path treats the informational `lease_owner` label as authority | `multi_session.rs` `force_unlock_replaces_only_the_same_operators_active_run` (foreign operator refused with live control, same operator succeeds, old run loses heartbeat, unattributed single-user control); hooks `workstream.rs` `force_unlock_cannot_evict_another_operator` exercises the HTTP actor boundary; CLI `run.rs` `force_unlock_asks_once_instead_of_waiting_out_the_lease`; existing controls `managed_workstream_batches_are_idempotent_and_release_the_lease`, `a_renewed_lease_is_reported_as_a_live_owner_not_taken_over`, `interactive_launch_waits_out_a_lapsing_lease_then_proceeds`, and `ctrl_c_aborts_the_held_lease_wait_immediately` | STRONG for exclusivity. The `lease_owner` label (`host:pid`) is informational only and not unique inside ai-jail (every jailed launcher reports `ai-sandbox:<ns-pid>`), so it must never become an ownership key. A shared unattributed/root context is intentionally one operator bucket; deployments needing person-level refusal must authenticate DB users |+| 13c | A native store shared by every project yields only this checkout's records (Antigravity `history.jsonl`) | `ai-memory-workstream/src/transcript.rs` `export_antigravity` — the conversation must first pass `locate_session_file` (UUID file name + `.db` workspace == `cwd`); every `history.jsonl` line must match both that `conversationId` and, by `same_path`, this `cwd` (a conversation resumed in another directory keeps its id but records that directory, so the id alone is not enough); an id-less opening prompt is attributed only when exactly one same-workspace candidate falls within 5 s of the `.db` creation time and no other conversation of this checkout was created within 5 s of it; slash/shell entries and oversized lines are dropped, and only records naming this conversation are reported as its losses | `transcript.rs` `antigravity_history_exports_only_this_conversations_prompts_in_this_workspace` (other conversation in this and another workspace refused; own control passes), `antigravity_history_drops_prompts_from_a_resumed_conversation_in_another_workspace` (same id, foreign workspace refused), `antigravity_export_rejects_a_conversation_from_another_workspace`, `antigravity_export_rejects_unknown_or_traversal_ids`, `antigravity_opening_prompt_is_refused_when_another_conversation_started_with_it` (an opener another conversation of this checkout could own is refused), `antigravity_opening_prompt_covers_a_single_prompt_conversation` (control), `antigravity_history_reports_no_losses_for_foreign_records`; `backfill_e2e.rs` `backfill_imports_antigravity_prompts_from_history_and_nothing_foreign` (foreign prompts absent from wiki and db) | STRONG || 14 | Per-project authorization (#708) | `ai-memory-store/src/project_authz.rs` `authorize_project` / `ProjectAuthz::authorize` choke point (V68 `project_grants` + `projects.access_mode`, default `open`; V69 `projects.created_by` feeds `is_creator`); `scope.rs` `ScopeResolver::with_project_authz` (reader pool for reads, writer actor for writes) and its free forms `authorize_scope_for` / `*_guarded`, attached for every DB user by `ai-memory-mcp` `scope_resolver_as`, `ai-memory-web` `authorize_read` / `lookup_project`, and `ai-memory-hooks` (`grants.rs` `authorize_resolved` for run/workstream ids, the capture check in `router.rs`); read-shaped mutations resolve at `ProjectAccess::Write` (`resolve_existing_args(.., need)`); unscoped reads filtered before `LIMIT` by `reader.rs` `readable_repository_sql`; `WriterHandle::authorize_project` as defense in depth. Page ids are never taken from a caller (only derived from already-authorized hits), so there is no page-id entry point to guard; incremental `recent` resolves and authorizes the requested project before decoding its scope-bound cursor; batch outcomes do not bypass the capture grant or session owner check | `tests/suite/project_authz.rs` (decision matrix, ship-inert, resolver gate); `tests/suite/access_mode.rs` `every_caller_against_both_modes`, `a_restricted_project_admits_the_team_and_refuses_the_outsider`, `new_projects_follow_the_server_default_and_admit_their_creator`; `scope.rs` `the_argument_shape_does_not_decide_the_level`, `a_user_reaches_only_what_they_were_granted`, `creating_authorizes_against_a_project_that_already_exists`, `a_refused_scope_fails_the_search_instead_of_shortening_it`; `grants.rs` `search_finds_only_what_the_viewer_may_read`, `the_limit_counts_only_what_the_viewer_may_see`, `the_workspace_handoff_comes_only_from_readable_repositories`; `ai-memory-mcp` `server.rs` `a_reader_may_read_everything_and_change_nothing`, `bob_cannot_read_alices_page_in_a_restricted_project`, `bob_cannot_find_alices_page_by_searching_in_a_restricted_project`, `bob_cannot_consolidate_a_session_in_alices_repository`, `a_restricted_projects_queues_need_write`; `ai-memory-hooks` `a_capture_needs_writer_on_the_repository_it_lands_in`, `session_start_delivers_nothing_from_a_repository_the_viewer_cannot_read`, `run_and_workstream_ids_only_answer_someone_who_may_reach_the_repository` (foreign raw workstream id refuses populated provenance; granted reader receives the source record and tool correlation); `ai-memory-web` `web_reads_honour_grants_in_a_restricted_project`, `metadata_shows_only_what_the_viewer_may_read` — each with a granted or open-project control, and proven to fail with the choke point (18 tests) or the SQL filter (9 tests) neutralized; `routes.rs` `api_recent_incremental_rechecks_restricted_scope_on_cursor` (a valid cursor cannot bypass a revoked grant; granted control passes), `api_recent_incremental_paginates_over_120_pages_and_binds_cursor`; `mcp_stateless_http.rs` `generic_machine_client_writes_queries_and_claims_a_handoff` (real machine key, foreign scope and partial scope refused); `router.rs` `an_unauthorized_batch_item_is_consumed_rather_than_retried`, `batch_acknowledges_foreign_collision_then_commits_next_item` (refused foreign item followed by a legitimate stored item) | STRONG — slice 3 closed both bypass classes (unscoped reads, raw-id entry points) and added the root-only management surface. Out of scope by design: per-project administrators (granting/restricting is root-only), and access modes, creators and grants live only in SQLite, so `reindex` resets them || 14aa | Companion capture privacy before persistence and delivery | `companions/ai-memory-client/src/lib.rs` `reject_sensitive`, `sanitize_external_value`, `check_body`; `src/scrub.rs` `sanitize_external_text`; relay `identity::validate`, `check_scope`, `relay::init` and `flush_locked`; importer `plan_external_conversation`, `planned_hook_event`, `truncate_utf8`, `ImportClient::post_hook_batch` | SDK secret-position, nested key, numeric metric (11 names plus numeric `*_tokens` counts; string values still redact), bounded schema and literal-marker tests; relay `sensitive_envelope_is_rejected_before_enqueue_with_clean_control`, `new_native_bodies_are_sanitized_before_hash_enqueue_and_exact_retry`, `unsafe_legacy_pending_is_dropped_locally_and_never_blocks_its_session`, `unsafe_legacy_native_identity_is_dropped_and_its_session_continues`, `legacy_binding_scope_spaces_are_preserved_on_delivery`; importer `sensitive_native_identity_is_refused_before_manifest_and_http_with_unicode_control`, `shared_key_sanitation_precedes_plan_manifest_transport_and_exact_retry`, `preview_and_manifest_use_scrubbed_bodies_and_digest`, `forged_body_authority_is_refused_before_transport`, `sanitized_redaction_markers_survive_utf8_byte_caps_in_bodies_and_titles`. Controls preserve opaque native Unicode IDs, metric representation and untrusted provenance. An unsafe item queued before these checks never reaches the wire (body or request line); it leaves the queue as a local `dropped_policy` receipt so the rest of its session ships. Guard-removal runtime probes target the final formatted sources and restore identical hashes. | Client-side privacy only; server scope, auth, admission and capture-policy guards remain authoritative || 14ap | Managed-run mutation owner and current project access (finish, cancel, heartbeat, native-session link, context acceptance) | `hooks/src/workstream.rs` `authorize_run_owner` derives `ManagedRunAuthority` only from authenticated middleware extensions and checks the real run before any mutation — for finish before status/payload/segment handling; cancel, heartbeat, link and context acceptance go through the same helper, so none is a side door around finish's rule; `ReaderPool::authorize_managed_run` and `store/src/workstream.rs` `authorize_run_mutation` use a read snapshot and the existing writer transaction, respectively. Strict resolution (`project_authz.rs` `resolve_project_authz_strict`, the canonical decision with a fail-closed resolution mode, so the reserved global-scope write gate applies here too) requires the actual workspace/project pair, current user existence and current Write decision for DB principals. The private trusted-proxy policy requires authenticated User capability, a canonical identity, no DB principal and server proxy configuration; it retains legacy project admission after strict scope validation, with ownership still enforced. Human-login disable does not revoke API keys. The legacy resolver and auth middleware remain unchanged. Both writer entry paths, including finished retries, require the same authority; no cross-owner bypass | MCP `auth.rs` `owner_finish_http_refuses_another_writer_before_payload_or_segment`, `owner_finish_http_trusted_proxy_preserves_restricted_lifecycle`, `owner_finish_http_active_api_key_survives_disabled_human_login`; store `multi_session.rs` `owner_finish_store_refuses_another_writer_including_finished_retry`, `owner_finish_store_rechecks_write_after_preflight`, `owner_finish_store_refuses_a_deleted_user_after_preflight`, `owner_finish_store_proxy_policy_requires_server_and_authenticated_identity`, `owner_finish_store_raw_ids_use_the_actual_project_and_workspace`, `owner_finish_store_topology_reload_and_phase_controls`, `owner_finish_store_strict_resolution_refuses_pairing_missing_and_sql_failure`, `owner_finish_store_keeps_qualified_identity_namespaces`; hooks `workstream.rs` `owner_finish_revocation_between_preflight_and_both_writer_paths`; MCP `auth.rs` `owner_finish_http_real_auth_rungs_shared_and_project_controls`, `owner_finish_http_preserves_late_active_terminal_and_missing_phases`; hooks `workstream.rs` `run_mutation_routes_refuse_another_operator` (another operator and root under another identity refused on heartbeat/link/accept/cancel, run left active; owner admitted); store `project_authz.rs` `strict_resolution_shares_the_global_gate_and_fails_closed_on_a_gap`. Guard-neutralization recipes and results accompany the local frozen patch | SQL refusal is atomic within finish's writer transaction; a revocation after preflight can leave an unindexed raw segment. Cancel, heartbeat, link and acceptance check one read snapshot immediately before a single-row update (a run's owner never changes). A later hook event carrying `AI_MEMORY_RUN_ID` links its native session only through `link_native_session_in_scope`, which applies the SessionStart boundary (`ManagedRunBoundaryRow::admits`: the run's own project and operator) and refuses a foreign run id while the event itself is still captured (`router.rs` `hook_event_links_a_managed_run_only_from_its_operator`, `hook_event_in_another_project_does_not_link_the_run`, control `hook_event_links_a_shared_run_on_a_single_operator_server`; #1082). Shared NULL ownership, trusted-proxy project admission and late still-active finishes remain intentional |