Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
fix(wrapper): run lifecycle hooks through the native host client
AkitaOnRails5efdf789e1e4
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex ce6d9420..2b32b824 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -27,6 +27,14 @@ and [0.x](docs/changelog/0.x.md).that is being deleted, so the queue-file guard failed with `inspectrelay.sqlite-journal: Access is denied`. The guard now retries that answerfor up to about 100 ms; a file that stays unreadable is still refused.+- Fixed `ai-memory hook` running inside a helper container on Docker-wrapper+ installs (Linux/macOS). A hook config or plugin that called the wrapper paid+ about 280 ms per event, over the 200 ms hook budget, and its default+ `127.0.0.1` server URL pointed at the container instead of the host server.+ The wrapper now runs `hook` and `hook-drain` through the same+ checksum-verified native host client as `mcp-bridge` and `install-hooks`.+ When that client is unavailable, the wrapper reports why on stderr, skips+ the event and exits 0, so the agent is never blocked.## [2.6.6] - 2026-10-13diff --git a/bin/ai-memory b/bin/ai-memoryindex 274ae17e..0298e425 100755--- a/bin/ai-memory+++ b/bin/ai-memory@@ -18,6 +18,12 @@# ai-memory mcp-bridge ... Run the session-aware MCP bridge through that# client: its --server-url loopback only resolves on# the host, never inside the helper container.+# ai-memory hook ... Run lifecycle hooks (and the hidden hook-drain)+# through that client: they target the host's+# loopback server, read host markers and spool, and+# must stay inside the hook time bound. If the+# client is unavailable the event is dropped and+# the hook still exits 0.## Env overrides:# AI_MEMORY_IMAGE container image (default: docker.io/akitaonrails/ai-memory:latest)@@ -598,6 +604,21 @@ case "${1:-}" inNATIVE_HOST_BIN=$(native_host_binary)exec "${NATIVE_HOST_BIN}" "$@";;+ hook | hook-drain)+ # Lifecycle hooks are client-side for the same reason as mcp-bridge: the+ # default hook URL is the host's loopback server, and the cwd, marker+ # files, capture exclusions and hook spool all live on the host. A helper+ # container would also cost ~280 ms per event, past the hook time bound.+ # Resolve the same native client mcp-bridge uses and never fall back to the+ # container. If that client is unavailable, native_host_binary has already+ # said why on stderr; drop this one event and exit 0 so the agent is never+ # blocked or shown a failing hook.+ if NATIVE_HOST_BIN=$(native_host_binary) && [ -x "${NATIVE_HOST_BIN}" ]; then+ exec "${NATIVE_HOST_BIN}" "$@"+ fi+ echo "ai-memory: native host client unavailable; lifecycle hook event not captured" >&2+ exit 0+ ;;esac# ---- normal pass-through to the binary inside a container ----------------diff --git a/docs/install.md b/docs/install.mdindex 089fbe29..aa2a8210 100644--- a/docs/install.md+++ b/docs/install.md@@ -2714,10 +2714,16 @@ ai-memory install-hooks --agent claude-code --apply```The installed Docker wrapper runs CLI commands inside a short-lived-helper container for server/store operations, but runs `install-hooks` through-the stable checksum-verified native client under+helper container for server/store operations, but runs `install-hooks`,+`mcp-bridge`, and lifecycle `hook` commands through the stable+checksum-verified native client under`~/.local/share/ai-memory/native-runner/`. This keeps hook command paths valid-on the host and enforces capture exclusions before spooling. For local loopback+on the host and enforces capture exclusions before spooling. A hook command+that calls the wrapper (`ai-memory hook --event …`) therefore runs natively in+a few milliseconds and posts to the host's `127.0.0.1:49374`; it never starts a+container. If the native client is missing and cannot be installed, the wrapper+prints the reason on stderr, drops that one event, and exits 0 so the agent is+not blocked. For local loopbackservers, the wrapper automatically bridgesthat helper back to the host's `127.0.0.1:49374`, so `ai-memory status`,`ai-memory search`, and `ai-memory bootstrap` work with the same default@@ -2760,7 +2766,7 @@ inoperable. Docker documents `label=disable` in the[`docker run` security options](https://docs.docker.com/reference/cli/docker/container/run/#security-opt).`ai-memory run`, `ai-memory show`, `ai-memory continue`, `ai-memory resume`,-`ai-memory workstreams`, and `ai-memory rename-workstream` are the exceptions:+`ai-memory workstreams`, and `ai-memory rename-workstream` are exceptions too:the current wrapper intercepts them and starts a cached checksum-verified nativeclient on the host, where local checkouts, harness executables, and sessionstores exist. It preserves an explicit remote `AI_MEMORY_SERVER_URL`. If one ofdiff --git a/docs/security-boundaries.md b/docs/security-boundaries.mdindex b1abefa7..ad2fbb81 100644--- a/docs/security-boundaries.md+++ b/docs/security-boundaries.md@@ -73,7 +73,7 @@ means the boundary is not built yet.| 11e | Native self-upgrade archive confinement (#1025) | `ai-memory-cli/src/commands/upgrade.rs` normalizes every tar/zip path, rejects links and special entries, extracts only the binary and `hooks/`, accepts only the exact non-runtime support layout emitted by `release.yml`, and caps both entry count and declared expanded bytes before writing | `upgrade.rs` `release_workflow_support_assets_are_exact_and_ignored` (shipped support paths admitted, near-miss docs/packaging/template paths refused), `extracts_archive_built_like_release_yml` and `extracts_zip_happy_path_with_hooks` (real workflow-shaped archives succeed but support assets are not written), `validate_rejects_path_traversal`, `zip_extract_rejects_path_traversal`, `tar_extract_rejects_declared_expansion_past_limit`, with binary/hooks extraction as the legitimate control | STRONG || 11f | Native hook executable-path integrity after self-replacement (#1027) | `ai-memory-cli/src/commands/render_shared.rs` `hook_embedded_exe_path` is the single executable resolver for every persisted native hook form (POSIX/Windows command, Windows exec, Zero, ZCode, OpenCode 2 assistant capture): it prefers an existing literal path, strips Linux's trailing ` (deleted)` procfs marker only when the stripped installed binary exists, and otherwise falls back to the platform command name. Upgrade keeps the already-loaded config and refreshes in-process | `render_shared.rs` `embedded_exe_strips_the_marker_when_the_stripped_path_exists`, `embedded_exe_preserves_a_real_file_whose_name_has_the_marker_suffix`, `embedded_exe_falls_back_when_the_resolved_path_is_missing`, `hook_embedded_exe_path_names_an_existing_file_without_marker`; `upgrade_e2e.rs` `native_upgrade_refreshes_hooks_without_a_deleted_path_or_losing_cli_data_dir` reproduces the replacement window through the shipped binary, plants the corrupt suffix, and proves refresh removes it while preserving a CLI-only data dir | STRONG on Linux reproduction plus platform-neutral resolver/render coverage; Windows path rendering remains covered by its existing command/exec tests and full Windows CI || 11g | Cross-project tool attribution cannot bypass source or destination capture, or destination server routing (#932, #1199) | `ai-memory-hooks/src/capture_policy.rs` `absolute_tool_location_paths` accepts only bounded, recognized, absolute locations: file-tool targets, a search/list tool's explicit `path`, or a shell command's structured `workdir` (command text is never parsed for a location); `ai-memory-cli/src/commands/hook.rs` `tool_location_routing_cwd` requires every path to prove the same repository/marker boundary, refuses the reroute when the payload cwd's own capture policy would not keep the event (source `ignore_paths` may name paths outside the source checkout, and a command may name source files while its `workdir` points elsewhere), and resolves that destination before capture policy, allowlist marker, server profile, scope hints, or spool credentials. Relative, mixed-project, unsupported, and arbitrary non-project paths keep the payload cwd | `hook.rs` `absolute_file_target_uses_destination_scope_and_capture_policy` (hostile content under the destination exclusion is never spooled; a public control is routed there), `absolute_file_target_cannot_escape_a_destination_server_profile` (unregistered destination profile drops instead of falling back to source/install default), `file_target_routing_refuses_relative_external_and_mixed_project_paths`, `workdir_reroute_applies_the_destination_capture_policy` (a `workdir`-rerouted command reading a destination-excluded path is dropped, public control kept), **`a_reroute_cannot_escape_the_source_capture_exclusions`** (adversarial: a file read under a source pattern that names another project's directory, and a shell command naming an excluded source file with `workdir` in another project, are both refused; public file and shell controls still reroute; fails with the source-policy check removed), `hermes_tool_location_finds_the_marker_the_process_cwd_lacks` (Hermes `terminal` `workdir`, `read_file` `path` and `search_files` `path` reach the repository marker under allowlist; no-workdir, relative-workdir and relative-search controls stay with the unmarked process cwd); `capture_policy.rs` `absolute_tool_location_paths_cover_search_dirs_and_shell_workdirs` (relative workdir, command-text paths, workdir without a command, and Hermes `process` yield no location) | STRONG for native `ai-memory hook`; generated and compatibility hooks retain cwd-only routing |-| 11h | Docker-wrapper capture-policy capability (#1002) | `bin/ai-memory` routes ordinary Linux/macOS `install-hooks` and upgrade refreshes through the stable checksum-verified native host client, so the hook process executes the capture-policy boundary in 11b before spool/network. Only an explicit `AI_MEMORY_HOOK_PLATFORM=posix|windows` retains the documented script compatibility path and its warning; `AI_MEMORY_NATIVE_BIN` may not point back to the wrapper | `scripts/check-native-packaging.sh` `Checking host-launch wrapper routing`: a default allowlist install reaches the fake native client while a Docker executable that always fails proves no container fallback occurred; the explicit `posix` control reaches fake Docker and adds no native invocation; the self-referential native override is refused | STRONG on the shipped Linux/macOS wrapper. Windows and remote/manual script installers remain explicitly outside capture-policy v1 |+| 11h | Docker-wrapper capture-policy capability (#1002) | `bin/ai-memory` routes ordinary Linux/macOS `install-hooks`, upgrade refreshes, and every `hook`/`hook-drain` invocation through the stable checksum-verified native host client, so the hook process executes the capture-policy boundary in 11b before spool/network; a hook config that calls the wrapper never runs in the helper container, and an unavailable client drops the event with exit 0 instead of falling back to it. Only an explicit `AI_MEMORY_HOOK_PLATFORM=posix|windows` retains the documented script compatibility path and its warning; `AI_MEMORY_NATIVE_BIN` may not point back to the wrapper | `scripts/check-native-packaging.sh` `Checking host-launch wrapper routing`: a default allowlist install reaches the fake native client while a Docker executable that always fails proves no container fallback occurred; the explicit `posix` control reaches fake Docker and adds no native invocation; the self-referential native override is refused. `tests/wrapper_upgrade.sh` `lifecycle hook native host routing`: `hook --event session-start` and `hook-drain` reach the fake native client with the stdin payload while `docker`/`podman` stubs on PATH and in `AI_MEMORY_DOCKER` stay unused, and a non-executable `AI_MEMORY_NATIVE_BIN` exits 0 without reaching the engine | STRONG on the shipped Linux/macOS wrapper. Windows and remote/manual script installers remain explicitly outside capture-policy v1 || 11i | OpenCode executable probing and generated-config cleanup (#1097) | `ai-memory-cli/src/config.rs` captures the full process environment once in typed `RuntimeEnv`; `run.rs` overlays explicit `run --env` values with platform key semantics and uses that same effective environment for executable resolution, probing, and launch without rereading process environment; `opencode_dialect.rs` invokes only direct argv `--version`, caps output and time, and rejects malformed/future majors. Automatic launch skips only OpenCode on probe failure; explicit OpenCode fails. `install_hooks.rs` quarantines the incompatible sibling before reading its exact ownership markers, restores unowned content with an atomic no-replace link, and never deletes or overwrites a concurrent replacement; `install_mcp.rs` owns the shared dual-schema endpoint/token inference used by both installers, rejects conflicting generated V1/V2 entries before mutation, and migrates an alternate-major entry only when endpoint, generated shape, and absent-or-Authorization-only headers match, preserving custom headers and ambiguous entries; auto-wire migrates MCP before hooks infer from that exact config path and skips hook mutation when transition preflight fails | `run.rs` `automatic_opencode_probe_failure_keeps_another_harness_available`, `opencode_executable_probe_selects_v2_for_launch_and_autowire`, effective-child-environment PATH casing controls, fake V1/V2/env-derived-major/malformed/future/timeout executable tests, `setup-agent` V1/V2 no-probe integration controls; `install_hooks.rs` `opencode_plugin_cleanup_removes_only_exact_owned_incompatible_sibling`, `opencode_plugin_cleanup_preserves_a_concurrent_user_replacement`, `opencode_plugin_restore_moves_quarantine_when_destination_is_free`, `opencode_plugin_restore_preserves_an_existing_replacement`, `opencode_plugin_restore_does_not_replace_a_file_created_at_restore_window`, `opencode_plugin_cleanup_does_not_escape_custom_target_directory`; `install_mcp.rs` `opencode_install_reuses_owned_endpoint_and_token_across_major_transitions`, `opencode_install_refuses_conflicting_owned_transition_entries`, `opencode_install_does_not_infer_from_a_user_owned_transition_entry`, `opencode_mcp_migration_moves_only_owned_alternate_shape`, `opencode_mcp_migration_preserves_ambiguous_user_entry`, `opencode_mcp_migration_preserves_custom_headers`, `opencode_mcp_migration_accepts_only_historical_generated_headers` | STRONG for Unix process probes and platform-neutral ownership gates; exact Windows executable resolution remains covered by `run.rs` resolver tests and requires the PR `windows` label || 12 | Network/auth posture | `config.rs` loopback `DEFAULT_BIND`; `serve.rs` `validate_http_exposure`, `require_allowed_host`; `auth.rs` `require_bearer`; `serve.rs` mounts `/identity` inside the host and machine-auth gates even with web disabled; `nix/nixos-module.nix` treats only literal `127.0.0.1`/`::1` as loopback, requires exactly one secret environment source for other binds, keeps secret auth and `llm_headers` values out of the Nix store, and brackets IPv6 in `ExecStart` | `serve.rs` host-guard (missing→400 / forged→403), non-loopback-requires-token; `auth.rs` wrong-token 401; `serve.rs` `machine_identity_remains_authenticated_with_web_disabled_and_expired_keys` (wrong and expired keys refused, valid machine key accepted, web remains disabled); `flake.nix` `nixos-module-eval` refuses `localhost` and `0.0.0.0` without a secret source, refuses two secret sources, `settings.auth` secrets, and `settings.llm_headers`, with literal IPv4/IPv6 loopback and one-source controls | STRONG || 12b | Managed-workstream provenance privacy, stable replay identity and server event namespace | `ai-memory-core/src/workstream.rs` `scrub_workstream_provenance`; `ai-memory-hooks/src/workstream.rs` `finish_run` reserved client-id check and `sanitize_events` before immutable segments, with `sanitize_events` and `append_boundary_events` scrubbing event/checkpoint/losses content *before* the 64 KiB cap so a cap-straddling secret is never cut into an unmatchable prefix (#1113); `ai-memory-store/src/workstream.rs` `finish_run` stable identity check and `stored_event`, scrubbing content with the explicitly supplied configured sanitizer before its 16 KiB store-boundary bound; no changes to the typed observation boundary | `store/tests/suite/workstream_provenance.rs` `workstream_provenance_is_bounded_scrubbed_and_idempotent_before_storage` (hostile and oversized source ids, unknown metadata, legitimate correlation, retry and distinct event ids), `workstream_provenance_legacy_reads_strip_secrets_and_unknown_metadata` (search and tail, malformed/oversized legacy JSON), `workstream_provenance_reused_event_id_retains_data_and_refuses_identity_change` (first indexed version retained, identity refusal and batch rollback), `workstream_provenance_replay_survives_sanitizer_changes` (new extra patterns and removal of a built-in-pattern allowlist exemption, with identity-refusal control), `workstream_provenance_replay_preserves_legacy_rows_above_bounds`, `workstream_provenance_finished_run_ignores_late_events`, `workstream_provenance_retry_with_marker_matching_pattern_is_idempotent`, `workstream_content_is_scrubbed_before_the_store_cap` (a credential straddling the 16 KiB store cap is redacted, not persisted as an unmatchable prefix; benign control verbatim; #1113); `hooks` `workstream_provenance_finish_scrubs_before_raw_segment_and_search` (custom sanitizer, immutable segment, SQL recovery and finished-run no-op), `workstream_provenance_client_cannot_use_server_event_ids` (reserved ids refused before raw writes, normal client event and server-generated checkpoint/loss controls), `finish_run_scrubs_event_content_before_the_byte_cap` and `finish_run_scrubs_boundary_events_before_the_byte_cap` (a credential straddling the 64 KiB event cap is redacted in the raw segment and never reaches the index; checkpoint/losses and benign controls; #1113); `cli/tests/suite/workstream_provenance.rs` (text/JSON and old server DTOs) | STRONG: privacy, stable-identity and reserved-namespace regressions fail with their guards bypassed; an isolated allowlist bypass fails on the unknown key, and bypassing the raw-id search authorization returns 200 where its test requires 403. No provenance is emitted in startup packets, and old raw segments are not rewritten |diff --git a/tests/wrapper_upgrade.sh b/tests/wrapper_upgrade.shindex a460ee1d..f1e0e18e 100755--- a/tests/wrapper_upgrade.sh+++ b/tests/wrapper_upgrade.sh@@ -278,4 +278,76 @@ if [ -s "${BRIDGE_CASE_DIR}/docker.log" ]; thenfiprintf 'wrapper mcp-bridge native routing checks passed\n'+# ---- lifecycle hook native host routing ----------------------------------++# `ai-memory hook` is what every generated hook config and plugin calls. Inside+# the helper container it costs ~280 ms per event and its default loopback URL+# is the container, not the host server. The wrapper must exec the native+# client with the payload on stdin and never touch the container engine: not+# through AI_MEMORY_DOCKER, and not through a `docker` found on PATH.+HOOK_CASE_DIR="${TMP_ROOT}/hook"+mkdir -p "${HOOK_CASE_DIR}/home" "${HOOK_CASE_DIR}/cache" "${HOOK_CASE_DIR}/path"++HOOK_NATIVE="${HOOK_CASE_DIR}/native-ai-memory"+cat >"${HOOK_NATIVE}" <<'NATIVE'+#!/usr/bin/env bash+printf 'argv: %s\n' "$*" >>"${AI_MEMORY_WRAPPER_TEST_LOG}"+printf 'stdin: %s\n' "$(cat)" >>"${AI_MEMORY_WRAPPER_TEST_LOG}"+NATIVE+chmod 0755 "${HOOK_NATIVE}"++for engine in docker podman; do+ cat >"${HOOK_CASE_DIR}/path/${engine}" <<'DOCKER'+#!/usr/bin/env bash+printf '%s %s\n' "$(basename "$0")" "$*" >>"${AI_MEMORY_DOCKER_TEST_LOG}"+exit 0+DOCKER+ chmod 0755 "${HOOK_CASE_DIR}/path/${engine}"+done++run_hook_case() {+ local native_bin="$1"+ shift+ HOME="${HOOK_CASE_DIR}/home" \+ XDG_CACHE_HOME="${HOOK_CASE_DIR}/cache" \+ PATH="${HOOK_CASE_DIR}/path:${PATH}" \+ AI_MEMORY_DOCKER="${HOOK_CASE_DIR}/path/docker" \+ AI_MEMORY_NATIVE_BIN="${native_bin}" \+ AI_MEMORY_WRAPPER_TEST_LOG="${HOOK_CASE_DIR}/native.log" \+ AI_MEMORY_DOCKER_TEST_LOG="${HOOK_CASE_DIR}/docker.log" \+ "${ROOT}/bin/ai-memory" "$@"+}++assert_no_container() {+ if [ -s "${HOOK_CASE_DIR}/docker.log" ]; then+ fail "$1 unexpectedly reached the container engine: $(cat "${HOOK_CASE_DIR}/docker.log")"+ fi+}++: >"${HOOK_CASE_DIR}/native.log"+: >"${HOOK_CASE_DIR}/docker.log"+printf '{"session_id":"s-1"}' \+ | run_hook_case "${HOOK_NATIVE}" hook --agent claude-code --event session-start+assert_contains "${HOOK_CASE_DIR}/native.log" "argv: hook --agent claude-code --event session-start"+assert_contains "${HOOK_CASE_DIR}/native.log" 'stdin: {"session_id":"s-1"}'+assert_no_container "hook"++: >"${HOOK_CASE_DIR}/native.log"+run_hook_case "${HOOK_NATIVE}" hook-drain </dev/null+assert_contains "${HOOK_CASE_DIR}/native.log" "argv: hook-drain"+assert_no_container "hook-drain"++# Without a usable native client the hook drops the event, says why on stderr,+# and still exits 0 at once; it never falls back to the container.+: >"${HOOK_CASE_DIR}/native.log"+hook_status=0+printf '{}' | run_hook_case "${HOOK_CASE_DIR}/missing-ai-memory" \+ hook --agent claude-code --event session-start 2>"${HOOK_CASE_DIR}/missing.err" \+ || hook_status=$?+[ "${hook_status}" -eq 0 ] || fail "hook without a native client exited ${hook_status}, want 0"+assert_contains "${HOOK_CASE_DIR}/missing.err" "AI_MEMORY_NATIVE_BIN is not executable"+assert_contains "${HOOK_CASE_DIR}/missing.err" "lifecycle hook event not captured"+assert_no_container "hook without a native client"+printf 'wrapper hook native routing checks passed\n'+printf 'wrapper upgrade ownership checks passed\n'