Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
Merge latest PR #989 review corrections
AkitaOnRailsbec5e952abc5
diff --cc .github/workflows/nix.ymlindex dc7ba0ea,1e3cd337..986b3602--- a/.github/workflows/nix.yml+++ b/.github/workflows/nix.yml@@@ -16,9 -19,10 +19,10 @@@ name: ni# demand.on:push:- branches: [main]+ branches: [main, release/2.6]paths:- 'flake.nix'+ - 'flake.lock'- 'nix/**'- 'Cargo.lock'- 'Cargo.toml'@@@ -77,24 -104,12 +83,29 @@@ jobs# - Soft fake-systemd without a real unit start# - Claiming the app Docker image covers the NixOS module# - Darwin / multi-arch NixOS OCI (Linux x86_64 only by design)+ # - Re-asserting sandbox keys (covered by nixos-sandbox-parity)- if: >-- matrix.system == 'x86_64-linux' &&- (github.event_name == 'schedule' ||- github.event_name == 'workflow_dispatch' ||- (github.event_name == 'pull_request' &&- (contains(github.event.pull_request.labels.*.name, 'nix') ||- contains(github.event.pull_request.labels.*.name, 'full-ci'))))+ github.event_name == 'schedule' ||+ github.event_name == 'workflow_dispatch' ||- (github.event_name == 'pull_request' && contains(github.event.pull_request.labels.*.name, 'full-ci'))++ (github.event_name == 'pull_request' &&++ (contains(github.event.pull_request.labels.*.name, 'nix') ||++ contains(github.event.pull_request.labels.*.name, 'full-ci')))run: scripts/test-nixos-systemd-container.sh++ build-darwin:+ name: nix build (aarch64-darwin)+ if: >-+ github.event_name == 'schedule' ||+ github.event_name == 'workflow_dispatch' ||- (github.event_name == 'pull_request' && contains(github.event.pull_request.labels.*.name, 'full-ci'))++ (github.event_name == 'pull_request' &&++ (contains(github.event.pull_request.labels.*.name, 'nix') ||++ contains(github.event.pull_request.labels.*.name, 'full-ci')))+ runs-on: macos-latest+ steps:+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1+ - uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31.11.1+ with:+ extra_nix_config: |+ experimental-features = nix-command flakes+ - run: nix build --print-build-logs '.#packages.aarch64-darwin.default'+ - run: scripts/check-nix-packaging.sh ./resultdiff --cc CHANGELOG.mdindex 3bdee810,6c4d7afb..b0f0e530--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -8,123 -8,36 +8,130 @@@ and this project adheres to [Semantic V## [Unreleased]### Added-- A NixOS module (`nix/nixos-module.nix`, exposed as `nixosModules.default`)- so NixOS hosts can run `services.ai-memory.enable = true` as a hardened- systemd service — dedicated `ai-memory` system user (`nologin`, gated by- `createUser`) plus the NixOS systemd sandbox in `nix/systemd-sandbox.nix`.- Typed `services.ai-memory.settings` renders declarative `config.toml`- (minimal typed keys + `freeformType`; values land in a world-readable store- path — keep secrets out, including `llm_headers`); secrets use- `ageSecret`, `sopsSecret`, or `environmentFile` (required for non-loopback- binds). The web UI defaults off (`enableWeb = false`, matching the CLI- flag's own default rather than the packaged unit's hardcoded- `--enable-web`). `flake.nix` adds eval checks for sandbox keys, settings →- `--config`, agenix/sops wiring, refusal contracts (age+sops mutex, secrets- in `settings.auth`, forbidden `settings.bind`), escaped `ExecStart`- (`--data-dir` / `serve` / `--transport http`), default `StateDirectory` vs- custom `dataDir` tmpfiles/`ReadWritePaths`, and cheap- network-online / `openFirewall` / `MemoryMax` / `TasksMax` round-trips.- `packages.x86_64-linux.nixos-ai-memory-docker` builds one NixOS- `system.build.toplevel` (via the docker-image rootfs tarball) and- `scripts/test-nixos-systemd-container.sh` runs a privileged systemd- container smoke (`systemctl` + in-container `/healthz` + dataDir- restart/volume persistence). The existing `flake.nix` package/devShell/CLI- build is unchanged. (#989)+- Added a default-quit prompt after a successful interactive managed run so the+ operator can re-run the current harness or switch to another installed+ harness in the same workstream. Utility, failed, interrupted,+ non-interactive, and custom-`--executable` runs keep their existing exit+ behavior. Resolves #909. (#975)+- Added `ai-memory backup-agents` and dry-run-by-default+ `ai-memory restore-agents` commands for host agent configurations, skills,+ plugins, and project instructions. Archives are written atomically with+ private Unix permissions, MCP config text is sanitized by default, and+ restore bounds archive input, verifies manifest checksums and completeness,+ refuses duplicate or symlink-escaping targets, and keeps overwritten files+ as timestamped backups. Other asset types are copied verbatim and may contain+ secrets or active instructions/code. (#962)- - Added a NixOS module exposed as `nixosModules.default`, with a dedicated- non-login service user, a hardened systemd unit, declarative non-secret- settings, agenix/sops/environment-file secret sources, and opt-in web,- firewall, and resource settings. Non-loopback binds require exactly one- secret source; expensive Darwin and privileged container checks run only in- the full CI tier. (#989)++- Added a NixOS module (`nix/nixos-module.nix`, exposed as++ `nixosModules.default`) with a dedicated non-login system user and a hardened++ NixOS systemd sandbox. A minimal typed set plus `freeformType` renders++ declarative `config.toml`; values land in a world-readable store path, so++ secrets (including `llm_headers`) use `ageSecret`, `sopsSecret`, or++ `environmentFile`. Non-loopback binds require a secret source. The module++ includes opt-in web, firewall, and resource settings plus eval and privileged++ container checks. (#989)+- Added `ai-memory doctor` reporting for Claude Code's default native+ `memory/` store for the current repository: location, file count, and whether+ the nearest marker's `ignore_paths` would exclude a read. Repository-root+ resolution keeps worktrees and subdirectories on the same report, and the+ output warns that shell/PowerShell compatibility hooks do not enforce the+ exclusion. Resolves #1003. (#1005)+- Added per-event outcomes to `/hook/batch` acknowledgements and process-lifetime+ ingest counters for stored events, replays, recovery, ignored endings,+ collisions and failures. Legacy acknowledgement fields were preserved. (#1015)+- Added receipt outcomes and delivery counts to the companion relay. Opening a+ schema 1 queue now upgrades it atomically to schema 2; older relay binaries+ refuse schema 2, so downgrade requires a pre-upgrade backup with all queue+ users stopped. Pending order and existing delivery metadata were preserved.+ Receipts now retain their first known outcome. (#1015)+- Added `ai-memory list-projects [--workspace] [--json]`: a plain, scriptable+ listing of every workspace/project pair the server knows about, sorted by+ workspace then project. Previously the only way to see this from outside+ an interactive `show` session was to call `GET /api/v1/projects` directly.+ (#1022)+- Added scope and policy preflight to `ai-memory hook --check-capture`, including+ event eligibility, sanitized routing hints and server-remapping information.+ Partial scope, excluded events, rejected profiles and oversized hints refuse+ capture. Inspection neither ingests events nor claims handoffs. (#1019)+- Added the latest scoped consolidation job's state and attempt count to HTTP+ session summaries and MCP `memory_read_session_observations`. The field is+ `null` when no job exists; provider errors remain private and existing session+ ownership filters are preserved. (#1018)+- Added incremental paging to scoped HTTP `recent` reads with `updated_since`+ and an opaque, scope-bound cursor. Queries preserve equal-time ordering,+ exclude expired and superseded pages and recheck authorization on each+ request. Calls without incremental arguments retain their existing array+ response. Added a generic MCP/HTTP integration guide. (#1017)+- Added machine-authenticated `GET /identity`, available with the web UI+ disabled, and `ai-memory doctor` diagnostics for caller identity, external+ capture ownership and sessions with multiple capture sources (native events+ or distinct extensions, including backfill).+ Mixed-source counts preserve scope, session ownership and time-window filters;+ older servers leave unsupported fields unknown. (#1016)### Changed+- Documented FutureInfra as an endpoint for the existing `openai-compat`+ provider. (#1026)+ - `.github/workflows/nix.yml` builds the flake on `x86_64-linux` for path-+ filtered PRs and pushes (package smoke + NixOS module eval /+ sandbox-parity). The `aarch64-darwin` package smoke and the privileged+ NixOS container smoke run only on schedule, `workflow_dispatch`, or a PR+ labelled `nix` / `full-ci`. (#989)+### Fixed+- Fixed completed retries and no-op session endings advancing+ `last_persisted_ms` without a durable write. Recovery still advances the+ timestamp when it commits a new page or terminal effect. (#1015)+- Fixed `ai-memory upgrade` rewriting native hook commands with Linux's+ ` (deleted)` executable-path suffix after replacing its own binary. Every+ hook renderer now keeps a still-existing literal path, otherwise strips the+ kernel suffix only when the resulting installed binary exists, and finally+ falls back to the bare `ai-memory` command instead of embedding a dead path.+ The in-process refresh continues using the already-loaded CLI/config scope.+ (#1027)+- Fixed native `ai-memory upgrade` rejecting every current Linux, macOS, and+ Windows release archive when it encountered the packaging, config-template,+ or documentation entries shipped beside the binary. The extractor now+ validates the exact support-file layout emitted by `release.yml`, ignores+ those non-runtime files during self-upgrade, and still extracts only the+ binary and hooks. Archive entry-count and expanded-size caps now complement+ the existing compressed-download limit. (#1025)+- Fixed the recommended Linux/macOS Docker-wrapper `install-hooks` path+ silently installing shell hooks that cannot enforce client-side capture+ controls such as `[capture] ignore_paths` and allowlist mode. The wrapper now+ uses its existing checksum-verified native host client for hook installation+ and refresh, while an explicit `AI_MEMORY_HOOK_PLATFORM=posix|windows` keeps+ the documented compatibility fallback and warning. Wrapper upgrades refresh+ that stable native client before rewriting existing hook registrations.+ (#1002)+- Fixed managed Codex sessions losing their startup continuity when a shared+ Codex app-server daemon reports a finished run's stale `AI_MEMORY_RUN_ID`.+ SessionStart now recovers only the sole live, undelivered Codex run in the+ already-authorized repository, checkout cwd, and operator bucket, and links+ it atomically to the new native session. Active boundary mismatches,+ concurrent matching runs, cross-project or cross-worktree candidates,+ cross-operator candidates, and a second linker all fail closed instead of+ guessing or rebinding a run. (#987)+- `ai-memory doctor` now warns when the nearest `.ai-memory.toml`'s+ `[capture]` section is invalid. This fails closed today — every file and+ shell tool event is reduced to metadata until the marker is fixed, nothing+ leaks — but nothing surfaced that it had happened, so an operator could+ believe `ignore_paths` was excluding paths it no longer was. The check is+ based on the same resolution state the live hook path uses (not just+ whether the TOML parses), so it also catches a `[capture]` table that+ parses fine but is still rejected at compile time, e.g. an unsupported glob+ character in `ignore_paths`. (#1021)+- Fixed the shell and PowerShell hook scripts mishandling a `$HOME` that ends+ in a separator: the shell walks missed a marker between the checkout and+ home, and the PowerShell walks went past home and read a marker above it.+ Both now match the native `ai-memory hook` walk. (#1023)+- Fixed `ai-memory run --jail` leaving the `ssh` toggle unchecked when+ `git push` goes over SSH but `origin`'s fetch URL does not show it: an HTTPS+ `origin` with an SSH `pushurl` or `pushInsteadOf`, or a `host:path` remote+ using an `~/.ssh/config` alias. The interactive offer now also uses+ ai-jail's own dry-run preflight, so a backend that exists but fails ai-jail's+ trust checks is treated as unavailable instead of producing a broken offer.+ (#1024)+## [2.5.2] - 2026-10-01### Addeddiff --cc README.mdindex a2a081cc,0c78ebc2..b13fb13c--- a/README.md+++ b/README.md@@@ -416,19 -408,19 +417,20 @@@ under `packaging/systemd/` keep their e}```- Declarative non-secret config lives in `services.ai-memory.settings`. Common- keys are typed for discoverability and the TOML freeform type keeps every- other current and future `config.toml` key usable without duplicating the Rust- schema. The module renders a generated TOML file and passes `--config`;- network and web-listener controls stay in the module's top-level options.+ Declarative non-secret config lives in `services.ai-memory.settings` (a+ small typed set for common keys, plus `freeformType` for the rest of+ `config.toml`). The module renders a generated TOML file and passes+ `--config`. Top-level `bind`, `port`, and `enableWeb` win over duplicate+ settings keys. Anything in `settings` (including `llm_headers`) lands in a+ world-readable Nix store path — do not put API keys there.-Secrets such as `AI_MEMORY_AUTH_TOKEN` never go in `settings` or the Nix-store. Use `ageSecret` / `sopsSecret` (consume `config.age` / `config.sops`-when the host imports agenix or sops-nix) or `environmentFile` as an escape-hatch. Non-loopback binds require one of those sources; loopback may omit-them and tolerates a missing env file (systemd `EnvironmentFile=-…`). Put TLS-in front of a LAN/WAN bind — see [`docs/https-via-proxy.md`](docs/https-via-proxy.md).+Secrets such as `AI_MEMORY_AUTH_TOKEN` never go in `settings` or the+world-readable Nix store. This includes `llm_headers`, which can carry API+credentials; set `AI_MEMORY_LLM_HEADERS` in `ageSecret`, `sopsSecret`, or+`environmentFile` instead. These three secret sources are mutually exclusive.+Non-loopback binds require one; loopback may omit them and tolerates a missing+environment file (systemd `EnvironmentFile=-…`). Put TLS in front of a LAN/WAN+bind — see [`docs/https-via-proxy.md`](docs/https-via-proxy.md).All options and defaults are in [`nix/nixos-module.nix`](nix/nixos-module.nix).Entirely opt-in — `nix build`, `nix run`, `nix develop`, and the CLI arediff --cc nix/settings-options.nixindex 6f94d5d5,63284e84..6ce5ef6e--- a/nix/settings-options.nix+++ b/nix/settings-options.nix@@@ -1,8 -1,10 +1,8 @@@- # A small discoverability layer for common non-secret config.toml keys.+ # Typed `services.ai-memory.settings` options for common non-secret keys.#- # The parent submodule has `freeformType = pkgs.formats.toml.type`, so every- # current and future config key remains usable without copying Config's full- # Rust schema into Nix. Secrets belong in an environment file, never here.-# The settings submodule also sets `freeformType = pkgs.formats.toml.type`, so-# any other `config.toml` key can be set without listing it here. Keep this-# file small: only type keys we want docs/validation for. Nested schema-# sections (decay, auto_improve, dream, …) stay freeform to avoid drifting-# against the Rust config.++# The parent submodule has `freeformType = pkgs.formats.toml.type`, so any++# other `config.toml` key remains usable without copying the Rust schema into++# Nix. Keep nested and fast-moving sections freeform to avoid schema drift.{ lib, ... }:let@@@ -20,11 -22,11 +20,16 @@@ idefault = null;};+ server_url = lib.mkOption {+ type = types.nullOr types.str;+ default = null;+ };++ capture_assistant = lib.mkOption {+ type = types.nullOr types.bool;+ default = null;+ };+llm_provider = lib.mkOption {type = types.nullOr types.str;default = null;@@@ -35,27 -37,57 +40,56 @@@default = null;};+ llm_base_url = lib.mkOption {+ type = types.nullOr types.str;+ default = null;+ };++ llm_headers = lib.mkOption {+ type = types.nullOr (types.listOf types.str);+ default = null;+ description = ''+ Extra HTTP headers for the LLM provider (`Header: value` strings).++ Warning: values set here land in the generated `config.toml` Nix store+ path, which is world-readable. Do not put API keys or other secrets in- `llm_headers`; pass those via `ageSecret` / `sopsSecret` /++ `llm_headers`; pass those via `ageSecret`, `sopsSecret`, or+ `environmentFile` (for example `AI_MEMORY_LLM_HEADERS`) instead.+ '';+ };++ embedding_provider = lib.mkOption {+ type = types.nullOr types.str;+ default = null;+ };++ embedding_model = lib.mkOption {+ type = types.nullOr types.str;+ default = null;+ };+auth = lib.mkOption {type = types.nullOr (types.submodule {- # Freeform so a mistaken secret key reaches the module assertion- # ("settings.auth must not contain secrets") instead of a generic- # "option does not exist" type error.- freeformType = types.attrsOf types.str;+ # Keep unknown keys visible to the module's explicit secret-key+ # assertion, which gives a useful refusal instead of a type error.+ freeformType = types.attrsOf types.anything;options = {- secure_cookie = lib.mkOption {- type = types.nullOr types.bool;- default = null;- };- root_username = lib.mkOption {- type = types.nullOr types.str;+ secure_cookie = lib.mkOption { type = types.nullOr types.bool; default = null; };+ root_username = lib.mkOption { type = types.nullOr types.str; default = null; };+ root_issuer = lib.mkOption { type = types.nullOr types.str; default = null; };+ root_subject = lib.mkOption { type = types.nullOr types.str; default = null; };+ root_email = lib.mkOption { type = types.nullOr types.str; default = null; };+ trusted_proxy_cidrs = lib.mkOption {+ type = types.nullOr (types.listOf types.str);default = null;};};});default = null;description = ''- Non-secret auth settings only. Use ageSecret, sopsSecret, or- environmentFile for bearer tokens, password material, and other- credentials.+ Non-secret auth settings only. Never put bearer_token, token_pepper,- initial_root_password, recovery_token, or actor_proxy_bearer_token here —- use ageSecret / sopsSecret / environmentFile instead.++ initial_root_password, recovery_token, or actor_proxy_bearer_token here;++ use ageSecret, sopsSecret, or environmentFile instead.'';};}diff --cc nix/systemd-sandbox.nixindex 61b5b303,56453589..cd88e29a--- a/nix/systemd-sandbox.nix+++ b/nix/systemd-sandbox.nix@@@ -1,4 -1,9 +1,5 @@@- # Systemd hardening for the NixOS ai-memory service.-# Shared systemd hardening for the NixOS ai-memory module.-#-# Applied by `nix/nixos-module.nix`. Packaged FHS units under-# `packaging/systemd/` keep their existing lighter hardening for now;-# aligning those units with this sandbox is a follow-up (needs a real unit-# start test — `systemd-analyze verify` alone is not enough).++# Systemd hardening for the NixOS ai-memory module. Packaged FHS units under++# `packaging/systemd/` keep their existing lighter hardening.{ lib, ... }:{@@@ -31,4 -36,34 +32,5 @@@ProtectHome = true;ProtectSystem = "strict";};+- # User-unit sandbox (reserved for a future FHS user-unit hardening PR).- # Same confinement keys less ProtectHome/ProtectSystem, which conflict- # with %h data paths.- aiMemoryUserSandbox = {- CapabilityBoundingSet = [ ];- AmbientCapabilities = [ ];- MemoryDenyWriteExecute = true;- RestrictAddressFamilies = [- "AF_UNIX"- "AF_INET"- "AF_INET6"- ];- RestrictNamespaces = true;- RestrictRealtime = true;- RestrictSUIDSGID = true;- LockPersonality = true;- PrivateDevices = true;- RemoveIPC = true;- ProtectKernelTunables = true;- ProtectKernelModules = true;- ProtectKernelLogs = true;- ProtectControlGroups = true;- ProtectClock = true;- ProtectHostname = true;- SystemCallArchitectures = "native";- UMask = "0077";- NoNewPrivileges = true;- PrivateTmp = true;- };}