Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
docs: add EU/enterprise-readiness reference docs (data handling, SSO, air-gapped install)
Murilo Pereira9605bcb5b70d
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex a52cf4b2..6c4b29e3 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+### Added+- `DATA_HANDLING.md`, `docs/sso.md`, and `docs/airgapped-install.md`,+ consolidating existing data-flow, OIDC, and offline-install facts from+ `SECURITY.md`/`docs/install.md`/`docs/local-embeddings.md` into the shape+ enterprise security/legal review typically asks for before approving a dev+ tool. No behavior change; a small addendum to `SECURITY.md`'s reporting+ section adds a fallback contact path for reporters who can't use GitHub's+ private-advisory flow.+### Fixed- `install-hooks --apply` no longer aborts the entire install when the hookbearer token cannot be persisted under the data dir. This bit the dockerdiff --git a/DATA_HANDLING.md b/DATA_HANDLING.mdnew file mode 100644index 00000000..524a093d--- /dev/null+++ b/DATA_HANDLING.md@@ -0,0 +1,98 @@+# Data Handling++This page exists for reviewers who need a single place to answer "what data+does ai-memory collect, where does it live, and what (if anything) leaves the+machine" — typically a security, legal, or data-protection review ahead of+internal approval. It restates facts that already live in+[`SECURITY.md`](SECURITY.md), cited by section below, rather than making new+claims; if the two ever disagree, `SECURITY.md` is authoritative.++## Default: fully local, no telemetry++ai-memory does not phone home. There is no analytics, crash reporting, or+usage telemetry built into the binary. Everything it stores — wiki pages and+the SQLite database — lives under a single, operator-controlled data+directory on the local filesystem (`SECURITY.md`, "Local data+confidentiality"). No data leaves the host unless one of the two opt-in+features below is explicitly enabled, or you point ai-memory's own LLM/+embedding-provider config at a cloud API for its core functionality (the same+way any AI coding tool needs a model to call).++There is no encryption at rest in v1; ai-memory relies on OS filesystem+permissions (owner-only directories/files on Unix, ACLs on Windows —+`SECURITY.md`, same section).++## The two opt-in paths that send data externally++Both are off by default and each requires a deliberate step to turn on.++1. **Assistant/Stop capture** (`SECURITY.md`, "Assistant/Stop capture is+ opt-in and sanitized"). Persisting the coding assistant's final-turn text+ requires a double opt-in: `capture_assistant` on the server *and*+ `install-hooks --capture-assistant` on the client. Once enabled, captured+ text flows into consolidation/reviewer prompts and — only if you have+ separately configured a cloud LLM provider — is sent to that provider. The+ flag is global to the install; there is no per-project exclusion once it's+ on.+2. **LLM-based search reranking** (`SECURITY.md`, "Search reranking is an+ outbound-data opt-in"). Setting `AI_MEMORY_RERANKER=llm` sends each live+ query plus bounded page titles/snippets to the configured LLM provider.+ Leave this unset, or point it at a local provider, to keep queries and+ recalled snippets on-host.++Both paths pass through a sanitizer that strips obvious credentials before+anything is transmitted, but the sanitizer is documented as best-effort, not+a guarantee — see the "Stored-content prompt injection" and reranking notes+in `SECURITY.md` for the exact caveats.++## Is this "personal data" under GDPR?++ai-memory stores developer-authored artifacts — wiki pages, session+handoffs, entity extractions — derived from coding-agent sessions. It has no+purpose-built handling for personal data (no user-profile fields, no+consent-tracking, no data-subject-request tooling) because it isn't designed+to process personal data as such. In practice this means:++- If your team's session content incidentally contains personal data (e.g. a+ commit message with a name, or free text a developer typed), that data is+ stored and retained exactly like any other memory content, under your own+ organization's control on infrastructure you run.+- ai-memory does not transmit that content anywhere on its own; the two+ opt-in paths above are the only export mechanisms, and both are under your+ control.+- Treat this the same way you'd treat any local database your team+ operates — your organization is the data controller and decides retention,+ access, and disclosure. This document does not constitute legal advice;+ confirm classification with your own data-protection function.++## Processor relationship++Self-hosted with no opt-in features enabled: there is no third party in the+data path, so ai-memory does not create a data-processor relationship under+GDPR Art. 28. If you configure a cloud LLM or embedding provider (for core+functionality, capture, or reranking), **that provider** becomes your+processor for the data you send it, under whatever agreement you have with+them directly — ai-memory is a conduit, not a party to that relationship.++## Deletion / retention++There is no built-in retention-expiry policy; data persists until removed.+Deletion is filesystem-level: removing the data directory removes everything+in it. For scoped deletion, a per-project purge operation exists and is+isolation-safe — it cannot delete files or rows belonging to a different+`(workspace_id, project_id)` (`SECURITY.md`, "Per-project isolation").++## Network exposure (if you run the server non-loopback)++Binding the server beyond `127.0.0.1` is a separate, opt-in operational+choice unrelated to the data-export paths above — it controls who can reach+the API, not what ai-memory sends externally. See `SECURITY.md`'s "Network+exposure when binding to non-loopback addresses" section and+[`docs/https-via-proxy.md`](docs/https-via-proxy.md) before doing this.++## Related documents++- [`SECURITY.md`](SECURITY.md) — threat model, scope, vulnerability reporting.+- [`docs/sso.md`](docs/sso.md) — enterprise identity/SSO integration.+- [`docs/airgapped-install.md`](docs/airgapped-install.md) — offline/air-gapped+ installation.diff --git a/README.md b/README.mdindex 6b1d5fdb..0af06819 100644--- a/README.md+++ b/README.md@@ -286,7 +286,10 @@ paths or invert to allowlist mode.The full model is in [`docs/security.md`](docs/security.md),[`docs/users.md`](docs/users.md), and-[`docs/https-via-proxy.md`](docs/https-via-proxy.md).+[`docs/https-via-proxy.md`](docs/https-via-proxy.md). For data-flow,+identity/SSO, and offline-install questions specifically, see+[`DATA_HANDLING.md`](DATA_HANDLING.md), [`docs/sso.md`](docs/sso.md), and+[`docs/airgapped-install.md`](docs/airgapped-install.md).## LLM providers@@ -340,6 +343,9 @@ diagram, crate breakdown, schema notes, and invariants.| [`docs/auto-improvement-loop.md`](docs/auto-improvement-loop.md) | Auto-improvement design notes: Hermes-inspired scheduled review, auto-approval default, manual review opt-in, pending proposal storage, and curator work. || [`docs/companion-crates.md`](docs/companion-crates.md) | Boundary and implementation plan for optional companion projects, including the standalone importer at [`companions/ai-memory-importer`](companions/ai-memory-importer), without widening core ai-memory. || [`docs/llm-provider-comparison.md`](docs/llm-provider-comparison.md) | Empirical notes behind the recommended LLM defaults. |+| [`DATA_HANDLING.md`](DATA_HANDLING.md) | **Data-flow reference for security/legal review.** What's stored, what's local-only, the two opt-in paths that send data externally, and how deletion/retention work. |+| [`docs/sso.md`](docs/sso.md) | Enterprise identity: the existing OIDC device-auth flow, what it does and doesn't cover, and how to front the server with an OIDC-aware gateway. |+| [`docs/airgapped-install.md`](docs/airgapped-install.md) | Offline/air-gapped install: self-contained build, checksum-verified release binaries, and the offline path for local embedding models. || [`docs/llm-provider-fallback.md`](docs/llm-provider-fallback.md) | Proposed opt-in fallback-chain design for transient LLM-provider failures; not yet a supported configuration surface. || [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Operational summary: data flow, crate layout, cross-cutting invariants, schema. || [`docs/design-decisions.md`](docs/design-decisions.md) | The full v1 spec. |diff --git a/SECURITY.md b/SECURITY.mdindex dad4ac06..40c95764 100644--- a/SECURITY.md+++ b/SECURITY.md@@ -9,6 +9,11 @@ on GitHub. You will receive a response within 7 days. If the issue is confirmedwe will aim to release a patch within 30 days and credit you in the changelog(unless you prefer to remain anonymous).+If you cannot use GitHub's private advisory flow (e.g. your organization's+vendor-security process requires a named point of contact independent of a+GitHub account), open an issue asking for an alternate contact channel and a+maintainer will follow up; do not put vulnerability details in that issue.+## Threat modelai-memory is a **single-tenant workstation/homelab service**. It supportsdiff --git a/docs/airgapped-install.md b/docs/airgapped-install.mdnew file mode 100644index 00000000..3bd58a92--- /dev/null+++ b/docs/airgapped-install.md@@ -0,0 +1,73 @@+# Air-gapped / offline install++Corporate IT-security review commonly asks for a documented offline+installation path before approving a dev tool. ai-memory's build and+runtime are already close to air-gap-friendly by design; this page collects+what already exists elsewhere in the docs into one answer, rather than+introducing a new mode.++## Build: no network required++The build is self-contained (`CONTRIBUTING.md`, "Dev setup"): SQLite is+bundled via `rusqlite`'s `bundled` feature and `libgit2` is vendored via+`git2`'s `vendored-libgit2` feature. No system libraries beyond a standard C+toolchain are required, and nothing is fetched from the network during+`cargo build`. Building from a vendored/mirrored crates.io cache (standard+practice for air-gapped Rust builds, e.g. `cargo vendor`) works the same way.++If you'd rather not build at all, tagged GitHub Releases publish prebuilt+binaries for Linux (`linux/amd64`, `linux/arm64`), macOS+(`ai-memory-macos-aarch64.tar.gz`, `ai-memory-macos-x86_64.tar.gz`), and+Windows (`ai-memory-windows-x86_64.zip`), each with a SHA-256 checksum+(`SECURITY.md`, "Published executable integrity") — download once on a+network-connected machine, verify the checksum, and transfer the artifact+into the air-gapped environment.++## Runtime: no network calls unless you configure one++Covered in full in [`DATA_HANDLING.md`](../DATA_HANDLING.md), summarized+here: ai-memory has no telemetry, analytics, or phone-home behavior. The+server, CLI, and lifecycle hooks run entirely against the local data+directory. The only things that make a network call are things you+explicitly configure:++- **A cloud LLM/embedding provider**, if you want AI-assisted consolidation,+ search, or provider-based embeddings. Skip this and use the `local`+ embedding provider (below) to stay fully offline.+- **`capture_assistant`** and **`AI_MEMORY_RERANKER=llm`**, both off by+ default — see `DATA_HANDLING.md`.++## Local embeddings without a network dependency++For semantic search without a cloud API key or a self-hosted inference+server, set the embedding provider to `local`+([`docs/local-embeddings.md`](local-embeddings.md)). On first use it fetches+three small model files (~87 MB total, Apache-2.0 licensed) into+`<data_dir>/models/all-MiniLM-L6-v2/`, each checked against a sha256 pinned+in source so a tampered or drifted file fails loudly rather than silently.++For a fully offline install, download those three files+(`model.safetensors`, `tokenizer.json`, `config.json`) from+`huggingface.co/sentence-transformers/all-MiniLM-L6-v2` on a+network-connected machine and place them in that directory before first+start; the loader verifies the same checksums and never touches the network+(`docs/local-embeddings.md`, "Offline installs"). After that, embeddings run+entirely on-host via the bundled `candle` (pure Rust) runtime — no ONNX+runtime or other native library to source separately.++## What still needs a decision from you++- **git remote sync**, if you use it to push the wiki repository somewhere,+ is your own channel to secure (`SECURITY.md`, "Remote sync security" —+ out of scope for ai-memory itself).+- **Update/patch delivery** in an air-gapped environment is manual: pull a+ new release and checksum on a connected machine, then transfer it in,+ the same as the initial install.++## Related documents++- [`DATA_HANDLING.md`](../DATA_HANDLING.md) — what leaves the host and when.+- [`docs/local-embeddings.md`](local-embeddings.md) — embedding provider+ choices, including the offline path in detail.+- [`docs/deploy.md`](deploy.md) — Docker/homelab deployment pattern.+- [`SECURITY.md`](../SECURITY.md) — release integrity and threat model.diff --git a/docs/sso.md b/docs/sso.mdnew file mode 100644index 00000000..0d5fc095--- /dev/null+++ b/docs/sso.md@@ -0,0 +1,67 @@+# SSO / enterprise identity++This page consolidates ai-memory's existing OIDC support into one place for+teams whose IT-security policy requires centralized identity (SSO) for any+tool touching a workstation, before they can approve local install. It does+not introduce new behavior — everything here already exists in+[`docs/install.md`](install.md) and [`SECURITY.md`](../SECURITY.md); this+page just answers "does it support our IdP?" without reading either in full.++## What's supported today++`ai-memory auth login oidc-device` performs an OIDC device-authorization+flow against any standards-compliant issuer (Keycloak, Okta, Entra ID, etc.):++```bash+ai-memory auth login oidc-device \+ --issuer "https://issuer.example.com/realms/team" \+ --client-id "ai-memory-cli"+```++The resulting access token is stored per-developer and used for two things:++- **Native lifecycle-hook auth** — the client-side hook that reports coding-+ session events to the server.+- **Thin-client HTTP commands** (`status`, `search`, `read-page`,+ `write-page`, `backup`, `embed`, and similar), when no static+ `AI_MEMORY_AUTH_TOKEN` / `[auth].bearer_token` is configured. Static bearer+ auth takes precedence if both are set.++See [`docs/install.md`](install.md) (the `auth login oidc-device` section)+for the full command reference and provider setup notes.++## What this is not++- It is **not** a login gate for the ai-memory server process itself. The+ server's own admin/root authorization model is the static root bearer+ token / database-user token ladder described in `SECURITY.md`'s+ "Authentication and administrative authorization" section. OIDC identifies+ the *human developer* to the CLI/hooks; it does not replace server-side+ admin auth.+- If you front the server with an OIDC-aware gateway or reverse proxy, that+ gateway can translate accepted OIDC identity into the auth ai-memory+ understands (static bearer / DB-user token) — ai-memory itself does not+ validate OIDC tokens against your IdP for server API access.+- OIDC/Keycloak `sid` (session) claims describe the identity provider's login+ session, not the coding-agent session ai-memory tracks for per-session+ isolation (`[auto_scope]`). Don't conflate the two when wiring a gateway.++## Deployment shape for SSO-required environments++1. Set up your OIDC issuer's device-authorization flow (most enterprise IdPs+ support this out of the box for CLI tools).+2. Have each developer run `ai-memory auth login oidc-device` once, pointed+ at your issuer.+3. If you need the *server's* HTTP/admin surface behind your IdP too (not+ just the CLI/hooks), put an OIDC-aware reverse proxy or gateway in front+ of it and translate accepted identities into ai-memory's static bearer /+ DB-user tokens — see `SECURITY.md`'s network-exposure guidance and+ [`docs/https-via-proxy.md`](https-via-proxy.md) for the TLS-termination+ side of that setup.++## Related documents++- [`docs/install.md`](install.md) — full `auth login oidc-device` reference.+- [`SECURITY.md`](../SECURITY.md) — server-side authentication/authorization+ model.+- [`DATA_HANDLING.md`](../DATA_HANDLING.md) — what data moves where.