Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
hardening: bounded buffers + admin root-only + LRU cwd cache
AkitaOnRails607d4bf72aad
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 37fc0780..305b19d6 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -42,6 +42,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0before removal ([#55]).### Fixed+- Backups no longer dereference symlinks under `wiki/`, preventing a planted+ wiki symlink from pulling arbitrary readable host-file contents into+ `backup.tar.gz`.+- `ai-memory restore` now validates tar entries before extraction and accepts+ only regular files/directories under the expected backup paths+ (`wiki/`, `db/memory.sqlite`, and `config.toml`), rejecting links, special+ files, unsafe paths, and unexpected archive entries.+- In multi-user mode (`[auth].token_pepper` configured), operational+ `/admin/*` endpoints now require the root token; DB-user tokens receive+ 403 while single-user installs keep the historical permissive admin behavior.+- LLM provider clients now cap provider response bodies before JSON, text, or+ SSE parsing, and truncate error bodies from bounded buffers instead of+ buffering arbitrary-size responses.+- Non-blocking admission webhooks now have a process-level in-flight cap and+ webhook timeouts are clamped to a safe maximum, preventing observer hooks+ from growing unbounded background work during write bursts.+- Hook cwd/project resolution caching is now bounded with LRU-style eviction,+ preventing unbounded process-lifetime growth from streams of unique cwd+ values.- `memory_write_page` tool description and routing prompts now steer agentstoward writing the page title as a `# H1` on the first line of `body` andomitting the `title` argument. ai-memory already auto-derived the title fromdiff --git a/README.md b/README.mdindex ee4acf6e..3f14bdf8 100644--- a/README.md+++ b/README.md@@ -307,10 +307,13 @@ The bearer token continues to authenticate at the wire level; userscreated via `ai-memory user add` get their own tokens that resolve totheir identity in audit logs (and, in subsequent milestones, pagefrontmatter + the web UI). Data stays single-tenant — there is no-RBAC. Existing single-user installs are not affected unless you opt-in by setting `[auth].token_pepper` (auto-generated for new installs-by `ai-memory init`). See [`docs/users.md`](docs/users.md) for the-full walkthrough and the four-rung auth ladder.+per-page RBAC — but once `[auth].token_pepper` enables multi-user+mode, operational `/admin/*` endpoints such as backup, purge, move,+embed, and commit require the root token. Existing single-user installs+are not affected unless you opt in by setting `[auth].token_pepper`+(auto-generated for new installs by `ai-memory init`). See+[`docs/users.md`](docs/users.md) for the full walkthrough and the+four-rung auth ladder.See [`docs/deploy.md`](docs/deploy.md) for the full homelab patternwith bearer auth, host allowlisting, and TLS/reverse-proxy options.diff --git a/crates/ai-memory-cli/src/commands/restore.rs b/crates/ai-memory-cli/src/commands/restore.rsindex 8ba883fb..4f3b04a9 100644--- a/crates/ai-memory-cli/src/commands/restore.rs+++ b/crates/ai-memory-cli/src/commands/restore.rs@@ -17,6 +17,7 @@use ai_memory_store::Store;use anyhow::{Context, Result, bail};use flate2::read::GzDecoder;+use std::path::{Component, Path};use tracing::info;use crate::cli::RestoreArgs;@@ -63,9 +64,7 @@ pub fn run(config: &Config, args: RestoreArgs) -> Result<()> {.with_context(|| format!("opening {}", args.from.display()))?;let decoder = GzDecoder::new(file);let mut archive = tar::Archive::new(decoder);- archive.set_preserve_permissions(false);- archive- .unpack(&config.data_dir)+ unpack_checked_archive(&mut archive, &config.data_dir).with_context(|| format!("extracting into {}", config.data_dir.display()))?;info!(from = %args.from.display(), into = %config.data_dir.display(), "tarball extracted");@@ -80,3 +79,136 @@ pub fn run(config: &Config, args: RestoreArgs) -> Result<()> {);Ok(())}++fn unpack_checked_archive<R: std::io::Read>(+ archive: &mut tar::Archive<R>,+ data_dir: &Path,+) -> Result<()> {+ archive.set_preserve_permissions(false);+ for entry in archive.entries()? {+ let mut entry = entry?;+ let path = entry.path()?.into_owned();+ let entry_type = entry.header().entry_type();+ validate_restore_entry(&path, entry_type)?;+ entry+ .unpack_in(data_dir)+ .with_context(|| format!("extracting {}", path.display()))?;+ }+ Ok(())+}++fn validate_restore_entry(path: &Path, entry_type: tar::EntryType) -> Result<()> {+ if !path.components().all(|c| matches!(c, Component::Normal(_))) {+ bail!("backup contains unsafe path: {}", path.display());+ }+ if entry_type.is_symlink() || entry_type.is_hard_link() {+ bail!("backup contains unsupported link entry: {}", path.display());+ }+ if !(entry_type.is_file() || entry_type.is_dir()) {+ bail!("backup contains unsupported entry type: {}", path.display());+ }+ let path_str = path.to_string_lossy();+ let allowed = if entry_type.is_dir() {+ path_str == "wiki" || path_str.starts_with("wiki/") || path_str == "db"+ } else {+ path_str == "config.toml" || path_str == "db/memory.sqlite" || path_str.starts_with("wiki/")+ };+ if !allowed {+ bail!("backup contains unexpected path: {}", path.display());+ }+ Ok(())+}++#[cfg(test)]+mod tests {+ use super::*;++ fn archive_with_entry(path: &str, entry_type: tar::EntryType) -> Vec<u8> {+ let mut bytes = Vec::new();+ {+ let mut builder = tar::Builder::new(&mut bytes);+ let mut header = tar::Header::new_gnu();+ header.set_path(path).unwrap();+ header.set_entry_type(entry_type);+ if entry_type.is_symlink() || entry_type.is_hard_link() {+ header.set_link_name("/etc/passwd").unwrap();+ }+ let body: &[u8] = if entry_type.is_file() { b"body" } else { b"" };+ header.set_size(body.len() as u64);+ header.set_cksum();+ builder.append(&header, body).unwrap();+ builder.finish().unwrap();+ }+ bytes+ }++ #[test]+ fn restore_accepts_expected_backup_paths() {+ let tmp = tempfile::TempDir::new().unwrap();+ let mut bytes = Vec::new();+ {+ let mut builder = tar::Builder::new(&mut bytes);+ builder.append_dir("wiki", tmp.path()).unwrap();+ let mut header = tar::Header::new_gnu();+ header.set_path("wiki/default/project/notes/x.md").unwrap();+ header.set_size(4);+ header.set_cksum();+ builder.append(&header, &b"body"[..]).unwrap();+ let mut header = tar::Header::new_gnu();+ header.set_path("db/memory.sqlite").unwrap();+ header.set_size(0);+ header.set_cksum();+ builder.append(&header, &b""[..]).unwrap();+ let mut header = tar::Header::new_gnu();+ header.set_path("config.toml").unwrap();+ header.set_size(0);+ header.set_cksum();+ builder.append(&header, &b""[..]).unwrap();+ builder.finish().unwrap();+ }++ let restore_dir = tempfile::TempDir::new().unwrap();+ let mut archive = tar::Archive::new(bytes.as_slice());+ unpack_checked_archive(&mut archive, restore_dir.path()).unwrap();+ assert!(+ restore_dir+ .path()+ .join("wiki/default/project/notes/x.md")+ .is_file()+ );+ assert!(restore_dir.path().join("db/memory.sqlite").is_file());+ assert!(restore_dir.path().join("config.toml").is_file());+ }++ #[test]+ fn restore_rejects_link_entries() {+ for entry_type in [tar::EntryType::symlink(), tar::EntryType::hard_link()] {+ let bytes = archive_with_entry("wiki/link.md", entry_type);+ let restore_dir = tempfile::TempDir::new().unwrap();+ let mut archive = tar::Archive::new(bytes.as_slice());+ let err = unpack_checked_archive(&mut archive, restore_dir.path()).unwrap_err();+ assert!(err.to_string().contains("unsupported link entry"));+ }+ }++ #[test]+ fn restore_rejects_unexpected_or_unsafe_paths() {+ for path in ["../config.toml", "/tmp/x"] {+ let err = validate_restore_entry(Path::new(path), tar::EntryType::file()).unwrap_err();+ assert!(+ err.to_string().contains("unsafe path"),+ "unexpected error for {path}: {err}"+ );+ }++ let path = "db/extra.sqlite";+ let bytes = archive_with_entry(path, tar::EntryType::file());+ let restore_dir = tempfile::TempDir::new().unwrap();+ let mut archive = tar::Archive::new(bytes.as_slice());+ let err = unpack_checked_archive(&mut archive, restore_dir.path()).unwrap_err();+ assert!(+ err.to_string().contains("unexpected path"),+ "unexpected error for {path}: {err}"+ );+ }+}diff --git a/crates/ai-memory-cli/src/commands/serve.rs b/crates/ai-memory-cli/src/commands/serve.rsindex 81f005ce..f6ecd9fc 100644--- a/crates/ai-memory-cli/src/commands/serve.rs+++ b/crates/ai-memory-cli/src/commands/serve.rs@@ -7,7 +7,9 @@ use std::time::Duration;use ai_memory_consolidate::{Consolidator, run_lint, run_sweep};use ai_memory_core::{ActiveProject, ProjectId, Sanitizer, WorkspaceId};-use ai_memory_hooks::{DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT, HookState, hook_router};+use ai_memory_hooks::{+ DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT, HookState, ProjectCacheStore, hook_router,+};use ai_memory_llm::{Embedder, LlmProvider, ProviderHealth, build_embedder, build_provider};use ai_memory_mcp::{AdminState, AiMemoryServer, admin_router};use ai_memory_store::{EmbeddingWrite, ReaderPool, Store, WriterHandle, f32_vec_to_bytes};@@ -196,7 +198,7 @@ pub async fn run(config: &Config, args: ServeArgs) -> Result<()> {// router gets a fire-and-forget eviction hook so a `move-project`// can proactively drop the moved project's stale entries.let project_cache: ai_memory_hooks::ProjectCache =- std::sync::Arc::new(tokio::sync::Mutex::new(std::collections::HashMap::new()));+ std::sync::Arc::new(tokio::sync::Mutex::new(ProjectCacheStore::default()));let on_project_moved: std::sync::Arc<dyn Fn(ProjectId) + Send + Sync> = {let cache = project_cache.clone();std::sync::Arc::new(move |proj: ProjectId| {diff --git a/crates/ai-memory-hooks/src/lib.rs b/crates/ai-memory-hooks/src/lib.rsindex 0a5ec060..b7aba0e9 100644--- a/crates/ai-memory-hooks/src/lib.rs+++ b/crates/ai-memory-hooks/src/lib.rs@@ -27,5 +27,8 @@ pub mod synth;// pointing at this crate's `sanitize` module keep working.pub use ai_memory_core::{SanitizeConfig, Sanitized, Sanitizer};pub use payload::{HookEnvelope, HookEvent};-pub use router::{DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT, HookState, ProjectCache, hook_router};+pub use router::{+ DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT, DEFAULT_PROJECT_CACHE_MAX_ENTRIES, HookState, ProjectCache,+ ProjectCacheStore, hook_router,+};pub use synth::synthesize_session_page;diff --git a/crates/ai-memory-hooks/src/router.rs b/crates/ai-memory-hooks/src/router.rsindex f0d23f32..865587b8 100644--- a/crates/ai-memory-hooks/src/router.rs+++ b/crates/ai-memory-hooks/src/router.rs@@ -8,7 +8,7 @@//! basic-memory #763). The agent never blocks on us thanks to the//! fire-and-forget client side.-use std::collections::HashMap;+use std::collections::{HashMap, VecDeque};use std::str::FromStr;use std::sync::Arc;@@ -41,11 +41,100 @@ use crate::synth::synthesize_session_page;/// callers can drop or retry instead of growing memory without bound.pub const DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT: usize = 1024;-/// Resolved-project cache: `(cwd, workspace_override, project_override, project_strategy)`-/// keyed map, shared behind a Tokio mutex so the router can be cloned-/// freely without losing the in-flight cache hits.-pub type ProjectCache =- Arc<tokio::sync::Mutex<HashMap<(String, String, String, String), (WorkspaceId, ProjectId)>>>;+/// Maximum cwd-resolution cache entries kept per server process. The cache is+/// an optimization only; evicted entries are re-resolved through the writer.+pub const DEFAULT_PROJECT_CACHE_MAX_ENTRIES: usize = 4096;++/// Resolved-project cache key:+/// `(cwd, workspace_override, project_override, project_strategy)`.+pub type ProjectCacheKey = (String, String, String, String);++/// Shared bounded resolved-project cache.+pub type ProjectCache = Arc<tokio::sync::Mutex<ProjectCacheStore>>;++/// Bounded cwd-resolution cache used by the hook router.+#[derive(Debug)]+pub struct ProjectCacheStore {+ entries: HashMap<ProjectCacheKey, (WorkspaceId, ProjectId)>,+ order: VecDeque<ProjectCacheKey>,+ max_entries: usize,+}++impl Default for ProjectCacheStore {+ fn default() -> Self {+ Self::new(DEFAULT_PROJECT_CACHE_MAX_ENTRIES)+ }+}++impl ProjectCacheStore {+ #[must_use]+ fn new(max_entries: usize) -> Self {+ Self {+ entries: HashMap::new(),+ order: VecDeque::new(),+ max_entries: max_entries.max(1),+ }+ }++ fn get(&mut self, key: &ProjectCacheKey) -> Option<(WorkspaceId, ProjectId)> {+ let ids = self.entries.get(key).copied()?;+ self.touch(key);+ Some(ids)+ }++ fn insert(&mut self, key: ProjectCacheKey, ids: (WorkspaceId, ProjectId)) {+ if self.entries.contains_key(&key) {+ self.entries.insert(key.clone(), ids);+ self.touch(&key);+ return;+ }+ self.entries.insert(key.clone(), ids);+ self.order.push_back(key);+ while self.entries.len() > self.max_entries {+ if let Some(oldest) = self.order.pop_front() {+ self.entries.remove(&oldest);+ } else {+ break;+ }+ }+ }++ fn remove(&mut self, key: &ProjectCacheKey) {+ self.entries.remove(key);+ self.order.retain(|k| k != key);+ }++ #[must_use]+ #[cfg(test)]+ fn len(&self) -> usize {+ self.entries.len()+ }++ #[must_use]+ #[cfg(test)]+ fn contains_key(&self, key: &ProjectCacheKey) -> bool {+ self.entries.contains_key(key)+ }++ #[cfg(test)]+ fn values(&self) -> impl Iterator<Item = &(WorkspaceId, ProjectId)> {+ self.entries.values()+ }++ /// Retain only cache entries that match `keep`.+ pub fn retain<F>(&mut self, mut keep: F)+ where+ F: FnMut(&ProjectCacheKey, &(WorkspaceId, ProjectId)) -> bool,+ {+ self.entries.retain(|key, ids| keep(key, ids));+ self.order.retain(|key| self.entries.contains_key(key));+ }++ fn touch(&mut self, key: &ProjectCacheKey) {+ self.order.retain(|k| k != key);+ self.order.push_back(key.clone());+ }+}/// Shared state passed to the hook handler.#[derive(Clone)]@@ -302,13 +391,13 @@ async fn resolve_project_ids();{- let cache = state.project_cache.lock().await;+ let mut cache = state.project_cache.lock().await;if let Some(ids) = cache.get(&cache_key) {// Republish on every hit: a cache hit still means the agent// is active in this project *now*, which is exactly what the// MCP read tools need as their default.state.active_project.set(ids.0, ids.1);- return Ok(*ids);+ return Ok(ids);}}@@ -736,7 +825,6 @@ const fn importance_for(event: HookEvent) -> u8 {#[cfg(test)]mod tests {- use std::collections::HashMap;use std::sync::Arc;use ai_memory_core::Sanitizer;@@ -770,7 +858,7 @@ mod tests {wiki,consolidator: None,sanitizer,- project_cache: Arc::new(tokio::sync::Mutex::new(HashMap::new())),+ project_cache: Arc::new(tokio::sync::Mutex::new(ProjectCacheStore::default())),active_project: ActiveProject::new(),consolidate_on_session_end: false,ingest_semaphore: Arc::new(tokio::sync::Semaphore::new(@@ -957,6 +1045,27 @@ mod tests {}}+ #[test]+ fn project_cache_store_evicts_oldest_untouched_entry() {+ let mut cache = ProjectCacheStore::new(2);+ let key_a = ("/a".into(), String::new(), String::new(), "basename".into());+ let key_b = ("/b".into(), String::new(), String::new(), "basename".into());+ let key_c = ("/c".into(), String::new(), String::new(), "basename".into());++ cache.insert(key_a.clone(), (WorkspaceId::new(), ProjectId::new()));+ cache.insert(key_b.clone(), (WorkspaceId::new(), ProjectId::new()));+ assert!(+ cache.get(&key_a).is_some(),+ "touch key_a so key_b is oldest"+ );+ cache.insert(key_c.clone(), (WorkspaceId::new(), ProjectId::new()));++ assert!(cache.contains_key(&key_a));+ assert!(!cache.contains_key(&key_b));+ assert!(cache.contains_key(&key_c));+ assert_eq!(cache.len(), 2);+ }+/// If the cached project is deleted out from under the router (e.g./// `purge-project` on a live server), the next event must self-heal:/// evict the stale slot, recreate the project, and ingest — instead ofdiff --git a/crates/ai-memory-llm/src/anthropic.rs b/crates/ai-memory-llm/src/anthropic.rsindex ea0d8380..f3b3123a 100644--- a/crates/ai-memory-llm/src/anthropic.rs+++ b/crates/ai-memory-llm/src/anthropic.rs@@ -10,7 +10,7 @@ use tracing::debug;use crate::error::{LlmError, LlmResult};use crate::provider::LlmProvider;-use crate::text::truncate_with_ellipsis;+use crate::response::{provider_error_body, response_json_limited};use crate::types::{ChatRequest, ChatResponse, Role, Usage};/// Default Anthropic API base.@@ -264,13 +264,13 @@ impl AnthropicProvider {let resp = builder.json(body).send().await?;let status = resp.status();if !status.is_success() {- let body = resp.text().await.unwrap_or_default();+ let body = provider_error_body(resp).await;return Err(LlmError::Provider {status: status.as_u16(),- body: truncate_with_ellipsis(&body, 1024),+ body,});}- resp.json::<R>().await.map_err(LlmError::from)+ response_json_limited::<R>(resp).await}/// The auth headers for this provider instance: `x-api-key` for a staticdiff --git a/crates/ai-memory-llm/src/copilot.rs b/crates/ai-memory-llm/src/copilot.rsindex 492d759d..97ce2e4e 100644--- a/crates/ai-memory-llm/src/copilot.rs+++ b/crates/ai-memory-llm/src/copilot.rs@@ -19,7 +19,7 @@ use crate::auth_file::{load_entry, now_ms, save_entry};use crate::error::{LlmError, LlmResult};use crate::openai::{STRUCTURED_OUTPUT_SCHEMA_NAME, enforce_strict_object_schemas};use crate::provider::LlmProvider;-use crate::text::truncate_with_ellipsis;+use crate::response::{provider_error_body, response_json_limited};use crate::types::{ChatRequest, ChatResponse, Role, Usage};/// GitHub Copilot's public OAuth client id used by Copilot clients.@@ -285,15 +285,13 @@ impl CopilotProvider {.map_err(LlmError::from)?;let status = resp.status();if !status.is_success() {- let body = resp.text().await.unwrap_or_default();+ let body = provider_error_body(resp).await;return Err(LlmError::Provider {status: status.as_u16(),- body: truncate_with_ellipsis(&body, 1024),+ body,});}- resp.json::<CopilotChatResponse>()- .await- .map_err(LlmError::from)+ response_json_limited::<CopilotChatResponse>(resp).await}}@@ -369,15 +367,12 @@ async fn exchange_copilot_token(.map_err(LlmError::from)?;let status = resp.status();if !status.is_success() {- let body = resp.text().await.unwrap_or_default();+ let body = provider_error_body(resp).await;return Err(LlmError::Auth(format!("copilot token exchange failed ({status}): {body}")));}- let exchanged = resp- .json::<CopilotExchangeResponse>()- .await- .map_err(LlmError::from)?;+ let exchanged = response_json_limited::<CopilotExchangeResponse>(resp).await?;let expires_at_ms = unix_epoch_to_ms(exchanged.expires_at);Ok(CopilotApiToken {base_url: derive_copilot_api_base_url_from_token(&exchanged.token)diff --git a/crates/ai-memory-llm/src/embedding.rs b/crates/ai-memory-llm/src/embedding.rsindex 19025d64..c0da0323 100644--- a/crates/ai-memory-llm/src/embedding.rs+++ b/crates/ai-memory-llm/src/embedding.rs@@ -21,6 +21,7 @@ use tracing::debug;use crate::error::{LlmError, LlmResult};use crate::openai::normalize_openai_base;+use crate::response::{provider_error_body, response_json_limited, response_text_limited};use crate::text::{truncate_for_embedding, truncate_with_ellipsis};/// Conservative per-request input cap for OpenAI-compatible embedding APIs@@ -229,20 +230,20 @@ impl Embedder for OpenAiEmbedder {}// Client errors (e.g. input > 8192 tokens) are not retried.if status.as_u16() == 400 {- let body = resp.text().await.unwrap_or_default();+ let body = provider_error_body(resp).await;return Err(LlmError::Provider {status: status.as_u16(),- body: truncate_with_ellipsis(&body, 1024),+ body,});}if !status.is_success() {- let body = resp.text().await.unwrap_or_default();+ let body = provider_error_body(resp).await;return Err(LlmError::Provider {status: status.as_u16(),- body: truncate_with_ellipsis(&body, 1024),+ body,});}- let body = resp.text().await.unwrap_or_default();+ let body = response_text_limited(resp).await?;let values = parse_openai_embedding_values(&body, status.as_u16())?;if values.len() as u32 != self.dim {return Err(LlmError::UnexpectedShape(format!(@@ -341,13 +342,13 @@ impl Embedder for VoyageEmbedder {.await?;let status = resp.status();if !status.is_success() {- let body = resp.text().await.unwrap_or_default();+ let body = provider_error_body(resp).await;return Err(LlmError::Provider {status: status.as_u16(),- body: truncate_with_ellipsis(&body, 1024),+ body,});}- let parsed: VoyageResponse = resp.json().await?;+ let parsed: VoyageResponse = response_json_limited(resp).await?;let first =parsed.data.into_iter().next().ok_or_else(|| {LlmError::UnexpectedShape("voyage response had no data[0]".into())diff --git a/crates/ai-memory-llm/src/gemini.rs b/crates/ai-memory-llm/src/gemini.rsindex 12732ce2..654f1097 100644--- a/crates/ai-memory-llm/src/gemini.rs+++ b/crates/ai-memory-llm/src/gemini.rs@@ -18,7 +18,7 @@ use tracing::debug;use crate::error::{LlmError, LlmResult};use crate::provider::LlmProvider;-use crate::text::truncate_with_ellipsis;+use crate::response::{provider_error_body, response_json_limited};use crate::types::{ChatRequest, ChatResponse, Role, Usage};/// Default Gemini API base.@@ -235,13 +235,13 @@ impl GeminiProvider {.await?;let status = resp.status();if !status.is_success() {- let body = resp.text().await.unwrap_or_default();+ let body = provider_error_body(resp).await;return Err(LlmError::Provider {status: status.as_u16(),- body: truncate_with_ellipsis(&body, 1024),+ body,});}- resp.json::<R>().await.map_err(LlmError::from)+ response_json_limited::<R>(resp).await}}diff --git a/crates/ai-memory-llm/src/google.rs b/crates/ai-memory-llm/src/google.rsindex d1ca1cfd..72296a85 100644--- a/crates/ai-memory-llm/src/google.rs+++ b/crates/ai-memory-llm/src/google.rs@@ -11,7 +11,7 @@ use tracing::debug;use crate::embedding::{Embedder, normalise};use crate::error::{LlmError, LlmResult};-use crate::text::truncate_with_ellipsis;+use crate::response::{provider_error_body, response_json_limited};/// Default Gemini API host.pub const DEFAULT_BASE_URL: &str = "https://generativelanguage.googleapis.com";@@ -106,13 +106,13 @@ impl GoogleEmbedder {continue;}if !status.is_success() {- let body_text = resp.text().await.unwrap_or_default();+ let body = provider_error_body(resp).await;return Err(LlmError::Provider {status: status.as_u16(),- body: truncate_with_ellipsis(&body_text, 1024),+ body,});}- let parsed: GeminiEmbedResponse = resp.json().await?;+ let parsed: GeminiEmbedResponse = response_json_limited(resp).await?;let values = parsed.embedding.values;if values.len() as u32 != self.dim {return Err(LlmError::UnexpectedShape(format!(diff --git a/crates/ai-memory-llm/src/lib.rs b/crates/ai-memory-llm/src/lib.rsindex 1d730f9b..eecd599f 100644--- a/crates/ai-memory-llm/src/lib.rs+++ b/crates/ai-memory-llm/src/lib.rs@@ -42,6 +42,7 @@ pub mod provider;pub mod types;mod auth_file;+mod response;mod text;pub use anthropic::AnthropicProvider;diff --git a/crates/ai-memory-llm/src/openai.rs b/crates/ai-memory-llm/src/openai.rsindex b27fcdc0..26fa3a21 100644--- a/crates/ai-memory-llm/src/openai.rs+++ b/crates/ai-memory-llm/src/openai.rs@@ -10,7 +10,7 @@ use tracing::debug;use crate::error::{LlmError, LlmResult};use crate::provider::LlmProvider;-use crate::text::truncate_with_ellipsis;+use crate::response::{provider_error_body, response_json_limited};use crate::types::{ChatRequest, ChatResponse, Role, Usage};/// Default OpenAI API base.@@ -323,13 +323,13 @@ impl OpenAiProvider {.await?;let status = resp.status();if !status.is_success() {- let body = resp.text().await.unwrap_or_default();+ let body = provider_error_body(resp).await;return Err(LlmError::Provider {status: status.as_u16(),- body: truncate_with_ellipsis(&body, 1024),+ body,});}- resp.json::<OpenAiResponse>().await.map_err(LlmError::from)+ response_json_limited::<OpenAiResponse>(resp).await}}diff --git a/crates/ai-memory-llm/src/openai_oauth.rs b/crates/ai-memory-llm/src/openai_oauth.rsindex b0a76eae..f86fee63 100644--- a/crates/ai-memory-llm/src/openai_oauth.rs+++ b/crates/ai-memory-llm/src/openai_oauth.rs@@ -19,6 +19,7 @@ use crate::auth_file::{load_entry, now_ms, save_entry};use crate::error::{LlmError, LlmResult};use crate::openai::{STRUCTURED_OUTPUT_SCHEMA_NAME, enforce_strict_object_schemas};use crate::provider::LlmProvider;+use crate::response::{provider_error_body, response_json_limited, response_text_limited};use crate::text::truncate_with_ellipsis;use crate::types::{ChatRequest, ChatResponse, Role, Usage};@@ -241,18 +242,16 @@ impl OpenAiOAuthProvider {let resp = request.send().await.map_err(LlmError::from)?;let status = resp.status();if !status.is_success() {- let body = resp.text().await.unwrap_or_default();+ let body = provider_error_body(resp).await;return Err(LlmError::Provider {status: status.as_u16(),- body: truncate_with_ellipsis(&body, 1024),+ body,});}if body.stream {- parse_sse_response(&resp.text().await.map_err(LlmError::from)?)+ parse_sse_response(&response_text_limited(resp).await?)} else {- resp.json::<CodexResponsesResponse>()- .await- .map_err(LlmError::from)+ response_json_limited::<CodexResponsesResponse>(resp).await}}}@@ -312,15 +311,12 @@ async fn refresh_access_token(.map_err(LlmError::from)?;let status = resp.status();if !status.is_success() {- let body = resp.text().await.unwrap_or_default();+ let body = provider_error_body(resp).await;return Err(LlmError::Auth(format!("openai-oauth refresh failed ({status}): {body}. Run `ai-memory auth login openai-oauth` again.")));}- let token_response = resp- .json::<OpenAiOAuthTokenResponse>()- .await- .map_err(LlmError::from)?;+ let token_response = response_json_limited::<OpenAiOAuthTokenResponse>(resp).await?;Ok(OpenAiOAuthToken::from_token_response(token_response.access_token,token_responsediff --git a/crates/ai-memory-llm/src/response.rs b/crates/ai-memory-llm/src/response.rsnew file mode 100644index 00000000..10adf6aa--- /dev/null+++ b/crates/ai-memory-llm/src/response.rs@@ -0,0 +1,52 @@+use serde::de::DeserializeOwned;++use crate::error::{LlmError, LlmResult};+use crate::text::truncate_with_ellipsis;++/// Generous cap for successful provider responses. Normal chat, structured+/// output, SSE transcripts, and embedding payloads are far smaller; this only+/// protects us from broken or hostile provider endpoints buffering forever.+pub(crate) const MAX_PROVIDER_RESPONSE_BYTES: usize = 16 * 1024 * 1024;+const MAX_PROVIDER_ERROR_BYTES: usize = 1024 * 1024;+const DISPLAY_ERROR_BYTES: usize = 1024;++pub(crate) async fn response_bytes_limited(+ mut resp: reqwest::Response,+ max_bytes: usize,+) -> LlmResult<Vec<u8>> {+ let status = resp.status().as_u16();+ let mut bytes = Vec::new();+ while let Some(chunk) = resp.chunk().await.map_err(LlmError::from)? {+ if bytes.len().saturating_add(chunk.len()) > max_bytes {+ return Err(LlmError::Provider {+ status,+ body: format!("provider response exceeded {max_bytes} bytes"),+ });+ }+ bytes.extend_from_slice(&chunk);+ }+ Ok(bytes)+}++pub(crate) async fn response_text_limited(resp: reqwest::Response) -> LlmResult<String> {+ let bytes = response_bytes_limited(resp, MAX_PROVIDER_RESPONSE_BYTES).await?;+ Ok(String::from_utf8_lossy(&bytes).into_owned())+}++pub(crate) async fn response_json_limited<T: DeserializeOwned>(+ resp: reqwest::Response,+) -> LlmResult<T> {+ let bytes = response_bytes_limited(resp, MAX_PROVIDER_RESPONSE_BYTES).await?;+ serde_json::from_slice(&bytes).map_err(LlmError::from)+}++pub(crate) async fn provider_error_body(resp: reqwest::Response) -> String {+ match response_bytes_limited(resp, MAX_PROVIDER_ERROR_BYTES).await {+ Ok(bytes) => {+ let body = String::from_utf8_lossy(&bytes);+ truncate_with_ellipsis(body.as_ref(), DISPLAY_ERROR_BYTES)+ }+ Err(LlmError::Provider { body, .. }) => body,+ Err(e) => format!("<failed to read provider error body: {e}>"),+ }+}diff --git a/crates/ai-memory-mcp/src/admin.rs b/crates/ai-memory-mcp/src/admin.rsindex 4e996a5a..561ef800 100644--- a/crates/ai-memory-mcp/src/admin.rs+++ b/crates/ai-memory-mcp/src/admin.rs@@ -45,6 +45,7 @@ use axum::Router;use axum::body::Body;use axum::extract::{Query, State};use axum::http::{HeaderMap, StatusCode, header};+use axum::middleware::Next;use axum::response::{IntoResponse, Response};use axum::routing::{get, post};use flate2::Compression;@@ -160,7 +161,8 @@ fn default_chunk_input_tokens() -> usize {/// - `POST /admin/move-project`/// - `POST /admin/write-page`pub fn admin_router(state: AdminState) -> Router {- Router::new()+ let state = Arc::new(state);+ let operational = Router::new().route("/admin/backup", post(handle_backup)).route("/admin/bootstrap", post(handle_bootstrap)).route("/admin/status", get(handle_status))@@ -175,6 +177,11 @@ pub fn admin_router(state: AdminState) -> Router {.route("/admin/rename-project", post(handle_rename_project)).route("/admin/move-project", post(handle_move_project)).route("/admin/write-page", post(handle_write_page))+ .route_layer(axum::middleware::from_fn_with_state(+ state.clone(),+ require_root_for_multiuser_admin,+ ));+ let users = Router::new().route("/admin/users",get(handle_list_users).post(handle_create_user),@@ -184,8 +191,38 @@ pub fn admin_router(state: AdminState) -> Router {.route("/admin/users/{username}/rotate-token",post(handle_rotate_user_token),- )- .with_state(Arc::new(state))+ );+ operational.merge(users).with_state(state)+}++async fn require_root_for_multiuser_admin(+ State(state): State<Arc<AdminState>>,+ req: axum::http::Request<Body>,+ next: Next,+) -> Response {+ if state.token_pepper.is_none() {+ return next.run(req).await;+ }+ let level = req+ .extensions()+ .get::<ai_memory_core::AuthLevel>()+ .copied()+ .unwrap_or(ai_memory_core::AuthLevel::Anonymous);+ if level.is_root() {+ return next.run(req).await;+ }+ let (code, msg) = match level {+ ai_memory_core::AuthLevel::Anonymous => (+ StatusCode::UNAUTHORIZED,+ "admin operation requires authentication in multi-user mode",+ ),+ ai_memory_core::AuthLevel::User => (+ StatusCode::FORBIDDEN,+ "admin operation is root-only in multi-user mode",+ ),+ ai_memory_core::AuthLevel::Root => unreachable!("guarded above"),+ };+ (code, Json(serde_json::json!({ "error": msg }))).into_response()}// ---------------------------------------------------------------------@@ -247,6 +284,7 @@ async fn build_backup_tarball_file(state: &AdminState) -> anyhow::Result<tokio::let encoder = GzEncoder::new(&mut tar_file, Compression::default());let mut tar = tar::Builder::new(encoder);tar.mode(tar::HeaderMode::Deterministic);+ tar.follow_symlinks(false);let wiki_dir = state.data_dir.join("wiki");if wiki_dir.is_dir() {@@ -3188,6 +3226,136 @@ mod tests {.unwrap()}+ #[tokio::test]+ async fn multiuser_operational_admin_routes_reject_db_user_tier() {+ let (_tmp, router) = user_admin_test_router("root-token");+ let routes = [+ ("POST", "/admin/backup", serde_json::Value::Null),+ (+ "POST",+ "/admin/bootstrap",+ serde_json::json!({+ "workspace": "default",+ "project": "scratch",+ "sources": [],+ "dry_run": true+ }),+ ),+ ("GET", "/admin/status", serde_json::Value::Null),+ ("GET", "/admin/search?q=test", serde_json::Value::Null),+ (+ "GET",+ "/admin/read-page?workspace=default&project=scratch&path=notes/x.md",+ serde_json::Value::Null,+ ),+ ("POST", "/admin/reorg", serde_json::json!({"dry_run": true})),+ (+ "POST",+ "/admin/lint",+ serde_json::json!({"workspace": "default", "project": "scratch", "dry_run": true}),+ ),+ (+ "POST",+ "/admin/forget-sweep",+ serde_json::json!({"workspace": "default", "project": "scratch", "dry_run": true}),+ ),+ (+ "POST",+ "/admin/embed",+ serde_json::json!({"workspace": "default", "project": "scratch", "dry_run": true}),+ ),+ (+ "POST",+ "/admin/commit",+ serde_json::json!({"message": "test"}),+ ),+ (+ "POST",+ "/admin/purge-project",+ serde_json::json!({"workspace": "default", "project": "scratch", "confirm": true}),+ ),+ (+ "POST",+ "/admin/rename-project",+ serde_json::json!({"workspace": "default", "from": "scratch", "to": "renamed"}),+ ),+ (+ "POST",+ "/admin/move-project",+ serde_json::json!({+ "from_workspace": "default",+ "to_workspace": "archive",+ "project": "scratch",+ "confirm": true+ }),+ ),+ (+ "POST",+ "/admin/write-page",+ serde_json::json!({+ "workspace": "default",+ "project": "scratch",+ "path": "notes/x.md",+ "body": "body"+ }),+ ),+ ];++ for (method, uri, payload) in routes {+ let mut builder = Request::builder()+ .method(method)+ .uri(uri)+ .header("authorization", "Bearer db-user-token");+ let body = if payload.is_null() {+ Body::empty()+ } else {+ builder = builder.header("content-type", "application/json");+ Body::from(serde_json::to_vec(&payload).unwrap())+ };+ let resp = router+ .clone()+ .oneshot(builder.body(body).unwrap())+ .await+ .unwrap();+ assert_eq!(+ resp.status(),+ StatusCode::FORBIDDEN,+ "{method} {uri} must be root-only in multi-user mode"+ );+ }+ }++ #[tokio::test]+ async fn multiuser_operational_admin_routes_reject_anonymous() {+ let (_tmp, router) = user_admin_test_router("root-token");+ let resp = router+ .oneshot(+ Request::builder()+ .uri("/admin/status")+ .body(Body::empty())+ .unwrap(),+ )+ .await+ .unwrap();+ assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);+ }++ #[tokio::test]+ async fn multiuser_operational_admin_routes_allow_root() {+ let (_tmp, router) = user_admin_test_router("root-token");+ let resp = router+ .oneshot(+ Request::builder()+ .uri("/admin/status")+ .header("authorization", "Bearer root-token")+ .body(Body::empty())+ .unwrap(),+ )+ .await+ .unwrap();+ assert_eq!(resp.status(), StatusCode::OK);+ }+#[tokio::test]async fn create_user_happy_path_returns_token_once() {let (_tmp, router) = user_admin_test_router("root-token");diff --git a/crates/ai-memory-mcp/tests/admin_backup.rs b/crates/ai-memory-mcp/tests/admin_backup.rsindex f452a2a3..04c80e54 100644--- a/crates/ai-memory-mcp/tests/admin_backup.rs+++ b/crates/ai-memory-mcp/tests/admin_backup.rs@@ -180,3 +180,52 @@ async fn backup_empty_store_still_succeeds() {.unwrap();assert!(!bytes.is_empty());}++#[cfg(unix)]+#[tokio::test]+async fn backup_does_not_dereference_wiki_symlinks() {+ let tmp = TempDir::new().unwrap();+ let (state, _store) = make_state(&tmp).await;++ let wiki_dir = tmp.path().join("wiki");+ std::fs::create_dir_all(&wiki_dir).unwrap();+ let secret = tmp.path().join("outside-secret.md");+ let secret_body = "outside secret must not enter backup";+ std::fs::write(&secret, secret_body).unwrap();+ std::os::unix::fs::symlink(&secret, wiki_dir.join("leak.md")).unwrap();++ let router = admin_router(state);+ let resp = router+ .oneshot(+ Request::builder()+ .method("POST")+ .uri("/admin/backup")+ .body(Body::empty())+ .unwrap(),+ )+ .await+ .unwrap();+ assert_eq!(resp.status(), StatusCode::OK);++ let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)+ .await+ .unwrap();+ let decoder = GzDecoder::new(bytes.as_ref());+ let mut archive = Archive::new(decoder);+ for entry in archive.entries().expect("tarball must be readable") {+ let mut entry = entry.expect("entry must be readable");+ if !entry.header().entry_type().is_file() {+ continue;+ }+ let mut body = Vec::new();+ entry+ .read_to_end(&mut body)+ .expect("regular file entry must be readable");+ assert!(+ !body+ .windows(secret_body.len())+ .any(|window| window == secret_body.as_bytes()),+ "backup must not include symlink target contents"+ );+ }+}diff --git a/crates/ai-memory-wiki/src/admission.rs b/crates/ai-memory-wiki/src/admission.rsindex 220419f2..997729ef 100644--- a/crates/ai-memory-wiki/src/admission.rs+++ b/crates/ai-memory-wiki/src/admission.rs@@ -216,6 +216,16 @@ struct WebhookResponsePage {/// loop) and would be better served by an out-of-band fan-out service.pub const MAX_ADMISSION_WEBHOOKS: usize = 16;+/// Maximum timeout a single webhook request may consume. Operators can set a+/// lower value per hook; larger values are clamped so misconfiguration cannot+/// pin write-path or async admission work indefinitely.+pub const MAX_WEBHOOK_TIMEOUT_MS: u64 = 30_000;++/// Maximum non-blocking webhook requests in flight for one process. Beyond this+/// cap observer hooks are dropped with a warning; the durable wiki write has+/// already completed, so backpressure here should never stall the caller.+pub const MAX_ASYNC_ADMISSION_IN_FLIGHT: usize = 256;+/// Maximum bytes read from a single webhook response body. Webhooks/// only need to return the mutated `page` envelope; multi-megabyte/// responses are pathological (faulty webhook or hostile peer) and@@ -229,6 +239,7 @@ pub const MAX_RESPONSE_BYTES: usize = 1024 * 1024;pub struct AdmissionChain {webhooks: Arc<Vec<WebhookConfig>>,client: reqwest::Client,+ async_semaphore: Arc<tokio::sync::Semaphore>,}impl AdmissionChain {@@ -252,6 +263,7 @@ impl AdmissionChain {Ok(Self {webhooks: Arc::new(webhooks),client,+ async_semaphore: Arc::new(tokio::sync::Semaphore::new(MAX_ASYNC_ADMISSION_IN_FLIGHT)),})}@@ -305,7 +317,7 @@ impl AdmissionChain {.client.post(&hook.url).header("X-Memory-Op", ctx.op.as_header_value())- .timeout(Duration::from_millis(hook.timeout_ms))+ .timeout(webhook_timeout(hook.timeout_ms)).json(&payload).send().await;@@ -415,7 +427,7 @@ impl AdmissionChain {.client.post(&hook.url).header("X-Memory-Op", ctx.op.as_header_value())- .timeout(Duration::from_millis(hook.timeout_ms))+ .timeout(webhook_timeout(hook.timeout_ms)).json(&payload).send().await;@@ -471,6 +483,14 @@ impl AdmissionChain {if !hook.events.contains(&ctx.op) {continue;}+ let Ok(permit) = self.async_semaphore.clone().try_acquire_owned() else {+ tracing::warn!(+ webhook = %hook.name,+ cap = MAX_ASYNC_ADMISSION_IN_FLIGHT,+ "admission async queue saturated; dropping observer webhook",+ );+ continue;+ };let client = self.client.clone();let url = hook.url.clone();let name = hook.name.clone();@@ -481,6 +501,7 @@ impl AdmissionChain {let frontmatter = frontmatter.clone();let body = body.to_string();tokio::spawn(async move {+ let _permit = permit;let payload = WebhookRequestBody {page: WebhookPagePayload {path: path.as_deref().unwrap_or(""),@@ -492,7 +513,7 @@ impl AdmissionChain {match client.post(&url).header("X-Memory-Op", op.as_header_value())- .timeout(Duration::from_millis(timeout))+ .timeout(webhook_timeout(timeout)).json(&payload).send().await@@ -512,6 +533,10 @@ impl AdmissionChain {}}+fn webhook_timeout(timeout_ms: u64) -> Duration {+ Duration::from_millis(timeout_ms.clamp(1, MAX_WEBHOOK_TIMEOUT_MS))+}+async fn read_response_limited(resp: reqwest::Response) -> Result<Option<Vec<u8>>, reqwest::Error> {let mut stream = resp.bytes_stream();let mut bytes = Vec::new();@@ -545,6 +570,16 @@ mod tests {assert!(matches!(op, AdmissionOp::WritePage));}+ #[test]+ fn webhook_timeout_is_clamped_to_smart_bounds() {+ assert_eq!(webhook_timeout(0), Duration::from_millis(1));+ assert_eq!(webhook_timeout(2_000), Duration::from_millis(2_000));+ assert_eq!(+ webhook_timeout(MAX_WEBHOOK_TIMEOUT_MS + 1),+ Duration::from_millis(MAX_WEBHOOK_TIMEOUT_MS)+ );+ }+/// `partial_failure` serialises only when `true`, so existing webhook/// consumers see no extra key on the happy path. Mirrors that care/// about filesystem-vs-DB drift (a git-push mirror) opt in by readingdiff --git a/docs/users.md b/docs/users.mdindex 82a9ce53..244293b6 100644--- a/docs/users.md+++ b/docs/users.md@@ -7,10 +7,12 @@ai-memory is **single-tenant data** with **optional multi-userattribution**. Every authenticated request sees the same wiki pages —-there is no RBAC, no per-user scoping, no group permissions. What-multi-user mode adds is *who-did-this*: every write attributes to a-named user, audit-log rows carry that identity, and the web UI can-show "Last edited by Alice Smith" instead of the anonymous default.+there is no per-page RBAC, no per-user data scoping, no group+permissions. What multi-user mode adds is *who-did-this*: every write+attributes to a named user, audit-log rows carry that identity, and the+web UI can show "Last edited by Alice Smith" instead of the anonymous+default. Operational `/admin/*` endpoints stay root-only once+multi-user mode is configured.If you run ai-memory alone, you can skip this page — your installkeeps working unchanged.@@ -47,7 +49,7 @@ Every HTTP request is resolved to one of four authentication tiers:|---|---|---|| **0 — Anonymous** | No `[auth].bearer_token` set. | Allowed, no identity. Same as pre-multi-user defaults. || **1 — Root** | Bearer matches `[auth].bearer_token`. | Allowed as **root**. When `[auth].root_username` is set, writes attribute to that name; otherwise attribution stays anonymous. |-| **2 — DB user** | Bearer doesn't match root, matches a `users.token_hash` row (via SHA-256 of token + `[auth].token_pepper`). | Allowed as **that user**. The audit log records the username/email/name. |+| **2 — DB user** | Bearer doesn't match root, matches a `users.token_hash` row (via SHA-256 of token + `[auth].token_pepper`). | Allowed as **that user** for normal read/write APIs. Operational `/admin/*` endpoints are root-only in multi-user mode. The audit log records the username/email/name. || **3 — 401** | Bearer present but matches nothing. | Rejected. Closes the bypass — unknown bearers can't slip through as anonymous. |The rungs are sticky: a request is matched at the lowest tier that@@ -170,6 +172,10 @@ If you're upgrading from a pre-v0.8 ai-memory:user-management endpoints return **503** with a clear`multi-user not enabled` message. Existing installs never tripthis because they never call `user add`.+- Operational `/admin/*` endpoints are open to the configured bearer+ token in single-user mode, matching historical behavior. After you+ configure `[auth].token_pepper`, those endpoints require the root+ token; DB-user tokens receive **403**.### Migrating an existing single-user install@@ -268,18 +274,17 @@ the bearer authenticates, attribution flows from the token's owner## Limitations-- **No RBAC.** Every authenticated user sees every page in the- workspace. If you need data isolation, run separate ai-memory- servers (per-user data dirs) and front them with a reverse+- **No per-page RBAC.** Every authenticated user sees every page in+ the workspace. Operational `/admin/*` endpoints are still root-only+ in multi-user mode. If you need data isolation, run separate+ ai-memory servers (per-user data dirs) and front them with a reverseproxy.- **One token per user.** Rotation issues a new token andinvalidates the old in the same transaction. There's nonotion of multiple device-bound tokens per user.-- **Root token is single.** `[auth].bearer_token` is a single- scalar. If you want multiple "admin"-class users, add them via- `user add` and accept that they currently have the same- capabilities as the root token for everything *except*- user-management endpoints, which stay root-only.+- **Root token is single.** `[auth].bearer_token` is the admin token+ for operational `/admin/*` endpoints. DB users created with+ `user add` are normal users, not additional admins.- **No federation / OIDC / OAuth.** Per[`design-decisions.md`](design-decisions.md) §13, ai-memory v1stays self-contained. If you need SSO, terminate it at a