Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
fix(embeddings): reserve local query capacity
AkitaOnRails107ada8db79a
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 46bdb924..0c10d013 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -143,6 +143,13 @@ version in [`docs/changelog/`](docs/changelog/):descriptor's real path. (#1416)### Fixed+- Reserved local-embedding capacity for interactive search queries instead of+ allowing concurrent page writes and backfill work to occupy every inference+ slot. Excess queries fall back immediately to non-vector retrieval, and a+ cancelled request retains its slot until its non-cancellable CPU inference+ actually ends. An opt-in real-model contention probe records idle,+ unbounded-load, and bounded-load p50/p95 query latency; hosted embedding+ providers are unchanged.- Added exact hover timestamps to `/web` page metadata, made the OKF upgradedialog keyboard accessible, and added a labelled main-navigation landmarkwith current-page state and narrow-screen wrapping.diff --git a/crates/ai-memory-cli/src/commands/serve.rs b/crates/ai-memory-cli/src/commands/serve.rsindex bf1c3f28..a5ee3477 100644--- a/crates/ai-memory-cli/src/commands/serve.rs+++ b/crates/ai-memory-cli/src/commands/serve.rs@@ -16,7 +16,7 @@ use ai_memory_hooks::{CLIENT_MARKER_HEADER, DEFAULT_HOOK_INGEST_MAX_IN_FLIGHT, HookState, ProjectCacheStore,WorkstreamState, hook_router, workstream_router,};-use ai_memory_llm::{Embedder, LlmProvider, ProviderHealth, build_embedder};+use ai_memory_llm::{Embedder, EmbeddingCapacity, LlmProvider, ProviderHealth, build_embedder};use ai_memory_mcp::human_auth::{Cidr, HumanAuthRuntime, LoginLimiter};use ai_memory_mcp::{AdminState, AiMemoryServer, ScopeInvalidation, admin_router_with_sweep_tuning,@@ -2916,6 +2916,7 @@ async fn configure_embedder(return Ok((wiki, None));};let provider_name = cfg.provider.name().to_string();+ let is_local = cfg.provider == ai_memory_llm::EmbedderChoice::Local;let model = cfg.model.clone();let dim = cfg.dim;let defaulted = cfg.defaulted;@@ -3007,6 +3008,14 @@ async fn configure_embedder("embedder enabled");let embedder = provider_health.wrap_embedder(embedder, provider_name, model, dim);+ let embedder: Arc<dyn Embedder> = if is_local {+ // all-MiniLM inference parallelizes internally. A single background+ // document slot plus one reserved query slot avoids CPU oversubscription+ // while keeping interactive vector retrieval available during writes.+ Arc::new(EmbeddingCapacity::new(embedder, 2)?)+ } else {+ embedder+ };Ok((wiki.with_embedder(embedder.clone()), Some(embedder)))}diff --git a/crates/ai-memory-consolidate/tests/suite/local_embeddings.rs b/crates/ai-memory-consolidate/tests/suite/local_embeddings.rsindex 00a61187..15696401 100644--- a/crates/ai-memory-consolidate/tests/suite/local_embeddings.rs+++ b/crates/ai-memory-consolidate/tests/suite/local_embeddings.rs@@ -18,9 +18,10 @@use std::path::Path;use std::sync::Arc;+use std::time::{Duration, Instant};use ai_memory_core::{PagePath, Tier};-use ai_memory_llm::{Embedder, LocalEmbedder};+use ai_memory_llm::{Embedder, EmbeddingCapacity, LocalEmbedder};use ai_memory_store::Store;use ai_memory_wiki::{Wiki, WritePageRequest};use tempfile::TempDir;@@ -268,6 +269,131 @@ async fn embedding_is_deterministic() {assert_eq!(a, b, "same input must produce identical vectors");}+fn percentile(mut samples: Vec<Duration>, percentile: usize) -> Duration {+ samples.sort_unstable();+ let index = (samples.len() * percentile).div_ceil(100).saturating_sub(1);+ samples[index]+}++async fn measure_queries(embedder: &Arc<dyn Embedder>, count: usize) -> Vec<Duration> {+ let mut samples = Vec::with_capacity(count);+ for sequence in 0..count {+ let started = Instant::now();+ embedder+ .embed_query(&format!(+ "where is the deployment runbook for production? sample {sequence}"+ ))+ .await+ .unwrap();+ samples.push(started.elapsed());+ }+ samples+}++async fn measure_queries_under_document_load(+ embedder: &Arc<dyn Embedder>,+ workers: usize,+ document: &str,+ query_samples: usize,+ documents_per_worker: usize,+) -> Vec<Duration> {+ let barrier = Arc::new(tokio::sync::Barrier::new(workers + 1));+ let mut background = Vec::with_capacity(workers);+ for worker in 0..workers {+ let embedder = Arc::clone(embedder);+ let barrier = Arc::clone(&barrier);+ let document = document.to_owned();+ background.push(tokio::spawn(async move {+ barrier.wait().await;+ for sequence in 0..documents_per_worker {+ embedder+ .embed_document(&format!("{document} worker {worker} item {sequence}"))+ .await+ .unwrap();+ }+ }));+ }++ barrier.wait().await;+ let loaded = measure_queries(embedder, query_samples).await;+ for task in background {+ task.await.unwrap();+ }+ loaded+}++/// Manual capacity probe for the shared local embedder. It is deliberately a+/// benchmark, not a CI assertion: CPU topology and scheduler load determine+/// the ratio. Run it before adding an admission-control policy so the policy+/// responds to measured contention rather than an assumed bottleneck.+#[tokio::test(flavor = "multi_thread")]+#[ignore = "manual local-model contention benchmark"]+async fn local_query_latency_under_document_load() {+ const QUERY_SAMPLES: usize = 9;+ const DOCUMENTS_PER_WORKER: usize = 12;++ let embedder: Arc<dyn Embedder> =+ Arc::new(LocalEmbedder::load(Path::new(&models_root())).unwrap());+ embedder.embed_query("warm the local model").await.unwrap();++ let idle = measure_queries(&embedder, QUERY_SAMPLES).await;+ let workers = std::env::var("AI_MEMORY_EMBED_BENCH_WORKERS")+ .ok()+ .and_then(|value| value.parse().ok())+ .unwrap_or_else(|| {+ std::thread::available_parallelism()+ .map(usize::from)+ .unwrap_or(2)+ .clamp(2, 8)+ });+ assert!(workers > 0, "benchmark needs at least one worker");+ let document = "A long-lived project memory page describes architecture decisions, \+ deployment procedures, incident recovery, database migrations, authentication \+ boundaries, contributor workflow, release validation, and operational runbooks. "+ .repeat(32);+ let loaded = measure_queries_under_document_load(+ &embedder,+ workers,+ &document,+ QUERY_SAMPLES,+ DOCUMENTS_PER_WORKER,+ )+ .await;+ let bounded: Arc<dyn Embedder> = Arc::new(EmbeddingCapacity::new(embedder, 2).unwrap());+ let bounded_loaded = measure_queries_under_document_load(+ &bounded,+ workers,+ &document,+ QUERY_SAMPLES,+ DOCUMENTS_PER_WORKER,+ )+ .await;++ let idle_p50 = percentile(idle.clone(), 50);+ let idle_p95 = percentile(idle, 95);+ let loaded_p50 = percentile(loaded.clone(), 50);+ let loaded_p95 = percentile(loaded, 95);+ let bounded_p50 = percentile(bounded_loaded.clone(), 50);+ let bounded_p95 = percentile(bounded_loaded, 95);+ eprintln!(+ "local embedding contention: workers={workers} idle_p50_ms={} \+ idle_p95_ms={} loaded_p50_ms={} loaded_p95_ms={} \+ loaded_p50_ratio={:.2} loaded_p95_ratio={:.2} \+ bounded_p50_ms={} bounded_p95_ms={} bounded_p50_ratio={:.2} \+ bounded_p95_ratio={:.2}",+ idle_p50.as_millis(),+ idle_p95.as_millis(),+ loaded_p50.as_millis(),+ loaded_p95.as_millis(),+ loaded_p50.as_secs_f64() / idle_p50.as_secs_f64(),+ loaded_p95.as_secs_f64() / idle_p95.as_secs_f64(),+ bounded_p50.as_millis(),+ bounded_p95.as_millis(),+ bounded_p50.as_secs_f64() / idle_p50.as_secs_f64(),+ bounded_p95.as_secs_f64() / idle_p95.as_secs_f64(),+ );+}+/// A handful of synonym/paraphrase orderings beyond the single case in/// the unit test — cheap confidence that the pooling isn't degenerate.#[tokio::test]diff --git a/crates/ai-memory-llm/src/embedding.rs b/crates/ai-memory-llm/src/embedding.rsindex ac8c699a..1bc2252e 100644--- a/crates/ai-memory-llm/src/embedding.rs+++ b/crates/ai-memory-llm/src/embedding.rs@@ -120,6 +120,14 @@ pub async fn embed_search_query(embedder: &dyn Embedder, query: &str) -> Option<model: embedder.model_identity(),dim: embedder.dim(),}),+ Err(LlmError::CapacityBusy(_)) => {+ debug!(+ provider = embedder.provider(),+ model = embedder.model(),+ "embedding capacity busy; degrading search to FTS5 + entity + graph"+ );+ None+ }Err(e) => {tracing::warn!(provider = embedder.provider(),diff --git a/crates/ai-memory-llm/src/embedding_capacity.rs b/crates/ai-memory-llm/src/embedding_capacity.rsnew file mode 100644index 00000000..c825fef1--- /dev/null+++ b/crates/ai-memory-llm/src/embedding_capacity.rs@@ -0,0 +1,336 @@+//! Capacity isolation for in-process embedding inference.+//!+//! Local model inference is CPU-heavy and already parallelizes internally.+//! Letting background page writes fill every inference slot can more than+//! double interactive query latency. [`EmbeddingCapacity`] bounds total work+//! while reserving one slot for query embeddings.++use std::sync::Arc;++use async_trait::async_trait;+use tokio::sync::{OwnedSemaphorePermit, Semaphore};++use crate::{Embedder, LlmError, LlmResult};++/// Embedder wrapper that bounds inference and reserves foreground capacity.+///+/// `capacity` must be at least two. Document and generic embeddings may use+/// at most `capacity - 1` slots, so an interactive query can enter without+/// waiting for background indexing work to finish.+pub struct EmbeddingCapacity {+ inner: Arc<dyn Embedder>,+ total: Arc<Semaphore>,+ documents: Arc<Semaphore>,+}++impl EmbeddingCapacity {+ /// Wrap `inner` with a bounded scheduler.+ ///+ /// # Errors+ /// Returns [`LlmError::NotConfigured`] when fewer than two slots are+ /// requested because that cannot reserve foreground capacity.+ pub fn new(inner: Arc<dyn Embedder>, capacity: usize) -> LlmResult<Self> {+ if capacity < 2 {+ return Err(LlmError::NotConfigured(+ "embedding capacity must be at least 2".into(),+ ));+ }+ Ok(Self {+ inner,+ total: Arc::new(Semaphore::new(capacity)),+ documents: Arc::new(Semaphore::new(capacity - 1)),+ })+ }++ async fn acquire_document_capacity(+ &self,+ ) -> LlmResult<(OwnedSemaphorePermit, OwnedSemaphorePermit)> {+ let document = self+ .documents+ .clone()+ .acquire_owned()+ .await+ .map_err(|_| capacity_closed())?;+ let total = self+ .total+ .clone()+ .acquire_owned()+ .await+ .map_err(|_| capacity_closed())?;+ Ok((document, total))+ }++ async fn finish_task(+ task: tokio::task::JoinHandle<LlmResult<Vec<f32>>>,+ ) -> LlmResult<Vec<f32>> {+ task.await+ .map_err(|e| LlmError::UnexpectedShape(format!("embedding task: {e}")))?+ }+}++fn capacity_closed() -> LlmError {+ LlmError::UnexpectedShape("embedding capacity scheduler closed".into())+}++#[async_trait]+impl Embedder for EmbeddingCapacity {+ fn provider(&self) -> &'static str {+ self.inner.provider()+ }++ fn model(&self) -> &str {+ self.inner.model()+ }++ fn model_identity(&self) -> String {+ self.inner.model_identity()+ }++ fn dim(&self) -> u32 {+ self.inner.dim()+ }++ async fn embed(&self, text: &str) -> LlmResult<Vec<f32>> {+ let capacity = self.acquire_document_capacity().await?;+ let inner = Arc::clone(&self.inner);+ let text = text.to_owned();+ Self::finish_task(tokio::spawn(async move {+ let _capacity = capacity;+ inner.embed(&text).await+ }))+ .await+ }++ async fn embed_document(&self, text: &str) -> LlmResult<Vec<f32>> {+ let capacity = self.acquire_document_capacity().await?;+ let inner = Arc::clone(&self.inner);+ let text = text.to_owned();+ Self::finish_task(tokio::spawn(async move {+ let _capacity = capacity;+ inner.embed_document(&text).await+ }))+ .await+ }++ async fn embed_query(&self, text: &str) -> LlmResult<Vec<f32>> {+ let capacity = self+ .total+ .clone()+ .try_acquire_owned()+ .map_err(|_| LlmError::CapacityBusy("embedding"))?;+ let inner = Arc::clone(&self.inner);+ let text = text.to_owned();+ Self::finish_task(tokio::spawn(async move {+ let _capacity = capacity;+ inner.embed_query(&text).await+ }))+ .await+ }+}++#[cfg(test)]+mod tests {+ use std::time::Duration;++ use tokio::sync::mpsc;++ use super::*;++ struct BlockingEmbedder {+ entered: mpsc::UnboundedSender<&'static str>,+ releases: Arc<Semaphore>,+ }++ impl BlockingEmbedder {+ async fn block(&self, operation: &'static str) -> LlmResult<Vec<f32>> {+ self.entered.send(operation).unwrap();+ self.releases+ .acquire()+ .await+ .map_err(|_| capacity_closed())?+ .forget();+ Ok(vec![1.0])+ }+ }++ #[async_trait]+ impl Embedder for BlockingEmbedder {+ fn provider(&self) -> &'static str {+ "blocking"+ }++ fn model(&self) -> &str {+ "test"+ }++ fn model_identity(&self) -> String {+ "document-identity".into()+ }++ fn dim(&self) -> u32 {+ 1+ }++ async fn embed(&self, _text: &str) -> LlmResult<Vec<f32>> {+ self.block("generic").await+ }++ async fn embed_document(&self, _text: &str) -> LlmResult<Vec<f32>> {+ self.block("document").await+ }++ async fn embed_query(&self, _text: &str) -> LlmResult<Vec<f32>> {+ self.block("query").await+ }+ }++ #[test]+ fn requires_a_foreground_reservation() {+ let (entered, _entered_rx) = mpsc::unbounded_channel();+ let inner = Arc::new(BlockingEmbedder {+ entered,+ releases: Arc::new(Semaphore::new(0)),+ });+ assert!(EmbeddingCapacity::new(inner, 1).is_err());+ }++ #[tokio::test]+ async fn query_enters_while_second_document_waits() {+ let (entered, mut entered_rx) = mpsc::unbounded_channel();+ let releases = Arc::new(Semaphore::new(0));+ let inner = Arc::new(BlockingEmbedder {+ entered,+ releases: Arc::clone(&releases),+ });+ let stage: Arc<dyn Embedder> = Arc::new(EmbeddingCapacity::new(inner, 2).unwrap());++ let first_document = {+ let stage = Arc::clone(&stage);+ tokio::spawn(async move { stage.embed_document("one").await })+ };+ assert_eq!(entered_rx.recv().await, Some("document"));++ let second_document = {+ let stage = Arc::clone(&stage);+ tokio::spawn(async move { stage.embed_document("two").await })+ };+ let query = {+ let stage = Arc::clone(&stage);+ tokio::spawn(async move { stage.embed_query("find it").await })+ };++ assert_eq!(+ tokio::time::timeout(Duration::from_secs(1), entered_rx.recv())+ .await+ .unwrap(),+ Some("query"),+ "the reserved query slot must remain available"+ );+ assert!(+ crate::embed_search_query(stage.as_ref(), "another query")+ .await+ .is_none(),+ "a saturated query must degrade immediately to non-vector search"+ );+ assert!(+ tokio::time::timeout(Duration::from_millis(25), entered_rx.recv())+ .await+ .is_err(),+ "the second document must wait for the document slot"+ );++ releases.add_permits(1);+ assert_eq!(+ tokio::time::timeout(Duration::from_secs(1), entered_rx.recv())+ .await+ .unwrap(),+ Some("document")+ );+ releases.add_permits(2);+ first_document.await.unwrap().unwrap();+ second_document.await.unwrap().unwrap();+ query.await.unwrap().unwrap();+ }++ #[tokio::test]+ async fn cancelled_caller_does_not_release_capacity_before_inference_finishes() {+ let (entered, mut entered_rx) = mpsc::unbounded_channel();+ let releases = Arc::new(Semaphore::new(0));+ let stage: Arc<dyn Embedder> = Arc::new(+ EmbeddingCapacity::new(+ Arc::new(BlockingEmbedder {+ entered,+ releases: Arc::clone(&releases),+ }),+ 2,+ )+ .unwrap(),+ );++ let cancelled = {+ let stage = Arc::clone(&stage);+ tokio::spawn(async move { stage.embed_document("cancel me").await })+ };+ assert_eq!(entered_rx.recv().await, Some("document"));+ cancelled.abort();+ assert!(cancelled.await.unwrap_err().is_cancelled());++ let next = {+ let stage = Arc::clone(&stage);+ tokio::spawn(async move { stage.embed_document("wait for capacity").await })+ };+ assert!(+ tokio::time::timeout(Duration::from_millis(25), entered_rx.recv())+ .await+ .is_err(),+ "cancelled caller must not free a still-running inference slot"+ );++ releases.add_permits(1);+ assert_eq!(+ tokio::time::timeout(Duration::from_secs(1), entered_rx.recv())+ .await+ .unwrap(),+ Some("document")+ );+ releases.add_permits(1);+ next.await.unwrap().unwrap();+ }++ #[tokio::test]+ async fn generic_embedding_keeps_inner_method_semantics() {+ let (entered, mut entered_rx) = mpsc::unbounded_channel();+ let releases = Arc::new(Semaphore::new(0));+ let stage = Arc::new(+ EmbeddingCapacity::new(+ Arc::new(BlockingEmbedder {+ entered,+ releases: Arc::clone(&releases),+ }),+ 2,+ )+ .unwrap(),+ );+ let task = tokio::spawn(async move { stage.embed("generic input").await });+ assert_eq!(entered_rx.recv().await, Some("generic"));+ releases.add_permits(1);+ task.await.unwrap().unwrap();+ }++ #[test]+ fn delegates_embedding_identity() {+ let (entered, _entered_rx) = mpsc::unbounded_channel();+ let stage = EmbeddingCapacity::new(+ Arc::new(BlockingEmbedder {+ entered,+ releases: Arc::new(Semaphore::new(0)),+ }),+ 2,+ )+ .unwrap();+ assert_eq!(stage.provider(), "blocking");+ assert_eq!(stage.model(), "test");+ assert_eq!(stage.model_identity(), "document-identity");+ assert_eq!(stage.dim(), 1);+ }+}diff --git a/crates/ai-memory-llm/src/error.rs b/crates/ai-memory-llm/src/error.rsindex 66cde7c9..5c13a9dc 100644--- a/crates/ai-memory-llm/src/error.rs+++ b/crates/ai-memory-llm/src/error.rs@@ -65,6 +65,12 @@ pub enum LlmError {#[error("auth: {0}")]Auth(String),+ /// Optional local work could not start without waiting behind already+ /// active work. Callers should use their non-provider fallback instead of+ /// retrying or reporting the provider unhealthy.+ #[error("{0} capacity is busy")]+ CapacityBusy(&'static str),+/// JSON schema for structured output could not be derived.#[error("schema: {0}")]Schema(String),@@ -127,6 +133,7 @@ impl LlmError {Self::EmptyContent { .. } => "empty-content",Self::NotConfigured(_) => "not-configured",Self::Auth(_) => "auth",+ Self::CapacityBusy(_) => "capacity-busy",Self::Schema(_) => "schema",Self::AllCandidatesFailed { .. } => "all-candidates-failed",}@@ -178,6 +185,7 @@ mod tests {#[test]fn deterministic_errors_are_not_transient() {assert!(!LlmError::Auth("expired".into()).is_transient());+ assert!(!LlmError::CapacityBusy("embedding").is_transient());assert!(!LlmError::Schema("bad".into()).is_transient());assert!(!LlmError::Serde("nope".into()).is_transient());assert!(!LlmError::UnexpectedShape("no tool block".into()).is_transient());@@ -274,6 +282,7 @@ mod tests {),(LlmError::NotConfigured("no key".into()), "not-configured"),(LlmError::Auth("expired".into()), "auth"),+ (LlmError::CapacityBusy("embedding"), "capacity-busy"),(LlmError::Schema("bad".into()), "schema"),(LlmError::AllCandidatesFailed {diff --git a/crates/ai-memory-llm/src/lib.rs b/crates/ai-memory-llm/src/lib.rsindex ce9a70c7..6b47ec5c 100644--- a/crates/ai-memory-llm/src/lib.rs+++ b/crates/ai-memory-llm/src/lib.rs@@ -79,6 +79,7 @@ pub mod codex;pub mod copilot;pub mod cursor_agent;pub mod embedding;+pub mod embedding_capacity;pub mod error;pub mod factory;pub mod fallback;@@ -121,6 +122,7 @@ pub use embedding::{Embedder, OpenAiCompatEmbedder, OpenAiEmbedder, QueryVector, SyntheticEmbedder, VoyageEmbedder,cosine, embed_search_query,};+pub use embedding_capacity::EmbeddingCapacity;pub use error::{LlmError, LlmResult};pub use factory::{EmbedderChoice, EmbedderConfig, ProviderChoice, ProviderConfig, build_embedder, build_provider,diff --git a/docs/comparison.md b/docs/comparison.mdindex 7634f2aa..e0148095 100644--- a/docs/comparison.md+++ b/docs/comparison.md@@ -364,9 +364,10 @@ ships something ai-memory does not, it is named.packaging, its `memu-agent detect` for unknown agents, and its hosted cloud,and EverOS's ANN vector index, optional Milvus backend and publishedend-to-end benchmark. EverOS v1.4.2 also reserves query-embedding capacity- from background writes; ai-memory's reranker fails open under saturation,- but embedding backfill and foreground query embedding do not yet have an- equivalent measured capacity split.+ from background writes. ai-memory now does the same for in-process local+ inference, backed by an opt-in real-model contention probe; hosted embedding+ providers keep their existing concurrency, and the reranker remains bounded+ and fail-open.- **From Caura (governed multi-agent fleet memory):** a different buyer. Cauraoptimizes org-scale fleet governance (agent/team/org visibility scopes, fourtrust tiers, an audit log, PII flagging) on Postgres+pgvector with LLM@@ -392,19 +393,19 @@ ships something ai-memory does not, it is named.## Where we're behind, or different by choice-- **No human workstream operations view.** Managed workstreams, run leases,- handoffs, and portable event history exist, but `/web` does not turn them- into a scannable queue of active, blocked, review-ready, and completed work.- claude-mem v14's local “My Work” view is better for a human supervising- several requests.+- **Workstream operations are now visible to operators.** `/web/workstreams`+ presents trusted lifecycle states for managed workstreams without exposing+ checkout paths, native session ids, or transcript content. It deliberately+ does not infer blocked or review-ready states from untrusted prose.- **No structured page filters on search.** Scope and `as_of` are typed, but`memory_query` cannot narrow by path prefix, kind, tier, tag, or authorbefore ranking. Mem0 has directory/category scopes; basic-memory'sunreleased v0.24 adds a broader structured filter set.-- **Foreground embedding has no measured reservation from backfill.** The LLM- reranker is bounded and fails open, but query embedding and administrative- embedding backfill share the configured embedder without a dedicated- foreground budget. EverOS v1.4.2 shows the latency value of separating them.+- **Local foreground embedding is isolated from background writes.** A+ real-model probe demonstrated CPU contention, so local document inference is+ limited to one slot and a second slot remains available for query embedding.+ Hosted providers retain their existing concurrency policy; the LLM reranker+ remains separately bounded and fail-open.- **Marketplace reach is narrower than harness support.** ai-memory now shipsits own Claude Code and Codex marketplace plugins and `ai-memory setup`detects and wires installed agents across Linux, macOS, and Windows. It isdiff --git a/docs/competitive-parity.md b/docs/competitive-parity.mdindex f6b15993..4d6ed14e 100644--- a/docs/competitive-parity.md+++ b/docs/competitive-parity.md@@ -281,7 +281,7 @@ invariants), **ALREADY** (we have it and should say so).| Capability seen in the field | Verdict | Why ||---|---|---|| Human workstream operations view (claude-mem v14 “My Work”) | ADOPT | The managed-workstream ledger already has runs, leases, events, and handoff evidence, but humans cannot scan active/attention/review-ready work in `/web`. Build a read-only derived view; add only a minimal review marker, not a second task system |-| Foreground query capacity isolated from embedding backfill (EverOS v1.4.2) | MEASURE-FIRST, then ADOPT if contention is real | The reranker already has a shared fail-open gate. Query embedding and administrative backfill share the embedder without a foreground reservation. Add a contention benchmark; preserve lexical/entity/graph results under saturation |+| Foreground query capacity isolated from embedding backfill (EverOS v1.4.2) | SHIPPED (3.2) for local inference | A real-model probe measured eight-worker query p50/p95 rising from 36/41 ms idle to 74/76 ms. Limiting local document inference to one slot while reserving one query slot reduced loaded latency to 39/51 ms. Hosted providers keep their existing concurrency; provider failures still degrade to lexical/entity/graph retrieval || Structured pre-rank filters (Mem0; basic-memory v0.24 unreleased) | ADOPT | Bounded path-prefix, kind, tier, tag, author, and valid-time filters improve monorepo/tool ergonomics without changing ranking. All paths must reuse `ScopeResolver` and fail closed on incomplete scope || Per-operation LLM provider/model profiles (Graphiti v0.31; Hindsight `main`) | MEASURE-FIRST | Potentially lowers cost and latency for consolidation, rerank, and auto-improvement, but expands config and auth surface. Prototype against the evals before adding public settings || Retrieval recall through capture (several tools rank whole raw sessions) | SHIPPED (3.0) | The 2026-10-12 decomposition showed our zero-LLM session page indexed only 80-character turn titles; the observation stream recovered it (FTS hit@5 0.662 → 0.951, local 0.800 → 0.945, all 470). Chasing another tool's headline number is declined: those numbers measure different things |@@ -323,8 +323,8 @@ which should be checked before any work is scoped.2. **Structured filters before ranking** for path prefix, kind, tier, tag,author, and valid time (Mem0; basic-memory v0.24, currently unreleased).3. **Foreground embedding isolation backed by a contention benchmark**- (EverOS v1.4.2). Reserve query capacity only if the benchmark shows- administrative backfill can harm interactive latency.+ (EverOS v1.4.2) is now shipped for the in-process local model. The manual+ probe remains in-tree so the capacity can be rechecked on other CPU classes.4. **One reproducible end-to-end QA number** beside hit@5, with reader,judge, context tokens and latency stated, and more benchmarks thanLongMemEval (Mem0, Zep, Hindsight and EverOS publish QA numbers; Hindsight'sdiff --git a/docs/local-embeddings.md b/docs/local-embeddings.mdindex 3c9de46a..e4d2fbfc 100644--- a/docs/local-embeddings.md+++ b/docs/local-embeddings.md@@ -104,9 +104,26 @@ cargo run --release -p ai-memory-eval -- retrieval --embeddings local- Inference is CPU, off the async runtime (`spawn_blocking`);first-load reads ~87 MB into memory once per server process.+- Local inference has two process-wide slots. Background document embeddings+ use one slot, leaving the other available to search queries. This avoids+ page writes and administrative backfill crowding interactive recall off the+ CPU. A query that arrives while both slots are serving queries falls back+ immediately to lexical/entity/graph retrieval instead of building an+ unbounded queue. Hosted embedding providers are not subject to this local+ scheduler.- The `local-embeddings` cargo feature (default on) carries the MLdependency tree; a slim build can disable it and keep every otherprovider.- Tokenization truncates at 512 tokens, the model's positional limit.Longer page bodies contribute only their head, the same contract as thehosted embedders.++The opt-in contention probe compares the raw local embedder with that capacity+split. It needs the downloaded model and is not part of the routine test tier:++```bash+AI_MEMORY_TEST_MODELS_DIR=$PWD/evals/models \+AI_MEMORY_EMBED_BENCH_WORKERS=8 \+cargo test -p ai-memory-consolidate \+ local_query_latency_under_document_load -- --ignored --nocapture+```diff --git a/docs/research-2026-landscape.md b/docs/research-2026-landscape.mdindex 31568718..30370e3f 100644--- a/docs/research-2026-landscape.md+++ b/docs/research-2026-landscape.md@@ -688,7 +688,7 @@ explicitly.| **Memori** | No commits since 2026-10-03 and no new release. | No material change. || **memvid** | No commits since 2026-07-14 and no new release. | Remains quiet; its relative benchmark claim is still not independently useful. || **memU** | No feature release. Four Windows path/idempotency fixes landed. | No material change. |-| **EverOS** | **Released v1.4.2.** Fixes an FD leak, separates query embedding capacity from background writes, batches embeddings, keys indexes by the full scope tuple, self-repairs damaged indexes, and hardens path containment. | The capacity split is worth copying in principle. ai-memory's reranker already fails open behind a shared gate, but query embedding and administrative backfill share the embedder without a foreground reservation. Add a contention benchmark before changing it. |+| **EverOS** | **Released v1.4.2.** Fixes an FD leak, separates query embedding capacity from background writes, batches embeddings, keys indexes by the full scope tuple, self-repairs damaged indexes, and hardens path containment. | Adopted for the in-process local embedder after measurement: eight background workers raised query p50/p95 from 36/41 ms to 74/76 ms on the reference host; one document slot plus one reserved query slot brought them to 39/51 ms. Hosted providers and the reranker retain their existing policies. || **MemOS** | No commits or release after the prior cutoff. Its current headline remains DeepSeek Harness integration, multimodal memory, memory cubes, asynchronous ingestion, and feedback/correction. | No material change. The memory-OS and multimodal tradeoffs remain different from ai-memory's scope. || **Honcho** | **Released v3.3.0.** Adds an optional Chroma backend, scoped custom instructions, cursor pagination, prompt-budget handling, and performance fixes; stable object `uid` work followed on `main`. | No core work. Its theory-of-mind buyer remains different. Operation-scoped instructions are a useful reference if per-operation LLM profiles are designed. || **Engram** | No new tag after v3.4.0. `main` adds local aliases/routing preview, project provenance, compact search/history clients, capture guards, tombstones, and sync isolation. | No new gap. ai-memory already has stronger repository identity, routing diagnostics, tombstone/supersession semantics, and conflict-aware wiki sync. |@@ -870,11 +870,12 @@ Each item needs its own design and approval before implementation.workstreams when the schema has no such relationship. This borrowsclaude-mem's useful human visibility without turning ai-memory into a kanbansystem or cloud collaboration product.-2. **P1 - Protect interactive recall from background embedding.** Build a- contention benchmark first. If administrative backfill can delay query- embedding, give foreground queries reserved capacity or separate gates.- Preserve the current fail-open behavior: lexical/entity/graph retrieval- must still answer when embedding or reranking is saturated.+2. **P1 - Protect interactive recall from background embedding. Completed.**+ The opt-in real-model probe found material local CPU contention, so local+ inference now admits one document embedding and reserves a second slot for+ queries. The same probe verifies the improvement; hosted providers remain+ unchanged, and provider failures still degrade to lexical/entity/graph+ retrieval.3. **P2 - Structured query filters.** Expose the existing pre-rank kind/tierfilters through `memory_query`, then add bounded path-prefix and exact-tagfilters. Keep `as_of` as the valid-time control and defer author until null