Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
fix(serve): stop retrying a deterministic provider failure on the session consolidation queue
Rudy Celeklibca9540250ce
diff --git a/crates/ai-memory-cli/src/commands/serve.rs b/crates/ai-memory-cli/src/commands/serve.rsindex 0e30237f..ab9cafe3 100644--- a/crates/ai-memory-cli/src/commands/serve.rs+++ b/crates/ai-memory-cli/src/commands/serve.rs@@ -855,27 +855,31 @@ fn session_consolidation_retry_delay(attempt: u32) -> Duration {/// Failures that must terminate a SessionEnd queue job: re-sending the same/// expensive prompt cannot clear them.///-/// Deterministic structured-response failures — malformed JSON (`Serde`), an-/// unexpected response shape (`UnexpectedShape`), an output-budget stop-/// (`TruncatedResponse`), or empty content (`EmptyContent`) — reproduce on-/// identical inputs, so the job goes terminal and the heuristic page the-/// SessionEnd hook already wrote remains. Pre-send connection failures and-/// provider statuses a later attempt may plausibly clear (`429`, a `5xx`-/// such as a capacity rejection) never reached the model yet and keep the-/// queue's bounded, backoff-scheduled retry up to the attempt cap.+/// An LLM failure defers to [`ai_memory_llm::LlmError::is_transient`], which+/// is the same predicate the two in-process consolidation retry loops already+/// gate on — `complete_structured_with_retry` in `ai-memory-consolidate`'s+/// `consolidator.rs` and the bootstrap chunk loop in its `bootstrap.rs`. One+/// definition of "worth another expensive call" then covers all three retry+/// sites: a provider `429` or `5xx` and an `Http` timeout/connect failure+/// never reached the model, so they keep the queue's bounded,+/// backoff-scheduled retry up to the attempt cap, while everything else — a+/// malformed (`Serde`) or unexpected (`UnexpectedShape`) structured response,+/// an output-budget stop (`TruncatedResponse`), empty content+/// (`EmptyContent`), a rejected or expired credential (`Auth`), a missing+/// provider configuration (`NotConfigured`), a schema failure (`Schema`), a+/// malformed-request `4xx` — reproduces on identical inputs, so the job goes+/// terminal and the heuristic page the SessionEnd hook already wrote remains.+///+/// A `ConsolidatorError::Serde` is not an `LlmError`: it is this crate's own+/// JSON failure on an unchanged input and stays terminal on its own.fn is_terminal_session_consolidation_error(error: &ai_memory_consolidate::ConsolidatorError,) -> bool {- matches!(- error,- ai_memory_consolidate::ConsolidatorError::Serde(_)- | ai_memory_consolidate::ConsolidatorError::Llm(- ai_memory_llm::LlmError::Serde(_)- | ai_memory_llm::LlmError::UnexpectedShape(_)- | ai_memory_llm::LlmError::TruncatedResponse { .. }- | ai_memory_llm::LlmError::EmptyContent { .. }- )- )+ match error {+ ai_memory_consolidate::ConsolidatorError::Llm(llm) => !llm.is_transient(),+ ai_memory_consolidate::ConsolidatorError::Serde(_) => true,+ _ => false,+ }}/// Delay before the first profile pass, so it never competes with migration@@ -4980,6 +4984,14 @@ mod tests {TruncatedResponse,/// The provider returned HTTP 2xx with no usable content.EmptyContent,+ /// The configured credential is rejected or expired: the same key+ /// fails identically on every later attempt (deterministic).+ ExpiredAuth,+ /// The provider rejected the request itself with a 4xx other than+ /// `429`, which an unchanged prompt reproduces (deterministic).+ MalformedRequest,+ /// The provider is rate-limiting: a later attempt may clear it.+ RateLimited,}/// Structured-output failure fixture: counts calls (proving the prompt@@ -5037,6 +5049,15 @@ mod tests {ConsolidationFailure::EmptyContent => LlmError::EmptyContent {model: "test-model".into(),},+ ConsolidationFailure::ExpiredAuth => LlmError::Auth("api key expired".into()),+ ConsolidationFailure::MalformedRequest => LlmError::Provider {+ status: 400,+ body: "bad request".into(),+ },+ ConsolidationFailure::RateLimited => LlmError::Provider {+ status: 429,+ body: "rate limited".into(),+ },};Err(error)})@@ -5044,7 +5065,9 @@ mod tests {}// The classifier decides the queue's terminal set by variant, never by- // message text: every deterministic structured-response failure is+ // message text, and for an LLM failure it decides it with+ // `LlmError::is_transient` — the predicate the in-process consolidation+ // and bootstrap retry loops already use. Every deterministic failure is// terminal at any queue attempt, and every failure a later attempt may// plausibly clear is not.#[tokio::test]@@ -5062,6 +5085,21 @@ mod tests {ConsolidatorError::Llm(LlmError::EmptyContent {model: "test-model".into(),}),+ // Deterministic provider-side failures the queue must not re-send+ // either: the credential, the missing configuration, the derived+ // schema and a malformed-request 4xx all fail identically on the+ // next attempt.+ ConsolidatorError::Llm(LlmError::Auth("expired".into())),+ ConsolidatorError::Llm(LlmError::NotConfigured("no key".into())),+ ConsolidatorError::Llm(LlmError::Schema("bad".into())),+ ConsolidatorError::Llm(LlmError::Provider {+ status: 401,+ body: "invalid api key".into(),+ }),+ ConsolidatorError::Llm(LlmError::Provider {+ status: 400,+ body: "bad request".into(),+ }),];for error in &terminal {assert!(@@ -5086,6 +5124,10 @@ mod tests {// Pre-send and provider-status failures a later attempt may plausibly// clear keep the queue's bounded backoff, whatever the message says.let not_terminal = [+ ConsolidatorError::Llm(LlmError::Provider {+ status: 429,+ body: "rate limited".into(),+ }),ConsolidatorError::Llm(LlmError::Provider {status: 503,body: "capacity".into(),@@ -5095,8 +5137,6 @@ mod tests {body: "bad gateway".into(),}),ConsolidatorError::Llm(LlmError::Http(connect)),- ConsolidatorError::Llm(LlmError::NotConfigured("no key".into())),- ConsolidatorError::Llm(LlmError::Auth("expired".into())),];for error in ¬_terminal {assert!(@@ -5594,8 +5634,8 @@ mod tests {// Mutation captured: persisting the error's `Display` writes the private// provider body into the queue row; the worker must persist only the- // redacted class/status summary, and a failed first attempt keeps the- // queue's scheduled retry instead of flipping terminal.+ // redacted class/status summary, and a deterministic `400` settles the+ // job terminal instead of re-sending the same prompt.#[tokio::test]async fn session_end_worker_persists_redacted_summary_not_provider_body() {let tmp = TempDir::new().unwrap();@@ -5677,10 +5717,10 @@ mod tests {// lands.let db = rusqlite::Connection::open(store.db_path()).unwrap();let deadline = std::time::Instant::now() + Duration::from_secs(5);- let (state, last_error, next_attempt_at) = loop {+ let (state, last_error, completed_at) = loop {let row = db.query_row(- "SELECT state, last_error, next_attempt_at \+ "SELECT state, last_error, completed_at \FROM session_consolidation_jobs WHERE session_id = ?1",rusqlite::params![session_id.as_bytes()],|r| {@@ -5717,13 +5757,18 @@ mod tests {!last_error.contains("SENTINEL_PRIVATE_BODY"),"provider body leaked into the queue row: {last_error}");+ // A malformed-request `400` is deterministic (`LlmError::is_transient`+ // is false for every 4xx but `429`), so the job settles terminal on+ // the first attempt rather than re-sending the same prompt. The+ // terminal update stamps `completed_at` and leaves `state` out of+ // `pending`, which is what the claim query filters on.assert_eq!(- state, "pending",- "a failed first attempt must keep the queue's retry"+ state, "failed",+ "a deterministic 4xx must settle the job terminal");assert!(- next_attempt_at.is_some(),- "the retryable failure must keep a scheduled retry"+ completed_at.is_some(),+ "a terminal job must be stamped completed");}@@ -5907,16 +5952,36 @@ mod tests {assert!(body.contains("Durable worker completed"), "{body}");}+ /// What the SessionEnd queue must do with a failure.+ #[derive(Clone, Copy)]+ enum QueueOutcome {+ /// The job settles terminal: no later claim ever returns it again.+ Terminal,+ /// The job keeps the bounded backoff: a claim past the first 30s+ /// backoff returns it as the next attempt.+ Requeued,+ }+/// Run the real SessionEnd worker against a session whose structured LLM- /// call fails with `failure`, and prove the queue item goes terminal on- /// the first attempt: the prompt goes out exactly once (no inner replay,- /// no queue re-send), the job settles with no `next_attempt_at` — a claim- /// far enough in the future that a re-queued job (first backoff is 30s)- /// would be claimable finds none — and the heuristic page the SessionEnd- /// hook wrote before enqueuing is untouched. The settle window (5 × 100ms)- /// is measured: a writer round-trip is millisecond-scale, so probing past- /// it is deterministic for a single-row SQLite update.- async fn session_end_worker_terminates_on(failure: ConsolidationFailure) {+ /// call fails with `failure`, and prove the queue settles the job as+ /// `outcome` after exactly `expected_calls` prompts.+ ///+ /// For [`QueueOutcome::Terminal`] the job settles with no+ /// `next_attempt_at` — a claim far enough in the future that a re-queued+ /// job (first backoff is 30s) would be claimable finds none — and no+ /// further prompt goes out. The settle window (5 × 100ms) is measured: a+ /// writer round-trip is millisecond-scale, so probing past it is+ /// deterministic for a single-row SQLite update. For+ /// [`QueueOutcome::Requeued`] that same future-clock claim returns the+ /// job as attempt 2.+ ///+ /// Either way the heuristic page the SessionEnd hook wrote before+ /// enqueuing is untouched.+ async fn session_end_worker_settles_as(+ failure: ConsolidationFailure,+ outcome: QueueOutcome,+ expected_calls: usize,+ ) {let tmp = TempDir::new().unwrap();let store = Store::open(tmp.path()).unwrap();let workspace_id = store@@ -6015,35 +6080,65 @@ mod tests {));notify.notify_one();- // The prompt must go out exactly once.- let deadline = std::time::Instant::now() + Duration::from_secs(5);- while calls.load(Ordering::SeqCst) < 1 {+ // The prompt must go out exactly `expected_calls` times. A transient+ // failure also spends the consolidator's own bounded inner retries+ // (2s apart) before the queue ever sees it, so allow for those.+ let deadline = std::time::Instant::now() + Duration::from_secs(30);+ while calls.load(Ordering::SeqCst) < expected_calls {assert!(std::time::Instant::now() < deadline,- "worker never called the LLM"+ "worker never finished calling the LLM");tokio::time::sleep(Duration::from_millis(10)).await;}// Let the writer settle the failure row, then probe with a clock 60s- // ahead: a re-queued job (first backoff 30s) would be claimable, a- // terminal row is not.- for _ in 0..5 {- tokio::time::sleep(Duration::from_millis(100)).await;- let probe_now = jiff::Timestamp::now().as_microsecond() + 60 * 1_000_000;- assert!(- store- .writer- .claim_session_consolidation(probe_now, probe_now - 10 * 60 * 1_000_000)- .await- .unwrap()- .is_none(),- "the failure must not re-queue the same prompt"- );- assert_eq!(- calls.load(Ordering::SeqCst),- 1,- "no immediate replay or queue re-send of the same prompt"- );+ // ahead: a re-queued job (first backoff 30s) is claimable there, a+ // terminal row never is.+ match outcome {+ QueueOutcome::Terminal => {+ for _ in 0..5 {+ tokio::time::sleep(Duration::from_millis(100)).await;+ let probe_now = jiff::Timestamp::now().as_microsecond() + 60 * 1_000_000;+ assert!(+ store+ .writer+ .claim_session_consolidation(probe_now, probe_now - 10 * 60 * 1_000_000)+ .await+ .unwrap()+ .is_none(),+ "the failure must not re-queue the same prompt"+ );+ assert_eq!(+ calls.load(Ordering::SeqCst),+ expected_calls,+ "no immediate replay or queue re-send of the same prompt"+ );+ }+ }+ QueueOutcome::Requeued => {+ let deadline = std::time::Instant::now() + Duration::from_secs(10);+ let requeued = loop {+ let probe_now = jiff::Timestamp::now().as_microsecond() + 60 * 1_000_000;+ if let Some(job) = store+ .writer+ .claim_session_consolidation(probe_now, probe_now - 10 * 60 * 1_000_000)+ .await+ .unwrap()+ {+ break job;+ }+ assert!(+ std::time::Instant::now() < deadline,+ "a failure a later attempt may clear must stay on the queue",+ );+ tokio::time::sleep(Duration::from_millis(50)).await;+ };+ assert_eq!(+ requeued.attempts(),+ 2,+ "the re-queued job must be the second attempt"+ );+ }}cancel.cancel();task.await.unwrap();@@ -6064,16 +6159,64 @@ mod tests {// must settle the job terminal after the single send.#[tokio::test]async fn session_end_worker_makes_malformed_json_and_unexpected_shape_terminal() {- session_end_worker_terminates_on(ConsolidationFailure::MalformedJson).await;- session_end_worker_terminates_on(ConsolidationFailure::UnexpectedShape).await;+ session_end_worker_settles_as(+ ConsolidationFailure::MalformedJson,+ QueueOutcome::Terminal,+ 1,+ )+ .await;+ session_end_worker_settles_as(+ ConsolidationFailure::UnexpectedShape,+ QueueOutcome::Terminal,+ 1,+ )+ .await;}// Mutation captured: removing either provider error from the terminal// queue set causes a second claim of the same truncated or empty output.#[tokio::test]async fn session_end_worker_makes_truncated_and_empty_output_terminal() {- session_end_worker_terminates_on(ConsolidationFailure::TruncatedResponse).await;- session_end_worker_terminates_on(ConsolidationFailure::EmptyContent).await;+ session_end_worker_settles_as(+ ConsolidationFailure::TruncatedResponse,+ QueueOutcome::Terminal,+ 1,+ )+ .await;+ session_end_worker_settles_as(+ ConsolidationFailure::EmptyContent,+ QueueOutcome::Terminal,+ 1,+ )+ .await;+ }++ /// A rejected credential and a malformed-request 4xx are deterministic:+ /// the identical prompt fails identically on every later attempt. Before+ /// the queue gate consulted `LlmError::is_transient` it hand-enumerated+ /// only the structured-response shapes, so one expired key burned the+ /// full attempt cap (5 prompts over ~7.5 minutes) per ended session and+ /// logged "queued for retry" four times on the way.+ #[tokio::test]+ async fn session_end_worker_makes_an_expired_key_and_a_4xx_terminal() {+ session_end_worker_settles_as(ConsolidationFailure::ExpiredAuth, QueueOutcome::Terminal, 1)+ .await;+ session_end_worker_settles_as(+ ConsolidationFailure::MalformedRequest,+ QueueOutcome::Terminal,+ 1,+ )+ .await;+ }++ /// The other direction, which the polarity of the gate decides: a `429`+ /// never reached the model, so the queue must keep its bounded,+ /// backoff-scheduled retry. The consolidator spends its own three inner+ /// attempts first, then the queue re-queues the job as attempt 2.+ #[tokio::test]+ async fn session_end_worker_requeues_a_rate_limited_provider() {+ session_end_worker_settles_as(ConsolidationFailure::RateLimited, QueueOutcome::Requeued, 3)+ .await;}async fn two_project_wiki() -> (TempDir, Store, Wiki, WorkspaceId, ProjectId, ProjectId) {