Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
fix(consolidate): reconcile the session job row after a manual memory_consolidate (#890)
AkitaOnRails18e7936ca2a9
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 39186f2d..1785e1ed 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -20,6 +20,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0query before falling back. (#873)### Fixed+- A manual `memory_consolidate` now reconciles the session's durable+ consolidation job row. The MCP handler wrote the page directly through the+ consolidator without touching `session_consolidation_jobs`, so a session+ whose automatic SessionEnd job had reached the terminal `failed` state (that+ the worker never re-claims) kept showing `failed` even though the operator+ had just consolidated it. After a successful, non-dry consolidate the handler+ flips a `failed`/`pending`/`superseded` row for the session to `completed`; a+ live `running` lease is never touched, so a concurrent automatic worker+ attempt is left to settle its own row. (#890)- `ai-memory run`'s auto-wire no longer overwrites an installed session-awareClaude Code MCP bridge with the static HTTP registration. The auto-wiresentinel is keyed by client version, so the MCP step re-ran on every upgradediff --git a/crates/ai-memory-mcp/src/server.rs b/crates/ai-memory-mcp/src/server.rsindex fb6210ae..10d4e0ee 100644--- a/crates/ai-memory-mcp/src/server.rs+++ b/crates/ai-memory-mcp/src/server.rs@@ -3217,16 +3217,41 @@ impl AiMemoryServer {.consolidate_session_multi(session_id, dry, actor, author_id, instructions).await.map_err(|e| McpError::internal_error(e.to_string(), None))?;+ if !dry {+ self.reconcile_consolidation_job(session_id).await;+ }ok_json(&serde_json::json!({ "outcomes": outcomes }))} else {let outcome = consolidator.consolidate_session(session_id, dry, actor, author_id, instructions).await.map_err(|e| McpError::internal_error(e.to_string(), None))?;+ if !dry {+ self.reconcile_consolidation_job(session_id).await;+ }ok_json(&outcome)}}+ /// Reconcile the durable SessionEnd job row after a successful manual+ /// `memory_consolidate` so the operator does not see a `failed` job for a+ /// session that is now consolidated. The automatic worker owns the job's+ /// lease, so the reconcile never touches a `running` row; a best-effort+ /// failure here must not fail the consolidate that already wrote the page.+ async fn reconcile_consolidation_job(&self, session_id: SessionId) {+ if let Err(error) = self+ .writer+ .reconcile_session_consolidation_completed(session_id)+ .await+ {+ tracing::warn!(+ %session_id,+ %error,+ "failed to reconcile session consolidation job after manual consolidate"+ );+ }+ }+/// Stage durable wiki edit proposals for a completed session.#[tool(description = "Run manual auto-improvement for one completed session and apply or stage validated wiki edit proposals through the auto-improvement approval path. Use when the user asks what durable lessons should be captured, what memory pages this session suggests, or at explicit wrap-up when a learning review is useful. Omit `session_id` to review the latest completed session that has not already produced an auto-improvement run in the current project; repeated implicit calls advance through the remaining sessions, including after a preflight skip. Pass `session_id` to rerun a specific session. The server also schedules background review for newly completed sessions in every project when an LLM provider is configured. Admins can set `[auto_improve.scheduler] enabled = false` to stop automatic review, or `[auto_improve] require_approval = true` to leave scheduled and manual proposals pending for review."diff --git a/crates/ai-memory-store/src/session_consolidation.rs b/crates/ai-memory-store/src/session_consolidation.rsindex 46d4e2a3..24e075f2 100644--- a/crates/ai-memory-store/src/session_consolidation.rs+++ b/crates/ai-memory-store/src/session_consolidation.rs@@ -280,6 +280,38 @@ pub fn release(conn: &mut Connection, job: &SessionConsolidationJob) -> StoreResrequire_claim_update(changed)}+/// Reconcile the durable job row after a manual `memory_consolidate` produced+/// the page out-of-band from the automatic SessionEnd worker.+///+/// A manual consolidate writes the page directly through the consolidator and+/// never touches this queue, so the session's job row can sit in a terminal+/// `failed` (or `pending`/`superseded`) state for a session that is, in fact,+/// consolidated — a two-sources-of-truth inconsistency the operator sees. This+/// flips those rows to `completed`.+///+/// It deliberately never touches a `running` row: that state is an active+/// worker lease, and stomping it would violate the single-writer/collaboration+/// invariants (a live automatic attempt must finish and settle its own row+/// through the claim-guarded `complete`/`fail`). `completed` rows are already+/// consistent and left untouched. Unlike `complete`/`fail`, this is not+/// claim-guarded: it is a session-scoped reconcile, not a lease transition.+///+/// Returns the number of job rows updated (0 when there was no job, or the only+/// row was already `completed` or `running`).+pub fn reconcile_session_consolidation_completed(+ conn: &mut Connection,+ session_id: SessionId,+) -> StoreResult<usize> {+ let now = Timestamp::now().as_microsecond();+ let changed = conn.execute(+ "UPDATE session_consolidation_jobs \+ SET state = 'completed', completed_at = ?1, claim_id = NULL, last_error = NULL \+ WHERE session_id = ?2 AND state IN ('failed', 'pending', 'superseded')",+ params![now, session_id.as_bytes()],+ )?;+ Ok(changed)+}+fn require_claim_update(changed: usize) -> StoreResult<()> {if changed == 1 {Ok(())@@ -580,6 +612,112 @@ mod tests {.unwrap();}+ #[tokio::test]+ async fn manual_reconcile_completes_a_terminally_failed_job() {+ let (_tmp, store, workspace_id, project_id, session_id) = ended_session().await;+ store+ .writer+ .enqueue_session_consolidation(workspace_id, project_id, session_id)+ .await+ .unwrap();+ let now = Timestamp::now().as_microsecond();+ // Drive the job to terminal `failed`: spend every attempt but the last+ // with a scheduled retry, then fail the final attempt with no retry.+ for _ in 1..SESSION_CONSOLIDATION_MAX_ATTEMPTS {+ let job = store+ .writer+ .claim_session_consolidation(now, now - 1)+ .await+ .unwrap()+ .unwrap();+ store+ .writer+ .fail_session_consolidation(job, "retry".into(), Some(now))+ .await+ .unwrap();+ }+ let terminal = store+ .writer+ .claim_session_consolidation(now, now - 1)+ .await+ .unwrap()+ .unwrap();+ assert_eq!(terminal.attempts(), SESSION_CONSOLIDATION_MAX_ATTEMPTS);+ store+ .writer+ .fail_session_consolidation(terminal, "terminal".into(), None)+ .await+ .unwrap();+ // A terminally failed job is never re-claimed by the worker.+ assert!(+ store+ .writer+ .claim_session_consolidation(now + 1_000, now - 1)+ .await+ .unwrap()+ .is_none(),+ "a terminally failed job must not be re-claimed"+ );++ // The manual reconcile flips the terminal row to `completed`.+ let updated = store+ .writer+ .reconcile_session_consolidation_completed(session_id)+ .await+ .unwrap();+ assert_eq!(updated, 1, "the failed job row must be reconciled");+ assert!(+ store+ .writer+ .claim_session_consolidation(now + 1_000, now - 1)+ .await+ .unwrap()+ .is_none(),+ "a reconciled job stays completed and is never re-claimed"+ );+ }++ #[tokio::test]+ async fn manual_reconcile_leaves_a_live_running_lease_intact() {+ // Invariant #16 / #2 guard: a manual reconcile must never stomp a live+ // worker lease. This control fails if the reconcile WHERE clause is+ // widened to touch `state = 'running'`.+ let (_tmp, store, workspace_id, project_id, session_id) = ended_session().await;+ store+ .writer+ .enqueue_session_consolidation(workspace_id, project_id, session_id)+ .await+ .unwrap();+ let now = Timestamp::now().as_microsecond();+ let leased = store+ .writer+ .claim_session_consolidation(now, now - 1)+ .await+ .unwrap()+ .unwrap();+ assert_eq!(leased.attempts(), 1);++ // Reconcile while the automatic worker holds the lease: it must not+ // touch the `running` row.+ let updated = store+ .writer+ .reconcile_session_consolidation_completed(session_id)+ .await+ .unwrap();+ assert_eq!(+ updated, 0,+ "a live running lease must be left intact by the reconcile"+ );++ // The lease is still valid: its own claim-guarded completion succeeds,+ // proving the reconcile did not clear claim_id or change its state.+ store+ .writer+ .complete_session_consolidation(leased)+ .await+ .expect("the untouched lease must still complete through its own claim");+ }+#[tokio::test]async fn expired_final_attempt_replays_without_spending_an_extra_attempt() {let (_tmp, store, workspace_id, project_id, session_id) = ended_session().await;diff --git a/crates/ai-memory-store/src/writer.rs b/crates/ai-memory-store/src/writer.rsindex 3993eb56..50f793e1 100644--- a/crates/ai-memory-store/src/writer.rs+++ b/crates/ai-memory-store/src/writer.rs@@ -216,6 +216,10 @@ pub(crate) enum WriteCmd {job: SessionConsolidationJob,reply: oneshot::Sender<StoreResult<()>>,},+ ReconcileSessionConsolidationCompleted {+ session_id: SessionId,+ reply: oneshot::Sender<StoreResult<usize>>,+ },InsertHandoff {handoff: NewHandoff,reply: oneshot::Sender<StoreResult<HandoffId>>,@@ -1200,6 +1204,22 @@ impl WriterHandle {rx.await.map_err(|_| StoreError::WriterClosed)?}+ /// Reconcile a session's durable consolidation job row to `completed` after+ /// a manual `memory_consolidate` produced the page out-of-band. Never+ /// touches a `running` lease. Returns the number of rows updated.+ pub async fn reconcile_session_consolidation_completed(+ &self,+ session_id: SessionId,+ ) -> StoreResult<usize> {+ let (tx, rx) = oneshot::channel();+ self.send(WriteCmd::ReconcileSessionConsolidationCompleted {+ session_id,+ reply: tx,+ })+ .await?;+ rx.await.map_err(|_| StoreError::WriterClosed)?+ }+/// Insert a new handoff in `open` state.////// # Errors@@ -2954,6 +2974,13 @@ fn worker_loop(mut conn: Connection, mut rx: mpsc::Receiver<WriteCmd>) {let result = crate::session_consolidation::release(&mut conn, &job);send_or_warn(reply, result, "release_session_consolidation");}+ WriteCmd::ReconcileSessionConsolidationCompleted { session_id, reply } => {+ let result =+ crate::session_consolidation::reconcile_session_consolidation_completed(+ &mut conn, session_id,+ );+ send_or_warn(reply, result, "reconcile_session_consolidation_completed");+ }WriteCmd::InsertHandoff { handoff, reply } => {let result = ops::insert_handoff(&mut conn, &handoff);send_or_warn(reply, result, "insert_handoff");diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex 7396f265..e95ecb9b 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -282,7 +282,7 @@ separately gated Claude Code assistant/Stop excerpt remains capped at 2 KB.| `pages` | Versioned wiki pages with `is_latest` + `supersedes` chain. M8 columns: `last_accessed_at`, `access_count`, and decay-only tombstone marker `superseded_at`. M9 cols: `embedding_provider`, `embedding_model`, `embedding_dim`. V36: `expires_at` (frontmatter TTL). V37: `salience` (NULL = `salience_default`; derived from `page_feedback`). || `pages_fts` | FTS5 virtual table over `(title, body)`, auto-synced by triggers. || `sessions`, `observations` | Sanitized, bounded lifecycle-hook projections. `sessions.ended_observation_count` is the stable generation watermark for resumed-session re-end eligibility; wall clocks are not used for that decision. They are an operational audit trail, not a complete native transcript. |-| `session_consolidation_jobs` | Durable, observation-generation-idempotent queue for opt-in SessionEnd LLM consolidation. One bounded server worker leases jobs, retries provider failures with backoff, and recovers expired leases after restart. |+| `session_consolidation_jobs` | Durable, observation-generation-idempotent queue for the *automatic* SessionEnd LLM consolidation worker. One bounded server worker leases jobs, retries provider failures with backoff, and recovers expired leases after restart. A manual `memory_consolidate` writes its page out-of-band and then reconciles this table (flipping a `failed`/`pending`/`superseded` row for the session to `completed`, never touching a live `running` lease), so the operator does not see a `failed` job for a session that is in fact consolidated. || `observations_fts` | FTS5 virtual table over raw observation `(title, body)`, used only as bounded fallback. || `workstreams`, `managed_runs`, `workstream_native_sessions` | Optional lease state plus per-harness native source and delivery cursors for `ai-memory run`. || `workstream_events`, `workstream_events_fts` | Append-only normalized visible transcript events and full-text search; immutable sanitized source batches also live under `raw/workstreams/`. |