Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

feat(store): per-page TTL via frontmatter expires_at (V36)

Samir Hanna Verzaa0bc43db23c0
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 4e4e64f0..10c30fd8 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -8,6 +8,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
+- Per-page TTL via a frontmatter `expires_at:` key (RFC3339, or a bare
+ `YYYY-MM-DD` meaning end of that day UTC), mirrored into a new
+ `pages.expires_at` column (V36) and settable through a new optional
+ `expires_at` parameter on `memory_write_page`. Expired pages are hidden
+ from `memory_query`/`memory_recent`/briefing/session-brief surfaces —
+ `memory_query` gains `include_expired: true` to still see them — while
+ exact-path reads still return the page, annotated `expired: true`,
+ because an explicit read is not a search. The forget sweep hard-deletes
+ them through the wiki layer, so the markdown file goes too, not just
+ the rows. An explicit TTL outranks `pinned` (a pin means "don't decay
+ this", not "keep it past the date its author set"); `memory_lint`
+ flags pinned+expiring pages so the combination is visible rather than
+ silent.
- New `openai-compat` embedding provider for self-hosted engines
(Ollama, LM Studio, vLLM). Set
`AI_MEMORY_EMBEDDING_PROVIDER=openai-compat` together with explicit
diff --git a/README.md b/README.md
index b07b01bb..f8e6745e 100644
--- a/README.md
+++ b/README.md
@@ -203,6 +203,13 @@ priors are at the [bottom](#influences-and-prior-art).
auto-synthesised session page (rewritten on consolidation), a
write-page note is yours: it shows up in `memory_query`, renders in
`/web`, and stays until you change it.
+- **"Remember this, but only until the sprint ends."** Pass
+ `expires_at` to `memory_write_page` (RFC3339 or `YYYY-MM-DD` = end of
+ that day, UTC) — or put `expires_at:` in a page's frontmatter by
+ hand. Past the TTL the page disappears from search/recent/briefing
+ (pass `include_expired: true` to `memory_query` to still see it) and
+ the next forget sweep hard-deletes the file and its rows. A TTL beats
+ a pin; `memory_lint` warns about pinned+expiring combos.
- **"This new project has months of history before ai-memory."**
`cd /path/to/my-project && ai-memory bootstrap` collects
`git log`, README, `docs/`, module headers, project rules and
diff --git a/crates/ai-memory-cli/src/commands/reindex.rs b/crates/ai-memory-cli/src/commands/reindex.rs
index d6b0b9e6..d2c7afa8 100644
--- a/crates/ai-memory-cli/src/commands/reindex.rs
+++ b/crates/ai-memory-cli/src/commands/reindex.rs
@@ -120,6 +120,7 @@ mod tests {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
diff --git a/crates/ai-memory-cli/src/commands/serve.rs b/crates/ai-memory-cli/src/commands/serve.rs
index 7a6e92e7..f34abc80 100644
--- a/crates/ai-memory-cli/src/commands/serve.rs
+++ b/crates/ai-memory-cli/src/commands/serve.rs
@@ -759,6 +759,7 @@ async fn start_maintenance_scheduler(
if maintenance_enabled && forget_sweep_interval_secs > 0 {
let reader = reader.clone();
let writer = writer.clone();
+ let wiki = wiki.clone();
tasks.push(tokio::spawn(async move {
let interval = std::time::Duration::from_secs(forget_sweep_interval_secs);
run_persisted_maintenance_job(
@@ -782,10 +783,12 @@ async fn start_maintenance_scheduler(
move || {
let reader = reader.clone();
let writer = writer.clone();
+ let wiki = wiki.clone();
let decay = decay;
async move {
let started = std::time::Instant::now();
- let outcome = run_scheduled_sweep_tick(&reader, &writer, &decay).await?;
+ let outcome =
+ run_scheduled_sweep_tick(&reader, &writer, &wiki, &decay).await?;
if outcome.errors > 0 {
anyhow::bail!(
"scheduled forget sweep had {} scope errors",
@@ -796,6 +799,7 @@ async fn start_maintenance_scheduler(
scopes = outcome.scopes,
candidates_evaluated = outcome.candidates_evaluated,
evicted = outcome.evicted,
+ expired = outcome.expired,
hard_deleted = outcome.hard_deleted,
errors = outcome.errors,
elapsed_ms = started.elapsed().as_millis(),
@@ -1006,6 +1010,7 @@ struct ScheduledSweepTickOutcome {
scopes: usize,
candidates_evaluated: usize,
evicted: usize,
+ expired: usize,
hard_deleted: usize,
errors: usize,
}
@@ -1013,6 +1018,7 @@ struct ScheduledSweepTickOutcome {
async fn run_scheduled_sweep_tick(
reader: &ReaderPool,
writer: &WriterHandle,
+ wiki: &Wiki,
decay: &ai_memory_store::DecayParams,
) -> Result<ScheduledSweepTickOutcome> {
let scopes = reader.list_all_scopes().await?;
@@ -1025,6 +1031,7 @@ async fn run_scheduled_sweep_tick(
match run_sweep(
reader,
writer,
+ Some(wiki),
scope.workspace_id,
scope.project_id,
decay,
@@ -1035,6 +1042,7 @@ async fn run_scheduled_sweep_tick(
Ok(report) => {
outcome.candidates_evaluated += report.candidates_evaluated;
outcome.evicted += report.evicted.len();
+ outcome.expired += report.expired.len();
outcome.hard_deleted += report.hard_deleted;
}
Err(e) => {
@@ -2174,7 +2182,7 @@ mod tests {
cold_threshold: 2.0,
..ai_memory_store::DecayParams::default()
};
- let outcome = run_scheduled_sweep_tick(&store.reader, &store.writer, &decay)
+ let outcome = run_scheduled_sweep_tick(&store.reader, &store.writer, &wiki, &decay)
.await
.unwrap();
diff --git a/crates/ai-memory-consolidate/src/curator.rs b/crates/ai-memory-consolidate/src/curator.rs
index c31b561d..d605320e 100644
--- a/crates/ai-memory-consolidate/src/curator.rs
+++ b/crates/ai-memory-consolidate/src/curator.rs
@@ -506,6 +506,7 @@ mod tests {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
diff --git a/crates/ai-memory-consolidate/src/lint.rs b/crates/ai-memory-consolidate/src/lint.rs
index 60cd2fa5..51074a45 100644
--- a/crates/ai-memory-consolidate/src/lint.rs
+++ b/crates/ai-memory-consolidate/src/lint.rs
@@ -222,6 +222,29 @@ fn rule_based_findings(candidates: &[DecayCandidate]) -> Vec<LintFinding> {
.or_default()
.push(c.path.as_str().to_string());
}
+ // Pinned + expiring is contradictory: the pin says "never
+ // decay" but the TTL hard-deletes regardless of pin (an
+ // explicit expiry is the more explicit statement). Surface the
+ // combo so the operator resolves it before the sweep does.
+ let frontmatter_pinned = frontmatter
+ .as_ref()
+ .and_then(|fm| fm.get("pinned"))
+ .and_then(serde_json::Value::as_bool)
+ == Some(true);
+ if c.expires_at_us.is_some() && (c.pinned || frontmatter_pinned) {
+ out.push(LintFinding {
+ kind: "pinned_expiring".into(),
+ severity: "warning".into(),
+ message: format!(
+ "Page {} is pinned but carries an expires_at TTL; \
+ the forget sweep will hard-delete it when the TTL \
+ passes despite the pin. Remove one of the two.",
+ c.path,
+ ),
+ pages: vec![c.path.as_str().to_string()],
+ detail: None,
+ });
+ }
}
for (title, paths) in titles {
@@ -365,6 +388,7 @@ mod tests {
access_count: 0,
last_accessed_at_us: None,
frontmatter_json: "{}".into(),
+ expires_at_us: None,
}];
let findings = rule_based_findings(&candidates);
assert_eq!(findings.len(), 1);
@@ -382,6 +406,7 @@ mod tests {
access_count: 0,
last_accessed_at_us: None,
frontmatter_json: r#"{"title": "Karpathy Wiki"}"#.into(),
+ expires_at_us: None,
};
let b = DecayCandidate {
path: ai_memory_core::PagePath::new("concepts/b.md").unwrap(),
@@ -407,6 +432,7 @@ mod tests {
last_accessed_at_us: None,
frontmatter_json: r#"{"title": "Never ship code without a test", "kind": "rule"}"#
.into(),
+ expires_at_us: None,
};
let findings = rule_based_findings(&[candidate]);
let rules: Vec<_> = findings
@@ -431,6 +457,7 @@ mod tests {
access_count: 0,
last_accessed_at_us: None,
frontmatter_json: "{}".into(),
+ expires_at_us: None,
};
let findings = rule_based_findings(&[candidate]);
assert!(
@@ -453,6 +480,7 @@ mod tests {
access_count: 5,
last_accessed_at_us: None,
frontmatter_json: r#"{"title": "Karpathy Wiki", "kind": "fact"}"#.into(),
+ expires_at_us: None,
};
let findings = rule_based_findings(&[candidate]);
assert!(
@@ -507,6 +535,7 @@ mod tests {
access_count: 0,
last_accessed_at_us: None,
frontmatter_json: "{}".into(),
+ expires_at_us: None,
}];
// rule_based_findings is the exact code path that `use_llm=false`
// keeps active. Confirm it still fires.
diff --git a/crates/ai-memory-consolidate/src/sweep.rs b/crates/ai-memory-consolidate/src/sweep.rs
index 58295082..8fc9d069 100644
--- a/crates/ai-memory-consolidate/src/sweep.rs
+++ b/crates/ai-memory-consolidate/src/sweep.rs
@@ -7,6 +7,12 @@
//! episodic decays). Pinned pages (schema flag OR `pinned: true` in
//! frontmatter) are exempt regardless of tier.
//!
+//! TTL pass: pages whose frontmatter `expires_at:` is in the past are
+//! hard-deleted through the wiki layer (file + rows), regardless of
+//! tier or pin — an explicit expiry is a more explicit user statement
+//! than a pin. `memory_lint` warns about pinned+expiring combos so the
+//! contradiction is visible before the delete lands.
+//!
//! Hard-delete pass cleans up rows soft-deleted more than
//! `hard_delete_after_days` ago that received zero subsequent access.
//! M7 supersession rows are safe: they have `supersedes IS NOT NULL`
@@ -14,6 +20,7 @@
use ai_memory_core::{PageId, ProjectId, Tier, WorkspaceId};
use ai_memory_store::{DecayCandidate, DecayParams, ReaderPool, WriterHandle, retention_score};
+use ai_memory_wiki::Wiki;
use jiff::Timestamp;
use serde::Serialize;
use thiserror::Error;
@@ -33,6 +40,22 @@ pub struct EvictedPage {
pub access_count: u32,
}
+/// One TTL-expired page surfaced in the [`SweepReport`].
+#[derive(Debug, Clone, Serialize)]
+pub struct ExpiredPage {
+ /// Identifier of the expired page version.
+ pub id: PageId,
+ /// Relative wiki path.
+ pub path: String,
+ /// ISO-8601 instant the page expired at.
+ pub expired_at: String,
+ /// `true` when the delete landed (always `false` on `dry_run`; can
+ /// be `false` on a real run if an admission webhook rejected the
+ /// delete or the wiki/store errored — the page is retried on the
+ /// next sweep).
+ pub deleted: bool,
+}
+
/// Outcome of one sweep run.
#[derive(Debug, Clone, Serialize)]
pub struct SweepReport {
@@ -43,6 +66,9 @@ pub struct SweepReport {
/// Pages that fell below the cold threshold (soft-deleted unless
/// `dry_run`).
pub evicted: Vec<EvictedPage>,
+ /// Pages past their frontmatter `expires_at:` TTL (hard-deleted
+ /// through the wiki layer unless `dry_run`).
+ pub expired: Vec<ExpiredPage>,
/// Number of older soft-deleted rows hard-deleted on this pass.
pub hard_deleted: usize,
}
@@ -60,12 +86,20 @@ const US_PER_DAY: f64 = 86_400_000_000.0;
/// Run a sweep against the given workspace/project.
///
+/// `wiki` routes TTL deletions through the wiki layer so the markdown
+/// file is removed together with the rows (deleting only store rows
+/// would let the watcher re-index the file). Callers without a wiki
+/// handle (bare-store tests) pass `None` and get a store-only delete.
+///
/// # Errors
/// Propagates any store error encountered while reading candidates or
-/// writing soft-deletions.
+/// writing soft-deletions. Per-page TTL delete failures (rejecting
+/// admission webhook, IO error) are reported in the
+/// [`SweepReport::expired`] entries instead of aborting the sweep.
pub async fn run_sweep(
reader: &ReaderPool,
writer: &WriterHandle,
+ wiki: Option<&Wiki>,
workspace_id: WorkspaceId,
project_id: ProjectId,
params: &DecayParams,
@@ -76,8 +110,22 @@ pub async fn run_sweep(
let mut evicted = Vec::new();
let mut to_evict_ids: Vec<PageId> = Vec::new();
+ let mut expired: Vec<ExpiredPage> = Vec::new();
for c in &candidates {
+ if let Some(expires_us) = c.expires_at_us
+ && expires_us <= now_us
+ {
+ expired.push(ExpiredPage {
+ id: c.id,
+ path: c.path.as_str().to_string(),
+ expired_at: Timestamp::from_microsecond(expires_us)
+ .map(|ts| ts.to_string())
+ .unwrap_or_default(),
+ deleted: false,
+ });
+ continue;
+ }
if !is_decayable(c) {
continue;
}
@@ -98,6 +146,32 @@ pub async fn run_sweep(
let mut hard_deleted = 0usize;
if !dry_run {
+ for page in &mut expired {
+ let path = match ai_memory_core::PagePath::new(page.path.clone()) {
+ Ok(p) => p,
+ Err(_) => continue,
+ };
+ let result = match wiki {
+ Some(w) => w
+ .delete_page(workspace_id, project_id, &path, None, None)
+ .await
+ .map_err(|e| e.to_string()),
+ None => writer
+ .delete_page(workspace_id, project_id, path.clone(), None)
+ .await
+ .map_err(|e| e.to_string()),
+ };
+ match result {
+ Ok(()) => page.deleted = true,
+ Err(error) => {
+ tracing::warn!(
+ path = %page.path,
+ error,
+ "forget sweep: TTL delete failed; retrying on next sweep"
+ );
+ }
+ }
+ }
if !to_evict_ids.is_empty() {
writer.soft_delete_for_decay(to_evict_ids).await?;
}
@@ -110,6 +184,7 @@ pub async fn run_sweep(
dry_run,
candidates_evaluated: candidates.len(),
evicted,
+ expired,
hard_deleted,
})
}
@@ -150,6 +225,7 @@ mod tests {
access_count: 0,
last_accessed_at_us: None,
frontmatter_json: "{}".into(),
+ expires_at_us: None,
};
assert!(!is_decayable(&c));
}
@@ -165,6 +241,7 @@ mod tests {
access_count: 0,
last_accessed_at_us: None,
frontmatter_json: "{}".into(),
+ expires_at_us: None,
};
assert!(!is_decayable(&c));
}
@@ -180,6 +257,7 @@ mod tests {
access_count: 0,
last_accessed_at_us: None,
frontmatter_json: r#"{"pinned": true}"#.into(),
+ expires_at_us: None,
};
assert!(!is_decayable(&c));
}
@@ -195,6 +273,7 @@ mod tests {
access_count: 0,
last_accessed_at_us: None,
frontmatter_json: "{}".into(),
+ expires_at_us: None,
};
assert!(is_decayable(&c));
}
diff --git a/crates/ai-memory-consolidate/tests/embeddings.rs b/crates/ai-memory-consolidate/tests/embeddings.rs
index c580c1f8..6f49cc57 100644
--- a/crates/ai-memory-consolidate/tests/embeddings.rs
+++ b/crates/ai-memory-consolidate/tests/embeddings.rs
@@ -140,6 +140,7 @@ async fn m9_embeddings_roundtrip_via_synthetic() {
"bag-of-words-v1".into(),
64,
5,
+ None,
)
.await
.expect("hybrid search");
@@ -166,6 +167,7 @@ async fn m9_embeddings_roundtrip_via_synthetic() {
"bag-of-words-v1".into(),
64,
5,
+ None,
)
.await
.expect("hybrid (no query vec)");
diff --git a/crates/ai-memory-consolidate/tests/lifecycle.rs b/crates/ai-memory-consolidate/tests/lifecycle.rs
index 5bc4fec8..3eeab717 100644
--- a/crates/ai-memory-consolidate/tests/lifecycle.rs
+++ b/crates/ai-memory-consolidate/tests/lifecycle.rs
@@ -220,6 +220,7 @@ async fn m8_retention_lifecycle_end_to_end() {
let dry = run_sweep(
&store.reader,
&store.writer,
+ None,
ws,
proj,
&params,
@@ -262,6 +263,7 @@ async fn m8_retention_lifecycle_end_to_end() {
let real = run_sweep(
&store.reader,
&store.writer,
+ None,
ws,
proj,
&params,
@@ -422,3 +424,179 @@ async fn m8_retention_lifecycle_end_to_end() {
);
}
}
+
+/// TTL lifecycle: pages whose frontmatter `expires_at:` has passed are
+/// hidden from retrieval, listed by the sweep, and hard-deleted (file +
+/// rows) by a real sweep — including pinned pages, since an explicit
+/// expiry beats a pin. Future-dated TTLs change nothing.
+#[tokio::test]
+async fn ttl_expiry_lifecycle_end_to_end() {
+ let tmp = TempDir::new().expect("tempdir");
+ let store = Store::open(tmp.path()).expect("open store");
+ let ws = store
+ .writer
+ .get_or_create_workspace("default")
+ .await
+ .expect("ws");
+ let proj = store
+ .writer
+ .get_or_create_project(ws, "ttl-test", None)
+ .await
+ .expect("proj");
+ let wiki = Wiki::new(tmp.path(), store.writer.clone()).expect("wiki");
+
+ let write = |path: &str, body: &str, expires_at: Option<&str>, pinned: bool| {
+ let mut frontmatter = serde_json::Map::new();
+ if let Some(ts) = expires_at {
+ frontmatter.insert("expires_at".into(), serde_json::Value::String(ts.into()));
+ }
+ if pinned {
+ frontmatter.insert("pinned".into(), serde_json::Value::Bool(true));
+ }
+ wiki.write_page(WritePageRequest {
+ workspace_id: ws,
+ project_id: proj,
+ path: PagePath::new(path.to_string()).expect("page path"),
+ frontmatter: serde_json::Value::Object(frontmatter),
+ body: body.to_string(),
+ tier: Tier::Semantic,
+ pinned: false,
+ title: None,
+ admission_ctx: None,
+ author_id: None,
+ actor: ai_memory_core::ActorContext::anonymous(),
+ })
+ };
+
+ write(
+ "notes/expired.md",
+ "# Expired\nttlmarker expired sprint context",
+ Some("2020-01-01"),
+ false,
+ )
+ .await
+ .expect("write expired page");
+ write(
+ "notes/expired-pinned.md",
+ "# Expired pinned\nttlmarker pinned but past its expiry",
+ Some("2020-06-01T00:00:00Z"),
+ true,
+ )
+ .await
+ .expect("write expired pinned page");
+ write(
+ "notes/future.md",
+ "# Future\nttlmarker still valid for decades",
+ Some("2099-01-01"),
+ false,
+ )
+ .await
+ .expect("write future page");
+ write(
+ "notes/forever.md",
+ "# Forever\nttlmarker no ttl at all",
+ None,
+ false,
+ )
+ .await
+ .expect("write forever page");
+
+ // Invalid expires_at fails closed instead of meaning "never".
+ let invalid = write("notes/bad.md", "# Bad\nbody", Some("soonish"), false).await;
+ assert!(invalid.is_err(), "invalid expires_at must be rejected");
+
+ // Retrieval hides expired pages by default…
+ let hits = store
+ .reader
+ .search_pages_for_project(ws, proj, "ttlmarker".into(), 10, None)
+ .await
+ .expect("search");
+ let paths: Vec<&str> = hits.iter().map(|h| h.path.as_str()).collect();
+ assert!(paths.contains(&"notes/future.md"));
+ assert!(paths.contains(&"notes/forever.md"));
+ assert!(
+ !paths.contains(&"notes/expired.md"),
+ "expired hidden: {paths:?}"
+ );
+ assert!(!paths.contains(&"notes/expired-pinned.md"));
+ let recent = store
+ .reader
+ .recent_pages_for_project(ws, proj, 10)
+ .await
+ .expect("recent");
+ assert!(!recent.iter().any(|h| h.path.as_str() == "notes/expired.md"));
+
+ // …but an i64::MIN cutoff (memory_query include_expired) shows them.
+ let all_hits = store
+ .reader
+ .search_pages_for_project(ws, proj, "ttlmarker".into(), 10, Some(i64::MIN))
+ .await
+ .expect("search all");
+ assert!(
+ all_hits
+ .iter()
+ .any(|h| h.path.as_str() == "notes/expired.md"),
+ "include_expired must surface expired pages",
+ );
+
+ // Dry-run sweep lists the expired pages without touching disk.
+ let params = DecayParams::default();
+ let dry = run_sweep(
+ &store.reader,
+ &store.writer,
+ Some(&wiki),
+ ws,
+ proj,
+ &params,
+ true,
+ )
+ .await
+ .expect("dry sweep");
+ let dry_expired: HashSet<&str> = dry.expired.iter().map(|e| e.path.as_str()).collect();
+ assert_eq!(
+ dry_expired,
+ HashSet::from(["notes/expired.md", "notes/expired-pinned.md"]),
+ "dry sweep lists exactly the expired pages",
+ );
+ assert!(dry.expired.iter().all(|e| !e.deleted));
+ let ws_dir = tmp
+ .path()
+ .join("wiki")
+ .join(ws.to_string())
+ .join(proj.to_string());
+ assert!(ws_dir.join("notes/expired.md").exists());
+
+ // Real sweep hard-deletes them through the wiki layer.
+ let real = run_sweep(
+ &store.reader,
+ &store.writer,
+ Some(&wiki),
+ ws,
+ proj,
+ &params,
+ false,
+ )
+ .await
+ .expect("real sweep");
+ assert!(real.expired.iter().all(|e| e.deleted), "{:?}", real.expired);
+ assert!(
+ !ws_dir.join("notes/expired.md").exists(),
+ "markdown file must be removed, not just the rows",
+ );
+ assert!(
+ !ws_dir.join("notes/expired-pinned.md").exists(),
+ "explicit expiry beats pin",
+ );
+ assert!(ws_dir.join("notes/future.md").exists());
+
+ // Rows are gone too — even an include-everything search misses them.
+ let after = store
+ .reader
+ .search_pages_for_project(ws, proj, "ttlmarker".into(), 10, Some(i64::MIN))
+ .await
+ .expect("search after sweep");
+ let after_paths: Vec<&str> = after.iter().map(|h| h.path.as_str()).collect();
+ assert!(!after_paths.contains(&"notes/expired.md"));
+ assert!(after_paths.contains(&"notes/future.md"));
+ assert!(after_paths.contains(&"notes/forever.md"));
+}
diff --git a/crates/ai-memory-consolidate/tests/recall_eval.rs b/crates/ai-memory-consolidate/tests/recall_eval.rs
index 15c3002b..32ddc9ea 100644
--- a/crates/ai-memory-consolidate/tests/recall_eval.rs
+++ b/crates/ai-memory-consolidate/tests/recall_eval.rs
@@ -217,7 +217,7 @@ async fn graph_neighbor_expansion_recovers_linked_page() {
let fts_hits = store
.reader
- .search_pages_for_project(ws, proj, "graphseed".into(), 5)
+ .search_pages_for_project(ws, proj, "graphseed".into(), 5, None)
.await
.expect("fts search");
assert!(
@@ -238,6 +238,7 @@ async fn graph_neighbor_expansion_recovers_linked_page() {
String::new(),
0,
5,
+ None,
)
.await
.expect("hybrid search");
@@ -297,7 +298,7 @@ async fn raw_observation_fallback_recovers_detail_when_wiki_misses() {
let page_hits = store
.reader
- .search_pages_for_project(ws, proj, "capybara".into(), 5)
+ .search_pages_for_project(ws, proj, "capybara".into(), 5, None)
.await
.expect("page search");
assert!(page_hits.is_empty(), "compiled wiki should miss");
@@ -333,6 +334,7 @@ async fn measure_recall(
emb.model().to_string(),
emb.dim(),
5,
+ None,
)
.await
.expect("hybrid search")
diff --git a/crates/ai-memory-core/src/page.rs b/crates/ai-memory-core/src/page.rs
index fdeef007..a5617f26 100644
--- a/crates/ai-memory-core/src/page.rs
+++ b/crates/ai-memory-core/src/page.rs
@@ -68,6 +68,12 @@ pub struct NewPage {
/// gets the pre-multi-user behaviour.
#[serde(default)]
pub author_id: Option<crate::UserId>,
+ /// TTL derived from the frontmatter `expires_at:` key (markdown is
+ /// the source of truth; the wiki layer parses and validates it).
+ /// `None` = never expires. Expired pages are hidden from
+ /// search/recent/briefing and hard-deleted by the retention sweep.
+ #[serde(default)]
+ pub expires_at: Option<Timestamp>,
}
/// A link target discovered in a page body.
@@ -143,6 +149,8 @@ pub struct Page {
pub created_at: Timestamp,
/// Wall-clock last-update time of *this version*.
pub updated_at: Timestamp,
+ /// TTL instant, if the page carries a frontmatter `expires_at:`.
+ pub expires_at: Option<Timestamp>,
}
impl Tier {
diff --git a/crates/ai-memory-hooks/src/router.rs b/crates/ai-memory-hooks/src/router.rs
index 30bdfc40..d895c294 100644
--- a/crates/ai-memory-hooks/src/router.rs
+++ b/crates/ai-memory-hooks/src/router.rs
@@ -6157,6 +6157,7 @@ mod tests {
pinned,
links: Vec::new(),
author_id: None,
+ expires_at: None,
}
}
diff --git a/crates/ai-memory-hooks/src/synth.rs b/crates/ai-memory-hooks/src/synth.rs
index cd759735..d3145a16 100644
--- a/crates/ai-memory-hooks/src/synth.rs
+++ b/crates/ai-memory-hooks/src/synth.rs
@@ -47,6 +47,7 @@ pub fn synthesize_session_page(
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
}
}
diff --git a/crates/ai-memory-mcp/src/admin.rs b/crates/ai-memory-mcp/src/admin.rs
index 630e300e..4b5dbcce 100644
--- a/crates/ai-memory-mcp/src/admin.rs
+++ b/crates/ai-memory-mcp/src/admin.rs
@@ -943,7 +943,7 @@ async fn handle_search(
Ok((ws, proj)) => {
state
.reader
- .search_pages_for_project(ws, proj, query.q, limit)
+ .search_pages_for_project(ws, proj, query.q, limit, None)
.await
}
Err(e) => return e,
@@ -1044,7 +1044,7 @@ async fn handle_read_page(
} else if let Some(q) = query.q {
let hits = match state
.reader
- .search_pages_for_project(ws, proj, q.clone(), 1)
+ .search_pages_for_project(ws, proj, q.clone(), 1, None)
.await
{
Ok(h) => h,
@@ -2478,6 +2478,7 @@ async fn handle_forget_sweep(
run_sweep(
&state.reader,
&state.writer,
+ Some(&state.wiki),
ws,
proj,
&state.decay_params,
diff --git a/crates/ai-memory-mcp/src/server.rs b/crates/ai-memory-mcp/src/server.rs
index 337fb708..73ddfa07 100644
--- a/crates/ai-memory-mcp/src/server.rs
+++ b/crates/ai-memory-mcp/src/server.rs
@@ -402,6 +402,10 @@ struct QueryArgs {
/// workspace + project so you can tell where it came from.
#[serde(default)]
global: Option<bool>,
+ /// Also return pages whose `expires_at` TTL has passed (hidden by
+ /// default; they are deleted by the next forget sweep). Default false.
+ #[serde(default)]
+ include_expired: Option<bool>,
}
#[derive(Debug, Serialize, Deserialize, schemars::JsonSchema)]
@@ -769,6 +773,12 @@ struct WritePageArgs {
/// every project. Cannot be combined with `workspace`/`project`.
#[serde(default)]
scope: Option<String>,
+ /// Optional TTL: RFC3339 instant (`2026-09-01T12:00:00Z`) or bare
+ /// date (`2026-09-01` = end of that day, UTC). After this instant
+ /// the page is hidden from search/recent/briefing and hard-deleted
+ /// by the next forget sweep. Omit for pages that never expire.
+ #[serde(default)]
+ expires_at: Option<String>,
}
#[tool_router]
@@ -1049,7 +1059,12 @@ impl AiMemoryServer {
query: &str,
query_vec: Option<&[f32]>,
limit: usize,
+ include_expired: bool,
) -> ai_memory_store::StoreResult<Vec<PageHit>> {
+ // `i64::MIN` as the expiry cutoff makes every stored TTL pass
+ // the `expires_at > cutoff` guard, i.e. expired pages stay
+ // searchable when the caller opted in.
+ let expiry_cutoff = include_expired.then_some(i64::MIN);
if let (Some(embedder), Some(qv)) = (&self.embedder, query_vec) {
return self
.reader
@@ -1062,11 +1077,18 @@ impl AiMemoryServer {
embedder.model().to_string(),
embedder.dim(),
limit,
+ expiry_cutoff,
)
.await;
}
self.reader
- .search_pages_for_project(workspace_id, project_id, query.to_owned(), limit)
+ .search_pages_for_project(
+ workspace_id,
+ project_id,
+ query.to_owned(),
+ limit,
+ expiry_cutoff,
+ )
.await
}
@@ -1192,6 +1214,7 @@ impl AiMemoryServer {
let query = args.query.clone();
let query_vec = self.embed_query(&args.query).await;
+ let include_expired = args.include_expired.unwrap_or(false);
let resolved_scopes = if args.scopes.is_empty() {
None
} else {
@@ -1201,7 +1224,14 @@ impl AiMemoryServer {
let mut hits_by_id: HashMap<PageId, PageHit> = HashMap::new();
for &(ws, proj) in scopes {
let hits = self
- .search_project(ws, proj, &args.query, query_vec.as_deref(), limit)
+ .search_project(
+ ws,
+ proj,
+ &args.query,
+ query_vec.as_deref(),
+ limit,
+ include_expired,
+ )
.await
.map_err(|e| McpError::internal_error(e.to_string(), None))?;
for hit in hits {
@@ -1231,8 +1261,15 @@ impl AiMemoryServer {
&aps_actor,
)
.await?;
- self.search_project(ws, proj, &args.query, query_vec.as_deref(), limit)
- .await
+ self.search_project(
+ ws,
+ proj,
+ &args.query,
+ query_vec.as_deref(),
+ limit,
+ include_expired,
+ )
+ .await
};
let hits = hits.map_err(|e| McpError::internal_error(e.to_string(), None))?;
self.spawn_access_bump(hits.iter().map(|h| h.id).collect());
@@ -1306,6 +1343,7 @@ impl AiMemoryServer {
&args.query,
query_vec.as_deref(),
limit,
+ include_expired,
)
.await
.map_err(|e| McpError::internal_error(e.to_string(), None))?;
@@ -1401,6 +1439,7 @@ impl AiMemoryServer {
let report = run_sweep(
&self.reader,
&self.writer,
+ self.wiki.as_ref(),
ws,
proj,
&self.decay_params,
@@ -1834,6 +1873,19 @@ impl AiMemoryServer {
if args.pinned {
fm.insert("pinned".into(), serde_json::Value::Bool(true));
}
+ if let Some(expires_at) = args
+ .expires_at
+ .as_deref()
+ .map(str::trim)
+ .filter(|s| !s.is_empty())
+ {
+ // The wiki layer validates the format on write; landing the
+ // raw string in frontmatter keeps markdown the source of truth.
+ fm.insert(
+ "expires_at".into(),
+ serde_json::Value::String(expires_at.to_string()),
+ );
+ }
let frontmatter = if fm.is_empty() {
serde_json::Value::Null
} else {
@@ -1941,7 +1993,7 @@ impl AiMemoryServer {
} else if let Some(query) = args.query {
let hits = self
.reader
- .search_pages_for_project(ws, proj, query.clone(), 1)
+ .search_pages_for_project(ws, proj, query.clone(), 1, None)
.await
.map_err(|e| McpError::internal_error(e.to_string(), None))?;
match hits.into_iter().next() {
@@ -2956,6 +3008,7 @@ mod tests {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
@@ -3855,6 +3908,7 @@ mod tests {
project: Some("sibling".to_string()),
workspace: None,
scope: None,
+ expires_at: None,
}),
OptionalParts(parts),
)
@@ -3934,6 +3988,7 @@ mod tests {
scopes: Vec::new(),
workspace: None,
global: None,
+ include_expired: None,
}),
OptionalParts(test_parts_default()),
)
@@ -3967,6 +4022,7 @@ mod tests {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
@@ -3978,6 +4034,7 @@ mod tests {
scopes: Vec::new(),
workspace: workspace.map(str::to_string),
global: None,
+ include_expired: None,
};
let result = server
@@ -4028,6 +4085,7 @@ mod tests {
scopes: Vec::new(),
workspace: None,
global: None,
+ include_expired: None,
}),
OptionalParts(test_parts_default()),
)
@@ -4063,6 +4121,7 @@ mod tests {
project: project.map(str::to_string),
workspace: None,
scope: scope.map(str::to_string),
+ expires_at: None,
};
server
@@ -4151,6 +4210,7 @@ mod tests {
scopes: Vec::new(),
workspace: None,
global: None,
+ include_expired: None,
}),
OptionalParts(test_parts_default()),
)
@@ -4226,6 +4286,7 @@ mod tests {
}],
workspace: None,
global: None,
+ include_expired: None,
}),
test_optional_parts(),
)
@@ -4318,6 +4379,7 @@ mod tests {
],
workspace: None,
global: None,
+ include_expired: None,
}),
test_optional_parts(),
)
@@ -4381,6 +4443,7 @@ mod tests {
],
workspace: None,
global: None,
+ include_expired: None,
}),
test_optional_parts(),
)
@@ -4420,6 +4483,7 @@ mod tests {
}],
workspace: None,
global: None,
+ include_expired: None,
}),
test_optional_parts(),
)
@@ -4458,6 +4522,7 @@ mod tests {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
@@ -4489,6 +4554,7 @@ mod tests {
],
workspace: None,
global: None,
+ include_expired: None,
}),
test_optional_parts(),
)
@@ -4536,6 +4602,7 @@ mod tests {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
@@ -4549,6 +4616,7 @@ mod tests {
scopes: Vec::new(),
workspace: Some("practice".into()),
global: None,
+ include_expired: None,
}),
OptionalParts(test_parts_default()),
)
@@ -4747,6 +4815,7 @@ mod tests {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
@@ -4865,6 +4934,7 @@ mod tests {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
@@ -4881,6 +4951,7 @@ mod tests {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
@@ -4897,6 +4968,7 @@ mod tests {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
@@ -4919,6 +4991,7 @@ mod tests {
],
workspace: None,
global: None,
+ include_expired: None,
}),
OptionalParts(test_parts_default()),
)
@@ -4974,6 +5047,7 @@ mod tests {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
@@ -4988,6 +5062,7 @@ mod tests {
scopes: Vec::new(),
workspace: None,
global: Some(true),
+ include_expired: None,
}),
OptionalParts(test_parts_default()),
)
@@ -5046,6 +5121,7 @@ mod tests {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
@@ -5067,6 +5143,7 @@ mod tests {
scopes: Vec::new(),
workspace: None,
global: None,
+ include_expired: None,
}),
OptionalParts(test_parts_default()),
)
@@ -5096,6 +5173,7 @@ mod tests {
scopes: Vec::new(),
workspace: Some("ops".into()),
global: None,
+ include_expired: None,
}),
OptionalParts(test_parts_default()),
)
@@ -5129,6 +5207,7 @@ mod tests {
scopes: Vec::new(),
workspace: None,
global: Some(true),
+ include_expired: None,
}),
OptionalParts(test_parts_default()),
)
@@ -5323,6 +5402,7 @@ mod tests {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
@@ -5421,6 +5501,7 @@ mod tests {
project: None,
workspace: None,
scope: None,
+ expires_at: None,
}),
OptionalParts(parts),
)
@@ -5487,6 +5568,7 @@ mod tests {
project: None,
workspace: Some("default".into()),
scope: None,
+ expires_at: None,
}),
OptionalParts(test_parts_default()),
)
@@ -5553,6 +5635,7 @@ mod tests {
project: None,
workspace: None,
scope: None,
+ expires_at: None,
}),
OptionalParts(parts),
)
@@ -5601,6 +5684,7 @@ mod tests {
project: None,
workspace: None,
scope: None,
+ expires_at: None,
}),
OptionalParts(test_parts_default()),
)
@@ -5664,6 +5748,7 @@ mod tests {
project: None,
workspace: None,
scope: None,
+ expires_at: None,
}),
OptionalParts(parts()),
)
@@ -5752,6 +5837,7 @@ mod tests {
project: None,
workspace: None,
scope: None,
+ expires_at: None,
}),
OptionalParts(test_parts_default()),
)
@@ -5844,6 +5930,7 @@ mod tests {
project: Some("shared".into()),
workspace: Some("alpha".into()),
scope: None,
+ expires_at: None,
}),
OptionalParts(parts()),
)
@@ -5861,6 +5948,7 @@ mod tests {
project: Some("shared".into()),
workspace: Some("beta".into()),
scope: None,
+ expires_at: None,
}),
OptionalParts(parts()),
)
@@ -5959,6 +6047,7 @@ mod tests {
project: Some("other".into()),
workspace: None,
scope: None,
+ expires_at: None,
}),
OptionalParts(parts()),
)
@@ -6580,6 +6669,7 @@ mod tests {
project: Some("audited".into()),
workspace: None,
scope: None,
+ expires_at: None,
}),
OptionalParts(parts()),
)
@@ -6676,6 +6766,7 @@ mod tests {
scopes: Vec::new(),
workspace: None,
global: None,
+ include_expired: None,
}),
OptionalParts(test_parts_default()),
)
@@ -6707,6 +6798,7 @@ mod tests {
scopes: Vec::new(),
workspace: None,
global: None,
+ include_expired: None,
}),
OptionalParts(test_parts_default()),
)
diff --git a/crates/ai-memory-mcp/tests/admin_phase3.rs b/crates/ai-memory-mcp/tests/admin_phase3.rs
index ba534cf4..a6d30002 100644
--- a/crates/ai-memory-mcp/tests/admin_phase3.rs
+++ b/crates/ai-memory-mcp/tests/admin_phase3.rs
@@ -496,6 +496,7 @@ async fn reorg_live_graveyards_only_default_workspace_pages() {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
@@ -523,6 +524,7 @@ async fn reorg_live_graveyards_only_default_workspace_pages() {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
@@ -601,6 +603,7 @@ async fn lint_dry_run_returns_lint_report_shape() {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
diff --git a/crates/ai-memory-mcp/tests/admin_read_page.rs b/crates/ai-memory-mcp/tests/admin_read_page.rs
index a94e6ffd..f5067d4d 100644
--- a/crates/ai-memory-mcp/tests/admin_read_page.rs
+++ b/crates/ai-memory-mcp/tests/admin_read_page.rs
@@ -87,6 +87,7 @@ async fn read_page_falls_back_to_db_when_file_missing() {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
diff --git a/crates/ai-memory-mcp/tests/admin_status_search.rs b/crates/ai-memory-mcp/tests/admin_status_search.rs
index 4e5d5090..816b0b53 100644
--- a/crates/ai-memory-mcp/tests/admin_status_search.rs
+++ b/crates/ai-memory-mcp/tests/admin_status_search.rs
@@ -61,6 +61,7 @@ async fn seed_page(store: &Store, title: &str, path: &str, body: &str) {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
};
store.writer.upsert_page(page).await.unwrap();
}
@@ -150,6 +151,7 @@ async fn list_projects_returns_workspace_project_pairs() {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
diff --git a/crates/ai-memory-mcp/tests/autoscope_multiuser.rs b/crates/ai-memory-mcp/tests/autoscope_multiuser.rs
index 7314fede..bafe9489 100644
--- a/crates/ai-memory-mcp/tests/autoscope_multiuser.rs
+++ b/crates/ai-memory-mcp/tests/autoscope_multiuser.rs
@@ -113,6 +113,7 @@ impl MultiUserHarness {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.expect("seed page");
diff --git a/crates/ai-memory-mcp/tests/autoscope_stress.rs b/crates/ai-memory-mcp/tests/autoscope_stress.rs
index 850523e8..3ae0874b 100644
--- a/crates/ai-memory-mcp/tests/autoscope_stress.rs
+++ b/crates/ai-memory-mcp/tests/autoscope_stress.rs
@@ -121,6 +121,7 @@ impl Harness {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.expect("seed page");
diff --git a/crates/ai-memory-store/migrations/V36__page_expiry.sql b/crates/ai-memory-store/migrations/V36__page_expiry.sql
new file mode 100644
index 00000000..29a3cc60
--- /dev/null
+++ b/crates/ai-memory-store/migrations/V36__page_expiry.sql
@@ -0,0 +1,9 @@
+-- Per-page TTL. `expires_at` (microseconds since epoch, UTC) is derived
+-- from the frontmatter `expires_at:` key — markdown stays the source of
+-- truth and a reindex rebuilds this column. NULL = never expires.
+-- Expired pages are hidden from search/recent/briefing and hard-deleted
+-- (file + rows) by the retention sweep.
+ALTER TABLE pages ADD COLUMN expires_at INTEGER;
+
+CREATE INDEX idx_pages_expiry ON pages(expires_at)
+ WHERE is_latest = 1 AND expires_at IS NOT NULL;
diff --git a/crates/ai-memory-store/src/lib.rs b/crates/ai-memory-store/src/lib.rs
index 09432615..a045994f 100644
--- a/crates/ai-memory-store/src/lib.rs
+++ b/crates/ai-memory-store/src/lib.rs
@@ -145,6 +145,7 @@ mod tests {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
}
}
@@ -2177,7 +2178,7 @@ mod tests {
let query = "what is the current decision about LLM embeddings";
let scoped = store
.reader
- .search_pages_for_project(ws, proj, query.into(), 10)
+ .search_pages_for_project(ws, proj, query.into(), 10, None)
.await
.unwrap();
let scoped_paths: Vec<&str> = scoped.iter().map(|hit| hit.path.as_str()).collect();
@@ -2193,7 +2194,7 @@ mod tests {
let top_one = store
.reader
- .search_pages_for_project(ws, proj, query.into(), 1)
+ .search_pages_for_project(ws, proj, query.into(), 1, None)
.await
.unwrap();
assert_eq!(top_one[0].path.as_str(), "decisions/embedding-policy.md");
@@ -2216,6 +2217,7 @@ mod tests {
String::new(),
0,
1,
+ None,
)
.await
.unwrap();
@@ -2223,7 +2225,7 @@ mod tests {
let session_evidence = store
.reader
- .search_pages_for_project(ws, proj, "chronicleonlytoken".into(), 10)
+ .search_pages_for_project(ws, proj, "chronicleonlytoken".into(), 10, None)
.await
.unwrap();
assert_eq!(session_evidence.len(), 1);
@@ -2361,7 +2363,7 @@ mod tests {
let hits = store
.reader
- .search_pages_for_project(ws, proj, "ai-memory".into(), 10)
+ .search_pages_for_project(ws, proj, "ai-memory".into(), 10, None)
.await
.unwrap();
assert_eq!(hits.len(), 1);
@@ -2403,6 +2405,7 @@ mod tests {
String::new(),
0,
10,
+ None,
)
.await
.unwrap();
@@ -3366,11 +3369,12 @@ mod tests {
let ws = super::ops::get_or_create_workspace(&mut conn, "default").unwrap();
let proj =
super::ops::get_or_create_project(&mut conn, &ws, "ai-memory", None).unwrap();
- let page_id = super::ops::upsert_page(
- &mut conn,
+ // Raw-SQL seeding: the v19-era schema predates the V36
+ // `expires_at` column that current `upsert_page` writes.
+ let page_id = super::ops::tests::insert_page_pre_v36(
+ &conn,
&sample_page(ws, proj, "notes/v103.md", "v1.0.3 upgrade fixture"),
- )
- .unwrap();
+ );
super::ops::begin_session(
&mut conn,
&NewSession {
@@ -3745,6 +3749,7 @@ mod tests {
pinned: true,
links: Vec::new(),
author_id: None,
+ expires_at: None,
};
store.writer.upsert_page(page).await.unwrap();
diff --git a/crates/ai-memory-store/src/ops.rs b/crates/ai-memory-store/src/ops.rs
index 146a576e..9daf1ca4 100644
--- a/crates/ai-memory-store/src/ops.rs
+++ b/crates/ai-memory-store/src/ops.rs
@@ -567,8 +567,8 @@ pub(crate) fn upsert_page_in_tx(
"INSERT INTO pages \
(id, workspace_id, project_id, path, path_search, title, tier, body, body_sha256, \
frontmatter_json, is_latest, supersedes, pinned, author_id, \
- created_at, updated_at) \
- VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, 1, ?11, ?12, ?13, ?14, ?14)",
+ created_at, updated_at, expires_at) \
+ VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, 1, ?11, ?12, ?13, ?14, ?14, ?15)",
params![
new_id.as_bytes(),
page.workspace_id.as_bytes(),
@@ -584,6 +584,7 @@ pub(crate) fn upsert_page_in_tx(
i64::from(page.pinned),
page.author_id.map(|id| id.as_bytes().to_vec()),
now,
+ page.expires_at.map(|ts| ts.as_microsecond()),
],
)?;
replace_links_in_tx(tx, &new_id, page)?;
@@ -605,8 +606,8 @@ pub(crate) fn upsert_page_in_tx(
tx.execute(
"INSERT INTO pages \
(id, workspace_id, project_id, path, path_search, title, tier, body, body_sha256, \
- frontmatter_json, is_latest, pinned, author_id, created_at, updated_at) \
- VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, 1, ?11, ?12, ?13, ?13)",
+ frontmatter_json, is_latest, pinned, author_id, created_at, updated_at, expires_at) \
+ VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, 1, ?11, ?12, ?13, ?13, ?14)",
params![
new_id.as_bytes(),
page.workspace_id.as_bytes(),
@@ -621,6 +622,7 @@ pub(crate) fn upsert_page_in_tx(
i64::from(page.pinned),
page.author_id.map(|id| id.as_bytes().to_vec()),
now,
+ page.expires_at.map(|ts| ts.as_microsecond()),
],
)?;
replace_links_in_tx(tx, &new_id, page)?;
@@ -2188,7 +2190,7 @@ pub fn delete_stale_page_embeddings(
}
#[cfg(test)]
-mod tests {
+pub(crate) mod tests {
//! Focused unit tests for the load-bearing mutating SQL paths.
//!
//! `Store::open` exercises these incidentally through
@@ -2506,6 +2508,7 @@ mod tests {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
}
}
@@ -4335,6 +4338,41 @@ mod tests {
assert_eq!(scratch_handoffs, 1);
}
+ /// Seed a minimal `pages` row against a pre-V36 schema. Migration-era
+ /// tests run old schemas, where current `upsert_page` — which writes
+ /// the V36 `expires_at` column — cannot be used to seed fixtures.
+ pub(crate) fn insert_page_pre_v36(conn: &Connection, page: &NewPage) -> PageId {
+ let id = PageId::new();
+ let now = Timestamp::now().as_microsecond();
+ let body_sha256: [u8; 32] = {
+ let mut hasher = Sha256::new();
+ hasher.update(page.body.as_bytes());
+ hasher.finalize().into()
+ };
+ conn.execute(
+ "INSERT INTO pages \
+ (id, workspace_id, project_id, path, path_search, title, tier, body, \
+ body_sha256, frontmatter_json, is_latest, pinned, created_at, updated_at) \
+ VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, 1, ?11, ?12, ?12)",
+ params![
+ id.as_bytes(),
+ page.workspace_id.as_bytes(),
+ page.project_id.as_bytes(),
+ page.path.as_str(),
+ path_search_text(page.path.as_str()),
+ page.title,
+ page.tier.as_str(),
+ page.body,
+ body_sha256.as_slice(),
+ serde_json::to_string(&page.frontmatter_json).unwrap(),
+ i64::from(page.pinned),
+ now,
+ ],
+ )
+ .unwrap();
+ id
+ }
+
#[test]
fn v18_migration_refuses_existing_split_brain_rows() {
let tmp = TempDir::new().unwrap();
@@ -4347,7 +4385,7 @@ mod tests {
let proj = get_or_create_project(&mut conn, &src_ws, "scratch", None).unwrap();
let mut bad_page = page(src_ws, proj, "notes/split.md", "body");
bad_page.workspace_id = stale_ws;
- upsert_page(&mut conn, &bad_page).unwrap();
+ insert_page_pre_v36(&conn, &bad_page);
let err = crate::migrations::run_to(&mut conn, 18).unwrap_err();
assert!(
diff --git a/crates/ai-memory-store/src/reader.rs b/crates/ai-memory-store/src/reader.rs
index 86377d3c..379785f2 100644
--- a/crates/ai-memory-store/src/reader.rs
+++ b/crates/ai-memory-store/src/reader.rs
@@ -36,6 +36,25 @@ use crate::maintenance::MaintenanceJob;
use crate::users::TOKEN_HASH_LEN;
use crate::workstream::{ManagedRunContext, StoredManagedRunStatus};
+/// TTL guard for retrieval surfaces (search / recent / embedding hits /
+/// graph neighbours / briefing lists): appended to a WHERE clause that
+/// already constrains `is_latest = 1`. `table` is the pages alias in
+/// that query; `now_param` is the positional placeholder bound to the
+/// current wall-clock in microseconds. One definition so the
+/// NULL-handling never drifts between queries. Exact-path/id reads and
+/// the decay-candidate walk deliberately do NOT use this — reads
+/// annotate expiry instead of hiding it, and the sweep must see
+/// expired rows to delete them.
+fn not_expired(table: &str, now_param: &str) -> String {
+ format!(" AND ({table}.expires_at IS NULL OR {table}.expires_at > {now_param})")
+}
+
+/// Current wall-clock in microseconds, for binding against
+/// [`not_expired`] fragments.
+fn now_us() -> i64 {
+ Timestamp::now().as_microsecond()
+}
+
fn page_kind_expr(path_column: &str, frontmatter_column: &str) -> String {
format!(
"COALESCE( \
@@ -688,6 +707,15 @@ pub struct PageMeta {
/// NULL`); `Some` when JOIN resolves a `users` row.
#[serde(skip_serializing_if = "Option::is_none")]
pub author: Option<PageAuthor>,
+ /// ISO-8601 TTL instant from the frontmatter `expires_at:` key.
+ /// Exact-path/id reads return expired pages (an explicit read is
+ /// not search); callers annotate expiry via [`PageMeta::expired`].
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub expires_at: Option<String>,
+ /// `true` when [`PageMeta::expires_at`] is in the past — the page
+ /// is hidden from retrieval and awaiting the next forget sweep.
+ #[serde(skip_serializing_if = "std::ops::Not::not")]
+ pub expired: bool,
}
/// One resolved cross-project edge (a link whose endpoints live in
@@ -889,14 +917,15 @@ impl ReaderPool {
pages.frontmatter_json, {kind_expr} AS kind \
FROM pages_fts \
JOIN pages ON pages.rowid = pages_fts.rowid \
- WHERE pages_fts MATCH ?1 AND pages.is_latest = 1 \
+ WHERE pages_fts MATCH ?1 AND pages.is_latest = 1{not_expired} \
ORDER BY pages_fts.rank \
- LIMIT ?2"
+ LIMIT ?2",
+ not_expired = not_expired("pages", "?3"),
);
let mut stmt = conn.prepare(&sql)?;
#[allow(clippy::cast_possible_wrap)]
let rows = stmt.query_map(
- params![fts_query, authority_candidate_limit(limit) as i64],
+ params![fts_query, authority_candidate_limit(limit) as i64, now_us()],
|row| {
let id_bytes: Vec<u8> = row.get(0)?;
let path: String = row.get(1)?;
@@ -970,14 +999,15 @@ impl ReaderPool {
JOIN pages ON pages.rowid = pages_fts.rowid \
JOIN projects ON projects.id = pages.project_id \
JOIN workspaces ON workspaces.id = pages.workspace_id \
- WHERE pages_fts MATCH ?1 AND pages.is_latest = 1 \
+ WHERE pages_fts MATCH ?1 AND pages.is_latest = 1{not_expired} \
ORDER BY pages_fts.rank \
- LIMIT ?2"
+ LIMIT ?2",
+ not_expired = not_expired("pages", "?3"),
);
let mut stmt = conn.prepare(&sql)?;
#[allow(clippy::cast_possible_wrap)]
let rows = stmt.query_map(
- params![fts_query, authority_candidate_limit(limit) as i64],
+ params![fts_query, authority_candidate_limit(limit) as i64, now_us()],
|row| {
let workspace_name: String = row.get(0)?;
let project_name: String = row.get(1)?;
@@ -1039,6 +1069,11 @@ impl ReaderPool {
/// Run an authority-adjusted full-text search scoped to one project.
///
+ /// `expiry_cutoff_us`: `None` hides pages whose TTL has passed as of
+ /// now (the default retrieval behaviour); `Some(cutoff)` compares
+ /// against that instant instead — pass `i64::MIN` to include
+ /// expired pages.
+ ///
/// # Errors
/// Propagates any SQL or pool error.
pub async fn search_pages_for_project(
@@ -1047,6 +1082,7 @@ impl ReaderPool {
project_id: ProjectId,
query: String,
limit: usize,
+ expiry_cutoff_us: Option<i64>,
) -> StoreResult<Vec<PageHit>> {
if limit == 0 {
return Ok(Vec::new());
@@ -1057,6 +1093,7 @@ impl ReaderPool {
project_id,
query,
authority_candidate_limit(limit),
+ expiry_cutoff_us,
)
.await?;
Ok(rerank_page_hits(candidates, limit))
@@ -1068,11 +1105,13 @@ impl ReaderPool {
project_id: ProjectId,
query: String,
candidate_limit: usize,
+ expiry_cutoff_us: Option<i64>,
) -> StoreResult<Vec<(PageHit, PageAuthority)>> {
let fts_query = normalize_fts_query(&query);
if fts_query.is_empty() || candidate_limit == 0 {
return Ok(Vec::new());
}
+ let cutoff = expiry_cutoff_us.unwrap_or_else(now_us);
self.with_conn(move |conn| {
let kind_expr = page_kind_expr("pages.path", "pages.frontmatter_json");
let sql = format!(
@@ -1085,9 +1124,10 @@ impl ReaderPool {
WHERE pages_fts MATCH ?1 \
AND pages.workspace_id = ?2 \
AND pages.project_id = ?3 \
- AND pages.is_latest = 1 \
+ AND pages.is_latest = 1{not_expired} \
ORDER BY pages_fts.rank \
- LIMIT ?4"
+ LIMIT ?4",
+ not_expired = not_expired("pages", "?5"),
);
let mut stmt = conn.prepare(&sql)?;
#[allow(clippy::cast_possible_wrap)]
@@ -1096,7 +1136,8 @@ impl ReaderPool {
fts_query,
workspace_id.as_bytes(),
project_id.as_bytes(),
- candidate_limit as i64
+ candidate_limit as i64,
+ cutoff
],
|row| {
let id_bytes: Vec<u8> = row.get(0)?;
@@ -1228,17 +1269,19 @@ impl ReaderPool {
/// Propagates any SQL or pool error.
pub async fn recent_pages(&self, limit: usize) -> StoreResult<Vec<PageHit>> {
self.with_conn(move |conn| {
- let mut stmt = conn.prepare_cached(
+ let sql = format!(
"SELECT id, path, title, \
substr(body, 1, 240) AS snip, \
CAST(updated_at AS REAL) AS rank \
FROM pages \
- WHERE is_latest = 1 \
+ WHERE is_latest = 1{not_expired} \
ORDER BY updated_at DESC \
LIMIT ?1",
- )?;
+ not_expired = not_expired("pages", "?2"),
+ );
+ let mut stmt = conn.prepare_cached(&sql)?;
#[allow(clippy::cast_possible_wrap)]
- let rows = stmt.query_map(params![limit as i64], |row| {
+ let rows = stmt.query_map(params![limit as i64, now_us()], |row| {
let id_bytes: Vec<u8> = row.get(0)?;
let path: String = row.get(1)?;
let title: String = row.get(2)?;
@@ -1273,18 +1316,25 @@ impl ReaderPool {
limit: usize,
) -> StoreResult<Vec<PageHit>> {
self.with_conn(move |conn| {
- let mut stmt = conn.prepare_cached(
+ let sql = format!(
"SELECT id, path, title, \
substr(body, 1, 240) AS snip, \
CAST(updated_at AS REAL) AS rank \
FROM pages \
- WHERE workspace_id = ?1 AND project_id = ?2 AND is_latest = 1 \
+ WHERE workspace_id = ?1 AND project_id = ?2 AND is_latest = 1{not_expired} \
ORDER BY updated_at DESC \
LIMIT ?3",
- )?;
+ not_expired = not_expired("pages", "?4"),
+ );
+ let mut stmt = conn.prepare_cached(&sql)?;
#[allow(clippy::cast_possible_wrap)]
let rows = stmt.query_map(
- params![workspace_id.as_bytes(), project_id.as_bytes(), limit as i64],
+ params![
+ workspace_id.as_bytes(),
+ project_id.as_bytes(),
+ limit as i64,
+ now_us()
+ ],
|row| {
let id_bytes: Vec<u8> = row.get(0)?;
let path: String = row.get(1)?;
@@ -1324,19 +1374,21 @@ impl ReaderPool {
/// Propagates any SQL or pool error.
pub async fn recent_pages_global(&self, limit: usize) -> StoreResult<Vec<PageHitWithMeta>> {
self.with_conn(move |conn| {
- let mut stmt = conn.prepare_cached(
+ let sql = format!(
"SELECT workspaces.name, projects.name, pages.path, pages.title, \
substr(pages.body, 1, 240) AS snip, \
CAST(pages.updated_at AS REAL) AS rank \
FROM pages \
JOIN projects ON projects.id = pages.project_id \
JOIN workspaces ON workspaces.id = pages.workspace_id \
- WHERE pages.is_latest = 1 \
+ WHERE pages.is_latest = 1{not_expired} \
ORDER BY pages.updated_at DESC \
LIMIT ?1",
- )?;
+ not_expired = not_expired("pages", "?2"),
+ );
+ let mut stmt = conn.prepare_cached(&sql)?;
#[allow(clippy::cast_possible_wrap)]
- let rows = stmt.query_map(params![limit as i64], |row| {
+ let rows = stmt.query_map(params![limit as i64, now_us()], |row| {
let workspace_name: String = row.get(0)?;
let project_name: String = row.get(1)?;
let path: String = row.get(2)?;
@@ -1836,22 +1888,25 @@ impl ReaderPool {
model: String,
dim: u32,
limit: usize,
+ expiry_cutoff_us: i64,
) -> StoreResult<Vec<(PageId, PagePath, f32)>> {
if limit == 0 {
return Ok(Vec::new());
}
self.with_conn(move |conn| {
- let mut stmt = conn.prepare_cached(
+ let sql = format!(
"SELECT page_embeddings.page_id, page_embeddings.vector, pages.path \
FROM page_embeddings \
JOIN pages ON pages.id = page_embeddings.page_id \
WHERE pages.workspace_id = ?1 \
AND pages.project_id = ?2 \
- AND pages.is_latest = 1 \
+ AND pages.is_latest = 1{not_expired} \
AND page_embeddings.provider = ?3 \
AND page_embeddings.model = ?4 \
AND page_embeddings.dim = ?5",
- )?;
+ not_expired = not_expired("pages", "?6"),
+ );
+ let mut stmt = conn.prepare_cached(&sql)?;
let rows = stmt.query_map(
params![
workspace_id.as_bytes(),
@@ -1859,6 +1914,7 @@ impl ReaderPool {
provider,
model,
dim,
+ expiry_cutoff_us,
],
|row| {
let id_bytes: Vec<u8> = row.get(0)?;
@@ -1947,7 +2003,7 @@ impl ReaderPool {
self.with_conn(move |conn| {
let mut stmt = conn.prepare(
"SELECT id, path, tier, pinned, updated_at, access_count, last_accessed_at, \
- frontmatter_json \
+ frontmatter_json, expires_at \
FROM pages \
WHERE workspace_id = ?1 AND project_id = ?2 AND is_latest = 1",
)?;
@@ -1975,10 +2031,12 @@ impl ReaderPool {
project_id: ProjectId,
seed_ids: Vec<PageId>,
limit: usize,
+ expiry_cutoff_us: Option<i64>,
) -> StoreResult<Vec<PageHit>> {
if seed_ids.is_empty() || limit == 0 {
return Ok(Vec::new());
}
+ let now = expiry_cutoff_us.unwrap_or_else(now_us);
self.with_conn(move |conn| {
let mut seen = std::collections::HashSet::new();
let mut out = Vec::new();
@@ -1995,8 +2053,13 @@ impl ReaderPool {
}
sql_params.push(Value::Blob(workspace_id.as_bytes().to_vec()));
sql_params.push(Value::Blob(project_id.as_bytes().to_vec()));
+ sql_params.push(Value::Integer(now));
sql_params.push(Value::Blob(workspace_id.as_bytes().to_vec()));
sql_params.push(Value::Blob(project_id.as_bytes().to_vec()));
+ sql_params.push(Value::Integer(now));
+
+ let out_not_expired = not_expired("tp", "?");
+ let in_not_expired = not_expired("fp", "?");
let mut sql = String::with_capacity(values_clause.len() + 1_400);
write!(
@@ -2009,7 +2072,7 @@ impl ReaderPool {
FROM seeds \
JOIN links l ON l.from_page_id = seeds.seed_id \
JOIN pages tp ON tp.id = l.to_page_id \
- WHERE tp.workspace_id = ? AND tp.project_id = ? AND tp.is_latest = 1 \
+ WHERE tp.workspace_id = ? AND tp.project_id = ? AND tp.is_latest = 1{out_not_expired} \
UNION ALL \
SELECT fp.id AS id, fp.path AS path, fp.title AS title, \
substr(fp.body, 1, 240) AS snippet, \
@@ -2017,7 +2080,7 @@ impl ReaderPool {
FROM seeds \
JOIN links l ON l.to_page_id = seeds.seed_id \
JOIN pages fp ON fp.id = l.from_page_id \
- WHERE fp.workspace_id = ? AND fp.project_id = ? AND fp.is_latest = 1 \
+ WHERE fp.workspace_id = ? AND fp.project_id = ? AND fp.is_latest = 1{in_not_expired} \
) \
SELECT id, path, title, snippet \
FROM neighbors \
@@ -2125,6 +2188,10 @@ impl ReaderPool {
///
/// k=60 is the canonical RRF constant.
///
+ /// `expiry_cutoff_us`: see [`Self::search_pages_for_project`]. Every
+ /// stream shares one cutoff so a page cannot be filtered out of one
+ /// ranker while still seeding another.
+ ///
/// # Errors
/// Propagates any SQL or pool error.
#[allow(clippy::too_many_arguments)]
@@ -2138,7 +2205,9 @@ impl ReaderPool {
model: String,
dim: u32,
limit: usize,
+ expiry_cutoff_us: Option<i64>,
) -> StoreResult<Vec<PageHit>> {
+ let cutoff = expiry_cutoff_us.unwrap_or_else(now_us);
// Authority is applied after RRF, so every stream needs the same
// bounded candidate window used by FTS-only search. A `limit * 2`
// window was too narrow at small limits for a canonical page to enter
@@ -2147,7 +2216,13 @@ impl ReaderPool {
// Fetch FTS5 hits first.
let fts_candidates = self
- .search_page_candidates_for_project(workspace_id, project_id, query, candidate_limit)
+ .search_page_candidates_for_project(
+ workspace_id,
+ project_id,
+ query,
+ candidate_limit,
+ Some(cutoff),
+ )
.await?;
let mut authorities: std::collections::HashMap<PageId, PageAuthority> = fts_candidates
.iter()
@@ -2168,6 +2243,7 @@ impl ReaderPool {
model,
dim,
candidate_limit,
+ cutoff,
)
.await?;
}
@@ -2184,7 +2260,13 @@ impl ReaderPool {
}
}
let graph_hits = self
- .graph_neighbors_for_project(workspace_id, project_id, seed_ids, candidate_limit)
+ .graph_neighbors_for_project(
+ workspace_id,
+ project_id,
+ seed_ids,
+ candidate_limit,
+ Some(cutoff),
+ )
.await?;
// RRF fuse: score(d) = Σ 1/(k + rank_i(d)) over rankers.
@@ -2396,11 +2478,12 @@ impl ReaderPool {
"SELECT path, title, {kind_expr} AS kind, \
updated_at \
FROM pages \
- WHERE is_latest = 1 AND path GLOB '_rules/*' \
- ORDER BY updated_at DESC"
+ WHERE is_latest = 1 AND path GLOB '_rules/*'{not_expired} \
+ ORDER BY updated_at DESC",
+ not_expired = not_expired("pages", "?1"),
))?;
let rules: Vec<BriefingPage> = rules_stmt
- .query_map([], briefing_page_from_row)?
+ .query_map(params![now_us], briefing_page_from_row)?
.collect::<Result<Vec<_>, _>>()?
.into_iter()
.collect::<Result<Vec<_>, _>>()?;
@@ -2409,11 +2492,12 @@ impl ReaderPool {
"SELECT path, title, {kind_expr} AS kind, \
updated_at \
FROM pages \
- WHERE is_latest = 1 AND path GLOB '_slots/*' \
- ORDER BY path ASC"
+ WHERE is_latest = 1 AND path GLOB '_slots/*'{not_expired} \
+ ORDER BY path ASC",
+ not_expired = not_expired("pages", "?1"),
))?;
let slots: Vec<BriefingPage> = slots_stmt
- .query_map([], briefing_page_from_row)?
+ .query_map(params![now_us], briefing_page_from_row)?
.collect::<Result<Vec<_>, _>>()?
.into_iter()
.collect::<Result<Vec<_>, _>>()?;
@@ -2422,12 +2506,13 @@ impl ReaderPool {
"SELECT path, title, {kind_expr} AS kind, \
updated_at \
FROM pages \
- WHERE is_latest = 1 \
+ WHERE is_latest = 1{not_expired} \
ORDER BY updated_at DESC \
- LIMIT ?1"
+ LIMIT ?1",
+ not_expired = not_expired("pages", "?2"),
))?;
let recent_pages: Vec<BriefingPage> = recent_stmt
- .query_map(params![recent_limit], briefing_page_from_row)?
+ .query_map(params![recent_limit, now_us], briefing_page_from_row)?
.collect::<Result<Vec<_>, _>>()?
.into_iter()
.collect::<Result<Vec<_>, _>>()?;
@@ -2520,11 +2605,12 @@ impl ReaderPool {
"SELECT path, title, {kind_expr} AS kind, \
updated_at \
FROM pages \
- WHERE workspace_id = ?1 AND project_id = ?2 AND is_latest = 1 AND path GLOB '_rules/*' \
- ORDER BY updated_at DESC"
+ WHERE workspace_id = ?1 AND project_id = ?2 AND is_latest = 1 AND path GLOB '_rules/*'{not_expired} \
+ ORDER BY updated_at DESC",
+ not_expired = not_expired("pages", "?3"),
))?;
let rules: Vec<BriefingPage> = rules_stmt
- .query_map(params![workspace_id.as_bytes(), project_id.as_bytes()], briefing_page_from_row)?
+ .query_map(params![workspace_id.as_bytes(), project_id.as_bytes(), now_us], briefing_page_from_row)?
.collect::<Result<Vec<_>, _>>()?
.into_iter()
.collect::<Result<Vec<_>, _>>()?;
@@ -2533,11 +2619,12 @@ impl ReaderPool {
"SELECT path, title, {kind_expr} AS kind, \
updated_at \
FROM pages \
- WHERE workspace_id = ?1 AND project_id = ?2 AND is_latest = 1 AND path GLOB '_slots/*' \
- ORDER BY path ASC"
+ WHERE workspace_id = ?1 AND project_id = ?2 AND is_latest = 1 AND path GLOB '_slots/*'{not_expired} \
+ ORDER BY path ASC",
+ not_expired = not_expired("pages", "?3"),
))?;
let slots: Vec<BriefingPage> = slots_stmt
- .query_map(params![workspace_id.as_bytes(), project_id.as_bytes()], briefing_page_from_row)?
+ .query_map(params![workspace_id.as_bytes(), project_id.as_bytes(), now_us], briefing_page_from_row)?
.collect::<Result<Vec<_>, _>>()?
.into_iter()
.collect::<Result<Vec<_>, _>>()?;
@@ -2546,12 +2633,13 @@ impl ReaderPool {
"SELECT path, title, {kind_expr} AS kind, \
updated_at \
FROM pages \
- WHERE workspace_id = ?1 AND project_id = ?2 AND is_latest = 1 \
+ WHERE workspace_id = ?1 AND project_id = ?2 AND is_latest = 1{not_expired} \
ORDER BY updated_at DESC \
- LIMIT ?3"
+ LIMIT ?3",
+ not_expired = not_expired("pages", "?4"),
))?;
let recent_pages: Vec<BriefingPage> = recent_stmt
- .query_map(params![workspace_id.as_bytes(), project_id.as_bytes(), recent_limit], briefing_page_from_row)?
+ .query_map(params![workspace_id.as_bytes(), project_id.as_bytes(), recent_limit, now_us], briefing_page_from_row)?
.collect::<Result<Vec<_>, _>>()?
.into_iter()
.collect::<Result<Vec<_>, _>>()?;
@@ -2598,17 +2686,24 @@ impl ReaderPool {
let recent_limit = recent_pages_limit.clamp(1, 100) as i64;
self.with_conn(move |conn| {
let kind_expr = page_kind_expr("path", "frontmatter_json");
- let mut core_stmt = conn.prepare_cached(
+ let core_sql = format!(
"SELECT path, title, body, pinned, updated_at \
FROM pages \
- WHERE workspace_id = ?1 AND project_id = ?2 AND is_latest = 1 \
+ WHERE workspace_id = ?1 AND project_id = ?2 AND is_latest = 1{not_expired} \
AND (pinned = 1 OR path GLOB '_rules/*' OR path GLOB '_slots/*') \
ORDER BY pinned DESC, path ASC \
LIMIT ?3",
- )?;
+ not_expired = not_expired("pages", "?4"),
+ );
+ let mut core_stmt = conn.prepare_cached(&core_sql)?;
let core: Vec<BriefPageBody> = core_stmt
.query_map(
- params![workspace_id.as_bytes(), project_id.as_bytes(), core_limit],
+ params![
+ workspace_id.as_bytes(),
+ project_id.as_bytes(),
+ core_limit,
+ now_us()
+ ],
|row| {
let updated_us: i64 = row.get(4)?;
Ok((
@@ -2643,13 +2738,19 @@ impl ReaderPool {
"SELECT path, title, {kind_expr} AS kind, \
updated_at \
FROM pages \
- WHERE workspace_id = ?1 AND project_id = ?2 AND is_latest = 1 \
+ WHERE workspace_id = ?1 AND project_id = ?2 AND is_latest = 1{not_expired} \
ORDER BY updated_at DESC \
- LIMIT ?3"
+ LIMIT ?3",
+ not_expired = not_expired("pages", "?4"),
))?;
let recent: Vec<BriefingPage> = recent_stmt
.query_map(
- params![workspace_id.as_bytes(), project_id.as_bytes(), recent_limit],
+ params![
+ workspace_id.as_bytes(),
+ project_id.as_bytes(),
+ recent_limit,
+ now_us()
+ ],
briefing_page_from_row,
)?
.collect::<Result<Vec<_>, _>>()?
@@ -2807,11 +2908,15 @@ impl ReaderPool {
"SELECT path, title, {kind_expr} AS kind, \
updated_at \
FROM pages \
- WHERE workspace_id = ?1 AND is_latest = 1 AND path GLOB '_rules/*' \
- ORDER BY updated_at DESC"
+ WHERE workspace_id = ?1 AND is_latest = 1 AND path GLOB '_rules/*'{not_expired} \
+ ORDER BY updated_at DESC",
+ not_expired = not_expired("pages", "?2"),
))?;
let rules: Vec<BriefingPage> = rules_stmt
- .query_map(params![workspace_id.as_bytes()], briefing_page_from_row)?
+ .query_map(
+ params![workspace_id.as_bytes(), now_us],
+ briefing_page_from_row,
+ )?
.collect::<Result<Vec<_>, _>>()?
.into_iter()
.collect::<Result<Vec<_>, _>>()?;
@@ -2820,11 +2925,15 @@ impl ReaderPool {
"SELECT path, title, {kind_expr} AS kind, \
updated_at \
FROM pages \
- WHERE workspace_id = ?1 AND is_latest = 1 AND path GLOB '_slots/*' \
- ORDER BY path ASC"
+ WHERE workspace_id = ?1 AND is_latest = 1 AND path GLOB '_slots/*'{not_expired} \
+ ORDER BY path ASC",
+ not_expired = not_expired("pages", "?2"),
))?;
let slots: Vec<BriefingPage> = slots_stmt
- .query_map(params![workspace_id.as_bytes()], briefing_page_from_row)?
+ .query_map(
+ params![workspace_id.as_bytes(), now_us],
+ briefing_page_from_row,
+ )?
.collect::<Result<Vec<_>, _>>()?
.into_iter()
.collect::<Result<Vec<_>, _>>()?;
@@ -2833,13 +2942,14 @@ impl ReaderPool {
"SELECT path, title, {kind_expr} AS kind, \
updated_at \
FROM pages \
- WHERE workspace_id = ?1 AND is_latest = 1 \
+ WHERE workspace_id = ?1 AND is_latest = 1{not_expired} \
ORDER BY updated_at DESC \
- LIMIT ?2"
+ LIMIT ?2",
+ not_expired = not_expired("pages", "?3"),
))?;
let recent_pages: Vec<BriefingPage> = recent_stmt
.query_map(
- params![workspace_id.as_bytes(), recent_limit],
+ params![workspace_id.as_bytes(), recent_limit, now_us],
briefing_page_from_row,
)?
.collect::<Result<Vec<_>, _>>()?
@@ -3114,7 +3224,7 @@ impl ReaderPool {
{kind_expr}, \
pg.tier, pg.pinned, pg.created_at, pg.updated_at, \
sp.path AS supersedes_path, \
- au.username, au.name, au.email \
+ au.username, au.name, au.email, pg.expires_at \
FROM pages pg \
JOIN projects p ON p.id = pg.project_id \
JOIN workspaces w ON w.id = pg.workspace_id \
@@ -3147,7 +3257,7 @@ impl ReaderPool {
{kind_expr}, \
pg.tier, pg.pinned, pg.created_at, pg.updated_at, \
sp.path AS supersedes_path, \
- au.username, au.name, au.email \
+ au.username, au.name, au.email, pg.expires_at \
FROM pages pg \
JOIN projects p ON p.id = pg.project_id \
JOIN workspaces w ON w.id = pg.workspace_id \
@@ -3642,7 +3752,7 @@ impl ReaderPool {
{kind_expr}, \
pg.tier, pg.pinned, pg.created_at, pg.updated_at, \
sp.path AS supersedes_path, \
- au.username, au.name, au.email \
+ au.username, au.name, au.email, pg.expires_at \
FROM pages pg \
JOIN projects p ON p.id = pg.project_id \
JOIN workspaces w ON w.id = pg.workspace_id \
@@ -4666,6 +4776,7 @@ fn page_meta_from_row(row: &rusqlite::Row<'_>) -> rusqlite::Result<StoreResult<P
let author_username: Option<String> = row.get(12)?;
let author_name: Option<String> = row.get(13)?;
let author_email: Option<String> = row.get(14)?;
+ let expires_us: Option<i64> = row.get(15)?;
let author = author_username.map(|username| PageAuthor {
username,
name: author_name,
@@ -4683,6 +4794,10 @@ fn page_meta_from_row(row: &rusqlite::Row<'_>) -> rusqlite::Result<StoreResult<P
let updated_at = jiff::Timestamp::from_microsecond(updated_us)
.map(|ts| ts.to_string())
.unwrap_or_default();
+ let expired = expires_us.is_some_and(|us| us <= now_us());
+ let expires_at = expires_us
+ .and_then(|us| jiff::Timestamp::from_microsecond(us).ok())
+ .map(|ts| ts.to_string());
Ok(Ok(PageMeta {
workspace_name,
@@ -4698,6 +4813,8 @@ fn page_meta_from_row(row: &rusqlite::Row<'_>) -> rusqlite::Result<StoreResult<P
updated_at,
supersedes,
author,
+ expires_at,
+ expired,
}))
}
@@ -4855,6 +4972,9 @@ pub struct DecayCandidate {
/// Frontmatter JSON; the sweep peeks at it for an explicit
/// `pinned: true` (which overrides the schema flag).
pub frontmatter_json: String,
+ /// TTL instant in microseconds since epoch. The sweep hard-deletes
+ /// pages past this regardless of tier or pin.
+ pub expires_at_us: Option<i64>,
}
fn row_to_decay_candidate(
@@ -4868,6 +4988,7 @@ fn row_to_decay_candidate(
let access_count: i64 = row.get(5)?;
let last_accessed_at_us: Option<i64> = row.get(6)?;
let frontmatter_json: String = row.get(7)?;
+ let expires_at_us: Option<i64> = row.get(8)?;
Ok(materialise_decay_candidate(
id_bytes,
path,
@@ -4877,6 +4998,7 @@ fn row_to_decay_candidate(
access_count,
last_accessed_at_us,
frontmatter_json,
+ expires_at_us,
))
}
@@ -4890,6 +5012,7 @@ fn materialise_decay_candidate(
access_count: i64,
last_accessed_at_us: Option<i64>,
frontmatter_json: String,
+ expires_at_us: Option<i64>,
) -> StoreResult<DecayCandidate> {
Ok(DecayCandidate {
id: PageId::from_slice(&id_bytes)?,
@@ -4902,6 +5025,7 @@ fn materialise_decay_candidate(
access_count: u32::try_from(access_count.max(0)).unwrap_or(u32::MAX),
last_accessed_at_us,
frontmatter_json,
+ expires_at_us,
})
}
diff --git a/crates/ai-memory-store/src/writer.rs b/crates/ai-memory-store/src/writer.rs
index 0e3397fe..d602990e 100644
--- a/crates/ai-memory-store/src/writer.rs
+++ b/crates/ai-memory-store/src/writer.rs
@@ -1929,6 +1929,7 @@ mod tests {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
}
}
diff --git a/crates/ai-memory-web/src/routes/api.rs b/crates/ai-memory-web/src/routes/api.rs
index f023425b..f520ffec 100644
--- a/crates/ai-memory-web/src/routes/api.rs
+++ b/crates/ai-memory-web/src/routes/api.rs
@@ -366,7 +366,13 @@ async fn search_scopes(
for scope in scopes {
let hits = state
.reader
- .search_pages_for_project(scope.workspace_id, scope.project_id, term.clone(), limit)
+ .search_pages_for_project(
+ scope.workspace_id,
+ scope.project_id,
+ term.clone(),
+ limit,
+ None,
+ )
.await?;
for hit in hits {
hits_by_id
diff --git a/crates/ai-memory-web/tests/routes.rs b/crates/ai-memory-web/tests/routes.rs
index 78d227b5..7154f022 100644
--- a/crates/ai-memory-web/tests/routes.rs
+++ b/crates/ai-memory-web/tests/routes.rs
@@ -38,6 +38,7 @@ fn new_page(
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
}
}
@@ -621,6 +622,7 @@ async fn api_pages_derives_kind_from_path_when_frontmatter_absent() {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
@@ -640,6 +642,7 @@ async fn api_pages_derives_kind_from_path_when_frontmatter_absent() {
pinned: false,
links: Vec::new(),
author_id: None,
+ expires_at: None,
})
.await
.unwrap();
diff --git a/crates/ai-memory-wiki/src/wiki.rs b/crates/ai-memory-wiki/src/wiki.rs
index 951ee1ce..8aee8bdf 100644
--- a/crates/ai-memory-wiki/src/wiki.rs
+++ b/crates/ai-memory-wiki/src/wiki.rs
@@ -376,7 +376,7 @@ impl Wiki {
let md = parse(&raw)?;
let title = derive_title(&md.frontmatter, &md.body, &path);
let links = extract_links(&md.body, &path);
- let (tier, pinned) = derive_index_metadata(&path, &md.frontmatter)?;
+ let meta = derive_index_metadata(&path, &md.frontmatter)?;
let _guard = self.mutation_lock.read().await;
self.ensure_project_workspace(workspace_id, project_id)
@@ -394,11 +394,12 @@ impl Wiki {
path,
title,
body: md.body,
- tier,
+ tier: meta.tier,
frontmatter_json: md.frontmatter,
- pinned,
+ pinned: meta.pinned,
links,
author_id: None,
+ expires_at: meta.expires_at,
})
.await?;
Ok(id)
@@ -766,6 +767,7 @@ impl Wiki {
let title = self.sanitizer.scrub(&detail.summary.title);
let links = extract_links(&markdown.body, &path);
let emitted = emit(&markdown)?;
+ let expires_at = parse_expires_at(&path, &markdown.frontmatter)?;
let page = NewPage {
workspace_id,
project_id,
@@ -777,6 +779,7 @@ impl Wiki {
pinned: is_slot_path(&path),
links,
author_id,
+ expires_at,
};
let result = {
@@ -856,7 +859,7 @@ impl Wiki {
let links = extract_links(&md.body, &path);
// Markdown is the source of truth: preserve explicit tier/pinned
// metadata on reindex instead of forcing every page back to semantic.
- let (tier, pinned) = derive_index_metadata(&path, &md.frontmatter)?;
+ let meta = derive_index_metadata(&path, &md.frontmatter)?;
let id = self
.writer
.upsert_page(NewPage {
@@ -865,11 +868,11 @@ impl Wiki {
path,
title,
body: md.body,
- tier,
+ tier: meta.tier,
frontmatter_json: md.frontmatter,
- pinned,
+ pinned: meta.pinned,
links,
-
+ expires_at: meta.expires_at,
author_id: None,
})
.await?;
@@ -1135,19 +1138,22 @@ impl Wiki {
// Build NewPage batch with the precomputed titles.
let pages: Vec<ai_memory_core::NewPage> = staged_files
.iter()
- .map(|(req, _, _, _)| ai_memory_core::NewPage {
- workspace_id: req.workspace_id,
- project_id: req.project_id,
- path: req.path.clone(),
- title: req.title.clone().unwrap_or_default(),
- body: req.body.clone(),
- tier: req.tier,
- frontmatter_json: req.frontmatter.clone(),
- pinned: req.pinned,
- links: extract_links(&req.body, &req.path),
- author_id: req.author_id,
+ .map(|(req, _, _, _)| {
+ Ok(ai_memory_core::NewPage {
+ workspace_id: req.workspace_id,
+ project_id: req.project_id,
+ path: req.path.clone(),
+ title: req.title.clone().unwrap_or_default(),
+ body: req.body.clone(),
+ tier: req.tier,
+ frontmatter_json: req.frontmatter.clone(),
+ pinned: req.pinned,
+ links: extract_links(&req.body, &req.path),
+ author_id: req.author_id,
+ expires_at: parse_expires_at(&req.path, &req.frontmatter)?,
+ })
})
- .collect();
+ .collect::<WikiResult<Vec<_>>>()?;
// Install files first so the DB is never ahead of markdown. If the
// SQL batch fails below, rollback restores the prior disk state;
@@ -1269,6 +1275,7 @@ impl Wiki {
.map(|t| self.sanitizer.scrub(&t))
.unwrap_or_else(|| derive_title(&markdown.frontmatter, &markdown.body, &path));
let links = extract_links(&markdown.body, &path);
+ let expires_at = parse_expires_at(&path, &markdown.frontmatter)?;
let Markdown {
frontmatter: final_frontmatter,
@@ -1304,6 +1311,7 @@ impl Wiki {
pinned,
links,
author_id,
+ expires_at,
})
.await
{
@@ -1410,7 +1418,7 @@ fn ai_memory_wiki_error(msg: &str) -> crate::WikiError {
fn derive_index_metadata(
path: &PagePath,
frontmatter: &serde_json::Value,
-) -> WikiResult<(Tier, bool)> {
+) -> WikiResult<IndexMetadata> {
let tier = match frontmatter.get("tier") {
None => Tier::Semantic,
Some(serde_json::Value::String(s)) => s.parse::<Tier>().map_err(|e| {
@@ -1431,7 +1439,65 @@ fn derive_index_metadata(
.get("pinned")
.and_then(|v| v.as_bool())
.unwrap_or(false);
- Ok((tier, pinned))
+ let expires_at = parse_expires_at(path, frontmatter)?;
+ Ok(IndexMetadata {
+ tier,
+ pinned,
+ expires_at,
+ })
+}
+
+/// Index-relevant metadata derived from a page's frontmatter. Markdown
+/// is the source of truth; every field here is rebuildable by a reindex.
+struct IndexMetadata {
+ tier: Tier,
+ pinned: bool,
+ expires_at: Option<jiff::Timestamp>,
+}
+
+/// Parse the frontmatter `expires_at:` key into a TTL instant.
+///
+/// Accepts RFC3339 (`2026-08-01T12:00:00Z`) or a bare `YYYY-MM-DD`,
+/// which means the *end* of that day in UTC — a date-only TTL that
+/// expired at midnight would surprise anyone who wrote "expires
+/// 2026-08-01" meaning "good through the 1st".
+pub(crate) fn parse_expires_at(
+ path: &PagePath,
+ frontmatter: &serde_json::Value,
+) -> WikiResult<Option<jiff::Timestamp>> {
+ let raw = match frontmatter.get("expires_at") {
+ None | Some(serde_json::Value::Null) => return Ok(None),
+ Some(serde_json::Value::String(s)) => s.trim(),
+ Some(_) => {
+ return Err(ai_memory_wiki_error(&format!(
+ "invalid non-string expires_at in frontmatter for {}",
+ path.as_str()
+ )));
+ }
+ };
+ if raw.is_empty() {
+ return Ok(None);
+ }
+ if let Ok(ts) = raw.parse::<jiff::Timestamp>() {
+ return Ok(Some(ts));
+ }
+ if let Ok(date) = raw.parse::<jiff::civil::Date>() {
+ let ts = date
+ .at(23, 59, 59, 999_999_000)
+ .to_zoned(jiff::tz::TimeZone::UTC)
+ .map_err(|e| {
+ ai_memory_wiki_error(&format!(
+ "invalid expires_at date in frontmatter for {}: {e}",
+ path.as_str()
+ ))
+ })?
+ .timestamp();
+ return Ok(Some(ts));
+ }
+ Err(ai_memory_wiki_error(&format!(
+ "invalid expires_at in frontmatter for {} (want RFC3339 or YYYY-MM-DD): {raw}",
+ path.as_str()
+ )))
}
fn canonicalize_index_frontmatter(
@@ -1888,7 +1954,7 @@ mod tests {
assert!(
store
.reader
- .search_pages_for_project(ws, proj, "proposed body".into(), 10)
+ .search_pages_for_project(ws, proj, "proposed body".into(), 10, None)
.await
.unwrap()
.is_empty()
diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
index 2414bf91..632ca21d 100644
--- a/docs/ARCHITECTURE.md
+++ b/docs/ARCHITECTURE.md
@@ -120,7 +120,9 @@ from hook paths.
to at most once per page per minute, so a burst of overlapping searches
does not flood the writer actor with redundant reinforcement writes.
7. The forget sweep runs on demand and on the server's `[maintenance]`
- schedule: pages with `retention < cold_threshold` are soft-deleted;
+ schedule: pages past their frontmatter `expires_at:` TTL are
+ hard-deleted through the wiki layer (file + rows, pin or not);
+ pages with `retention < cold_threshold` are soft-deleted;
soft-deletions older than `hard_delete_after_days` with no subsequent
access get purged. Semantic / pinned / freshly-touched pages survive.
Scheduled sweep, rule-based lint, and opt-in embedding backfill ticks
@@ -232,7 +234,7 @@ separately gated Claude Code assistant/Stop excerpt remains capped at 2 KB.
| Table | What |
|---|---|
| `workspaces`, `projects` | Top of the 3-tuple identity coordinate. |
-| `pages` | Versioned wiki pages with `is_latest` + `supersedes` chain. M8 columns: `last_accessed_at`, `access_count`, `superseded_at`. M9 cols: `embedding_provider`, `embedding_model`, `embedding_dim`. |
+| `pages` | Versioned wiki pages with `is_latest` + `supersedes` chain. M8 columns: `last_accessed_at`, `access_count`, `superseded_at`. M9 cols: `embedding_provider`, `embedding_model`, `embedding_dim`. V36: `expires_at` (frontmatter TTL). |
| `pages_fts` | FTS5 virtual table over `(title, body)`, auto-synced by triggers. |
| `sessions`, `observations` | Sanitized, bounded lifecycle-hook projections. `sessions.ended_observation_count` is the stable generation watermark for resumed-session re-end eligibility; wall clocks are not used for that decision. They are an operational audit trail, not a complete native transcript. |
| `session_consolidation_jobs` | Durable, observation-generation-idempotent queue for opt-in SessionEnd LLM consolidation. One bounded server worker leases jobs, retries provider failures with backoff, and recovers expired leases after restart. |
@@ -313,7 +315,7 @@ invariants below.
| Tool | Hint | Purpose |
|---|---|---|
-| `memory_query` | read-only | FTS5 + graph RRF + optional vector RRF search, followed by bounded kind/tier/pinned/tag authority adjustment and raw fallback. Bumps access counters for page hits. Defaults to the current project; default-scoped calls also union the reserved `_global` preferences scope as `global_scope_hits`; `scopes` searches named sibling projects; `global=true` searches every project at once (each hit annotated with its workspace + project). |
+| `memory_query` | read-only | FTS5 + graph RRF + optional vector RRF search, followed by bounded kind/tier/pinned/tag authority adjustment and raw fallback. Bumps access counters for page hits. Defaults to the current project; default-scoped calls also union the reserved `_global` preferences scope as `global_scope_hits`; `scopes` searches named sibling projects; `global=true` searches every project at once (each hit annotated with its workspace + project). `include_expired=true` also returns TTL-expired pages. |
| `memory_recent` | read-only | Most-recently-updated `is_latest=1` pages. |
| `memory_read_page` | read-only | Fetch the FULL body of a single wiki page by `path` or by top FTS5 hit for a `query`; optional `workspace` + `project` targets a named sibling workspace/project. Use when an agent needs more than the 24-word snippets from `memory_query`. |
| `memory_status` | read-only | Counts, paths, version. |