Skip to content

armgit / macro

Public Imported from GitHub
0

Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.

feat(cache): read Mail pages through a row position that survives writes (#8909)

teo0f0db98628cb
diff --git a/apps/web/docs/local-predicate-pagination-plan.md b/apps/web/docs/local-predicate-pagination-plan.md
index 1958d5e380..ac97e0432d 100644
--- a/apps/web/docs/local-predicate-pagination-plan.md
+++ b/apps/web/docs/local-predicate-pagination-plan.md
@@ -3,7 +3,7 @@
Status: **Mail first slice implemented; other flat views retain server-baseline reconciliation**
The `soup-mail-v1` ALL/INBOX slice now uses exclusive keyset pages bound to a
-query fingerprint, engine generation and cache revision. Pages cover complete
+query fingerprint and engine generation; a cursor stays valid across cache revisions. Pages cover complete
cached Mail projections, not proof of whole-mailbox coverage. This is separate
from the all-or-network proposal below; other Soup partitions continue to use
server-baseline reconciliation. See `docs/agent-guide/email.md` (Cached Mail filtering) for the shipped scope and remaining filters.
@@ -122,12 +122,11 @@ Expose an opaque, versioned `EntityFilterCursor` through the browser protocol. I
- cursor format version;
- active `soup-flat-v1` profile version;
- cache-engine generation;
-- cache revision;
- canonical query fingerprint excluding page limit and cursor;
- sort value;
- normalized record key.
-The query fingerprint prevents a cursor from one filter, sort, or direction from being reused with another. Generation binding prevents revision `"3"` from an old engine from matching revision `"3"` in a replacement engine. The existing `onCacheGenerationChanged` lifecycle must clear frontend cursor state; the worker boundary must also reject stale-generation cursors rather than relying only on UI ordering.
+The query fingerprint prevents a cursor from one filter, sort, or direction from being reused with another. Generation binding keeps a cursor from an old engine from being read against a replacement engine's index. The existing `onCacheGenerationChanged` lifecycle must clear frontend cursor state; the worker boundary must also reject stale-generation cursors rather than relying only on UI ordering.
Extend the browser API conceptually to:
@@ -153,7 +152,7 @@ type EntityFilterCacheResult =
| { kind: 'stale-cursor'; revision: CacheRevision };
```
-A stale cursor is a recoverable control-flow result: clear the local page chain and request its first page at the reported current revision. It is not a user-visible error and must not silently continue from the stale boundary.
+A stale cursor means the engine generation or query changed; native engines that predate row positions also return it after any write. It is a recoverable control-flow result: clear the local page chain and request its first page at the reported current revision. It is not a user-visible error and must not silently continue from the stale boundary.
## Keyset semantics
@@ -194,9 +193,9 @@ The existing revision-qualified reads provide the required consistency checks:
4. a mismatch discards the page and retries from the current revision;
5. a generation-change notification invalidates all revision and cursor watermarks.
-Continuation requests additionally require that the cursor revision equals the engine's current revision before predicate execution. Because engine commands are serialized, a successful predicate call observes one revision. Record selection is a separate command and retains the existing post-selection revision check.
+A cursor is a row position, valid across writes for one engine generation and query. A page read after a write continues after the same row at the current revision. Rows that moved past the boundary since are picked up by the evaluation that write triggers, so the client re-reads its chain rather than stitching together pages from different revisions.
-Do not attempt MVCC snapshots across revisions. Revision change means reset and reevaluate, not continue reading an old snapshot.
+A Mail page request may pass `through`, a page's `endCursor`, instead of a cursor. The engine reads every row from the top through that position, and at least `limit` of them, within one engine command, paging the index past its per-query limit. A chain re-read this way ends at the same row when mail arrives above it. Because engine commands are serialized, a successful predicate call observes one revision. Record selection is a separate command and retains the existing post-selection revision check.
## Frontend local page state
@@ -264,7 +263,7 @@ In `cache-turso` and `cache-core`:
- fetch one bounded extra result for next-cursor detection;
- retain completeness and relevant-uncertainty checks in the same read transaction;
- return revision-qualified `PredicatePage` values;
-- validate expected cursor revision/generation before execution;
+- validate the cursor's generation and query before execution;
- return typed stale-cursor and incomplete outcomes;
- verify relevant authoritative and optimistic indexes with `EXPLAIN QUERY PLAN`;
- avoid scanning normalized record blobs, queue JSON, or projection blobs.
@@ -371,7 +370,7 @@ Track at least:
### WASM and protocol tests
- first and continuation pages cross the JS boundary with exact revisions;
-- stale revision and stale generation cursors are rejected;
+- stale generation and changed-query cursors are rejected, and a cursor continues after the same row across writes;
- cursor query fingerprints cannot be reused with changed filters or sort;
- filter page and selected records carry the same revision;
- realtime Soup items can appear on a later local page without a Soup query fetch;
diff --git a/apps/web/tauri/graphql_cache_plugin/src/engine.rs b/apps/web/tauri/graphql_cache_plugin/src/engine.rs
index 3ed13aa072..161bcbbe5c 100644
--- a/apps/web/tauri/graphql_cache_plugin/src/engine.rs
+++ b/apps/web/tauri/graphql_cache_plugin/src/engine.rs
@@ -607,7 +607,7 @@ impl EngineHandle {
.map_err(|error| error.to_string())
}
- /// Evaluates a Soup predicate or a revision-bound cached Mail page.
+ /// Evaluates a Soup predicate or a cached Mail page.
pub async fn entity_filter(
&self,
request: EntityFilterRequest,
diff --git a/apps/web/tauri/graphql_cache_plugin/src/engine/test/soup.rs b/apps/web/tauri/graphql_cache_plugin/src/engine/test/soup.rs
index 8d3aa20ec8..e0a4cea8f0 100644
--- a/apps/web/tauri/graphql_cache_plugin/src/engine/test/soup.rs
+++ b/apps/web/tauri/graphql_cache_plugin/src/engine/test/soup.rs
@@ -215,7 +215,7 @@ fn native_backfill_evaluates_unseen_mail_filters_without_query_baselines() {
}
#[test]
-fn native_mail_pagination_and_revision_bound_cursors() {
+fn native_mail_pagination_and_cursors_that_survive_writes() {
let handle = spawn_handle();
write_data(&handle, MAIL, seed(), Some(VIEWER));
let first = evaluate(&handle, request("ALL", None, 2, None));
@@ -239,8 +239,11 @@ fn native_mail_pagination_and_revision_bound_cursors() {
);
write_data(&handle, PARTIAL, patch(VIEWER, 4, false), None);
assert_eq!(
- evaluate(&handle, request("ALL", None, 2, Some(first_cursor)))["kind"],
- "stale-cursor"
+ keys(&evaluate(
+ &handle,
+ request("ALL", None, 2, Some(first_cursor))
+ )),
+ vec![key(9), key(8)]
);
assert!(block_on(handle.entity_filter(request("ALL", None, 2, Some("not a cursor")))).is_err());
}
diff --git a/crates/predicate_index/src/lib.rs b/crates/predicate_index/src/lib.rs
index 8687a37a36..34e5273fc5 100644
--- a/crates/predicate_index/src/lib.rs
+++ b/crates/predicate_index/src/lib.rs
@@ -338,9 +338,12 @@ impl IndexDocument {
.iter()
.find(|fact| &fact.attribute == attribute)
.is_some_and(|fact| {
- directional_cmp(fact.value.cmp(value), *direction)
- .then_with(|| directional_cmp(self.record_key.cmp(key), *tie_direction))
- == Ordering::Greater
+ sorts_after(
+ (fact.value, &self.record_key),
+ (*value, key),
+ *direction,
+ *tie_direction,
+ )
}),
PredicateExpr::And(left, right) => self.matches(left) && self.matches(right),
PredicateExpr::Or(left, right) => self.matches(left) || self.matches(right),
@@ -445,6 +448,16 @@ impl ValidatedIndexQuery {
Self::new(query)
}
+ /// Whether a row sorts after a keyset boundary, as [`Self::after`] decides.
+ pub fn sorts_after(&self, row: (i64, &RecordKey), boundary: (i64, &RecordKey)) -> bool {
+ sorts_after(
+ row,
+ boundary,
+ self.0.sort_direction,
+ self.0.tie_break_direction,
+ )
+ }
+
/// Whether this query can inspect documents in `profile` and `partition`.
pub fn includes_scope(&self, profile: &Profile, partition: &Token) -> bool {
self.0.profile == *profile
@@ -1057,6 +1070,17 @@ fn upper_matches(value: i64, bound: RangeBound) -> bool {
}
}
+fn sorts_after(
+ (value, key): (i64, &RecordKey),
+ (boundary_value, boundary_key): (i64, &RecordKey),
+ direction: SortDirection,
+ tie_direction: SortDirection,
+) -> bool {
+ directional_cmp(value.cmp(&boundary_value), direction)
+ .then_with(|| directional_cmp(key.cmp(boundary_key), tie_direction))
+ == Ordering::Greater
+}
+
fn directional_cmp(ordering: Ordering, direction: SortDirection) -> Ordering {
match direction {
SortDirection::Asc => ordering,
diff --git a/crates/soup_filter_cache_adapter/AGENTS.md b/crates/soup_filter_cache_adapter/AGENTS.md
index 3cf8152b2e..1c4e549148 100644
--- a/crates/soup_filter_cache_adapter/AGENTS.md
+++ b/crates/soup_filter_cache_adapter/AGENTS.md
@@ -49,6 +49,10 @@ understand. Runs both natively (Tauri) and compiled to wasm (web).
- This crate must stay wasm-compatible — no `sqlx`, no native-only crates in
default features; native-only test fixtures live under
`[target.'cfg(not(target_arch = "wasm32"))'.dev-dependencies]`.
+- Native apps run this crate from their binary while the web bundle updates
+ over the air, so web code meets older versions. Requests use
+ `deny_unknown_fields`: send a new field only once a result shows the engine
+ supports it, as `endCursor` does for a Mail page's `through`.
## Related
diff --git a/crates/soup_filter_cache_adapter/src/mail.rs b/crates/soup_filter_cache_adapter/src/mail.rs
index 39f6d4ca1c..16f780ea01 100644
--- a/crates/soup_filter_cache_adapter/src/mail.rs
+++ b/crates/soup_filter_cache_adapter/src/mail.rs
@@ -1,4 +1,4 @@
-//! Canonical email facts and revision-bound pagination over synchronized Mail.
+//! Canonical email facts and keyset pagination over synchronized Mail.
use super::*;
use cache_core::{
engine::{Engine, EngineError},
@@ -636,19 +636,26 @@ pub fn new_generation() -> String {
pub struct PageRequest {
/// ALL, INBOX, DRAFTS or SENT.
pub view: String,
- /// Opaque revision/query-bound local continuation.
+ /// Opaque query-bound local continuation.
pub cursor: Option<String>,
+ /// A page's `end_cursor`: read from the top through that row, and at
+ /// least `limit` rows (up to 5,000), so a list re-read after writes ends
+ /// where it did.
+ pub through: Option<String>,
}
+/// A row position, valid across writes for one engine generation and query.
#[derive(Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct Cursor {
generation: String,
- revision: String,
query: String,
key: RecordKey,
value: i64,
}
+/// Rows past this read through a position are left to the next page.
+const MAX_THROUGH_ROWS: usize = 5_000;
+
/// Local Mail results are complete only over known cached projections, not the mailbox.
#[derive(Serialize)]
#[serde(
@@ -665,8 +672,10 @@ pub enum PageResult {
keys: Vec<String>,
/// View-correct display timestamps, aligned with keys.
sort_timestamps: Vec<String>,
- /// Query/revision-bound local continuation.
+ /// Query-bound local continuation.
next_cursor: Option<String>,
+ /// Position of the last row, to read through it again after writes.
+ end_cursor: Option<String>,
/// Whether queued optimistic work contributes to this page.
optimistic: bool,
},
@@ -677,7 +686,7 @@ pub enum PageResult {
/// Revision whose catalog is unavailable.
revision: String,
},
- /// Generation, revision or query changed. Restart at page one.
+ /// Generation or query changed. Restart at page one.
StaleCursor {
/// Current revision to restart from.
revision: String,
@@ -756,7 +765,14 @@ async fn page_impl<S: PredicateIndexStorage>(
current: bool,
) -> Result<PageResult, PageError<S::Error>> {
let revision = engine.current_revision().to_string();
- if limit == 0 || limit >= predicate_index::MAX_QUERY_LIMIT {
+ // A read through a position takes `limit` as its fewest rows and pages
+ // the index at its own size.
+ let page_size = if request.through.is_some() {
+ predicate_index::MAX_QUERY_LIMIT - 1
+ } else {
+ limit
+ };
+ if limit == 0 || page_size >= predicate_index::MAX_QUERY_LIMIT {
return Err(error(format!(
"Mail page limit must be 1..{}",
predicate_index::MAX_QUERY_LIMIT - 1
@@ -811,7 +827,7 @@ async fn page_impl<S: PredicateIndexStorage>(
SoupFlatRequest {
sort,
direction,
- limit: limit + 1,
+ limit: page_size + 1,
has_cursor: false,
},
&request.view,
@@ -819,40 +835,171 @@ async fn page_impl<S: PredicateIndexStorage>(
&viewer,
)
.map_err(error)?;
- let LocalCompileOutcome::Supported(mut query) = outcome else {
+ let LocalCompileOutcome::Supported(query) = outcome else {
return Ok(PageResult::Unsupported);
};
+ // Positions name rows of this query whatever page size reads them.
let fingerprint = uuid::Uuid::new_v5(
&uuid::Uuid::NAMESPACE_OID,
- serde_json::to_string(&query).map_err(error)?.as_bytes(),
+ serde_json::to_string(&query.with_limit(1).map_err(error)?)
+ .map_err(error)?
+ .as_bytes(),
)
.to_string();
- if let Some(cursor) = request.cursor {
+ // A position stays valid across writes: rows that moved past it since are
+ // read again by the caller's next evaluation of the rows above it.
+ let decode = |cursor: &str| {
if cursor.len() > 4096 {
- return Err(error("oversized Mail cursor").into());
+ return Err(error("oversized Mail cursor"));
}
- let cursor: Cursor = serde_json::from_str(&cursor).map_err(error)?;
- if cursor.generation != generation
- || cursor.revision != revision
- || cursor.query != fingerprint
- {
- return Ok(PageResult::StaleCursor { revision });
+ let cursor: Cursor = serde_json::from_str(cursor).map_err(error)?;
+ Ok((cursor.generation == generation && cursor.query == fingerprint).then_some(cursor))
+ };
+ let window = match (request.cursor, request.through) {
+ (Some(_), Some(_)) => {
+ return Err(error("a Mail page takes a cursor or a through position, not both").into());
}
- query = query.after(cursor.value, cursor.key).map_err(error)?;
- }
+ (Some(cursor), None) => {
+ let Some(cursor) = decode(&cursor)? else {
+ return Ok(PageResult::StaleCursor { revision });
+ };
+ let query = query.after(cursor.value, cursor.key).map_err(error)?;
+ read_page(engine, &query, limit).await?
+ }
+ (None, Some(through)) => {
+ let Some(through) = decode(&through)? else {
+ return Ok(PageResult::StaleCursor { revision });
+ };
+ let fewest = usize::from(limit).min(MAX_THROUGH_ROWS);
+ read_through(engine, &query, page_size, fewest, &through).await?
+ }
+ (None, None) => read_page(engine, &query, limit).await?,
+ };
+ let sort_timestamps = window
+ .values
+ .iter()
+ .map(|value| {
+ chrono::DateTime::from_timestamp_micros(*value)
+ .map(|ts| ts.to_rfc3339())
+ .ok_or_else(|| error("invalid page timestamp"))
+ })
+ .collect::<Result<Vec<_>, _>>()?;
+ let end_cursor = window
+ .keys
+ .last()
+ .zip(window.values.last())
+ .map(|(key, value)| {
+ serde_json::to_string(&Cursor {
+ generation: generation.into(),
+ query: fingerprint.clone(),
+ key: key.clone(),
+ value: *value,
+ })
+ .map_err(error)
+ })
+ .transpose()?;
+ Ok(PageResult::MailPage {
+ revision,
+ keys: window
+ .keys
+ .into_iter()
+ .map(|key| key.as_str().to_owned())
+ .collect(),
+ sort_timestamps,
+ next_cursor: end_cursor.clone().filter(|_| window.more),
+ end_cursor,
+ optimistic: window.optimistic,
+ })
+}
+
+/// Matching rows in index order, with their sort values.
+struct Window {
+ keys: Vec<RecordKey>,
+ values: Vec<i64>,
+ /// Matching rows remain after the last one.
+ more: bool,
+ optimistic: bool,
+}
+
+/// One page of `limit` rows. `query` asks for one more, to tell whether rows remain.
+async fn read_page<S: PredicateIndexStorage>(
+ engine: &mut Engine<S>,
+ query: &ValidatedIndexQuery,
+ limit: u16,
+) -> Result<Window, PageError<S::Error>> {
// Unknown rows never become fabricated matches; the UI labels this cached mail.
- let result = engine.reconcile_predicate_index(&query, &[]).await?;
+ let result = engine.reconcile_predicate_index(query, &[]).await?;
let mut keys = result.value.keys;
let more = keys.len() > usize::from(limit);
keys.truncate(usize::from(limit));
+ let values = sort_values(engine, query, &keys).await?;
+ Ok(Window {
+ keys,
+ values,
+ more,
+ optimistic: result.value.optimistic,
+ })
+}
+
+/// Rows from the top through `through`, and at least `fewest` of them, in
+/// index pages of `page_size` that `query` reads. The pages are read within
+/// one engine borrow, so no write lands between them.
+async fn read_through<S: PredicateIndexStorage>(
+ engine: &mut Engine<S>,
+ query: &ValidatedIndexQuery,
+ page_size: u16,
+ fewest: usize,
+ through: &Cursor,
+) -> Result<Window, PageError<S::Error>> {
+ let mut window = read_page(engine, query, page_size).await?;
+ let mut checked = 0;
+ loop {
+ let end = window
+ .keys
+ .iter()
+ .zip(&window.values)
+ .enumerate()
+ .skip(checked)
+ .position(|(index, (key, value))| {
+ index >= fewest && query.sorts_after((*value, key), (through.value, &through.key))
+ })
+ .map(|offset| checked + offset)
+ .or_else(|| (window.keys.len() > MAX_THROUGH_ROWS).then_some(MAX_THROUGH_ROWS));
+ if let Some(end) = end.map(|end| end.min(MAX_THROUGH_ROWS)) {
+ window.keys.truncate(end);
+ window.values.truncate(end);
+ window.more = true;
+ return Ok(window);
+ }
+ let (Some(key), Some(value), true) =
+ (window.keys.last(), window.values.last(), window.more)
+ else {
+ return Ok(window);
+ };
+ checked = window.keys.len();
+ let next = query.after(*value, key.clone()).map_err(error)?;
+ let page = read_page(engine, &next, page_size).await?;
+ window.keys.extend(page.keys);
+ window.values.extend(page.values);
+ window.more = page.more;
+ window.optimistic |= page.optimistic;
+ }
+}
+
+/// View-correct sort values of `keys`, from their effective projections.
+async fn sort_values<S: PredicateIndexStorage>(
+ engine: &mut Engine<S>,
+ query: &ValidatedIndexQuery,
+ keys: &[RecordKey],
+) -> Result<Vec<i64>, PageError<S::Error>> {
let bases = engine
.storage()
- .load_projection_states(&keys)
+ .load_projection_states(keys)
.await
.map_err(EngineError::Storage)?;
let shadows = engine
.storage()
- .load_optimistic_projections(&keys)
+ .load_optimistic_projections(keys)
.await
.map_err(EngineError::Storage)?;
let values = bases
@@ -876,36 +1023,5 @@ async fn page_impl<S: PredicateIndexStorage>(
.ok_or_else(|| error("missing page sort"))
})
.collect::<Result<Vec<_>, _>>()?;
- let sort_timestamps = values
- .iter()
- .map(|value| {
- chrono::DateTime::from_timestamp_micros(*value)
- .map(|ts| ts.to_rfc3339())
- .ok_or_else(|| error("invalid page timestamp"))
- })
- .collect::<Result<Vec<_>, _>>()?;
- let next_cursor = if more {
- Some(
- serde_json::to_string(&Cursor {
- generation: generation.into(),
- revision: revision.clone(),
- query: fingerprint,
- key: keys.last().expect("nonempty page").clone(),
- value: *values.last().expect("page has sort evidence"),
- })
- .map_err(error)?,
- )
- } else {
- None
- };
- Ok(PageResult::MailPage {
- revision,
- keys: keys
- .into_iter()
- .map(|key| key.as_str().to_owned())
- .collect(),
- sort_timestamps,
- next_cursor,
- optimistic: result.value.optimistic,
- })
+ Ok(values)
}
diff --git a/crates/soup_filter_cache_adapter/src/mail/test.rs b/crates/soup_filter_cache_adapter/src/mail/test.rs
index 8e3a88a6d3..4cdfaed434 100644
--- a/crates/soup_filter_cache_adapter/src/mail/test.rs
+++ b/crates/soup_filter_cache_adapter/src/mail/test.rs
@@ -12,6 +12,7 @@ use soup_filter_projection::{
mod drafts;
mod optimistic;
mod reminders;
+mod through;
const VIEWER: &str = "macro|[email protected]";
const QUERY: &str = r#"query MailSeed { user { id emailLinks { id } soup(input: {initial:{limit:100,emailView:ALL}}) { items { __typename id cacheProjection ... on GraphqlSoupEmailThread { linkId ownerId inboxVisible isRead isSignal latestInboundMessageTs reminderReturnedAt mailAllPreview { id } mailDraftPreview { id } mailSentPreview { id } updatedAt } } } } }"#;
@@ -114,6 +115,7 @@ async fn read<S: PredicateIndexStorage>(
PageRequest {
view: view.into(),
cursor,
+ through: None,
},
)
.await
@@ -273,7 +275,8 @@ async fn lifecycle<S: PredicateIndexStorage>(storage: S) {
10,
PageRequest {
view: "INBOX".into(),
- cursor: Some(cursor.clone())
+ cursor: Some(cursor.clone()),
+ through: None,
}
)
.await
@@ -340,10 +343,17 @@ async fn lifecycle<S: PredicateIndexStorage>(storage: S) {
.await
.unwrap()
.0;
- assert!(matches!(
- read(&mut engine, filters(), "INBOX", Some(cursor)).await,
- PageResult::StaleCursor { .. }
- ));
+ let PageResult::MailPage { keys, .. } =
+ read(&mut engine, filters(), "INBOX", Some(cursor)).await
+ else {
+ panic!("a write keeps the cursor's position")
+ };
+ assert_eq!(
+ keys,
+ all_keys(&mut engine, filters(), "INBOX").await[10..10 + keys.len()],
+ "the page after a write continues after the same row"
+ );
+ assert!(!keys.is_empty());
let mut f = filters();
f["emailFilter"] = json!({"tree":{"literal":{"read":true}}});
let PageResult::MailPage { keys, .. } = read(&mut engine, f.clone(), "ALL", None).await else {
@@ -536,6 +546,7 @@ fn page_limit_bounds_match_the_index_limit() {
PageRequest {
view: "ALL".into(),
cursor: None,
+ through: None,
},
)
.await;
diff --git a/crates/soup_filter_cache_adapter/src/mail/test/drafts.rs b/crates/soup_filter_cache_adapter/src/mail/test/drafts.rs
index d89be5d69e..e5f9aed252 100644
--- a/crates/soup_filter_cache_adapter/src/mail/test/drafts.rs
+++ b/crates/soup_filter_cache_adapter/src/mail/test/drafts.rs
@@ -27,6 +27,7 @@ async fn cached_keys<S: PredicateIndexStorage>(
PageRequest {
view: view.into(),
cursor: None,
+ through: None,
},
)
.await
diff --git a/crates/soup_filter_cache_adapter/src/mail/test/through.rs b/crates/soup_filter_cache_adapter/src/mail/test/through.rs
new file mode 100644
index 0000000000..d414fd5802
--- /dev/null
+++ b/crates/soup_filter_cache_adapter/src/mail/test/through.rs
@@ -0,0 +1,238 @@
+use super::*;
+
+fn key(n: u128) -> String {
+ format!("{TYPE}:{}", id(n))
+}
+
+/// A thread in the viewer's first inbox, whatever `row` would place it in.
+fn readable(n: u128) -> Value {
+ let mut row = row(n);
+ row["linkId"] = json!(id(1000));
+ row["ownerId"] = json!(VIEWER);
+ row
+}
+
+/// The same thread moved to an inbox the viewer cannot read, so it leaves Mail.
+fn unreadable(n: u128) -> Value {
+ let mut row = row(n);
+ row["linkId"] = json!(id(9999));
+ row["ownerId"] = json!("macro|[email protected]");
+ row
+}
+
+async fn store<S: Storage>(engine: &mut Engine<S>, rows: Vec<Value>) {
+ let data = json!({"user":{"id":VIEWER,"emailLinks":[{"id":id(1000)},{"id":id(1001)}],"soup":{"items":rows}}});
+ write(engine, QUERY, &data).await;
+}
+
+async fn through<S: PredicateIndexStorage>(
+ engine: &mut Engine<S>,
+ generation: &str,
+ through: &str,
+) -> PageResult {
+ through_at_least(engine, generation, through, 10).await
+}
+
+async fn through_at_least<S: PredicateIndexStorage>(
+ engine: &mut Engine<S>,
+ generation: &str,
+ through: &str,
+ fewest: u16,
+) -> PageResult {
+ page(
+ engine,
+ generation,
+ filters(),
+ "UPDATED_AT",
+ "DESC",
+ fewest,
+ PageRequest {
+ view: "ALL".into(),
+ cursor: None,
+ through: Some(through.into()),
+ },
+ )
+ .await
+ .unwrap()
+}
+
+/// Two loaded pages and the position of their last row.
+async fn loaded<S: PredicateIndexStorage>(engine: &mut Engine<S>) -> (Vec<String>, String) {
+ let PageResult::MailPage {
+ mut keys,
+ next_cursor: Some(cursor),
+ ..
+ } = read(engine, filters(), "ALL", None).await
+ else {
+ panic!("first page")
+ };
+ let PageResult::MailPage {
+ keys: more,
+ end_cursor: Some(edge),
+ ..
+ } = read(engine, filters(), "ALL", Some(cursor)).await
+ else {
+ panic!("second page")
+ };
+ keys.extend(more);
+ (keys, edge)
+}
+
+async fn window_edge<S: PredicateIndexStorage>(storage: S) {
+ let mut engine = Engine::new(storage);
+ write(&mut engine, QUERY, &seed()).await;
+ let all = all_keys(&mut engine, filters(), "ALL").await;
+ let (shown, edge) = loaded(&mut engine).await;
+ assert_eq!(shown, all[..20]);
+
+ let PageResult::MailPage {
+ keys,
+ next_cursor,
+ end_cursor,
+ ..
+ } = through(&mut engine, "generation", &edge).await
+ else {
+ panic!("window")
+ };
+ assert_eq!(keys, shown, "reads both pages back in one call");
+ assert_eq!(end_cursor.as_deref(), Some(edge.as_str()));
+ assert_eq!(next_cursor, end_cursor, "rows remain past the window");
+ let PageResult::MailPage { keys, .. } =
+ through_at_least(&mut engine, "generation", &edge, 30).await
+ else {
+ panic!("a position from a 10-row page reads at another size")
+ };
+ assert_eq!(keys, all[..30], "`limit` is the fewest rows");
+
+ store(&mut engine, vec![readable(80)]).await;
+ let PageResult::MailPage { keys, .. } = through(&mut engine, "generation", &edge).await else {
+ panic!("window")
+ };
+ assert_eq!(keys[0], key(80));
+ assert_eq!(
+ keys[1..],
+ shown,
+ "a new row above the window keeps its last row"
+ );
+
+ store(&mut engine, vec![unreadable(66)]).await;
+ let PageResult::MailPage { keys, .. } = through(&mut engine, "generation", &edge).await else {
+ panic!("window")
+ };
+ assert!(!keys.contains(&key(66)));
+ assert_eq!(
+ keys.last(),
+ shown.last(),
+ "a removed row above the window pulls nothing up from below"
+ );
+
+ let last = shown.last().unwrap().clone();
+ let n = uuid::Uuid::parse_str(last.split_once(':').unwrap().1)
+ .unwrap()
+ .as_u128();
+ store(&mut engine, vec![unreadable(n)]).await;
+ let PageResult::MailPage {
+ keys, next_cursor, ..
+ } = through(&mut engine, "generation", &edge).await
+ else {
+ panic!("a position outlives its row")
+ };
+ assert_eq!(keys.last(), shown.get(shown.len() - 2));
+ assert!(next_cursor.is_some());
+
+ assert!(matches!(
+ through(&mut engine, "other-generation", &edge).await,
+ PageResult::StaleCursor { .. }
+ ));
+ let both = page(
+ &mut engine,
+ "generation",
+ filters(),
+ "UPDATED_AT",
+ "DESC",
+ 10,
+ PageRequest {
+ view: "ALL".into(),
+ cursor: Some(edge.clone()),
+ through: Some(edge),
+ },
+ )
+ .await;
+ assert!(both.is_err(), "a cursor and a position are exclusive");
+}
+
+#[test]
+fn memory_window_keeps_its_last_row() {
+ pollster::block_on(window_edge(InMemoryStorage::new()))
+}
+
+#[test]
+fn turso_window_keeps_its_last_row() {
+ pollster::block_on(window_edge(
+ cache_turso::TursoStorage::open_in_memory("mail-through-test").unwrap(),
+ ))
+}
+
+#[test]
+fn a_short_window_still_fills_one_page() {
+ pollster::block_on(async {
+ let mut engine = Engine::new(InMemoryStorage::new());
+ write(&mut engine, QUERY, &seed()).await;
+ let all = all_keys(&mut engine, filters(), "ALL").await;
+ let PageResult::MailPage {
+ end_cursor: Some(edge),
+ ..
+ } = read(&mut engine, filters(), "ALL", None).await
+ else {
+ panic!("first page")
+ };
+ let first = uuid::Uuid::parse_str(all[0].split_once(':').unwrap().1)
+ .unwrap()
+ .as_u128();
+ store(&mut engine, vec![unreadable(first)]).await;
+ let PageResult::MailPage { keys, .. } = through(&mut engine, "generation", &edge).await
+ else {
+ panic!("window")
+ };
+ assert_eq!(keys, all[1..11]);
+ });
+}
+
+#[test]
+fn a_window_spans_several_index_pages() {
+ pollster::block_on(async {
+ let mut engine = Engine::new(InMemoryStorage::new());
+ store(&mut engine, (1..=1_100).map(readable).collect()).await;
+ let all = all_keys(&mut engine, filters(), "ALL").await;
+ assert!(all.len() > 1_000);
+ let mut cursor = None;
+ let mut edge = None;
+ for _ in 0..100 {
+ let PageResult::MailPage {
+ next_cursor,
+ end_cursor,
+ ..
+ } = read(&mut engine, filters(), "ALL", cursor).await
+ else {
+ panic!("page")
+ };
+ cursor = next_cursor;
+ edge = end_cursor;
+ }
+ let edge = edge.unwrap();
+ let PageResult::MailPage {
+ keys, next_cursor, ..
+ } = through(&mut engine, "generation", &edge).await
+ else {
+ panic!("window")
+ };
+ assert_eq!(keys, all[..1_000]);
+ assert!(next_cursor.is_some());
+ let PageResult::MailPage { keys, .. } =
+ through_at_least(&mut engine, "generation", &edge, 1_050).await
+ else {
+ panic!("window")
+ };
+ assert_eq!(keys, all[..1_050], "fewest rows past one index page");
+ });
+}
diff --git a/docs/agent-guide/email.md b/docs/agent-guide/email.md
index d12dc4f30d..d5dd3ffa44 100644
--- a/docs/agent-guide/email.md
+++ b/docs/agent-guide/email.md
@@ -339,7 +339,7 @@ schema additions before the client: it selects canonical message eligibility/rec
fields, body-free canonical preview references, and viewer-relative share facts.
The `soup-mail-v3` property-aware profile and new backfill checkpoint rebuild Mail proof without
changing the persisted mutation queue format. Native Tauri maintains the same
-predicate projections and revision-bound local page contract as the browser.
+predicate projections and local page contract as the browser.
Checkpoint v14 restarts older scans to populate property-aware indexes without wiping
queued work. A background network failure does not hide a usable current-query
cached Mail page; server-reported GraphQL errors still surface.