Skip to content

armgit / macro

Public Imported from GitHub
0

Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.

Keep normalized cache records across compatible schema changes (#5275)

Sean Aye4a70b93d4870
diff --git a/apps/web/docs/graphql-normalized-cache-plan.md b/apps/web/docs/graphql-normalized-cache-plan.md
index 0ef072629c..0cba096a9d 100644
--- a/apps/web/docs/graphql-normalized-cache-plan.md
+++ b/apps/web/docs/graphql-normalized-cache-plan.md
@@ -54,9 +54,10 @@ entire cache in browser memory. With 10s of thousands of cached objects
10. **Eviction & GC** — active operations pin their dependencies; memory tier
is LRU with a byte budget; disk tier has a byte budget with orphan sweep
(records unreachable from any persisted operation root).
-11. **Cache identity & lifecycle** — namespace by `scope + schemaHash +
- cacheFormatVersion`, where **scope is an anonymous client-generated
- uuid** (localStorage), *not* user identity: construction is synchronous
+11. **Cache identity & lifecycle** — namespace by `scope +
+ schemaCompatibilityEpoch + cacheFormatVersion`, where **scope is an
+ anonymous client-generated uuid** (localStorage), *not* user identity:
+ construction is synchronous
and offline-capable (no identity waterfall), and no PII appears in
enumerable storage metadata (IDB database names / SQLite filenames).
User↔cache consistency is enforced by **identity witnessing**, split
@@ -69,7 +70,11 @@ entire cache in browser memory. With 10s of thousands of cached objects
triggering write (no stale-in-flight-write races). A mismatch wipes and
rebinds (“silent restart”) and all active operations re-execute. Eager
path: clear on logout.
- Discard on schema/format mismatch (cache is disposable, rebuild from
+ Additive GraphQL schema changes retain existing normalized records;
+ fragment reads can continue projecting fields already present. Manually
+ bump the schema compatibility epoch when identity, field storage shape, or
+ other schema-derived cache semantics become incompatible. Discard records
+ on compatibility-epoch/format mismatch (cache is disposable, rebuild from
network).
**Key policy — presence-of-id convention**: no client-side key config.
@@ -291,7 +296,7 @@ apps/web/src/lib/graphql-cache/ # JS glue
**Phase 2 — persistence** *(done — `cache-sqlite`, `cache-idb`)*
- Shared postcard record codec + `cache_namespace(scope)` embedding
- schema hash + format version.
+ schema compatibility epoch + format version.
- SQLite backend (Tauri native): WAL mode, batch txns, namespace
wipe-on-mismatch; tested natively incl. engine integration.
- IndexedDB backend via the `idb` crate: one DB per namespace, atomic
diff --git a/apps/web/tauri/graphql_cache_plugin/src/commands.rs b/apps/web/tauri/graphql_cache_plugin/src/commands.rs
index ab678fabc2..6e79fb2f62 100644
--- a/apps/web/tauri/graphql_cache_plugin/src/commands.rs
+++ b/apps/web/tauri/graphql_cache_plugin/src/commands.rs
@@ -43,9 +43,10 @@ fn engine_handle(state: &State<'_, CacheState>) -> Result<EngineHandle, String>
/// Opens (or creates) the cache for `scope`. Idempotent for the same scope;
/// errors on a scope mismatch (parity with the browser worker `init`). The
-/// database lives at `{app_data_dir}/graphql-cache/cache.sqlite`; the
-/// namespace check inside `cache-sqlite` (scope + schema hash + format
-/// version) wipes and rebuilds on mismatch — the cache is disposable.
+/// database lives at `{app_data_dir}/graphql-cache/cache.sqlite`. A scope
+/// change clears all cache state; for the same scope, a schema compatibility
+/// epoch or format mismatch clears disposable records but retains queued user
+/// intent.
#[tauri::command]
pub async fn graphql_cache_init<R: Runtime>(
app: AppHandle<R>,
diff --git a/crates/client/cache-core/build.rs b/crates/client/cache-core/build.rs
index b92797f22f..19860b4c8c 100644
--- a/crates/client/cache-core/build.rs
+++ b/crates/client/cache-core/build.rs
@@ -53,8 +53,9 @@ fn main() {
.as_ref()
.map(|subscription| subscription.name.to_string());
- // The schema hash namespaces persisted caches; key policy is derived
- // from the schema, so hashing the schema covers it.
+ // Expose the schema hash for build diagnostics. Persisted cache
+ // compatibility is versioned separately so additive schema changes can
+ // retain existing normalized records.
let mut hasher = Sha256::new();
hasher.update(schema_text.as_bytes());
let schema_hash = hex_string(&hasher.finalize());
diff --git a/crates/client/cache-core/src/codec.rs b/crates/client/cache-core/src/codec.rs
index b4ceaee245..38e75261d8 100644
--- a/crates/client/cache-core/src/codec.rs
+++ b/crates/client/cache-core/src/codec.rs
@@ -1,8 +1,8 @@
//! Binary record codec shared by all persistent backends (IndexedDB,
//! SQLite). Records are stored as postcard bytes; the cache namespace embeds
-//! [`CACHE_FORMAT_VERSION`] and [`meta::SCHEMA_HASH`](crate::meta) so a
-//! format or schema change starts a fresh cache instead of attempting
-//! migration (the cache is disposable by design).
+//! [`CACHE_FORMAT_VERSION`] and [`CACHE_SCHEMA_COMPATIBILITY_EPOCH`]. Additive
+//! GraphQL schema changes keep existing records, while incompatible schema or
+//! record-format changes start a fresh cache.
use crate::normalize::RecordUpdates;
use crate::queue::{PersistedOptimisticLayer, StoredMutation};
@@ -14,6 +14,14 @@ use thiserror::Error;
/// (or anything else persisted) changes incompatibly.
pub const CACHE_FORMAT_VERSION: u32 = 2;
+/// Bump when a GraphQL schema change makes existing normalized records unsafe.
+///
+/// Additive fields do not require a bump: fragment reads only project selected
+/// fields, so older records remain usable until a newly selected field is
+/// fetched. Bump this epoch for incompatible changes to normalized identity,
+/// field storage shape, or other schema-derived cache semantics.
+pub const CACHE_SCHEMA_COMPATIBILITY_EPOCH: u32 = 1;
+
#[derive(Debug, Error)]
pub enum CodecError {
#[error("corrupt cache payload: {0}")]
@@ -70,17 +78,15 @@ pub fn decode_record_updates(bytes: &[u8]) -> Result<RecordUpdates, CodecError>
/// Canonical database/namespace name for a cache instance.
/// `scope` identifies the user/workspace (host-provided).
pub fn cache_namespace(scope: &str) -> String {
- format!(
- "graphql-cache:{scope}:{}:v{CACHE_FORMAT_VERSION}",
- &crate::meta::SCHEMA_HASH[..12]
- )
+ format!("graphql-cache:{scope}:s{CACHE_SCHEMA_COMPATIBILITY_EPOCH}:v{CACHE_FORMAT_VERSION}")
}
/// Stable physical database name for a cache scope.
///
-/// Unlike [`cache_namespace`], this deliberately excludes the record schema
-/// and format versions. Normalized records are disposable on version changes,
-/// while queued mutations represent user intent and must remain discoverable.
+/// Unlike [`cache_namespace`], this deliberately excludes the schema
+/// compatibility epoch and cache format version. Normalized records are
+/// disposable on version changes, while queued mutations represent user intent
+/// and must remain discoverable.
pub fn cache_database_name(scope: &str) -> String {
format!("graphql-cache:{scope}")
}
@@ -126,9 +132,7 @@ mod tests {
}
#[test]
- fn namespace_shape() {
- let ns = cache_namespace("user-1");
- assert!(ns.starts_with("graphql-cache:user-1:"));
- assert!(ns.ends_with(":v2"));
+ fn namespace_uses_schema_compatibility_epoch_not_schema_hash() {
+ assert_eq!(cache_namespace("user-1"), "graphql-cache:user-1:s1:v2");
}
}
diff --git a/crates/client/cache-core/src/engine.rs b/crates/client/cache-core/src/engine.rs
index fa0d4e43d9..5fea5ce35f 100644
--- a/crates/client/cache-core/src/engine.rs
+++ b/crates/client/cache-core/src/engine.rs
@@ -1197,8 +1197,8 @@ impl<S: Storage> Engine<S> {
Ok(affected)
}
- /// Drops all cached state (logout, schema-hash mismatch), including any
- /// pending optimistic layers.
+ /// Drops all cached state (for example, on logout), including any pending
+ /// optimistic layers.
pub async fn clear(&mut self) -> Result<(), EngineError<S::Error>> {
self.hot.clear();
self.optimistic.clear();
diff --git a/crates/client/cache-core/tests/record_selection.rs b/crates/client/cache-core/tests/record_selection.rs
index c62d7c0681..e878e0442b 100644
--- a/crates/client/cache-core/tests/record_selection.rs
+++ b/crates/client/cache-core/tests/record_selection.rs
@@ -117,9 +117,11 @@ fn projects_cold_links_skips_incomplete_and_paginates_exclusively() {
}
#[test]
-fn reads_object_and_union_fragments_in_entity_key_order() {
+fn reads_schema_incomplete_objects_through_fragments_in_entity_key_order() {
block_on(async {
let mut storage = InMemoryStorage::new();
+ // These records intentionally omit other current, non-null schema
+ // fields. A fragment selecting only persisted fields remains complete.
storage
.put_batch(vec![
document("b", "Beta"),
diff --git a/crates/client/cache-idb/src/idb_storage.rs b/crates/client/cache-idb/src/idb_storage.rs
index 3c26815189..4feb71f635 100644
--- a/crates/client/cache-idb/src/idb_storage.rs
+++ b/crates/client/cache-idb/src/idb_storage.rs
@@ -78,8 +78,8 @@ fn claim_matches(mutation: &cache_core::queue::StoredMutation, claim: &MutationC
impl IdbStorage {
/// Opens the cache database for `scope` and initializes all stores.
- /// Record namespace changes clear only disposable records; a scope change
- /// also clears queued user intent.
+ /// Compatibility-epoch or cache-format changes clear only disposable
+ /// records; a scope change also clears queued user intent.
pub async fn open(scope: &str) -> Result<Self, IdbStorageError> {
let name = cache_database_name(scope);
let factory = Factory::new()?;
diff --git a/crates/client/cache-idb/src/lib.rs b/crates/client/cache-idb/src/lib.rs
index cd087dc67c..f652989e74 100644
--- a/crates/client/cache-idb/src/lib.rs
+++ b/crates/client/cache-idb/src/lib.rs
@@ -2,8 +2,9 @@
//! host (wasm module in a SharedWorker / dedicated worker).
//!
//! One stable IndexedDB database per cache scope. Disposable normalized
-//! records are versioned through metadata, while queued mutations and their
-//! optimistic layers remain discoverable across record-schema changes.
+//! records are versioned by schema compatibility epoch and cache format,
+//! while queued mutations and their optimistic layers remain discoverable
+//! across incompatible record-schema changes.
//! Postcard payloads are stored in indexable record envelopes whose `value`
//! property is a `Uint8Array`.
//!
diff --git a/crates/client/cache-sqlite/src/lib.rs b/crates/client/cache-sqlite/src/lib.rs
index 1e8f8cb9a1..8e13770ba8 100644
--- a/crates/client/cache-sqlite/src/lib.rs
+++ b/crates/client/cache-sqlite/src/lib.rs
@@ -1,7 +1,7 @@
//! SQLite [`Storage`] backend for the Tauri native host.
//!
//! One database file per cache; the `meta` table pins the cache namespace
-//! (scope + schema hash + format version, see
+//! (scope + schema compatibility epoch + format version, see
//! [`cache_core::codec::cache_namespace`]). On mismatch the store is wiped
//! and rebuilt — the cache is disposable by design, never migrated.
//!
@@ -116,9 +116,10 @@ impl SqliteStorage {
tx.execute("DELETE FROM mutation_queue", [])?;
tx.execute("DELETE FROM records", [])?;
} else if stored_namespace.as_deref() != Some(expected_namespace.as_str()) {
- // Record schema/format changes only invalidate disposable records.
- // The queue retains source GraphQL + optimistic JSON so the engine
- // can re-normalize it against the current schema.
+ // Incompatible schema/cache-format changes only invalidate
+ // disposable records. The queue retains source GraphQL +
+ // optimistic JSON so the engine can re-normalize it against the
+ // current schema.
tx.execute("DELETE FROM records", [])?;
}
tx.execute(
@@ -650,7 +651,28 @@ mod tests {
}
#[test]
- fn namespace_change_wipes() {
+ fn compatibility_epoch_change_wipes_records() {
+ block_on(async {
+ let dir = tempfile::tempdir().unwrap();
+ let path = dir.path().join("cache.db");
+
+ let mut s = SqliteStorage::open(&path, "user-1").unwrap();
+ s.put_batch(vec![(key("A:1"), record("a"))]).await.unwrap();
+ s.conn()
+ .execute(
+ "UPDATE meta SET v = 'graphql-cache:user-1:s0:v2' WHERE k = 'namespace'",
+ [],
+ )
+ .unwrap();
+ drop(s);
+
+ let s = SqliteStorage::open(&path, "user-1").unwrap();
+ assert_eq!(s.record_count().unwrap(), 0);
+ });
+ }
+
+ #[test]
+ fn scope_change_wipes() {
block_on(async {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("cache.db");
diff --git a/crates/client/cache-wasm/src/shell.rs b/crates/client/cache-wasm/src/shell.rs
index f0caa43982..c75e3b20c9 100644
--- a/crates/client/cache-wasm/src/shell.rs
+++ b/crates/client/cache-wasm/src/shell.rs
@@ -136,9 +136,9 @@ pub struct CacheEngine {
ops: Rc<RefCell<OpInterner>>,
}
-/// Opens (or creates) the cache for `scope`. See
-/// [`cache_core::codec::cache_namespace`] for how scope + schema hash +
-/// format version determine the underlying database.
+/// Opens (or creates) the cache for `scope`. The physical database is selected
+/// by [`cache_core::codec::cache_database_name`] from `scope` alone; the schema
+/// compatibility epoch and format version are record-compatibility inputs.
#[wasm_bindgen(js_name = openCache)]
pub async fn open_cache(scope: String, hot_capacity: Option<u32>) -> Result<CacheEngine, JsError> {
let storage = IdbStorage::open(&scope)
@@ -162,7 +162,7 @@ pub async fn destroy_cache(scope: String) -> Result<(), JsError> {
.map_err(|e| JsError::new(&e.to_string()))
}
-/// Schema hash baked into this build (namespace diagnostics).
+/// Schema hash baked into this build (build diagnostics).
#[wasm_bindgen(js_name = schemaHash)]
pub fn schema_hash() -> String {
cache_core::meta::SCHEMA_HASH.to_string()
diff --git a/docs/PROPERTY_TARGET_ENTITY_TYPE_PLAN.md b/docs/PROPERTY_TARGET_ENTITY_TYPE_PLAN.md
index 8fab23a8a8..5329f6068b 100644
--- a/docs/PROPERTY_TARGET_ENTITY_TYPE_PLAN.md
+++ b/docs/PROPERTY_TARGET_ENTITY_TYPE_PLAN.md
@@ -286,7 +286,9 @@ This work uses an atomic deployment strategy. Removing `TASK` from the GraphQL t
Create the target-only enum without `TASK` immediately, regenerate all clients in the same change, and deploy the backend and clients atomically. Do not add a deprecated `TASK` alias or a compatibility phase.
-Changing the committed GraphQL SDL rotates the normalized cache schema hash, so persisted client caches will rebuild automatically. Do not add manual cache-version logic.
+This input-only GraphQL change does not alter normalized record identity or
+field storage shape, so it does not require a normalized-cache schema
+compatibility epoch bump. Persisted records may remain in place.
## Test plan