Bento, the office suite that fits in a file
TypeScript mit-licenseofficeoffice-365office-suiteoffice-toolsoffice365open-sourcepowerpointself-hostedsingle-page-appslidesslideshareslideshowslideshow-galleryslideshow-makerslideshows
kernel: correct the text:false sync-stamp comment, guard it, and pin its data loss (#500)
nyblnetecb3a92e14b0
diff --git a/docs/DECISIONS.md b/docs/DECISIONS.mdindex 854fae48..4a4c930c 100644--- a/docs/DECISIONS.md+++ b/docs/DECISIONS.md@@ -7238,3 +7238,44 @@ The experimental alphabets stay in `scripts/lib/b86.mjs` behindthe b86 decoder alone, verified by grep on the built shell), and thediagnostic panel stays behind `--diag`. Default: b86, cascade, no panel.+## 2026-09-17 — A lean (`text: false`) sync stamp loses different-block edits on a snapshot reunion; guarded, not yet fixed++`SyncState.toJSON({ text: false })` omits the per-node token history (the RGA+`txt` generations) to shrink a saved file — meant for bento/spaces, whose typed+prose would make that history dwarf its own document. Finding, measured with the+kernel engine alone (spaces' convergence rig, reproduced in a throwaway): a lean+stamp **silently loses an edit to a DIFFERENT block** when a mailed-back copy is+reunited by `mergeSnapshot`.++Mechanism: a `txt` edit advances the node's text GENERATION (`txt[el].sd`) but+NOT its `html` LWW register (`regs[el+' html']`). A `text: false` stamp drops the+generation, so the only stamped carrier of that edit is gone; `mergeSnapshot`+compares registers, sees the html register unchanged, and keeps the base value.+Two forks that edited *different* blocks lose one edit — and it is silent because+the documents still CONVERGE. A live op-replay session is unaffected: a `set`/`txt`+op both carries the value and stamps its register. The loss is specific to a lean+stamp reunited by snapshot — which is exactly the rejoin path a lean stamp+requires (a re-seeded generation would split-brain against a peer still holding+the original, so op-resume is unsafe; see scripts/test-sync-shape.ts).++Status: LATENT, not live. No shipping path stamps lean. Every app stamps the full+history via `stampInto` → `toJSON()` default (`text: true`), the differ runs+`text: true` (session.ts), and bento/spaces' `SyncSession` extends the kernel one+without overriding the stamp. The prior crdt.ts comment claiming "bento/spaces+does NOT stamp text history" and "ABSENT MEANS BLOCK-LEVEL MERGING, not breakage …+the SAME paragraph" was stale AND wrong on the safety promise; corrected.++Done now (this change): the crdt.ts comment states reality; `toJSON` REFUSES+`text: false` without `unsafeOmitTextHistory: true`, so a future size optimisation+that wires `stampInto` lean hits a wall instead of a silent loss (the rigs pass+the flag); scripts/test-sync-shape.ts pins the different-block loss as a NEGATIVE+CONTROL (assert the loss, invert it when the engine is fixed) alongside a full+snapshot of the same edit that is kept — proving the loss is the missing history,+not the merge.++Deferred: making lean safe is a separate kernel change — a stamped html-register+carrier that survives dropping the generation, WITHOUT out-ranking a concurrent+generation (that would break the same-block RGA merge). Its gate is the full+convergence rig, `scripts/test-sync`. Where it is wanted: bento/spaces per-block+text under Markdown storage (working/design/spaces-pages.md follow-ups).+diff --git a/kernel/src/sync/crdt.ts b/kernel/src/sync/crdt.tsindex 7f6dff8f..2195fb98 100644--- a/kernel/src/sync/crdt.ts+++ b/kernel/src/sync/crdt.ts@@ -409,17 +409,29 @@ export interface SyncStateJSON {* history only ever grows. An emptied paragraph still carries everything ever* typed into it.*- * bento/slides stamps it: slide text is titles and bullets. bento/spaces- * does NOT — a space is typed prose, which is the whole app, and the state- * would outweigh the document many times over inside the plaintext- * #bento-doc block, re-serialized on every save and re-parsed on every open.+ * EVERY shipping app stamps it — the default, via `stampInto` → `toJSON()`.+ * bento/slides and bento/spaces both do (spaces' `SyncSession` extends the+ * kernel one and does not override the stamp), and the differ runs with+ * `text: true` (session.ts). The `text: false` option below OMITS the history+ * and was meant for a future size optimisation — a space is typed prose, so+ * its token history would outweigh its own document many times over inside+ * the plaintext #bento-doc block. But NO shipping path uses it, only the sync+ * rigs, because it is NOT SAFE YET:*- * ABSENT MEANS BLOCK-LEVEL MERGING, not breakage: `fromJSON` restores a- * state with no token history, the differ falls back to a whole-value `set`- * for that node's text, and the merge resolves last-writer-wins per block- * instead of per character. A LIVE session is unaffected — both replicas hold- * the tokens in memory for as long as they are connected; what degrades is- * two offline forks reunited later, editing the SAME paragraph.+ * ABSENT TOKEN HISTORY LOSES DATA ON A SNAPSHOT REUNION — and not only in the+ * same paragraph, the way this comment once claimed. A txt edit advances the+ * node's text GENERATION but NOT its `html` LWW register; a `text: false`+ * stamp drops the generation, so the only stamped carrier of that edit is+ * gone, and `mergeSnapshot` — the path a mailed-back copy reunites by — never+ * copies the edited value from the peer's document. Two offline forks that+ * edited DIFFERENT blocks lose one edit, silently (measured; convergence+ * still holds, which is exactly why it is silent — docs/DECISIONS.md+ * 2026-09-17). A LIVE op-replay session is unaffected: a `set`/`txt` op both+ * carries the value and stamps its register, so the loss is specific to a+ * lean stamp reunited by snapshot. `toJSON` therefore REFUSES `text: false`+ * without an explicit acknowledgement (see there). Making it safe — a stamped+ * carrier that survives dropping the generation — is a separate kernel change+ * gated on scripts/test-sync.** Garbage-collecting the tombstones instead (what Yjs does by default) needs* to know every replica has seen the delete. A file that people mail to each@@ -513,11 +525,25 @@ export class SyncEngine {* scripts/test-sync-equiv.ts compares these bytes against the engine as* shipped.*+ * `text: false` is REFUSED without `unsafeOmitTextHistory: true`. Omitting the+ * history silently loses edits to different blocks on a snapshot reunion (the+ * note on SyncStateJSON.txt has the mechanism); no shipping stamp path may use+ * it, so a lean stamp must be an explicit, acknowledged act — the rigs that+ * measure the lean shape pass the flag. This is the wall that stops a future+ * size optimisation from wiring `stampInto` to a silent data loss.+ ** KEY ORDER IS PART OF THE FORMAT. `txt` is emitted in its original position* rather than appended, so a state WITH text is byte-identical whichever* call site produced it.*/- toJSON(opts: { text?: boolean } = {}): SyncStateJSON {+ toJSON(opts: { text?: boolean; unsafeOmitTextHistory?: boolean } = {}): SyncStateJSON {+ if (opts.text === false && !opts.unsafeOmitTextHistory)+ throw new Error(+ 'SyncState.toJSON({ text: false }) omits per-node token history, which silently LOSES ' ++ 'edits to different blocks when a copy is reunited by snapshot (mergeSnapshot) — see the ' ++ 'SyncStateJSON.txt note and docs/DECISIONS.md 2026-09-17. No shipping stamp path may use ' ++ 'it; a rig that intends the lean shape passes { text: false, unsafeOmitTextHistory: true }.',+ )return {v: SYNC_V,lamport: this.lamport,diff --git a/scripts/test-sync-shape.ts b/scripts/test-sync-shape.tsindex 1d545ad5..7ef45322 100644--- a/scripts/test-sync-shape.ts+++ b/scripts/test-sync-shape.ts@@ -145,7 +145,7 @@ const spacesDoc = (blocks: unknown[]) => ({}const withText = JSON.stringify(A.toJSON())- const without = JSON.stringify(A.toJSON({ text: false }))+ const without = JSON.stringify(A.toJSON({ text: false, unsafeOmitTextHistory: true }))ok(withText.includes('"txt"'), 'the default stamp carries the token history')ok(!without.includes('"txt"'), 'and `text: false` leaves it out entirely')ok(without.length * 5 < withText.length,@@ -207,7 +207,7 @@ const spacesDoc = (blocks: unknown[]) => ({// here so it cannot be forgotten when it is written.{const full = JSON.parse(JSON.stringify(A.toJSON()))- const lean = JSON.parse(JSON.stringify(A.toJSON({ text: false })))+ const lean = JSON.parse(JSON.stringify(A.toJSON({ text: false, unsafeOmitTextHistory: true })))ok(!!full.txt && !lean.txt, 'the two stamps differ only in the token history')const keptGeneration = Object.keys(full.txt ?? {}).lengthok(keptGeneration > 0,@@ -215,5 +215,67 @@ const spacesDoc = (blocks: unknown[]) => ({}}+// ---------------------------------------------------------------------------+// PINNED (finding, 2026-09-17): the reunion path the section above PRESCRIBES+// for a lean stamp — "rejoin by taking a peer's snapshot (mergeSnapshot)" —+// LOSES an edit to a DIFFERENT block. A txt edit advances the node's text+// generation but not its html LWW register; a text:false stamp drops the+// generation, so mergeSnapshot has no stamped carrier for the edit and keeps+// the base value. This is a NEGATIVE CONTROL: it asserts the loss so the loss+// is not silent, and so the day the engine carries a stamped html register+// through a lean stamp, THESE assertions flip and must be inverted.+// docs/DECISIONS.md 2026-09-17. No shipping path stamps lean — this is latent.+{+ const clone = <T,>(v: T): T => JSON.parse(JSON.stringify(v))+ const threeBlocks = () => spacesDoc([+ { id: 'b0', type: 'p', html: 'zero' },+ { id: 'b1', type: 'p', html: 'one' },+ { id: 'b2', type: 'p', html: 'two' },+ ]) as never as { pages: { blocks: { id: string; html: string }[] }[] }+ const editBlock = (eng: SpacesSync, d: { pages: { blocks: { html: string }[] }[] }, i: number, html: string) => {+ const n = clone(d); n.pages[0].blocks[i].html = html+ eng.diff(d as never, n as never, { text: true })+ d.pages[0].blocks[i].html = html+ }++ const origin = new SpacesSync('origin'); origin.adopt(threeBlocks() as never)+ // the saved file both forks open — LEAN, spaces' would-be size optimisation+ const saved = JSON.stringify(origin.toJSON({ text: false, unsafeOmitTextHistory: true }))++ const X = SpacesSync.fromJSON('xavier', JSON.parse(saved))+ const Y = SpacesSync.fromJSON('yasmin', JSON.parse(saved))+ const dx = threeBlocks(), dy = threeBlocks()+ editBlock(X, dx, 1, 'one — FROM-X') // X edits block b1+ editBlock(Y, dy, 2, 'two — FROM-Y') // Y edits block b2 — a DIFFERENT block+ // X reunites by taking Y's LEAN snapshot (the prescribed rejoin for lean stamps)+ X.mergeSnapshot(dx as never, clone(dy) as never, Y.toJSON({ text: false, unsafeOmitTextHistory: true }))+ ok(dx.pages[0].blocks[1].html === 'one — FROM-X', "X keeps its own edit through the reunion")+ ok(!dx.pages[0].blocks[2].html.includes('FROM-Y'),+ `KNOWN LOSS: a lean-stamp snapshot reunion drops the peer's DIFFERENT-block edit ` ++ `(b2 is ${JSON.stringify(dx.pages[0].blocks[2].html)}) — invert when the engine fix lands`)++ // the SAME reunion with a FULL snapshot keeps it — proving the loss is the+ // dropped token history, not the merge.+ const X2 = SpacesSync.fromJSON('xavier', JSON.parse(saved))+ const dx2 = threeBlocks()+ editBlock(X2, dx2, 1, 'one — FROM-X')+ X2.mergeSnapshot(dx2 as never, clone(dy) as never, Y.toJSON())+ ok(dx2.pages[0].blocks[2].html.includes('FROM-Y'),+ 'a FULL (text:true) snapshot of the very same edit is KEPT — the loss is the missing history, not mergeSnapshot')+}++// ---- the guard: a lean stamp is REFUSED without acknowledgement -------------+// The wall that keeps a future size optimisation from wiring stampInto to the+// silent loss above.+{+ const A = new SpacesSync('a'); A.adopt(spacesDoc([{ id: 'b1', type: 'p', html: 'x' }]) as never)+ let threw = false+ try { A.toJSON({ text: false }) } catch { threw = true }+ ok(threw, 'toJSON({ text: false }) REFUSES without unsafeOmitTextHistory — the footgun hits a wall, not a silent loss')+ let ok2 = false+ try { A.toJSON({ text: false, unsafeOmitTextHistory: true }); ok2 = true } catch { /* */ }+ ok(ok2, 'and the acknowledged form is allowed — the rigs, and a future engine fix, opt in explicitly')+}+console.log(`\n${checks - failures}/${checks} checks passed`)if (failures) process.exit(1)