Skip to content

armgit / bento

Public Imported from GitHub
0

Bento, the office suite that fits in a file

kernel: correct the text:false sync-stamp comment, guard it, and pin its data loss (#500)

nyblnetecb3a92e14b0
diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md
index 854fae48..4a4c930c 100644
--- a/docs/DECISIONS.md
+++ b/docs/DECISIONS.md
@@ -7238,3 +7238,44 @@ The experimental alphabets stay in `scripts/lib/b86.mjs` behind
the b86 decoder alone, verified by grep on the built shell), and the
diagnostic panel stays behind `--diag`. Default: b86, cascade, no panel.
+## 2026-09-17 — A lean (`text: false`) sync stamp loses different-block edits on a snapshot reunion; guarded, not yet fixed
+
+`SyncState.toJSON({ text: false })` omits the per-node token history (the RGA
+`txt` generations) to shrink a saved file — meant for bento/spaces, whose typed
+prose would make that history dwarf its own document. Finding, measured with the
+kernel engine alone (spaces' convergence rig, reproduced in a throwaway): a lean
+stamp **silently loses an edit to a DIFFERENT block** when a mailed-back copy is
+reunited by `mergeSnapshot`.
+
+Mechanism: a `txt` edit advances the node's text GENERATION (`txt[el].sd`) but
+NOT its `html` LWW register (`regs[el+' html']`). A `text: false` stamp drops the
+generation, so the only stamped carrier of that edit is gone; `mergeSnapshot`
+compares registers, sees the html register unchanged, and keeps the base value.
+Two forks that edited *different* blocks lose one edit — and it is silent because
+the documents still CONVERGE. A live op-replay session is unaffected: a `set`/`txt`
+op both carries the value and stamps its register. The loss is specific to a lean
+stamp reunited by snapshot — which is exactly the rejoin path a lean stamp
+requires (a re-seeded generation would split-brain against a peer still holding
+the original, so op-resume is unsafe; see scripts/test-sync-shape.ts).
+
+Status: LATENT, not live. No shipping path stamps lean. Every app stamps the full
+history via `stampInto` → `toJSON()` default (`text: true`), the differ runs
+`text: true` (session.ts), and bento/spaces' `SyncSession` extends the kernel one
+without overriding the stamp. The prior crdt.ts comment claiming "bento/spaces
+does NOT stamp text history" and "ABSENT MEANS BLOCK-LEVEL MERGING, not breakage …
+the SAME paragraph" was stale AND wrong on the safety promise; corrected.
+
+Done now (this change): the crdt.ts comment states reality; `toJSON` REFUSES
+`text: false` without `unsafeOmitTextHistory: true`, so a future size optimisation
+that wires `stampInto` lean hits a wall instead of a silent loss (the rigs pass
+the flag); scripts/test-sync-shape.ts pins the different-block loss as a NEGATIVE
+CONTROL (assert the loss, invert it when the engine is fixed) alongside a full
+snapshot of the same edit that is kept — proving the loss is the missing history,
+not the merge.
+
+Deferred: making lean safe is a separate kernel change — a stamped html-register
+carrier that survives dropping the generation, WITHOUT out-ranking a concurrent
+generation (that would break the same-block RGA merge). Its gate is the full
+convergence rig, `scripts/test-sync`. Where it is wanted: bento/spaces per-block
+text under Markdown storage (working/design/spaces-pages.md follow-ups).
+
diff --git a/kernel/src/sync/crdt.ts b/kernel/src/sync/crdt.ts
index 7f6dff8f..2195fb98 100644
--- a/kernel/src/sync/crdt.ts
+++ b/kernel/src/sync/crdt.ts
@@ -409,17 +409,29 @@ export interface SyncStateJSON {
* history only ever grows. An emptied paragraph still carries everything ever
* typed into it.
*
- * bento/slides stamps it: slide text is titles and bullets. bento/spaces
- * does NOT — a space is typed prose, which is the whole app, and the state
- * would outweigh the document many times over inside the plaintext
- * #bento-doc block, re-serialized on every save and re-parsed on every open.
+ * EVERY shipping app stamps it — the default, via `stampInto` → `toJSON()`.
+ * bento/slides and bento/spaces both do (spaces' `SyncSession` extends the
+ * kernel one and does not override the stamp), and the differ runs with
+ * `text: true` (session.ts). The `text: false` option below OMITS the history
+ * and was meant for a future size optimisation — a space is typed prose, so
+ * its token history would outweigh its own document many times over inside
+ * the plaintext #bento-doc block. But NO shipping path uses it, only the sync
+ * rigs, because it is NOT SAFE YET:
*
- * ABSENT MEANS BLOCK-LEVEL MERGING, not breakage: `fromJSON` restores a
- * state with no token history, the differ falls back to a whole-value `set`
- * for that node's text, and the merge resolves last-writer-wins per block
- * instead of per character. A LIVE session is unaffected — both replicas hold
- * the tokens in memory for as long as they are connected; what degrades is
- * two offline forks reunited later, editing the SAME paragraph.
+ * ABSENT TOKEN HISTORY LOSES DATA ON A SNAPSHOT REUNION — and not only in the
+ * same paragraph, the way this comment once claimed. A txt edit advances the
+ * node's text GENERATION but NOT its `html` LWW register; a `text: false`
+ * stamp drops the generation, so the only stamped carrier of that edit is
+ * gone, and `mergeSnapshot` — the path a mailed-back copy reunites by — never
+ * copies the edited value from the peer's document. Two offline forks that
+ * edited DIFFERENT blocks lose one edit, silently (measured; convergence
+ * still holds, which is exactly why it is silent — docs/DECISIONS.md
+ * 2026-09-17). A LIVE op-replay session is unaffected: a `set`/`txt` op both
+ * carries the value and stamps its register, so the loss is specific to a
+ * lean stamp reunited by snapshot. `toJSON` therefore REFUSES `text: false`
+ * without an explicit acknowledgement (see there). Making it safe — a stamped
+ * carrier that survives dropping the generation — is a separate kernel change
+ * gated on scripts/test-sync.
*
* Garbage-collecting the tombstones instead (what Yjs does by default) needs
* to know every replica has seen the delete. A file that people mail to each
@@ -513,11 +525,25 @@ export class SyncEngine {
* scripts/test-sync-equiv.ts compares these bytes against the engine as
* shipped.
*
+ * `text: false` is REFUSED without `unsafeOmitTextHistory: true`. Omitting the
+ * history silently loses edits to different blocks on a snapshot reunion (the
+ * note on SyncStateJSON.txt has the mechanism); no shipping stamp path may use
+ * it, so a lean stamp must be an explicit, acknowledged act — the rigs that
+ * measure the lean shape pass the flag. This is the wall that stops a future
+ * size optimisation from wiring `stampInto` to a silent data loss.
+ *
* KEY ORDER IS PART OF THE FORMAT. `txt` is emitted in its original position
* rather than appended, so a state WITH text is byte-identical whichever
* call site produced it.
*/
- toJSON(opts: { text?: boolean } = {}): SyncStateJSON {
+ toJSON(opts: { text?: boolean; unsafeOmitTextHistory?: boolean } = {}): SyncStateJSON {
+ if (opts.text === false && !opts.unsafeOmitTextHistory)
+ throw new Error(
+ 'SyncState.toJSON({ text: false }) omits per-node token history, which silently LOSES ' +
+ 'edits to different blocks when a copy is reunited by snapshot (mergeSnapshot) — see the ' +
+ 'SyncStateJSON.txt note and docs/DECISIONS.md 2026-09-17. No shipping stamp path may use ' +
+ 'it; a rig that intends the lean shape passes { text: false, unsafeOmitTextHistory: true }.',
+ )
return {
v: SYNC_V,
lamport: this.lamport,
diff --git a/scripts/test-sync-shape.ts b/scripts/test-sync-shape.ts
index 1d545ad5..7ef45322 100644
--- a/scripts/test-sync-shape.ts
+++ b/scripts/test-sync-shape.ts
@@ -145,7 +145,7 @@ const spacesDoc = (blocks: unknown[]) => ({
}
const withText = JSON.stringify(A.toJSON())
- const without = JSON.stringify(A.toJSON({ text: false }))
+ const without = JSON.stringify(A.toJSON({ text: false, unsafeOmitTextHistory: true }))
ok(withText.includes('"txt"'), 'the default stamp carries the token history')
ok(!without.includes('"txt"'), 'and `text: false` leaves it out entirely')
ok(without.length * 5 < withText.length,
@@ -207,7 +207,7 @@ const spacesDoc = (blocks: unknown[]) => ({
// here so it cannot be forgotten when it is written.
{
const full = JSON.parse(JSON.stringify(A.toJSON()))
- const lean = JSON.parse(JSON.stringify(A.toJSON({ text: false })))
+ const lean = JSON.parse(JSON.stringify(A.toJSON({ text: false, unsafeOmitTextHistory: true })))
ok(!!full.txt && !lean.txt, 'the two stamps differ only in the token history')
const keptGeneration = Object.keys(full.txt ?? {}).length
ok(keptGeneration > 0,
@@ -215,5 +215,67 @@ const spacesDoc = (blocks: unknown[]) => ({
}
}
+// ---------------------------------------------------------------------------
+// PINNED (finding, 2026-09-17): the reunion path the section above PRESCRIBES
+// for a lean stamp — "rejoin by taking a peer's snapshot (mergeSnapshot)" —
+// LOSES an edit to a DIFFERENT block. A txt edit advances the node's text
+// generation but not its html LWW register; a text:false stamp drops the
+// generation, so mergeSnapshot has no stamped carrier for the edit and keeps
+// the base value. This is a NEGATIVE CONTROL: it asserts the loss so the loss
+// is not silent, and so the day the engine carries a stamped html register
+// through a lean stamp, THESE assertions flip and must be inverted.
+// docs/DECISIONS.md 2026-09-17. No shipping path stamps lean — this is latent.
+{
+ const clone = <T,>(v: T): T => JSON.parse(JSON.stringify(v))
+ const threeBlocks = () => spacesDoc([
+ { id: 'b0', type: 'p', html: 'zero' },
+ { id: 'b1', type: 'p', html: 'one' },
+ { id: 'b2', type: 'p', html: 'two' },
+ ]) as never as { pages: { blocks: { id: string; html: string }[] }[] }
+ const editBlock = (eng: SpacesSync, d: { pages: { blocks: { html: string }[] }[] }, i: number, html: string) => {
+ const n = clone(d); n.pages[0].blocks[i].html = html
+ eng.diff(d as never, n as never, { text: true })
+ d.pages[0].blocks[i].html = html
+ }
+
+ const origin = new SpacesSync('origin'); origin.adopt(threeBlocks() as never)
+ // the saved file both forks open — LEAN, spaces' would-be size optimisation
+ const saved = JSON.stringify(origin.toJSON({ text: false, unsafeOmitTextHistory: true }))
+
+ const X = SpacesSync.fromJSON('xavier', JSON.parse(saved))
+ const Y = SpacesSync.fromJSON('yasmin', JSON.parse(saved))
+ const dx = threeBlocks(), dy = threeBlocks()
+ editBlock(X, dx, 1, 'one — FROM-X') // X edits block b1
+ editBlock(Y, dy, 2, 'two — FROM-Y') // Y edits block b2 — a DIFFERENT block
+ // X reunites by taking Y's LEAN snapshot (the prescribed rejoin for lean stamps)
+ X.mergeSnapshot(dx as never, clone(dy) as never, Y.toJSON({ text: false, unsafeOmitTextHistory: true }))
+ ok(dx.pages[0].blocks[1].html === 'one — FROM-X', "X keeps its own edit through the reunion")
+ ok(!dx.pages[0].blocks[2].html.includes('FROM-Y'),
+ `KNOWN LOSS: a lean-stamp snapshot reunion drops the peer's DIFFERENT-block edit ` +
+ `(b2 is ${JSON.stringify(dx.pages[0].blocks[2].html)}) — invert when the engine fix lands`)
+
+ // the SAME reunion with a FULL snapshot keeps it — proving the loss is the
+ // dropped token history, not the merge.
+ const X2 = SpacesSync.fromJSON('xavier', JSON.parse(saved))
+ const dx2 = threeBlocks()
+ editBlock(X2, dx2, 1, 'one — FROM-X')
+ X2.mergeSnapshot(dx2 as never, clone(dy) as never, Y.toJSON())
+ ok(dx2.pages[0].blocks[2].html.includes('FROM-Y'),
+ 'a FULL (text:true) snapshot of the very same edit is KEPT — the loss is the missing history, not mergeSnapshot')
+}
+
+// ---- the guard: a lean stamp is REFUSED without acknowledgement -------------
+// The wall that keeps a future size optimisation from wiring stampInto to the
+// silent loss above.
+{
+ const A = new SpacesSync('a'); A.adopt(spacesDoc([{ id: 'b1', type: 'p', html: 'x' }]) as never)
+ let threw = false
+ try { A.toJSON({ text: false }) } catch { threw = true }
+ ok(threw, 'toJSON({ text: false }) REFUSES without unsafeOmitTextHistory — the footgun hits a wall, not a silent loss')
+ let ok2 = false
+ try { A.toJSON({ text: false, unsafeOmitTextHistory: true }); ok2 = true } catch { /* */ }
+ ok(ok2, 'and the acknowledged form is allowed — the rigs, and a future engine fix, opt in explicitly')
+}
+
console.log(`\n${checks - failures}/${checks} checks passed`)
if (failures) process.exit(1)