Skip to content

Node-based terminal manager for AI coding agents — tmux-backed terminals and parallel agent sessions as draggable nodes on an infinite pan/zoom canvas. macOS, Linux, and a browser Server Edition.

fix(kanban-ops): three or more concurrent board adds converge in every interleaving (D5)

review4888a27bbbc1
diff --git a/docs/team-presence.md b/docs/team-presence.md
index 30ba9ddab..2078e44b3 100644
--- a/docs/team-presence.md
+++ b/docs/team-presence.md
@@ -551,9 +551,12 @@ wrote your board over theirs. Board items are in the vocabulary now, one op per
- **Order ops are cast on additions too.** An item op only says the item exists, and a peer that
applies it alone appends it. So the publisher also casts the list's order op whenever the list
gained an id or the relative order of the ids both sides share changed (`orderChanged`), and every
- replica lands on the later order op's list. Residual: an order op lists only the ids its sender
- knew, so two ids added in the same window by two different clients can sit in different relative
- orders until the next order op.
+ replica lands on the later order op's list. An order op lists only the ids its sender knew, so
+ ids other clients added in the same window are not in it: `reorder` puts those after the listed
+ ones SORTED BY ID, and the op's own sender applies the echo of its last order op for that list
+ instead of dropping it as an ack (`CanvasOrder.accept`). Without both, three concurrent adds
+ converged in only 30 of the 90 reflector interleavings; with them, all do
+ (`kanban-ops.convergence.test.ts` enumerates every interleaving for three clients).
- **Names are repaired, not refused.** A column title, label name or assignee name is trimmed,
stripped of control and bidi-override characters and cut to its bound (`displayText`); only an
empty result is refused. The UI puts no length cap on them, so a refusal would be a rename that
diff --git a/src/shared/canvas-order.test.ts b/src/shared/canvas-order.test.ts
index c7328dff1..321f17308 100644
--- a/src/shared/canvas-order.test.ts
+++ b/src/shared/canvas-order.test.ts
@@ -117,6 +117,26 @@ describe('kanban ops in the order', () => {
expect(mutationNodeId({ op: 'kb-meta', meta: { nodeId: 'n1' } })).toBeNull()
})
+ // D5: an order op lists only the ids its sender knew, so its sender — the one replica that dropped
+ // it as an ack — kept its concurrent adds in arrival order while everyone else sorted them.
+ it('the echo of our LAST order op for a board list is applied; any other echo of ours is an ack', () => {
+ const o = createCanvasOrder('me')
+ const order = (ids: string[]): CanvasMutation => ({ op: 'kb-column-order', ids, src: 'me' })
+ o.onLocal(order(['a']), P)
+ o.onLocal(order(['a', 'b']), P)
+ // A later order op of ours is still in flight: this one loses anyway, and applying it would flicker.
+ expect(o.accept({ ...order(['a']), seq: 3 }, P)).toBe(false)
+ // Our last one: applied, so its unlisted ids are sorted here exactly as on every other replica.
+ expect(o.accept({ ...order(['a', 'b']), seq: 4 }, P)).toBe(true)
+ // The label list takes the same rule; an item op of ours stays a plain ack.
+ const labels: CanvasMutation = { op: 'kb-label-order', ids: ['l1'], src: 'me' }
+ o.onLocal(labels, P)
+ expect(o.accept({ ...labels, seq: 5 }, P)).toBe(true)
+ const colOp: CanvasMutation = { op: 'kb-column', column: { id: 'c1', title: 'T', color: '#fff' }, src: 'me' }
+ o.onLocal(colOp, P)
+ expect(o.accept({ ...colOp, seq: 6 }, P)).toBe(false)
+ })
+
it('two kb-card ops for the same card: the higher seq wins, the straggler is dropped', () => {
const o = createCanvasOrder('me')
expect(o.accept(card('n1', 'doing', 'a', 5), P)).toBe(true)
diff --git a/src/shared/canvas-order.ts b/src/shared/canvas-order.ts
index 0c97e4d65..40dc766b1 100644
--- a/src/shared/canvas-order.ts
+++ b/src/shared/canvas-order.ts
@@ -20,7 +20,8 @@
//
// 1. OUR OWN ECHO IS AN ACK, NOT AN EDIT. We already applied it optimistically; re-applying it
// would rubber-band a node we are still dragging (the echo carries the position from ~50 ms
-// ago). So we consume it for its `seq` and drop it.
+// ago). So we consume it for its `seq` and drop it. (One exception, a board's ORDER op once it
+// is our last in flight for that list: see `accept`.)
// 2. WHILE ONE OF OUR OWN MUTATIONS FOR A NODE IS UNACKED, WE IGNORE PEERS' MUTATIONS FOR THAT
// NODE. Not a heuristic — a consequence of FIFO delivery, which IPC and WebSocket both
// guarantee: if the reflector had ordered our mutation BEFORE the peer's, our ack would
@@ -174,6 +175,12 @@ function isRemove(m: CanvasMutation): boolean {
return m.op === 'remove' || m.op === 'edge-remove'
}
+/** A board's two ORDER ops: whole-list values (`kb-column-order`, `kb-label-order`) whose listed ids
+ * are only what their sender knew — see the own-echo exception in `accept`. */
+function isBoardOrderOp(m: CanvasMutation): boolean {
+ return m.op === 'kb-column-order' || m.op === 'kb-label-order'
+}
+
/** The order's own "drops its subject" predicate, for a caller that has to ask the same question —
* the re-creation gate (`hasPendingRemove`) holds back everything EXCEPT a remove. */
export const isRemoveOp = isRemove
@@ -413,10 +420,12 @@ export function createCanvasOrder(
// Rule 4 forecloses that: a value the delete superseded lost on every OTHER client, so
// replaying it here would resurrect — on this canvas alone — a node nobody else has.
const repair = superseded.has(id) && current && !stale
+ let settled = false
// Rule 1: otherwise our own echo is just an ack — consume it, apply nothing.
if (p && --p.count <= 0) {
pending.delete(id)
superseded.delete(id) // every cast of ours is accounted for; the node is settled
+ settled = true
}
// The re-creation gate: this remove is now in our `seen` (above), so a re-creation of the
// key stamped from here on carries it. Matched against our casts in the order we made them:
@@ -428,7 +437,14 @@ export function createCanvasOrder(
if (at !== -1) {
for (const c of casts.splice(0, at + 1)) if (c.remove) releaseRemove(c.key)
}
- return repair
+ // …with one exception: the echo of our LAST order op for a board list (every cast of ours
+ // for that list now acked) is APPLIED. An order op lists only the ids its sender knew, so
+ // items teammates added concurrently sit unlisted, in each replica's arrival order, until the
+ // winning order op sorts them (kanban-ops `reorder`). Every other replica applies that op;
+ // its sender, dropping its own echo, was the one that kept its arrival order and diverged
+ // (D5). It lists what this board already shows, so it moves nothing else. Not while a later
+ // cast of ours is in flight: that one wins, and applying this one first would flicker.
+ return repair || (settled && current && isBoardOrderOp(m))
}
// A straggler: a mutation the total order has already superseded on this client (applied, or
// deliberately dropped). Applying it would move the node BACKWARDS out of the total order.
diff --git a/src/shared/kanban-ops.convergence.test.ts b/src/shared/kanban-ops.convergence.test.ts
index dd8e59ef7..aca6b9bb3 100644
--- a/src/shared/kanban-ops.convergence.test.ts
+++ b/src/shared/kanban-ops.convergence.test.ts
@@ -99,3 +99,87 @@ describe('board convergence (ruling R3)', () => {
expect(labels(a.board)).toEqual(labels(w.authority()))
})
})
+
+// D5: THREE clients. An order op lists the ids its sender knew, so with three concurrent adds the
+// winning order op names neither of the other two new ids, and each replica used to leave them in its
+// own local (arrival) order — 30 of the 90 reflector interleavings converged. Every interleaving is
+// enumerated below, twice: every client edits before anything is reflected (the reflector may take
+// the three clients' ops in any order that keeps each client's own order: 90), and edits interleaved
+// with the reflection itself (a client may edit after it has already received some ops).
+describe('board convergence with three clients (D5)', () => {
+ type Edit = (b: ProjectKanban) => ProjectKanban
+ const labelIds = (k: ProjectKanban) => (k.labels ?? []).map((l) => l.id)
+ const addLabel = (id: string): Edit => (b) => ({ ...b, labels: [...(b.labels ?? []), { id, name: id, color: 'red' }] })
+
+ /** Run one schedule: `E:<c>` = client c edits and casts; `R:<c>` = the reflector takes c's next op. */
+ function run(schedule: string[], edits: Record<string, Edit>) {
+ let seq = 0
+ let authority: ProjectKanban = defaultKanbanFor(P)
+ const names = Object.keys(edits)
+ const replicas = names.map((name) => ({ name, order: createCanvasOrder(name), board: defaultKanbanFor(P) }))
+ const outbox = new Map<string, CanvasMutation[]>(names.map((n) => [n, []]))
+ for (const step of schedule) {
+ const [kind, who] = step.split(':')
+ const r = replicas.find((x) => x.name === who)!
+ if (kind === 'E') {
+ const after = edits[who](r.board)
+ for (const op of diffKanbanOps(r.board, after, P, new Set())) {
+ const m = r.order.stamp({ ...op, src: r.name } as CanvasMutation)
+ r.order.onLocal(m, P)
+ outbox.get(who)!.push(m)
+ }
+ r.board = after
+ continue
+ }
+ const m = { ...outbox.get(who)!.shift()!, seq: ++seq }
+ authority = applyKanbanOp(authority, m as unknown as KanbanOp, P)
+ for (const x of replicas) if (x.order.accept(m, P)) x.board = applyKanbanOp(x.board, m as unknown as KanbanOp, P)
+ }
+ return { replicas, authority }
+ }
+
+ /** Every schedule: each client edits once, then the reflector takes its `opsPer` ops, in any
+ * interleaving that keeps a client's own edit before its own ops. `editsFirst` = all edits happen
+ * before any op is reflected. */
+ function schedules(names: string[], opsPer: number, editsFirst: boolean): string[][] {
+ const out: string[][] = []
+ const walk = (acc: string[], edited: Set<string>, sent: Map<string, number>): void => {
+ const done = names.every((n) => edited.has(n) && sent.get(n) === opsPer)
+ if (done) return void out.push(acc)
+ for (const n of names) {
+ if (!edited.has(n)) walk([...acc, `E:${n}`], new Set([...edited, n]), sent)
+ else if ((sent.get(n) ?? 0) < opsPer && (!editsFirst || edited.size === names.length))
+ walk([...acc, `R:${n}`], edited, new Map([...sent, [n, (sent.get(n) ?? 0) + 1]]))
+ }
+ }
+ walk([], new Set(), new Map())
+ // With every edit first, the order of the edits themselves does not matter: keep one.
+ return editsFirst ? out.filter((s) => s.slice(0, names.length).join() === names.map((n) => `E:${n}`).join()) : out
+ }
+
+ const converged = (w: ReturnType<typeof run>, read: (k: ProjectKanban) => string[]): boolean =>
+ w.replicas.every((r) => same(read(r.board), read(w.authority)))
+ const same = (a: string[], b: string[]) => a.join() === b.join()
+
+ for (const [what, edits, read] of [
+ ['a column', { A: add('kcol-X', 'X'), B: add('kcol-Y', 'Y'), C: add('kcol-Z', 'Z') }, ids],
+ ['a label', { A: addLabel('l-x'), B: addLabel('l-y'), C: addLabel('l-z') }, labelIds]
+ ] as const) {
+ it(`three clients adding ${what} at once converge in EVERY reflector interleaving (90 of 90)`, () => {
+ const all = schedules(['A', 'B', 'C'], 2, true)
+ expect(all).toHaveLength(90)
+ const bad = all.filter((s) => !converged(run(s, edits), read))
+ expect(bad.length, `diverged: ${bad[0]?.join(' ')}`).toBe(0)
+ })
+
+ it(`three clients adding ${what}, edits interleaved with the reflection, always converge`, () => {
+ const all = schedules(['A', 'B', 'C'], 2, false)
+ expect(all.length).toBeGreaterThan(90)
+ const bad = all.filter((s) => !converged(run(s, edits), read))
+ expect(bad.length, `diverged: ${bad[0]?.join(' ')} (${bad.length} of ${all.length})`).toBe(0)
+ // Every new item is on every board.
+ const w = run(all[0], edits)
+ for (const r of w.replicas) expect(read(r.board)).toHaveLength(read(defaultKanbanFor(P)).length + 3)
+ })
+ }
+})
diff --git a/src/shared/kanban-ops.test.ts b/src/shared/kanban-ops.test.ts
index cc6fc8c3d..cfddb1d96 100644
--- a/src/shared/kanban-ops.test.ts
+++ b/src/shared/kanban-ops.test.ts
@@ -156,6 +156,20 @@ describe('applyKanbanOp', () => {
const b = applyKanbanOp(base(), { op: 'kb-column-order', ids: [ids[2], 'ghost', ids[0]] }, P)
expect(b.columns.map((c) => c.id)).toEqual([ids[2], ids[0], ids[1]])
})
+ // D5: the unlisted ids are ones teammates added concurrently, in each replica's own arrival order;
+ // sorted, every replica applying the same op to the same set lands on the same list.
+ it('order ops put the unlisted ids after the listed ones SORTED BY ID, whatever order they arrived in', () => {
+ const col = (id: string) => ({ id, title: id, color: '#123456' })
+ const listed = ['kcol-a']
+ const one = { columns: [col('kcol-a'), col('kcol-z'), col('kcol-m')], assignments: [] }
+ const two = { columns: [col('kcol-m'), col('kcol-a'), col('kcol-z')], assignments: [] }
+ const order = { op: 'kb-column-order', ids: listed } as const
+ expect(applyKanbanOp(one, order, P).columns.map((c) => c.id)).toEqual(['kcol-a', 'kcol-m', 'kcol-z'])
+ expect(applyKanbanOp(two, order, P).columns.map((c) => c.id)).toEqual(['kcol-a', 'kcol-m', 'kcol-z'])
+ const lab = (id: string) => ({ id, name: id, color: 'red' as const })
+ const labels = { columns: [], assignments: [], labels: [lab('l-b'), lab('l-z'), lab('l-a')] }
+ expect(applyKanbanOp(labels, { op: 'kb-label-order', ids: ['l-z'] }, P).labels?.map((l) => l.id)).toEqual(['l-z', 'l-a', 'l-b'])
+ })
})
describe('diffKanbanOps', () => {
diff --git a/src/shared/kanban-ops.ts b/src/shared/kanban-ops.ts
index f3e70f163..a6a626ac9 100644
--- a/src/shared/kanban-ops.ts
+++ b/src/shared/kanban-ops.ts
@@ -182,11 +182,20 @@ const viewsOf = (b: ProjectKanban): KanbanSavedView[] =>
const cardLabelIds = (x: KanbanCardMeta): string[] =>
Array.isArray(x.labels) ? x.labels.filter((l): l is string => typeof l === 'string') : []
+/**
+ * Order `list` by an order op's `ids`: the listed items first, in that order, then every item the op
+ * does not list, SORTED BY ID. An order op names only the ids its sender knew, so an item a teammate
+ * added concurrently is unlisted, and appending the unlisted ones in each replica's own (arrival)
+ * order left three concurrent adds in different orders on different boards (D5). Sorted, every
+ * replica that applies the winning op to the same set of items lands on the same list; the sender
+ * applies its own winning op too (canvas-order `accept`). Compared by code unit, never by locale.
+ */
function reorder<T extends { id: string }>(list: T[], ids: string[]): T[] {
const byId = new Map(list.map((x) => [x.id, x]))
const head = ids.map((id) => byId.get(id)).filter((x): x is T => !!x)
const listed = new Set(head.map((x) => x.id))
- return [...head, ...list.filter((x) => !listed.has(x.id))]
+ const rest = list.filter((x) => !listed.has(x.id)).sort((a, b) => (a.id < b.id ? -1 : a.id > b.id ? 1 : 0))
+ return [...head, ...rest]
}
function upsertById<T extends { id: string }>(list: T[] | undefined, item: T): T[] {
@@ -270,10 +279,10 @@ const same = (a: unknown, b: unknown): boolean => stable(a) === stable(b)
* UI's add-then-move, a git pull) landed last everywhere else, and two people adding a column at
* once ended A at …,X,Y and B at …,Y,X. With the order op every replica lands on the LATER order
* op's list. A pure removal needs none: dropping an id never reorders the rest.
- * Residual, stated: an order op lists the ids its sender knew. An id added concurrently by ANOTHER
- * client that the op does not list keeps its place after the listed ones in each replica's local
- * order — one such id converges (it is last everywhere), two from two different clients in the same
- * window can still sit in different relative orders until the next order op.
+ * An order op lists only the ids its sender knew, so ids added concurrently by OTHER clients are not
+ * in it. They follow the listed ones sorted by id (`reorder`), and the op's own sender re-applies it
+ * on its echo (canvas-order `accept`), so three or more concurrent adds converge on every replica
+ * in every interleaving (kanban-ops.convergence.test.ts enumerates them).
*/
function orderChanged(prevIds: string[], nextIds: string[]): boolean {
const had = new Set(prevIds)