Beautiful, AI-native markdown IDE and LLM wiki
TypeScript 2nd-brainagent-skillsclaudecodexcompany-braindocsknowledge-knowledge-baseknowledge-managementllmllm-wikillm-wiki-karpathymarkdownmarkdown-editormdmdxnotespkmsecond-brainskills
PRD-9710 Keep both versions when an outside edit collides (#7080)
mike-inkeep7f391002bb74
diff --git a/.changeset/keep-both-on-collision.md b/.changeset/keep-both-on-collision.mdnew file mode 100644index 000000000..798e3f6fc--- /dev/null+++ b/.changeset/keep-both-on-collision.md@@ -0,0 +1,5 @@+---+'@inkeep/open-knowledge': patch+---++When an outside edit and your own edit change the same paragraph, OpenKnowledge now keeps both versions in the document, yours first, instead of raising a conflict to resolve. Both earlier versions are saved in the document's history.diff --git a/docs/content/features/git/sync.mdx b/docs/content/features/git/sync.mdxindex b4c03a1d6..0c5a50959 100644--- a/docs/content/features/git/sync.mdx+++ b/docs/content/features/git/sync.mdx@@ -75,7 +75,7 @@ You can't edit a page with conflicts until they're resolved; the same applies to### Conflicts that don't come from Git-A conflict doesn't need a remote. When a file changes on disk while OpenKnowledge has it open, OpenKnowledge merges the two versions itself, and raises a conflict if it can't merge them cleanly.+A conflict doesn't need a remote, though most changes on disk never raise one. When a file changes on disk while OpenKnowledge has it open, OpenKnowledge merges the two versions itself. Where both sides changed the same paragraph, both versions are kept, the OpenKnowledge version first and the outside version directly under it, and both earlier versions are saved to the [timeline](/docs/features/timeline-and-recovery). Where one side deleted a paragraph the other changed, the changed version stays. Where the two can't be merged safely, the OpenKnowledge version stays and the outside version is saved to the timeline. It raises a conflict when the file on disk holds Git conflict markers, when it can't save the outside version to the timeline, or when an agent write has no acknowledged version to merge against.Another app can also save an exact older copy over an edit OpenKnowledge already acknowledged. In the diff view, **Current** is the protected OpenKnowledge version and **Incoming** is the rejected external save. Choose which to keep. An intentional exact revert can trigger the same confirmation. Both versions stay recoverable after a restart, and keeping current protects against repeated saves from the stale editor. That protection expires after 30 minutes, so update or close the other app rather than relying on that protection.diff --git a/docs/content/features/timeline-and-recovery.mdx b/docs/content/features/timeline-and-recovery.mdxindex 434f85526..49feb73e2 100644--- a/docs/content/features/timeline-and-recovery.mdx+++ b/docs/content/features/timeline-and-recovery.mdx@@ -17,11 +17,11 @@ Timeline entries include:- **Agent and human edits** with contributor attribution and change summaries- **Upstream sync** entries from Git- **File system** changes made outside of OpenKnowledge-- **Recovered content** entries OpenKnowledge saves itself when it cannot merge changes into the document, such as an external file change with no acknowledged base+- **Recovered content** entries OpenKnowledge saves itself. They hold a version it could not merge into the document, such as the pre-change version when an external file change arrives with no acknowledged base. When an outside edit and yours change the same paragraph, two **Recovered content** rows appear, one per side: your version and the outside version as they were before both were kept. When an outside edit cannot be merged safely, one **Recovered content** row holds the outside version. Like every entry, they are listed newest first.Project-wide checkpoints saved by agents via the [`checkpoint` MCP tool](/docs/reference/mcp#tools) do not appear on the timeline; agents find and restore them with the `history` and `restore_version` tools.-OpenKnowledge also saves recovery entries itself. When it detects an external file change on its own and has no acknowledged base to merge the change against, the disk version replaces the document and the pre-change version is preserved as a **Recovered content** entry, which you can restore like any other timeline entry. When an agent write collides with an unmergeable external change instead, the write is refused as a conflict: your current content is kept, the disk change is not applied, and no entry is saved.+OpenKnowledge also saves recovery entries itself. When it detects an external file change on its own and has no acknowledged base to merge the change against, the disk version replaces the document and the pre-change version is preserved as a **Recovered content** entry, which you can restore like any other timeline entry. When an outside edit and the document's open version change the same paragraph, both versions stay in the document, the open version first and the outside version directly under it, nothing is asked, and both earlier versions are saved as entries. When one side deleted a paragraph the other changed, the changed version stays in the document, and both earlier versions are saved as entries. When the two cannot be merged safely, for example a very large file or a merge OpenKnowledge cannot verify, the open version stays and the outside version is saved as an entry. An agent write that meets an outside edit follows the same rules and lands. OpenKnowledge raises a conflict, and changes nothing until it is resolved, when the file on disk holds Git conflict markers, when it cannot save the outside version to the timeline, when an agent write has no acknowledged version to merge against, or when another app restores an exact older copy over an acknowledged edit (see [Conflicts that don't come from Git](/docs/features/git/sync#conflicts-that-dont-come-from-git)).The timeline is not the editor's undo history. Each timeline entry is a snapshot of the whole document, while undo steps back through your own recent edits in the pane where you are working. Agent edits, human edits, and file-system changes are batched into an entry once activity settles. Upstream syncs get their own entry when the sync lands. For when source-mode undo starts fresh, see [Editor → Undo in source mode](/docs/features/editor#undo-in-source-mode).diff --git a/docs/content/reference/mcp.mdx b/docs/content/reference/mcp.mdxindex 1c6941c80..ad14e7567 100644--- a/docs/content/reference/mcp.mdx+++ b/docs/content/reference/mcp.mdx@@ -48,7 +48,7 @@ The write verbs nest their result under the same target key you wrote, mirroring## Advisory warnings-`write` and `edit` results may carry a `warnings` array on the document entry (per entry for a batch); `restore_version` carries it at the top level. Entries are discriminated by `kind`. `content-divergence` and `disk-edit-reconciled` are write-integrity signals — the stored doc differs from what the call composed, so re-read it before editing further. `mermaid-parse-error` means the write landed but that fence won't render — fix the fence and re-edit. `lint-violation` reports a content-rule finding at a line and column. `link-check-deferred` means the server was still building its link index after starting, or the index was busy with other writes, so links were not checked and an empty `brokenLinks` is not an all-clear; that document's links are checked again only when it is written or edited later, or by an `audit`, which can time out until startup finishes.+`write` and `edit` results may carry a `warnings` array on the document entry (per entry for a batch); `restore_version` carries it at the top level. Entries are discriminated by `kind`. `content-divergence` and `disk-edit-reconciled` are write-integrity signals — the stored doc differs from what the call composed, so re-read it before editing further. A `disk-edit-reconciled` entry carries `mergeOutcome`, one of `clean` (the outside edit was taken as is), `merged` (both sides' edits were merged), `both-kept` or `kept-live`. `both-kept` means an outside edit and edits made in OpenKnowledge changed the same blocks, and three counts say what landed: `duplicatedBlocks` blocks are in the document twice, the OpenKnowledge version first and the outside version directly under it; `restoredDeletedHere` blocks deleted in OpenKnowledge came back with the outside version because the outside edit changed them; `restoredDeletedOutside` blocks the outside edit deleted stayed with the OpenKnowledge version because they were changed there. `kept-live` means the outside edit could not be merged, so the OpenKnowledge version stayed and the outside version was saved to the document's history. Those two outcomes also carry `nextStep: "re-read"`. In both the write landed and should not be retried; after `both-kept`, remove a duplicate, or delete a block restored over a deletion in OpenKnowledge, with an ordinary edit if the task calls for it. `mergeOutcome` and `nextStep` can gain values; treat a value you do not know like `merged`: the write landed, re-read the document. `mermaid-parse-error` means the write landed but that fence won't render — fix the fence and re-edit. `lint-violation` reports a content-rule finding at a line and column. `link-check-deferred` means the server was still building its link index after starting, or the index was busy with other writes, so links were not checked and an empty `brokenLinks` is not an all-clear; that document's links are checked again only when it is written or edited later, or by an `audit`, which can time out until startup finishes.`delete`, `move` and `install` on a `skill` target carry a separate `warnings` array of plain strings, not `kind`-discriminated entries. It reports work that did not undo the operation: bookkeeping records the server could not update after the skill itself was already deleted, renamed or moved, and — on a delete — that the directory removed was one an earlier failed move had retained for recovery. The operation succeeded; the warning names what to check or repair.@@ -120,7 +120,7 @@ A write that first detects the stale file during its final disk flush returns a`conflict.kind` says where the conflict came from, and the kind decides what a resolve does. `merge-native` is a git merge that left unmerged stages in the index. Resolving one stages your choice with git and commits the merge once no `merge-native` conflict is left. `working-tree` is a pull-only overlay collision, pinned to the origin blob it hit. `reconcile` is OpenKnowledge's own three-way merge of the editor against disk, including protected stale saves; no git object holds those stages, so OpenKnowledge snapshots them when it detects the conflict. Resolving a `working-tree` or `reconcile` conflict commits nothing.-A `reconcile` conflict also carries a `reason`: `merged-with-markers`, `refused-conflict-markers`, `refused-no-base`, `refused-too-large`, `disk-markers`, or `stale-external-write`. On that last reason the compatibility field `conflictKind` is `stale-external-write`. Every other row uses `conflictKind: "git"`. `refused-too-large` means the document is past the merge size cap or, rarely, that the merged output failed OpenKnowledge's fidelity check; the server log names the failed check.+A `reconcile` conflict also carries a `reason`: `refused-conflict-markers`, `refused-no-base`, `refused-too-large`, `disk-markers`, or `stale-external-write`; `merged-with-markers` appears only on entries recorded by earlier versions, or when both kept versions could not be applied to the open document. On `stale-external-write` the compatibility field `conflictKind` is `stale-external-write`. Every other row uses `conflictKind: "git"`. An outside edit to the same block as unsaved edits is not a conflict: both versions are kept in the document. `refused-too-large` means a merge past the size cap or one that failed OpenKnowledge's fidelity check, and is raised only when the outside version could not be saved to history; otherwise the OpenKnowledge version stays and the outside version goes to the document's history. The server log names the failed check.A write to a document whose file was deleted, renamed or moved on disk returns `urn:ok:error:doc-removed` (409): the edit reached collaborative state, but the disk write was refused. Retrying the same write does not help, and the `detail` says which case applies. If the file was deleted, re-create the document with `write` if you still want the content. If it was renamed or moved outside the app, the `detail` names its new path: re-read the document there and re-apply your change. If a different file now sits at the old path and has not been loaded into the document, only a reload of that document from disk clears the refusal, so tell the user instead of retrying.diff --git a/packages/app/src/components/TimelinePanel.dom.test.tsx b/packages/app/src/components/TimelinePanel.dom.test.tsxindex 7a47961c2..6559d53cc 100644--- a/packages/app/src/components/TimelinePanel.dom.test.tsx+++ b/packages/app/src/components/TimelinePanel.dom.test.tsx@@ -132,6 +132,18 @@ const SURFACED_KINDS: ParsedCheckpoint[] = [size: 12,metadata: { diskBytes: 41, discardedBytes: 66 },},+ {+ kind: 'external-change-both-kept',+ docName: 'notes',+ size: 12,+ metadata: { side: 'theirs', keptBlocks: 1 },+ },+ {+ kind: 'external-change-kept-live',+ docName: 'notes',+ size: 12,+ metadata: { side: 'theirs', reason: 'check-failed' },+ },];const AUTO_CONSOLIDATION: ParsedCheckpoint = {kind: 'auto-consolidation',diff --git a/packages/app/tests/integration/conflict-authority-reconcile.test.ts b/packages/app/tests/integration/conflict-authority-reconcile.test.tsindex 616c6145b..5ee248a77 100644--- a/packages/app/tests/integration/conflict-authority-reconcile.test.ts+++ b/packages/app/tests/integration/conflict-authority-reconcile.test.ts@@ -13,6 +13,7 @@ import {const BASE_CONTENT = '# Doc\n\nalpha\n\nbeta\n';const DISK_CONTENT = '# Doc\n\nalpha-disk\n\nbeta\n';+const BOTH_KEPT_CONTENT = '# Doc\n\nalpha-editor\n\nalpha-disk\n\nbeta\n';const MARKER_CONTENT ='# Doc\n\n<<<<<<< ours\nalpha-ours\n=======\nalpha-theirs\n>>>>>>> theirs\n\nbeta\n';@@ -107,9 +108,10 @@ async function seedDoc(server: TestServer, docName: string): Promise<void> {);}-async function driveReconcileConflict(+async function driveDiskWriteOverEditorEdit(server: TestServer,docName: string,+ diskContent: string,): Promise<Awaited<ReturnType<typeof createTestClient>>> {const client = await createTestClient(server.port, docName);cleanups.push(() => client.cleanup());@@ -122,10 +124,31 @@ async function driveReconcileConflict(client.ytext.insert(at, 'alpha-editor');});await pollUntil(() => client.ytext.toString().includes('alpha-editor'), 5_000);- writeFileSync(join(server.contentDir, `${docName}.md`), DISK_CONTENT, 'utf-8');+ writeFileSync(join(server.contentDir, `${docName}.md`), diskContent, 'utf-8');return client;}+async function assertBothKeptWithoutEntry(+ server: TestServer,+ docName: string,+ client: Awaited<ReturnType<typeof createTestClient>>,+): Promise<void> {+ const file = `${docName}.md`;+ await pollUntil(+ () => client.ytext.toString() === BOTH_KEPT_CONTENT,+ 30_000,+ 100,+ `both versions of the colliding block in ${file}`,+ );+ expect((await listConflicts(server.port)).some((c) => c.file === file)).toBe(false);+ await pollUntil(+ async () => (await writeProbe(server.port, docName, '# after\n')) === 200,+ CONCURRENT_REPLACE_WINDOW_MS * 2,+ 100,+ `a write to ${file} to land once its overlap window clears`,+ );+}+async function assertReconcileConflictIsResolvable(server: TestServer,docName: string,@@ -175,16 +198,16 @@ async function assertReconcileConflictIsResolvable(}describe('PRD-7947: a reconcile-born conflict is tracked, inspectable and resolvable', () => {- test('shape 1 — a three-way reconcile that conflicts registers a reconcile entry', async () => {+ test('shape 1 — a three-way reconcile where both sides changed one block keeps both and registers no entry', async () => {const server = await createTestServer({debounce: STORE_DEBOUNCE_MS,maxDebounce: STORE_MAX_DEBOUNCE_MS,});cleanups.push(() => server.cleanup());- const docName = `reconcile-conflicts-${crypto.randomUUID()}`;+ const docName = `reconcile-both-kept-${crypto.randomUUID()}`;await seedDoc(server, docName);- await driveReconcileConflict(server, docName);- await assertReconcileConflictIsResolvable(server, docName, 'merged-with-markers');+ const client = await driveDiskWriteOverEditorEdit(server, docName, DISK_CONTENT);+ await assertBothKeptWithoutEntry(server, docName, client);}, 120_000);test('shape 2 — conflict markers landing on disk register a disk-markers entry', async () => {@@ -205,7 +228,7 @@ describe('PRD-7947: a reconcile-born conflict is tracked, inspectable and resolvawait assertReconcileConflictIsResolvable(server, docName, 'disk-markers');}, 120_000);- test('shape 3 — the same reconcile conflict is tracked with sync wired to a remote', async () => {+ test('shape 3 — the same reconcile keeps both and registers no entry with sync wired to a remote', async () => {const docName = `reconcile-sync-${crypto.randomUUID()}`;const server = await createSyncWiredTestServer({originSeed: { [`${docName}.md`]: BASE_CONTENT },@@ -225,8 +248,8 @@ describe('PRD-7947: a reconcile-born conflict is tracked, inspectable and resolv`${docName} to be indexed`,);- await driveReconcileConflict(server, docName);- await assertReconcileConflictIsResolvable(server, docName, 'merged-with-markers');+ const client = await driveDiskWriteOverEditorEdit(server, docName, DISK_CONTENT);+ await assertBothKeptWithoutEntry(server, docName, client);}, 180_000);});@@ -239,7 +262,7 @@ describe('PRD-7947: the editor-side edit survives into the conflict stages', ()cleanups.push(() => server.cleanup());const docName = `reconcile-stages-${crypto.randomUUID()}`;await seedDoc(server, docName);- await driveReconcileConflict(server, docName);+ await driveDiskWriteOverEditorEdit(server, docName, MARKER_CONTENT);const file = `${docName}.md`;await pollUntil(@@ -251,7 +274,7 @@ describe('PRD-7947: the editor-side edit survives into the conflict stages', ()const content = await conflictContent(server.port, file);expect(content.status).toBe(200);- expect(String(content.body.theirs ?? '')).toContain('alpha-disk');+ expect(String(content.body.theirs ?? '')).toContain('alpha-theirs');expect(String(content.body.ours ?? '')).toContain('alpha-editor');expect(content.body.resolutionOptions).toEqual(expect.arrayContaining(['mine', 'content', 'delete']),@@ -267,7 +290,7 @@ describe('PRD-7947: the editor-side edit survives into the conflict stages', ()const docName = `reconcile-live-ours-${crypto.randomUUID()}`;const file = `${docName}.md`;await seedDoc(server, docName);- const client = await driveReconcileConflict(server, docName);+ const client = await driveDiskWriteOverEditorEdit(server, docName, MARKER_CONTENT);await pollUntil(async () => (await listConflicts(server.port)).some((c) => c.file === file),@@ -322,7 +345,7 @@ describe('PRD-7947: resolving a reconcile conflict with `delete` keeps the fileclient.ytext.insert(at, 'alpha-editor');});await pollUntil(() => client.ytext.toString().includes('alpha-editor'), 5_000);- writeFileSync(absPath, DISK_CONTENT, 'utf-8');+ writeFileSync(absPath, MARKER_CONTENT, 'utf-8');await pollUntil(async () => (await listConflicts(server.port)).some((c) => c.file === file),diff --git a/packages/app/tests/integration/conflict-aware-write-surfaces.test.ts b/packages/app/tests/integration/conflict-aware-write-surfaces.test.tsindex cd30dc524..e6b01ed68 100644--- a/packages/app/tests/integration/conflict-aware-write-surfaces.test.ts+++ b/packages/app/tests/integration/conflict-aware-write-surfaces.test.ts@@ -169,8 +169,8 @@ describe('FR1 + FR2: conflict-gate swap-in / swap-out (server-observable contrac}, 30_000);});-describe('FR11: the reconciliation conflict path raises an entry and fires the FR9 gate', () => {- test('reconcile case "conflicts" raises a reconcile entry + mutating handler returns 409', async () => {+describe('FR11 under D34: a same-block reconcile keeps both versions and raises no entry', () => {+ test('a same-block reconcile raises no entry and the next mutating write lands', async () => {const cleanups: Array<() => Promise<void> | void> = [];try {const docName = `fr11-${crypto.randomUUID()}`;@@ -203,19 +203,21 @@ describe('FR11: the reconciliation conflict path raises an entry and fires the Fconst theirsContent = '# Heading\n\nTheir version of first paragraph.\n\nSecond paragraph.\n';writeFileSync(join(server.contentDir, `${docName}.md`), theirsContent, 'utf-8');- await pollUntil(- async () => (await listConflicts(server.port)).some((c) => c.file === `${docName}.md`),- 10_000,+ const live = () =>+ server.instance.hocuspocus.documents.get(docName)?.getText('source').toString() ?? '';+ await pollUntil(() => live().includes('Their version'), 10_000);+ expect(live()).toBe(+ '# Heading\n\nOur version of first paragraph.\n\nTheir version of first paragraph.\n\nSecond paragraph.\n',);- const entry = (await listConflicts(server.port)).find((c) => c.file === `${docName}.md`);- expect(entry?.conflict).toBe('reconcile');- expect(entry?.reason).toBe('merged-with-markers');- expect(entry?.docName).toBe(docName);+ expect(await listConflicts(server.port)).toEqual([]);- const res = await agentWrite(server.port, docName, '# Replacement\n');- expect(res.status).toBe(409);- const body = (await res.json()) as Record<string, unknown>;- expect(body.type).toBe('urn:ok:error:doc-in-conflict');+ const res = await fetch(`http://127.0.0.1:${server.port}/api/agent-write-md`, {+ method: 'POST',+ headers: { 'Content-Type': 'application/json' },+ body: JSON.stringify({ docName, markdown: 'Agent line.\n', position: 'append' }),+ });+ expect(res.status).toBe(200);+ expect(live()).toContain('Agent line.');} finally {while (cleanups.length > 0) await cleanups.pop()?.();}diff --git a/packages/app/tests/integration/disk-authority-reconcile.test.ts b/packages/app/tests/integration/disk-authority-reconcile.test.tsindex 28d42b50f..edabaaefa 100644--- a/packages/app/tests/integration/disk-authority-reconcile.test.ts+++ b/packages/app/tests/integration/disk-authority-reconcile.test.ts@@ -226,10 +226,10 @@ describe('PRD-6832 β L1: agent write reconciles a newer out-of-band disk edit',}});- test('overlapping-block conflict: agent write is refused 409 doc-in-conflict, neither side is silently dropped', async () => {+ test('overlapping-block edit: both versions are kept and the agent write lands on them', async () => {server = await createTestServer({ debounce: 300_000, maxDebounce: 600_000 });const { port, contentDir } = server;- const docName = `reconcile-conflict-${randomUUID()}`;+ const docName = `reconcile-both-kept-${randomUUID()}`;const filePath = join(contentDir, `${docName}.md`);writeFileSync(filePath, '# Doc\n\nshared-line\n', 'utf-8');@@ -257,13 +257,19 @@ describe('PRD-6832 β L1: agent write reconciles a newer out-of-band disk edit',headers: { 'Content-Type': 'application/json' },body: JSON.stringify({ docName, markdown: 'agent-line\n', position: 'append' }),});- expect(res.status).toBe(409);- const body = (await res.json()) as { type?: string };- expect(body.type).toBe('urn:ok:error:doc-in-conflict');+ expect(res.status).toBe(200);+ const body = (await res.json()) as {+ warnings?: Array<{ kind?: string; mergeOutcome?: string; nextStep?: string }>;+ };+ const reconciled = body.warnings?.find((w) => w.kind === 'disk-edit-reconciled');+ expect(reconciled?.mergeOutcome).toBe('both-kept');+ expect(reconciled?.nextStep).toBe('re-read');- expect(serverYtext()).toContain('crdt-version');- expect(serverYtext()).not.toContain('agent-line');- expect(readTestDoc(contentDir, docName)).toContain('disk-version');+ expect(serverYtext()).toContain(+ 'shared-line crdt-version\n\nshared-line disk-version\n\nagent-line',+ );+ expect(server.instance.conflicts.findByDocName(docName)).toBeUndefined();+ await pollUntil(() => readTestDoc(contentDir, docName) === serverYtext());} finally {for (const c of clients) {await c.cleanup();diff --git a/packages/app/tests/integration/keep-both-collision.test.ts b/packages/app/tests/integration/keep-both-collision.test.tsnew file mode 100644index 000000000..193b08e23--- /dev/null+++ b/packages/app/tests/integration/keep-both-collision.test.ts@@ -0,0 +1,316 @@+import { randomUUID } from 'node:crypto';+import { existsSync, readFileSync, writeFileSync } from 'node:fs';+import { join } from 'node:path';+import { getMetrics } from '@inkeep/open-knowledge-server';+import { afterEach, describe, expect, test, vi } from 'vitest';+import { createWatcherDouble } from '../../../server/src/watcher-double.test-helper';+import {+ agentWriteMd,+ type CreateTestServerOptions,+ createTestClient,+ createTestServer,+ pollUntil,+ type TestClient,+ type TestServer,+} from './test-harness';++let server: TestServer | undefined;+let client: TestClient | undefined;++afterEach(async () => {+ await client?.cleanup();+ client = undefined;+ await server?.cleanup();+ server = undefined;+});++interface HistoryVersion {+ side: string;+ content: string;+}++function liveText(s: TestServer, docName: string): string {+ return s.instance.hocuspocus.documents.get(docName)?.getText('source').toString() ?? '';+}++async function openWithUnsavedEdit(+ docName: string,+ saved: string,+ edit: { find: string; replace: string },+ mergeHistoryWriter?: CreateTestServerOptions['mergeHistoryWriter'],+): Promise<{ s: TestServer; c: TestClient; file: string }> {+ const s = await createTestServer({+ gitEnabled: true,+ watcherBackend: createWatcherDouble().backend,+ debounce: 300_000,+ maxDebounce: 600_000,+ mergeHistoryWriter,+ });+ server = s;+ const file = join(s.contentDir, `${docName}.md`);+ await agentWriteMd(s.port, saved, { docName, position: 'replace' });+ await pollUntil(+ () => existsSync(file) && readFileSync(file, 'utf-8') === saved,+ 10_000,+ 25,+ 'the seeded document to reach disk',+ );+ const c = await createTestClient(s.port, docName);+ client = c;+ await pollUntil(() => c.ytext.toString() === saved);+ c.doc.transact(() => {+ const at = c.ytext.toString().indexOf(edit.find);+ c.ytext.delete(at, edit.find.length);+ c.ytext.insert(at, edit.replace);+ });+ await pollUntil(() => liveText(s, docName).includes(edit.replace));+ return { s, c, file };+}++async function flushPendingStore(s: TestServer, docName: string): Promise<void> {+ const id = `onStoreDocument-${docName}`;+ if (s.instance.hocuspocus.debouncer.isDebounced(id)) {+ await s.instance.hocuspocus.debouncer.executeNow(id);+ }+}++async function historyVersions(+ s: TestServer,+ docName: string,+ kind: string,+ count: number,+): Promise<HistoryVersion[]> {+ let rows: Array<{ sha: string; checkpoint?: { kind?: string; metadata?: { side?: string } } }> =+ [];+ await pollUntil(+ async () => {+ const res = await fetch(+ `http://127.0.0.1:${s.port}/api/history?docName=${encodeURIComponent(docName)}&limit=100`,+ );+ const body = (await res.json()) as { entries?: typeof rows };+ rows = (body.entries ?? []).filter((entry) => entry.checkpoint?.kind === kind);+ return rows.length >= count;+ },+ 10_000,+ 50,+ `${count} ${kind} history entries`,+ );+ const versions: HistoryVersion[] = [];+ for (const row of rows) {+ const res = await fetch(+ `http://127.0.0.1:${s.port}/api/history/${row.sha}?docName=${encodeURIComponent(docName)}`,+ );+ const body = (await res.json()) as { content: string };+ versions.push({ side: row.checkpoint?.metadata?.side ?? '', content: body.content });+ }+ return versions.sort((a, b) => a.side.localeCompare(b.side));+}++const SAVED = '# Doc\n\nShared paragraph.\n\nTail.\n';+const EDITED_HERE = '# Doc\n\nShared paragraph, edited here.\n\nTail.\n';+const EDITED_OUTSIDE = '# Doc\n\nShared paragraph, edited outside.\n\nTail.\n';+const BOTH_KEPT =+ '# Doc\n\nShared paragraph, edited here.\n\nShared paragraph, edited outside.\n\nTail.\n';+const PARAGRAPH_EDIT = { find: 'Shared paragraph.', replace: 'Shared paragraph, edited here.' };++const CHECK_SAVED = '# Doc\n\nA\n\nB\n\nC\n';+const CHECK_HERE = '# Doc\n\nA\n\nX\n\nB edited\n\nC\n';+const CHECK_OUTSIDE = '# Doc\n\nA\n\nX\n\nB\n\nC\n';+const CHECK_EDIT = { find: 'B\n\nC', replace: 'X\n\nB edited\n\nC' };++describe('an outside edit to the block being edited here keeps both versions', () => {+ test('the watcher lands both versions live and on disk, raises nothing, and keeps both originals in history', async () => {+ const docName = `keep-both-${randomUUID()}`;+ const { s, c, file } = await openWithUnsavedEdit(docName, SAVED, PARAGRAPH_EDIT);+ const conflictsBefore = getMetrics().conflictCount;++ writeFileSync(file, EDITED_OUTSIDE);+ expect((await s.instance.recheckDisk()).dispatched).toBeGreaterThanOrEqual(1);++ await pollUntil(() => liveText(s, docName) === BOTH_KEPT, 10_000, 25, 'both versions live');+ await pollUntil(+ () => c.ytext.toString() === BOTH_KEPT,+ 5_000,+ 25,+ 'both versions on the client',+ );+ await flushPendingStore(s, docName);+ await pollUntil(() => readFileSync(file, 'utf-8') === BOTH_KEPT, 5_000, 25, 'both on disk');++ expect(s.instance.conflicts.list()).toEqual([]);+ expect(getMetrics().conflictCount).toBe(conflictsBefore);+ expect(await historyVersions(s, docName, 'external-change-both-kept', 2)).toEqual([+ { side: 'ours', content: EDITED_HERE },+ { side: 'theirs', content: EDITED_OUTSIDE },+ ]);+ }, 60_000);++ test('a merge that fails the fidelity check keeps the version here and moves the outside version to history', async () => {+ const docName = `keep-live-${randomUUID()}`;+ const { s, file } = await openWithUnsavedEdit(docName, CHECK_SAVED, CHECK_EDIT);+ const keptLiveBefore = getMetrics().reconcileKeptLive;++ writeFileSync(file, CHECK_OUTSIDE);+ await s.instance.recheckDisk();+ await pollUntil(+ () => getMetrics().reconcileKeptLive > keptLiveBefore,+ 10_000,+ 25,+ 'the keep-live path to run',+ );+ await flushPendingStore(s, docName);+ await pollUntil(() => readFileSync(file, 'utf-8') === CHECK_HERE, 5_000, 25, 'live on disk');++ expect(liveText(s, docName)).toBe(CHECK_HERE);+ expect(s.instance.conflicts.list()).toEqual([]);+ expect(await historyVersions(s, docName, 'external-change-kept-live', 1)).toEqual([+ { side: 'theirs', content: CHECK_OUTSIDE },+ ]);+ }, 60_000);+});++describe('an agent write over an outside edit to the block being edited here', () => {+ test('lands on both versions and reports both-kept with re-read as the next step', async () => {+ const docName = `agent-keep-both-${randomUUID()}`;+ const { s, file } = await openWithUnsavedEdit(docName, SAVED, PARAGRAPH_EDIT);+ writeFileSync(file, EDITED_OUTSIDE);++ const res = await fetch(`http://127.0.0.1:${s.port}/api/agent-write-md`, {+ method: 'POST',+ headers: { 'Content-Type': 'application/json' },+ body: JSON.stringify({ docName, markdown: 'Agent line.\n', position: 'append' }),+ });+ expect(res.status).toBe(200);+ const body = (await res.json()) as {+ warnings?: Array<{ kind?: string; mergeOutcome?: string; nextStep?: string }>;+ };+ expect(body.warnings?.find((w) => w.kind === 'disk-edit-reconciled')).toMatchObject({+ mergeOutcome: 'both-kept',+ nextStep: 're-read',+ });++ const live = liveText(s, docName);+ expect(live.startsWith(BOTH_KEPT.trimEnd())).toBe(true);+ expect(live).toContain('Agent line.');+ await pollUntil(() => readFileSync(file, 'utf-8') === liveText(s, docName), 5_000, 25, 'disk');+ expect(s.instance.conflicts.list()).toEqual([]);+ expect(await historyVersions(s, docName, 'external-change-both-kept', 2)).toEqual([+ { side: 'ours', content: EDITED_HERE },+ { side: 'theirs', content: EDITED_OUTSIDE },+ ]);+ }, 60_000);++ test('a merge that fails the fidelity check lands the write on the version here and reports kept-live', async () => {+ const docName = `agent-keep-live-${randomUUID()}`;+ const { s, file } = await openWithUnsavedEdit(docName, CHECK_SAVED, CHECK_EDIT);+ writeFileSync(file, CHECK_OUTSIDE);++ const res = await fetch(`http://127.0.0.1:${s.port}/api/agent-write-md`, {+ method: 'POST',+ headers: { 'Content-Type': 'application/json' },+ body: JSON.stringify({ docName, markdown: 'Agent line.\n', position: 'append' }),+ });+ expect(res.status).toBe(200);+ const body = (await res.json()) as {+ warnings?: Array<{ kind?: string; mergeOutcome?: string; nextStep?: string }>;+ };+ expect(body.warnings?.find((w) => w.kind === 'disk-edit-reconciled')).toMatchObject({+ mergeOutcome: 'kept-live',+ nextStep: 're-read',+ });++ const live = liveText(s, docName);+ expect(live.startsWith(CHECK_HERE.trimEnd())).toBe(true);+ expect(live).toContain('Agent line.');+ await pollUntil(() => readFileSync(file, 'utf-8') === liveText(s, docName), 5_000, 25, 'disk');+ expect(s.instance.conflicts.list()).toEqual([]);+ expect(await historyVersions(s, docName, 'external-change-kept-live', 1)).toEqual([+ { side: 'theirs', content: CHECK_OUTSIDE },+ ]);+ }, 60_000);+});++const failingHistoryWriter = () =>+ vi.fn<NonNullable<CreateTestServerOptions['mergeHistoryWriter']>>(() =>+ Promise.reject(new Error('injected history write failure')),+ );++describe('when the history write fails', () => {+ test('a merge that fails the fidelity check raises the reconcile conflict and leaves the outside bytes on disk', async () => {+ const docName = `keep-live-no-history-${randomUUID()}`;+ const writer = failingHistoryWriter();+ const { s, file } = await openWithUnsavedEdit(docName, CHECK_SAVED, CHECK_EDIT, writer);+ const failuresBefore = getMetrics().mergeHistoryCheckpointFailures;++ writeFileSync(file, CHECK_OUTSIDE);+ await s.instance.recheckDisk();+ await pollUntil(+ () => s.instance.conflicts.findByDocName(docName) !== undefined,+ 10_000,+ 25,+ 'the reconcile conflict',+ );+ await flushPendingStore(s, docName);++ expect(s.instance.conflicts.findByDocName(docName)).toMatchObject({+ kind: 'reconcile',+ reason: 'refused-too-large',+ });+ expect(writer).toHaveBeenCalledTimes(1);+ expect(getMetrics().mergeHistoryCheckpointFailures).toBe(failuresBefore + 1);+ expect(liveText(s, docName)).toBe(CHECK_HERE);+ expect(readFileSync(file, 'utf-8')).toBe(CHECK_OUTSIDE);+ }, 60_000);++ test('an agent write over a merge that fails the fidelity check is refused and the outside bytes stay on disk', async () => {+ const docName = `agent-keep-live-no-history-${randomUUID()}`;+ const writer = failingHistoryWriter();+ const { s, file } = await openWithUnsavedEdit(docName, CHECK_SAVED, CHECK_EDIT, writer);+ writeFileSync(file, CHECK_OUTSIDE);++ const res = await fetch(`http://127.0.0.1:${s.port}/api/agent-write-md`, {+ method: 'POST',+ headers: { 'Content-Type': 'application/json' },+ body: JSON.stringify({ docName, markdown: 'Agent line.\n', position: 'append' }),+ });+ expect(res.status).toBe(409);+ expect(((await res.json()) as { type?: string }).type).toBe('urn:ok:error:doc-in-conflict');+ expect(writer).toHaveBeenCalledTimes(1);+ expect(s.instance.conflicts.findByDocName(docName)).toMatchObject({+ kind: 'reconcile',+ reason: 'refused-too-large',+ });+ expect(liveText(s, docName)).not.toContain('Agent line.');+ expect(readFileSync(file, 'utf-8')).toBe(CHECK_OUTSIDE);+ }, 60_000);++ test('a same-block collision still keeps both versions, counting the failed history writes', async () => {+ const docName = `keep-both-no-history-${randomUUID()}`;+ const writer = failingHistoryWriter();+ const { s, file } = await openWithUnsavedEdit(docName, SAVED, PARAGRAPH_EDIT, writer);+ const failuresBefore = getMetrics().mergeHistoryCheckpointFailures;++ writeFileSync(file, EDITED_OUTSIDE);+ await s.instance.recheckDisk();+ await pollUntil(() => liveText(s, docName) === BOTH_KEPT, 10_000, 25, 'both versions live');+ await flushPendingStore(s, docName);+ await pollUntil(() => readFileSync(file, 'utf-8') === BOTH_KEPT, 5_000, 25, 'both on disk');+ await pollUntil(+ () => getMetrics().mergeHistoryCheckpointFailures === failuresBefore + 2,+ 5_000,+ 25,+ 'both injected history writes to fail',+ );++ expect(s.instance.conflicts.list()).toEqual([]);+ const attempted = writer.mock.calls.map(([, , params]) =>+ params.kind === 'external-change-both-kept'+ ? `${params.kind}:${params.metadata.side}:${params.metadata.keptBlocks}`+ : params.kind,+ );+ expect(attempted.sort()).toEqual([+ 'external-change-both-kept:ours:1',+ 'external-change-both-kept:theirs:1',+ ]);+ }, 60_000);+});diff --git a/packages/app/tests/integration/persistence-staleness-watchdog.test.ts b/packages/app/tests/integration/persistence-staleness-watchdog.test.tsindex b084a1656..fea67c07c 100644--- a/packages/app/tests/integration/persistence-staleness-watchdog.test.ts+++ b/packages/app/tests/integration/persistence-staleness-watchdog.test.ts@@ -74,11 +74,10 @@ describe('persistence staleness watchdog (integration)', () => {expect(getMetrics().persistenceStalenessForcedStores).toBe(forcedBefore);});- test('stands down instead of clobbering an external native edit', async () => {+ test('never writes over an external native edit that lands under unflushed edits', async () => {server = await createTestServer({ stalenessGraceMs: 250, stalenessSweepIntervalMs: 100 });const docName = `staleness-external-${randomUUID()}`;const filePath = join(server.contentDir, `${docName}.md`);- const forcedBefore = getMetrics().persistenceStalenessForcedStores;const first = await writeMd(server.port, docName, '# version one\n');expect(first.status).toBe(200);@@ -104,15 +103,14 @@ describe('persistence staleness watchdog (integration)', () => {`http://127.0.0.1:${server.port}/api/document?docName=${encodeURIComponent(docName)}`,);expect(state.ok).toBe(true);- const watcherReconciled = (await state.text()).includes('external native edit');- const entry = server.instance.conflicts.findByDocName(docName);- const watcherSurfacedConflict =- entry?.kind === 'reconcile' && entry.reason === 'merged-with-markers';- expect(watcherReconciled || watcherSurfacedConflict).toBe(true);+ expect((await state.text()).includes('external native edit')).toBe(true);}- await wait(500);- expect(readFileSync(filePath, 'utf-8')).toBe('# external native edit\n');- expect(getMetrics().persistenceStalenessForcedStores).toBe(forcedBefore);+ const deadline = Date.now() + 1_500;+ while (Date.now() < deadline) {+ expect(readFileSync(filePath, 'utf-8')).toContain('# external native edit\n');+ await wait(50);+ }+ expect(server.instance.conflicts.findByDocName(docName)).toBeUndefined();});});diff --git a/packages/app/tests/integration/sync-conflict-resolution-crdt.test.ts b/packages/app/tests/integration/sync-conflict-resolution-crdt.test.tsindex 39fcfb72a..8f0a0c24c 100644--- a/packages/app/tests/integration/sync-conflict-resolution-crdt.test.ts+++ b/packages/app/tests/integration/sync-conflict-resolution-crdt.test.ts@@ -124,8 +124,8 @@ describe('case "conflict" disk event -> conflict ledger', () => {}, 60_000);});-describe('case "conflicts" reconcile branch -> conflict ledger', () => {- test('case "conflicts" raises a reconcile entry with reason "merged-with-markers"', async () => {+describe('same-block reconcile -> both versions kept, nothing in the conflict ledger', () => {+ test('a same-block reconcile keeps both versions in the editor and raises no ledger entry', async () => {const docName = `reconcile-conflicts-${crypto.randomUUID()}`;const baseContent = '# Heading\n\nFirst paragraph.\n\nSecond paragraph.\n';const server = await createTestServer({ debounce: 60_000, maxDebounce: 60_000 });@@ -160,12 +160,11 @@ describe('case "conflicts" reconcile branch -> conflict ledger', () => {const theirsContent = '# Heading\n\nTheir version of first paragraph.\n\nSecond paragraph.\n';writeFileSync(filePath, theirsContent, 'utf-8');- await pollUntil(() => server.instance.conflicts.has(docName), 30_000);+ const bothKept =+ '# Heading\n\nOur version of first paragraph.\n\nTheir version of first paragraph.\n\nSecond paragraph.\n';+ await pollUntil(() => client.ytext.toString() === bothKept, 30_000);- expect(server.instance.conflicts.findByDocName(docName)).toMatchObject({- kind: 'reconcile',- reason: 'merged-with-markers',- });+ expect(server.instance.conflicts.has(docName)).toBe(false);}, 60_000);});diff --git a/packages/app/tests/integration/test-harness.ts b/packages/app/tests/integration/test-harness.tsindex 2db7269dc..6707743b4 100644--- a/packages/app/tests/integration/test-harness.ts+++ b/packages/app/tests/integration/test-harness.ts@@ -110,6 +110,7 @@ export interface CreateTestServerOptions {seedSkillHostRoot?: boolean;watcherBackend?: ServerOptions['watcherBackend'];watcherClock?: ServerOptions['watcherClock'];+ mergeHistoryWriter?: ServerOptions['mergeHistoryWriter'];}export async function createTestServer(options: CreateTestServerOptions = {}): Promise<TestServer> {@@ -188,6 +189,7 @@ export async function createTestServer(options: CreateTestServerOptions = {}): PagentSessionOptions: options.agentSessionOptions,watcherBackend: options.watcherBackend,watcherClock: options.watcherClock,+ mergeHistoryWriter: options.mergeHistoryWriter,...(ephemeral ? { ephemeral: true, singleDocRelPath: options.singleDocRelPath } : {}),skipStateManifestCheck: true,});diff --git a/packages/core/src/checkpoint-kinds.ts b/packages/core/src/checkpoint-kinds.tsindex 6d47dc8ae..c3dc902f3 100644--- a/packages/core/src/checkpoint-kinds.ts+++ b/packages/core/src/checkpoint-kinds.ts@@ -81,6 +81,18 @@ export type ParsedCheckpoint =size: number | null;metadata: { diskBytes: number; discardedBytes: number };}+ | {+ kind: 'external-change-both-kept';+ docName: string | null;+ size: number | null;+ metadata: { side: 'ours' | 'theirs'; keptBlocks: number };+ }+ | {+ kind: 'external-change-kept-live';+ docName: string | null;+ size: number | null;+ metadata: { side: 'ours' | 'theirs'; reason: 'check-failed' | 'too-large' };+ }| {kind: 'auto-consolidation';docName: string | null;@@ -88,6 +100,10 @@ export type ParsedCheckpoint =metadata: { foldedRefs: number; trigger: string };};+function isMergeSide(value: unknown): value is 'ours' | 'theirs' {+ return value === 'ours' || value === 'theirs';+}+export function parseCheckpoint(body: string): ParsedCheckpoint | null {if (!body) return null;for (const line of body.split('\n')) {@@ -269,6 +285,34 @@ export function parseCheckpoint(body: string): ParsedCheckpoint | null {}return null;}+ if (kind === 'external-change-both-kept') {+ const m = metadata as { side?: unknown; keptBlocks?: unknown };+ if (+ isMergeSide(m.side) &&+ typeof m.keptBlocks === 'number' &&+ Number.isFinite(m.keptBlocks)+ ) {+ return {+ kind: 'external-change-both-kept',+ docName,+ size,+ metadata: { side: m.side, keptBlocks: m.keptBlocks },+ };+ }+ return null;+ }+ if (kind === 'external-change-kept-live') {+ const m = metadata as { side?: unknown; reason?: unknown };+ if (isMergeSide(m.side) && (m.reason === 'check-failed' || m.reason === 'too-large')) {+ return {+ kind: 'external-change-kept-live',+ docName,+ size,+ metadata: { side: m.side, reason: m.reason },+ };+ }+ return null;+ }if (kind === 'auto-consolidation') {const m = metadata as { foldedRefs?: unknown; trigger?: unknown };if (@@ -401,6 +445,18 @@ export const CHECKPOINT_KIND_REGISTRY = {bundleExposure: 'metadata',chainAnchor: false,},+ 'external-change-both-kept': {+ visibility: 'surfaced',+ gcBucket: 'external-change-both-kept',+ bundleExposure: 'metadata',+ chainAnchor: false,+ },+ 'external-change-kept-live': {+ visibility: 'surfaced',+ gcBucket: 'external-change-kept-live',+ bundleExposure: 'metadata',+ chainAnchor: false,+ },'auto-consolidation': {visibility: 'hidden',gcBucket: 'auto-consolidation',@@ -490,6 +546,18 @@ export const CHECKPOINT_SAMPLE_BY_KIND = {size: 12,metadata: { diskBytes: 41, discardedBytes: 66 },},+ 'external-change-both-kept': {+ kind: 'external-change-both-kept',+ docName: 'notes',+ size: 12,+ metadata: { side: 'theirs', keptBlocks: 1 },+ },+ 'external-change-kept-live': {+ kind: 'external-change-kept-live',+ docName: 'notes',+ size: 12,+ metadata: { side: 'theirs', reason: 'check-failed' },+ },'auto-consolidation': {kind: 'auto-consolidation',docName: null,diff --git a/packages/core/src/index.ts b/packages/core/src/index.tsindex 4e02e4b32..c1e90ce96 100644--- a/packages/core/src/index.ts+++ b/packages/core/src/index.ts@@ -1410,6 +1410,7 @@ export {interpretSkillMoveFailure,isBranchNotFoundGitError,isLoginFixableGitAuthError,+ isReconcileMergeOutcome,isSemanticSearchOffered,isSkillMoveRetainedDestinationCode,isSkillMoveStateCode,@@ -1511,9 +1512,11 @@ export {PushPermissionSchema,type PushPermissionWire,QueueSuccessSchema,+ RECONCILE_MERGE_OUTCOMES,RECONCILE_REASONS,REFUSED_SYMLINK_PATHS_CAP,RESOLVE_STRATEGIES,+ type ReconcileMergeOutcome,ReconcileReasonSchema,type ReconcileReasonWire,type RenamedAssetMapping,diff --git a/packages/core/src/schemas/api/agent-write.ts b/packages/core/src/schemas/api/agent-write.tsindex 93fafa95e..a8c75e749 100644--- a/packages/core/src/schemas/api/agent-write.ts+++ b/packages/core/src/schemas/api/agent-write.ts@@ -107,6 +107,13 @@ export const OrphanHintSchema = z.loose() satisfies StandardSchemaV1;export type OrphanHint = z.infer<typeof OrphanHintSchema>;+export const RECONCILE_MERGE_OUTCOMES = ['clean', 'merged', 'both-kept', 'kept-live'] as const;+export type ReconcileMergeOutcome = (typeof RECONCILE_MERGE_OUTCOMES)[number];++export function isReconcileMergeOutcome(value: unknown): value is ReconcileMergeOutcome {+ return (RECONCILE_MERGE_OUTCOMES as readonly unknown[]).includes(value);+}+export const DiskEditReconciledWarningSchema = z.object({kind: z.literal('disk-edit-reconciled'),@@ -114,6 +121,10 @@ export const DiskEditReconciledWarningSchema = zactualBytes: z.number().int().nonnegative(),byteDelta: z.number().int(),mergeOutcome: z.string().optional(),+ duplicatedBlocks: z.number().int().nonnegative().optional(),+ restoredDeletedHere: z.number().int().nonnegative().optional(),+ restoredDeletedOutside: z.number().int().nonnegative().optional(),+ nextStep: z.string().optional(),hint: z.string().optional(),}).loose() satisfies StandardSchemaV1;diff --git a/packages/server/assets/skills/project/references/conflict-resolution.md b/packages/server/assets/skills/project/references/conflict-resolution.mdindex 786ed3bff..4d5b53a4f 100644--- a/packages/server/assets/skills/project/references/conflict-resolution.md+++ b/packages/server/assets/skills/project/references/conflict-resolution.md@@ -28,7 +28,7 @@ A write that first detects a stale file during its final disk flush returns `urn| `working-tree` | a pull-only overlay collision, pinned to the origin blob it collided with | writes the chosen bytes to disk; nothing is committed || `reconcile` | OK's own three-way merge of the editor against disk, including stale external saves. No git object holds the stages, so OK snapshots them at detection | writes the chosen bytes to disk and into the loaded document; no git command runs |-A `reconcile` conflict carries a `reason`: `merged-with-markers` (the three-way merge produced markers), `refused-conflict-markers` (the disk bytes already carried markers), `refused-too-large` (the document is past the merge size cap or, rarely, the merged output failed OpenKnowledge's fidelity check; the server log names the failed check), `disk-markers` (markers landed on disk under an open doc), or `stale-external-write` (an older displaced version returned on disk). The compatibility `conflictKind` field is `stale-external-write` for that last reason and `git` otherwise.+A `reconcile` conflict carries a `reason`: `refused-conflict-markers` (the disk bytes already carried markers), `disk-markers` (markers landed on disk under an open doc), `refused-no-base` (there was no acknowledged version to merge against), `refused-too-large` (a merge past the size cap or one that failed OpenKnowledge's fidelity check, raised only when the outside version could not be saved to history), or `stale-external-write` (an older displaced version returned on disk). `merged-with-markers` appears only on entries recorded by earlier versions, or in the rare case where both versions were merged but could not be applied to the open document. An outside edit to the same block as edits made here is not a conflict: both versions are kept, this document's version first, and your write reports `disk-edit-reconciled` with `mergeOutcome: "both-kept"` (see `references/doc-editing.md`). The compatibility `conflictKind` field is `stale-external-write` for that last reason and `git` otherwise.## What the user choosesdiff --git a/packages/server/assets/skills/project/references/doc-editing.md b/packages/server/assets/skills/project/references/doc-editing.mdindex 60dbe244d..87f42c7ba 100644--- a/packages/server/assets/skills/project/references/doc-editing.md+++ b/packages/server/assets/skills/project/references/doc-editing.md@@ -10,4 +10,4 @@**MDX.** To author an MDX doc (the KB renders MDX/JSX components), set `extension: ".mdx"` on the create: `write({ document: { path: "guides/widget", content, extension: ".mdx", position: "replace" } })` lands `guides/widget.mdx`. A `.mdx` suffix typed into `path` works too (the `extension` field wins if you pass both); omit both and it lands `.md`. An existing doc keeps its on-disk extension regardless — changing it in place isn't available via the MCP today.-**Advisory warnings on writes.** `write` and `edit` responses may include `structuredContent.document.warnings` (batch: per-doc `structuredContent.documents[].warnings`) — advisory entries discriminated by `kind`, each also summarized as a `⚠` line in the response text. The write always landed; the entries tell you what to do next. Write-integrity kinds mean re-read the doc (`exec("cat <path>")`) before continuing: `content-divergence` (`{ kind, intendedBytes, actualBytes, byteDelta, hint }` — the converged Y.Text doesn't match what the payload composed to: concurrent peer residue, or — rare — a primitive regression) and `disk-edit-reconciled` (an out-of-band disk edit was folded in before your write landed on top). The renderability kind `mermaid-parse-error` (`{ kind, fenceIndex, fenceFirstLine, message, line? }`) means that mermaid fence will not render — fix the fence and re-edit. The top-level `warning` field on write tools is unrelated: it is the preview-attach hint (`action: "attach-preview-once" | "start-ui"`), not an advisory — a separate key from `document.warnings`, and the two can coexist on one response.+**Advisory warnings on writes.** `write` and `edit` responses may include `structuredContent.document.warnings` (batch: per-doc `structuredContent.documents[].warnings`) — advisory entries discriminated by `kind`, each also summarized as a `⚠` line in the response text. The write always landed; the entries tell you what to do next. Write-integrity kinds mean re-read the doc (`exec("cat <path>")`) before continuing: `content-divergence` (`{ kind, intendedBytes, actualBytes, byteDelta, hint }` — the converged Y.Text doesn't match what the payload composed to: concurrent peer residue, or — rare — a primitive regression) and `disk-edit-reconciled` (an out-of-band disk edit was folded in before your write landed on top; `mergeOutcome` says how: `clean`, `merged`, `both-kept` or `kept-live`). With `"both-kept"` the out-of-band edit changed the same blocks as edits made here, and three counts say what landed: `duplicatedBlocks` blocks are in the doc twice, this doc's version first and the out-of-band version directly under it; `restoredDeletedHere` blocks deleted here came back with the out-of-band version because it changed them; `restoredDeletedOutside` blocks the out-of-band edit deleted stayed with this doc's version because they were changed here. With `"kept-live"` it could not be merged, so the version here stayed and the out-of-band version was saved to the doc's history. Both carry `nextStep: "re-read"`, and in both your write landed: do not retry it. Re-read, and after `both-kept` remove a duplicate, or delete a block restored over a deletion here, with an ordinary `edit` if your task calls for it. A `mergeOutcome` or `nextStep` you do not know means the same as `merged`: re-read. The renderability kind `mermaid-parse-error` (`{ kind, fenceIndex, fenceFirstLine, message, line? }`) means that mermaid fence will not render — fix the fence and re-edit. The top-level `warning` field on write tools is unrelated: it is the preview-attach hint (`action: "attach-preview-once" | "start-ui"`), not an advisory — a separate key from `document.warnings`, and the two can coexist on one response.diff --git a/packages/server/src/api-extension.ts b/packages/server/src/api-extension.tsindex c526bff2b..07ddf0b97 100644--- a/packages/server/src/api-extension.ts+++ b/packages/server/src/api-extension.ts@@ -36,7 +36,6 @@ import {createCodeFenceTracker,createWikiAssetResolver,DEFAULT_LINTER_CONFIG,- type DiskEditReconciledWarning,type DocumentListEntry,estimateSkillCost,type HeadingEntry,@@ -234,7 +233,7 @@ import {type StoreFailure,} from './document-durability-state.ts';import {- type ReconcileBeforeWriteResult,+ buildReconcileWarning,reconcileDiskBeforeAgentWrite,reviveFrozenResidentDoc,} from './external-change.ts';@@ -354,6 +353,7 @@ import {type WikiRenameContext,wikiLinkRenameDestination,} from './managed-rename-rewrite.ts';+import { createMergeHistoryRecorder, type MergeHistoryWriter } from './merge-history.ts';import {incrementAgentWriteCalls,incrementSummariesProvided,@@ -1340,6 +1340,7 @@ interface ApiExtensionBaseOptions {getConfigDiagnostics?: () => ConfigDiagnosticsReport;resolveEmbed?: (basename: string, sourcePath: string) => string | null;getBridgeLossReporter?: () => BridgeDeriveLossReporter | undefined;+ mergeHistoryWriter?: MergeHistoryWriter;getPrincipal?: () => Principal | null;homeDirOverride?: string;resetSettingsForTest?: () => void;@@ -2006,22 +2007,12 @@ export function createApiExtension(options: ApiExtensionOptions): Extension & {errorResponse(res, 409, type, title, { handler, detail, extensions });}- function buildReconcileWarning(- reconcile: ReconcileBeforeWriteResult,- ): DiskEditReconciledWarning | undefined {- if (!reconcile.reconciled) return undefined;- return {- kind: 'disk-edit-reconciled',- intendedBytes: reconcile.baseBytes,- actualBytes: reconcile.diskBytes,- byteDelta: reconcile.diskBytes - reconcile.baseBytes,- ...(reconcile.mergeOutcome ? { mergeOutcome: reconcile.mergeOutcome } : {}),- hint:- reconcile.mergeOutcome === 'merged'- ? 'An out-of-band edit was three-way merged into this document before your edit was applied on top; the merge may have interleaved content blocks. Re-read it (e.g. `exec("cat <path>")`) and review the combined result carefully before continuing.'- : 'An out-of-band edit was reconciled into this document before your edit was applied on top; the document now reflects that edit plus yours. Re-read it (e.g. `exec("cat <path>")`) to see the combined result before continuing.',- };- }+ const mergeHistory = createMergeHistoryRecorder({+ getShadow: () => shadowRef?.current,+ contentRoot: contentRoot ?? '',+ getBranch: () => getCurrentBranch?.() ?? null,+ writeCheckpoint: options.mergeHistoryWriter,+ });/*** Content-scope exclusion for a docName, mirroring the file-watcher's markdown admission@@ -2817,7 +2808,7 @@ export function createApiExtension(options: ApiExtensionOptions): Extension & {span.setAttribute('rename.rewrite_candidates', pendingRewrites.length);assertRewriteTargetsNotConflicted(pendingRewrites.map((entry) => entry.docName));- reconcileDiskBeforeAgentWrite(+ await reconcileDiskBeforeAgentWrite(durabilityState,hocuspocus,sourceDocName,@@ -2825,6 +2816,7 @@ export function createApiExtension(options: ApiExtensionOptions): Extension & {undefined,getBridgeLossReporter?.(),conflicts,+ mergeHistory,);if (recentlyRemovedDocs && !isSystemDoc(sourceDocName) && !isConfigDoc(sourceDocName)) {recentlyRemovedDocs.setDeleted(sourceDocName);@@ -3069,7 +3061,7 @@ export function createApiExtension(options: ApiExtensionOptions): Extension & {throw error;}if (!existsSync(candidatePath)) continue;- reconcileDiskBeforeAgentWrite(+ await reconcileDiskBeforeAgentWrite(durabilityState,hocuspocus,docName,@@ -3077,6 +3069,7 @@ export function createApiExtension(options: ApiExtensionOptions): Extension & {undefined,getBridgeLossReporter?.(),conflicts,+ mergeHistory,);const content = readCurrentDocumentContent(docName);if (typeof content !== 'string') continue;@@ -3105,7 +3098,7 @@ export function createApiExtension(options: ApiExtensionOptions): Extension & {}}- reconcileDiskBeforeAgentWrite(+ await reconcileDiskBeforeAgentWrite(durabilityState,hocuspocus,docName,@@ -3113,6 +3106,7 @@ export function createApiExtension(options: ApiExtensionOptions): Extension & {undefined,getBridgeLossReporter?.(),conflicts,+ mergeHistory,);const content = readCurrentDocumentContent(docName);if (typeof content === 'string') {@@ -3648,7 +3642,7 @@ export function createApiExtension(options: ApiExtensionOptions): Extension & {clientName,});- const agentWriteReconcile = reconcileDiskBeforeAgentWrite(+ const agentWriteReconcile = await reconcileDiskBeforeAgentWrite(durabilityState,hocuspocus,docName,@@ -3656,6 +3650,7 @@ export function createApiExtension(options: ApiExtensionOptions): Extension & {options.resolveEmbed,getBridgeLossReporter?.(),conflicts,+ mergeHistory,);const timestamp = new Date().toISOString();@@ -3937,7 +3932,7 @@ export function createApiExtension(options: ApiExtensionOptions): Extension & {clientName,});- const reconcile = reconcileDiskBeforeAgentWrite(+ const reconcile = await reconcileDiskBeforeAgentWrite(durabilityState,hocuspocus,resolvedDocName,@@ -3945,6 +3940,7 @@ export function createApiExtension(options: ApiExtensionOptions): Extension & {options.resolveEmbed,getBridgeLossReporter?.(),conflicts,+ mergeHistory,);const entryEmbedResolver = options.resolveEmbed@@ -5502,6 +5498,7 @@ export function createApiExtension(options: ApiExtensionOptions): Extension & {hocuspocus,options,getBridgeLossReporter,+ mergeHistory,agentPresenceBroadcaster,recordContentDivergenceGate,buildAgentActor,diff --git a/packages/server/src/external-change-private-state.test.ts b/packages/server/src/external-change-private-state.test.tsindex 2b9700776..c673caad2 100644--- a/packages/server/src/external-change-private-state.test.ts+++ b/packages/server/src/external-change-private-state.test.ts@@ -15,6 +15,7 @@ import type * as Y from 'yjs';import { RECONCILE_TEST_CONFLICTS } from './conflict-authority.test-helper.ts';import { DocumentDurabilityState } from './document-durability-state.ts';import { reconcileDiskBeforeAgentWrite } from './external-change.ts';+import { NO_SHADOW_MERGE_HISTORY } from './merge-history.test-helper.ts';describe('reconcile before an agent write', () => {let contentDir = '';@@ -46,7 +47,7 @@ describe('reconcile before an agent write', () => {unlinkSync(join(contentDir, 'note.md'));symlinkSync('.ok/local/secret.md', join(contentDir, 'note.md'));- const result = reconcileDiskBeforeAgentWrite(+ const result = await reconcileDiskBeforeAgentWrite(durabilityState,hp,'note',@@ -54,6 +55,7 @@ describe('reconcile before an agent write', () => {undefined,undefined,RECONCILE_TEST_CONFLICTS,+ NO_SHADOW_MERGE_HISTORY,);expect(result.reconciled).toBe(false);diff --git a/packages/server/src/external-change.test.ts b/packages/server/src/external-change.test.tsindex b38f7cba2..2772e8ce4 100644--- a/packages/server/src/external-change.test.ts+++ b/packages/server/src/external-change.test.ts@@ -1,4 +1,4 @@-import { mkdirSync, mkdtempSync, renameSync, rmSync } from 'node:fs';+import { mkdirSync, mkdtempSync, realpathSync, renameSync, rmSync, writeFileSync } from 'node:fs';import { tmpdir } from 'node:os';import { join } from 'node:path';import { Hocuspocus } from '@hocuspocus/server';@@ -7,15 +7,19 @@ import {BridgeMergeContentLossError,stripFrontmatter,} from '@inkeep/open-knowledge-core';-import { beforeEach, describe, expect, test, vi } from 'vitest';-import type * as Y from 'yjs';+import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest';+import * as Y from 'yjs';import { DocumentDurabilityState } from './document-durability-state.ts';import {applyExternalChange,+ buildReconcileWarning,createExternalChangeHandler,+ reconcileDiskBeforeAgentWrite,+ reportReconcileBothKept,reportReconcileCheckFailed,} from './external-change.ts';import { getLogger } from './logger.ts';+import type { MergeHistoryRecorder } from './merge-history.ts';import { getMetrics, resetMetrics } from './metrics.ts';import { reconcile } from './reconciliation.ts';@@ -482,3 +486,159 @@ describe('reportReconcileCheckFailed', () => {}});});++describe('reconcileDiskBeforeAgentWrite keeps the live side only once the outside version is in history', () => {+ const docName = 'kept-live-doc';+ const base = 'A\n\nB\n\nC\n';+ const ours = 'A\n\nX\n\nB edited\n\nC\n';+ const theirs = 'A\n\nX\n\nB\n\nC\n';+ let contentDir: string;+ let document: Y.Doc;+ let durabilityState: DocumentDurabilityState;+ const conflicts = {+ raise: vi.fn(),+ dissolveReconcile: vi.fn(),+ fileOf: (name: string) => `${name}.md`,+ };++ const recorder = (keptLive: MergeHistoryRecorder['keptLive']): MergeHistoryRecorder => ({+ bothKept: vi.fn(),+ keptLive: vi.fn(keptLive),+ });++ const guard = (history: MergeHistoryRecorder) =>+ reconcileDiskBeforeAgentWrite(+ durabilityState,+ { documents: new Map([[docName, document]]) } as unknown as Hocuspocus,+ docName,+ contentDir,+ undefined,+ undefined,+ conflicts,+ history,+ );++ beforeEach(() => {+ contentDir = realpathSync(mkdtempSync(join(tmpdir(), 'ok-kept-live-')));+ document = new Y.Doc();+ document.getText('source').insert(0, ours);+ durabilityState = new DocumentDurabilityState();+ durabilityState.setReconciledBase(docName, base);+ writeFileSync(join(contentDir, `${docName}.md`), theirs, 'utf-8');+ conflicts.raise.mockClear();+ });++ afterEach(() => {+ document.destroy();+ rmSync(contentDir, { recursive: true, force: true });+ });++ test('a confirmed history write moves the base to the outside bytes and reports kept-live', async () => {+ const history = recorder(() => Promise.resolve(true));+ const result = await guard(history);++ expect(result).toMatchObject({ reconciled: true, mergeOutcome: 'kept-live' });+ expect(history.keptLive).toHaveBeenCalledWith(+ docName,+ { side: 'theirs', bytes: theirs },+ 'check-failed',+ );+ expect(durabilityState.getReconciledBase(docName)).toBe(theirs);+ expect(conflicts.raise).not.toHaveBeenCalled();+ });++ test('a history write that was not saved raises the reconcile conflict with all three stages and keeps the base', async () => {+ const result = await guard(recorder(() => Promise.resolve(false)));++ expect(result.reconciled).toBe(false);+ expect(conflicts.raise).toHaveBeenCalledWith({+ kind: 'reconcile',+ file: `${docName}.md`,+ reason: 'refused-too-large',+ stages: { base, ours, theirs },+ });+ expect(durabilityState.getReconciledBase(docName)).toBe(base);+ });+});++describe('buildReconcileWarning', () => {+ const reconciled = { reconciled: true, baseBytes: 10, diskBytes: 14 } as const;++ test('asks for a re-read only on the both-kept and kept-live outcomes', () => {+ expect(+ buildReconcileWarning({ ...reconciled, mergeOutcome: 'merged' })?.nextStep,+ ).toBeUndefined();+ expect(+ buildReconcileWarning({ ...reconciled, mergeOutcome: 'clean' })?.nextStep,+ ).toBeUndefined();+ expect(+ buildReconcileWarning({+ ...reconciled,+ mergeOutcome: 'both-kept',+ keptBlocks: { duplicated: 1, restoredDeletedHere: 0, restoredDeletedOutside: 0 },+ })?.nextStep,+ ).toBe('re-read');+ expect(buildReconcileWarning({ ...reconciled, mergeOutcome: 'kept-live' })?.nextStep).toBe(+ 're-read',+ );+ expect(+ buildReconcileWarning({ reconciled: false, baseBytes: 0, diskBytes: 0 }),+ ).toBeUndefined();+ });++ const bothKept = (+ duplicated: number,+ restoredDeletedHere: number,+ restoredDeletedOutside: number,+ ) =>+ buildReconcileWarning({+ ...reconciled,+ mergeOutcome: 'both-kept',+ keptBlocks: { duplicated, restoredDeletedHere, restoredDeletedOutside },+ });++ test('duplicates only: names the duplicate and asks to remove it', () => {+ const warning = bothKept(2, 0, 0);+ expect(warning).toMatchObject({+ duplicatedBlocks: 2,+ restoredDeletedHere: 0,+ restoredDeletedOutside: 0,+ });+ expect(warning?.hint).toBe(+ 'An out-of-band edit changed 2 blocks also edited here, so both versions were kept: the version here first, the out-of-band version directly under it. Your write landed; do not retry it. Re-read the document (e.g. `exec("cat <path>")`) and, if your task calls for it, remove the duplicate with an ordinary edit.',+ );+ });++ test('a deletion here undone: names this side as the one whose deletion was undone', () => {+ const warning = bothKept(0, 1, 0);+ expect(warning).toMatchObject({ duplicatedBlocks: 0, restoredDeletedHere: 1 });+ expect(warning?.hint).toBe(+ 'The deletion here of 1 block was undone because the out-of-band edit changed it: the out-of-band version was restored. Your write landed; do not retry it. Re-read the document (e.g. `exec("cat <path>")`) and, if your task calls for it, delete the restored block again with an ordinary edit.',+ );+ });++ test('an out-of-band deletion undone beside duplicates: both sentences, no invitation to delete', () => {+ const warning = bothKept(1, 0, 1);+ expect(warning).toMatchObject({ duplicatedBlocks: 1, restoredDeletedOutside: 1 });+ expect(warning?.hint).toBe(+ 'An out-of-band edit changed 1 block also edited here, so both versions were kept: the version here first, the out-of-band version directly under it. The out-of-band deletion of 1 block edited here was undone: the version here was kept. Your write landed; do not retry it. Re-read the document (e.g. `exec("cat <path>")`) and, if your task calls for it, remove the duplicate with an ordinary edit.',+ );+ });+});++describe('reportReconcileBothKept', () => {+ test('counts the merge once and only the blocks that landed twice', () => {+ resetMetrics();+ reportReconcileBothKept(+ 'counted-doc',+ [+ { blockIndex: 0, base: 'A', ours: 'A ours', theirs: 'A theirs', landed: 'both' },+ { blockIndex: 1, base: 'B', ours: '', theirs: 'B theirs', landed: 'theirs' },+ ],+ 'watcher',+ );+ expect(getMetrics().reconcileBothKept).toBe(1);+ expect(getMetrics().reconcileBothKeptBlocks).toBe(1);+ resetMetrics();+ });+});diff --git a/packages/server/src/external-change.ts b/packages/server/src/external-change.tsindex 9d7b783de..63068c02d 100644--- a/packages/server/src/external-change.ts+++ b/packages/server/src/external-change.ts@@ -3,8 +3,10 @@ import type { Hocuspocus } from '@hocuspocus/server';import {BridgeInvariantViolationError,BridgeMergeContentLossError,+ type DiskEditReconciledWarning,normalizeBridge,prependFrontmatter,+ type ReconcileMergeOutcome,stripFrontmatter,toBridgeInvariantLog,} from '@inkeep/open-knowledge-core';@@ -35,14 +37,24 @@ import { takeExternalChangeAttribution } from './external-change-attribution.ts'import { recordFrontmatterEditSurface } from './frontmatter-telemetry.ts';import { canonicalContentPathIsRefused } from './fs-safety.ts';import { getLogger } from './logger.ts';+import type { MergeHistoryRecorder } from './merge-history.ts';import {incrementExternalChangeHandlerErrors,+ incrementReconcileBothKept,incrementReconcileCheckFailed,incrementReconcileInFlightFallthroughs,incrementReconcileInsertDedupSkipped,+ incrementReconcileKeptLive,incrementReconcileOwnFlushSkips,} from './metrics.ts';-import { type ReconcileOutcome, type ReconcileRefusalReason, reconcile } from './reconciliation.ts';+import {+ countKeptBlocks,+ type KeptBlockCounts,+ type KeptBlockPair,+ type ReconcileOutcome,+ type ReconcileRefusalReason,+ reconcile,+} from './reconciliation.ts';import { FILE_SYSTEM_WRITER } from './shadow-repo.ts';export { FILE_WATCHER_ORIGIN } from './disk-content-intake.ts';@@ -257,6 +269,31 @@ export function reportReconcileCheckFailed();}+export function reportReconcileBothKept(+ docName: string,+ kept: readonly KeptBlockPair[],+ site: ReconcileCheckSite,+): void {+ const counts = countKeptBlocks(kept);+ incrementReconcileBothKept(counts.duplicated);+ getLogger('reconcile').info(+ { docName, site, ...counts },+ `[reconcile] ${site} ${docName} kept both versions of ${counts.duplicated} block(s), restored ${counts.restoredDeletedHere} block(s) deleted here and ${counts.restoredDeletedOutside} block(s) deleted outside`,+ );+}++export function reportReconcileKeptLive(+ docName: string,+ reason: 'check-failed' | 'too-large',+ site: ReconcileCheckSite,+): void {+ incrementReconcileKeptLive();+ getLogger('reconcile').warn(+ { docName, site, reason },+ `[reconcile] ${site} ${docName} could not be merged (${reason}); the live version stays and the other version was saved to history`,+ );+}+export function wireReasonForRefusal(reason: ReconcileRefusalReason): ReconcileReason {switch (reason) {case 'conflict-markers':@@ -332,7 +369,93 @@ export interface ReconcileBeforeWriteResult {reconciled: boolean;baseBytes: number;diskBytes: number;- mergeOutcome?: 'clean' | 'merged';+ mergeOutcome?: ReconcileMergeOutcome;+ keptBlocks?: KeptBlockCounts;+}++const RE_READ = 'Re-read the document (e.g. `exec("cat <path>")`)';++function blocks(count: number): string {+ return count === 1 ? '1 block' : `${count} blocks`;+}++function bothKeptHint(counts: KeptBlockCounts): string {+ const sentences: string[] = [];+ if (counts.duplicated > 0) {+ sentences.push(+ `An out-of-band edit changed ${blocks(counts.duplicated)} also edited here, so both versions were kept: the version here first, the out-of-band version directly under it.`,+ );+ }+ if (counts.restoredDeletedHere > 0) {+ sentences.push(+ `The deletion here of ${blocks(counts.restoredDeletedHere)} was undone because the out-of-band edit changed ${counts.restoredDeletedHere === 1 ? 'it' : 'them'}: the out-of-band version was restored.`,+ );+ }+ if (counts.restoredDeletedOutside > 0) {+ sentences.push(+ `The out-of-band deletion of ${blocks(counts.restoredDeletedOutside)} edited here was undone: the version here was kept.`,+ );+ }+ sentences.push('Your write landed; do not retry it.');+ const tidy = [+ ...(counts.duplicated > 0 ? ['remove the duplicate'] : []),+ ...(counts.restoredDeletedHere > 0 ? ['delete the restored block again'] : []),+ ];+ sentences.push(+ tidy.length > 0+ ? `${RE_READ} and, if your task calls for it, ${tidy.join(' or ')} with an ordinary edit.`+ : `${RE_READ} before continuing.`,+ );+ return sentences.join(' ');+}++function reconcileWarningHint(reconcile: ReconcileBeforeWriteResult): string {+ const outcome = reconcile.mergeOutcome;+ switch (outcome) {+ case 'merged':+ return 'An out-of-band edit was three-way merged into this document before your edit was applied on top; the merge may have interleaved content blocks. Re-read it (e.g. `exec("cat <path>")`) and review the combined result carefully before continuing.';+ case 'both-kept':+ return bothKeptHint(+ reconcile.keptBlocks ?? {+ duplicated: 1,+ restoredDeletedHere: 0,+ restoredDeletedOutside: 0,+ },+ );+ case 'kept-live':+ return 'An out-of-band edit to this file could not be merged safely, so the version here was kept and the out-of-band version was saved to the history of this document. Your write landed; do not retry it. Re-read the document (e.g. `exec("cat <path>")`) before continuing.';+ case 'clean':+ case undefined:+ return 'An out-of-band edit was reconciled into this document before your edit was applied on top; the document now reflects that edit plus yours. Re-read it (e.g. `exec("cat <path>")`) to see the combined result before continuing.';+ default: {+ const unhandled: never = outcome;+ throw new Error(`unhandled merge outcome: ${String(unhandled)}`);+ }+ }+}++export function buildReconcileWarning(+ reconcile: ReconcileBeforeWriteResult,+): DiskEditReconciledWarning | undefined {+ if (!reconcile.reconciled) return undefined;+ const asksReRead =+ reconcile.mergeOutcome === 'both-kept' || reconcile.mergeOutcome === 'kept-live';+ return {+ kind: 'disk-edit-reconciled',+ intendedBytes: reconcile.baseBytes,+ actualBytes: reconcile.diskBytes,+ byteDelta: reconcile.diskBytes - reconcile.baseBytes,+ ...(reconcile.mergeOutcome ? { mergeOutcome: reconcile.mergeOutcome } : {}),+ ...(reconcile.keptBlocks === undefined+ ? {}+ : {+ duplicatedBlocks: reconcile.keptBlocks.duplicated,+ restoredDeletedHere: reconcile.keptBlocks.restoredDeletedHere,+ restoredDeletedOutside: reconcile.keptBlocks.restoredDeletedOutside,+ }),+ ...(asksReRead ? { nextStep: 're-read' as const } : {}),+ hint: reconcileWarningHint(reconcile),+ };}const NOT_RECONCILED: ReconcileBeforeWriteResult = {@@ -349,7 +472,7 @@ export function serializeYDocSource(document: {return prependFrontmatter(frontmatter, body);}-export function reconcileDiskBeforeAgentWrite(+export async function reconcileDiskBeforeAgentWrite(durabilityState: DocumentDurabilityState,hocuspocus: Hocuspocus,docName: string,@@ -357,7 +480,8 @@ export function reconcileDiskBeforeAgentWrite(resolveEmbed: ((basename: string, sourcePath: string) => string | null) | undefined,bridgeLossReporter: BridgeDeriveLossReporter | undefined,conflicts: Pick<ConflictAuthority, 'dissolveReconcile' | 'fileOf' | 'raise'>,-): ReconcileBeforeWriteResult {+ history: MergeHistoryRecorder,+): Promise<ReconcileBeforeWriteResult> {if (isSystemDoc(docName) ||isConfigDoc(docName) ||@@ -449,29 +573,46 @@ export function reconcileDiskBeforeAgentWrite(`[reconcile] before-agent-write ${docName} result=${outcome.kind}`,);+ const reconciledResult = (+ mergeOutcome: ReconcileMergeOutcome,+ keptBlocks?: KeptBlockCounts,+ ): ReconcileBeforeWriteResult => ({+ reconciled: true,+ baseBytes: Buffer.byteLength(base, 'utf8'),+ diskBytes: Buffer.byteLength(diskContent, 'utf8'),+ mergeOutcome,+ ...(keptBlocks === undefined ? {} : { keptBlocks }),+ });+switch (outcome.kind) {case 'noop':return NOT_RECONCILED;- case 'conflicts':case 'refused': {reportReconcileCheckFailed(docName, outcome, 'agent-write');+ if (outcome.reason === 'check-failed' || outcome.reason === 'too-large') {+ if (+ await history.keptLive(docName, { side: 'theirs', bytes: diskContent }, outcome.reason)+ ) {+ durabilityState.setReconciledBase(docName, diskContent);+ reportReconcileKeptLive(docName, outcome.reason, 'agent-write');+ return reconciledResult('kept-live');+ }+ }conflicts.raise({kind: 'reconcile',file: conflicts.fileOf(docName),- reason:- outcome.kind === 'conflicts'- ? 'merged-with-markers'- : wireReasonForRefusal(outcome.reason),+ reason: wireReasonForRefusal(outcome.reason),stages: { base, ours, theirs: diskContent },});return NOT_RECONCILED;}case 'clean':- case 'merged': {+ case 'merged':+ case 'both-kept': {const ingest = outcome.kind === 'clean' ? diskContent : outcome.newContent;- if (outcome.kind === 'merged' && outcome.dedupSkipped) {+ if (outcome.kind !== 'clean' && outcome.dedupSkipped) {incrementReconcileInsertDedupSkipped();getLogger('reconcile').warn({ docName },@@ -487,15 +628,14 @@ export function reconcileDiskBeforeAgentWrite(undefined,bridgeLossReporter,);- if (outcome.kind === 'merged') {+ if (outcome.kind !== 'clean') {durabilityState.setReconciledBase(docName, diskContent);}- return {- reconciled: true,- baseBytes: Buffer.byteLength(base, 'utf8'),- diskBytes: Buffer.byteLength(diskContent, 'utf8'),- mergeOutcome: outcome.kind,- };+ if (outcome.kind !== 'both-kept') return reconciledResult(outcome.kind);+ const keptBlocks = countKeptBlocks(outcome.kept);+ history.bothKept(docName, { ours, theirs: diskContent }, keptBlocks.duplicated);+ reportReconcileBothKept(docName, outcome.kept, 'agent-write');+ return reconciledResult('both-kept', keptBlocks);}}}diff --git a/packages/server/src/http/agent-write-routes.ts b/packages/server/src/http/agent-write-routes.tsindex 7747e49c3..feb6f9ab3 100644--- a/packages/server/src/http/agent-write-routes.ts+++ b/packages/server/src/http/agent-write-routes.ts@@ -80,6 +80,7 @@ import {import { recordFrontmatterEditSurface } from '../frontmatter-telemetry.ts';import type { LinkAdvisoryPolicy, PrepareWriteLinkAdvisory } from '../link-advisory-policy.ts';import { getLogger } from '../logger.ts';+import type { MergeHistoryRecorder } from '../merge-history.ts';import { validateMermaidFences } from '../mermaid-validator.ts';import { incrementAgentPatchFindMismatches, incrementAgentWriteCalls } from '../metrics.ts';import { precomputeParse } from '../parse-pool.ts';@@ -167,6 +168,7 @@ export interface AgentWriteRouteDeps {resolveEmbed?: (basename: string, sourcePath: string) => string | null;};getBridgeLossReporter: (() => BridgeDeriveLossReporter | undefined) | undefined;+ mergeHistory: MergeHistoryRecorder;agentPresenceBroadcaster: AgentPresenceBroadcaster | undefined;recordContentDivergenceGate: (handler: 'agent-write-md' | 'agent-write-batch' | 'agent-patch' | 'rollback',@@ -242,6 +244,7 @@ export function createAgentWriteRoutes(deps: AgentWriteRouteDeps): ApiRouteGrouphocuspocus,options,getBridgeLossReporter,+ mergeHistory,agentPresenceBroadcaster,recordContentDivergenceGate,buildAgentActor,@@ -369,7 +372,7 @@ export function createAgentWriteRoutes(deps: AgentWriteRouteDeps): ApiRouteGroupcolorSeed,clientName,});- const writeMdReconcile = reconcileDiskBeforeAgentWrite(+ const writeMdReconcile = await reconcileDiskBeforeAgentWrite(durabilityState,hocuspocus,resolvedDocName,@@ -377,6 +380,7 @@ export function createAgentWriteRoutes(deps: AgentWriteRouteDeps): ApiRouteGroupoptions.resolveEmbed,getBridgeLossReporter?.(),conflicts,+ mergeHistory,);const writeMdEmbedResolver = options.resolveEmbed? { resolveEmbed: options.resolveEmbed, sourcePath: resolvedDocName }@@ -593,7 +597,7 @@ export function createAgentWriteRoutes(deps: AgentWriteRouteDeps): ApiRouteGroupcolorSeed,clientName,});- const fmReconcile = reconcileDiskBeforeAgentWrite(+ const fmReconcile = await reconcileDiskBeforeAgentWrite(durabilityState,hocuspocus,resolvedDocName,@@ -601,6 +605,7 @@ export function createAgentWriteRoutes(deps: AgentWriteRouteDeps): ApiRouteGroupoptions.resolveEmbed,getBridgeLossReporter?.(),conflicts,+ mergeHistory,);const fmPatchPrecomputed = await prepareFrontmatterPatchParse(session.dc.document, patch);const timestamp = new Date().toISOString();@@ -839,7 +844,7 @@ export function createAgentWriteRoutes(deps: AgentWriteRouteDeps): ApiRouteGroupcolorSeed,clientName,});- const patchReconcile = reconcileDiskBeforeAgentWrite(+ const patchReconcile = await reconcileDiskBeforeAgentWrite(durabilityState,hocuspocus,docName,@@ -847,6 +852,7 @@ export function createAgentWriteRoutes(deps: AgentWriteRouteDeps): ApiRouteGroupoptions.resolveEmbed,getBridgeLossReporter?.(),conflicts,+ mergeHistory,);const patchEmbedResolver = options.resolveEmbed? { resolveEmbed: options.resolveEmbed, sourcePath: docName }diff --git a/packages/server/src/index.ts b/packages/server/src/index.tsindex 1a5baea4d..c7f22b46e 100644--- a/packages/server/src/index.ts+++ b/packages/server/src/index.ts@@ -497,9 +497,9 @@ export {ProjectGitInitError,} from './project-git.ts';export {- type BlockConflict,CONFLICT_MARKER_RE,containsConflictMarkers,+ type KeptBlockPair,type ReconcileInput,type ReconcileOutcome,reconcile,diff --git a/packages/server/src/mcp/tools/advisory-warnings.test.ts b/packages/server/src/mcp/tools/advisory-warnings.test.tsindex d134df664..1dbf60931 100644--- a/packages/server/src/mcp/tools/advisory-warnings.test.ts+++ b/packages/server/src/mcp/tools/advisory-warnings.test.ts@@ -121,6 +121,76 @@ describe('formatAdvisoryBriefs', () => {const briefs = formatAdvisoryBriefs([mermaidWarning(), mermaidWarning({ fenceIndex: 2 })]);expect(briefs[0]).toContain('2 mermaid fences');});++ test('a both-kept or kept-live reconcile says the write landed and names its outcome', () => {+ const counts = (duplicated: number, here: number, outside: number) => ({+ ...RECONCILED,+ mergeOutcome: 'both-kept',+ nextStep: 're-read',+ duplicatedBlocks: duplicated,+ restoredDeletedHere: here,+ restoredDeletedOutside: outside,+ });+ const [+ duplicatesOnly,+ restoredHereOnly,+ restoredOutsideAndDuplicates,+ restoredHereAndDuplicates,+ restoredOutsideOnly,+ keptLive,+ ] = formatAdvisoryBriefs([+ counts(2, 0, 0),+ counts(0, 1, 0),+ counts(1, 0, 1),+ counts(1, 1, 0),+ counts(0, 0, 1),+ { ...RECONCILED, mergeOutcome: 'kept-live', nextStep: 're-read' },+ ]);+ expect(duplicatesOnly).toBe(+ '⚠ An out-of-band edit to the same block was kept beside the version here; the write landed. Re-read and remove the duplicate if needed.',+ );+ expect(restoredHereOnly).toBe(+ '⚠ A deletion here was undone and the out-of-band version restored; the write landed. Re-read and delete the restored block again if needed.',+ );+ expect(restoredOutsideAndDuplicates).toBe(+ '⚠ An out-of-band edit to the same block was kept beside the version here; an out-of-band deletion was undone and the version here kept; the write landed. Re-read and remove the duplicate if needed.',+ );+ expect(restoredHereAndDuplicates).toBe(+ '⚠ An out-of-band edit to the same block was kept beside the version here; a deletion here was undone and the out-of-band version restored; the write landed. Re-read and remove the duplicate or delete the restored block again if needed.',+ );+ expect(restoredOutsideOnly).toBe(+ '⚠ An out-of-band deletion was undone and the version here kept; the write landed. Re-read before continuing.',+ );+ expect(keptLive).toContain('saved to history');+ expect(keptLive).toContain('the write landed');+ });+});++describe('a merge outcome this tool copy does not know', () => {+ test('still parses and reaches the summary line through the generic reconcile brief', () => {+ const future = { ...RECONCILED, mergeOutcome: 'future-outcome', nextStep: 'future-step' };+ const parsed = parseAdvisoryWarnings([future]);++ expect(parsed).toEqual([future]);+ expect(formatAdvisoryBriefs(parsed ?? [])).toEqual([+ '⚠ Out-of-band disk edit reconciled before this write — re-read for the combined result.',+ ]);+ expect(formatAdvisoryLines(parsed ?? [])).toEqual([+ '⚠ An out-of-band edit was reconciled into this document before your edit landed on top — re-read for the combined result.',+ ]);+ });+});++describe('the reconcile next step', () => {+ test('parses on a reconcile warning and survives into structured content', () => {+ const parsed = parseAdvisoryWarnings([+ { ...RECONCILED, mergeOutcome: 'both-kept', nextStep: 're-read', hint: 'Re-read.' },+ ]);+ expect(parsed).toEqual([+ { ...RECONCILED, mergeOutcome: 'both-kept', nextStep: 're-read', hint: 'Re-read.' },+ ]);+ expect(formatAdvisoryLines(parsed ?? [])).toEqual(['⚠ Re-read.']);+ });});describe('render-family bounds phrasing', () => {diff --git a/packages/server/src/mcp/tools/advisory-warnings.ts b/packages/server/src/mcp/tools/advisory-warnings.tsindex 1ddc3468a..8a67caef1 100644--- a/packages/server/src/mcp/tools/advisory-warnings.ts+++ b/packages/server/src/mcp/tools/advisory-warnings.ts@@ -6,6 +6,8 @@ import {BrokenLinkSchema,type BrokenLinkSuppression,BrokenLinkSuppressionSchema,+ type DiskEditReconciledWarning,+ isReconcileMergeOutcome,isWriteWarningKind,type LinkCheckDeferredWarning,type LintViolationWarning,@@ -139,12 +141,55 @@ function formatIntegrityLine(d: WriteWarning): string {}}+function formatBothKeptBrief(d: DiskEditReconciledWarning): string {+ const restoredHere = d.restoredDeletedHere ?? 0;+ const restoredOutside = d.restoredDeletedOutside ?? 0;+ const duplicated = d.duplicatedBlocks ?? (restoredHere + restoredOutside > 0 ? 0 : 1);+ const parts = [+ ...(duplicated > 0+ ? ['an out-of-band edit to the same block was kept beside the version here']+ : []),+ ...(restoredHere > 0+ ? ['a deletion here was undone and the out-of-band version restored']+ : []),+ ...(restoredOutside > 0+ ? ['an out-of-band deletion was undone and the version here kept']+ : []),+ ];+ const summary = parts.join('; ');+ const tidy = [+ ...(duplicated > 0 ? ['remove the duplicate'] : []),+ ...(restoredHere > 0 ? ['delete the restored block again'] : []),+ ];+ const next =+ tidy.length > 0 ? `Re-read and ${tidy.join(' or ')} if needed.` : 'Re-read before continuing.';+ return `⚠ ${summary.charAt(0).toUpperCase()}${summary.slice(1)}; the write landed. ${next}`;+}++function formatReconciledBrief(d: DiskEditReconciledWarning): string {+ const outcome = isReconcileMergeOutcome(d.mergeOutcome) ? d.mergeOutcome : undefined;+ switch (outcome) {+ case 'both-kept':+ return formatBothKeptBrief(d);+ case 'kept-live':+ return '⚠ An out-of-band edit could not be merged and was saved to history; the write landed. Re-read before continuing.';+ case 'clean':+ case 'merged':+ case undefined:+ return '⚠ Out-of-band disk edit reconciled before this write — re-read for the combined result.';+ default: {+ const unhandled: never = outcome;+ return `⚠ Out-of-band disk edit reconciled before this write (${String(unhandled)}) — re-read for the combined result.`;+ }+ }+}+function formatIntegrityBrief(d: WriteWarning): string {switch (d.kind) {case 'content-divergence':return `⚠ Content divergence: ${d.actualBytes} actual vs ${d.intendedBytes} intended (byteDelta=${d.byteDelta}).`;case 'disk-edit-reconciled':- return '⚠ Out-of-band disk edit reconciled before this write — re-read for the combined result.';+ return formatReconciledBrief(d);default:return assertNeverWriteWarning(d);}diff --git a/packages/server/src/merge-history.test-helper.ts b/packages/server/src/merge-history.test-helper.tsnew file mode 100644index 000000000..0a1470184--- /dev/null+++ b/packages/server/src/merge-history.test-helper.ts@@ -0,0 +1,7 @@+import { createMergeHistoryRecorder } from './merge-history.ts';++export const NO_SHADOW_MERGE_HISTORY = createMergeHistoryRecorder({+ getShadow: () => undefined,+ contentRoot: '',+ getBranch: () => null,+});diff --git a/packages/server/src/merge-history.test.ts b/packages/server/src/merge-history.test.tsnew file mode 100644index 000000000..5fbde9744--- /dev/null+++ b/packages/server/src/merge-history.test.ts@@ -0,0 +1,98 @@+import { mkdirSync, mkdtempSync, realpathSync, rmSync } from 'node:fs';+import { tmpdir } from 'node:os';+import { join } from 'node:path';+import { parseCheckpoint } from '@inkeep/open-knowledge-core/shadow-repo-layout';+import simpleGit from 'simple-git';+import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest';+import { configureTestGitRepository } from '../../../test-support/configure-git-fixture.test-helper.ts';+import { createMergeHistoryRecorder } from './merge-history.ts';+import { getMetrics, resetMetrics } from './metrics.ts';+import { initShadowRepo, type ShadowHandle, shadowGit } from './shadow-repo.ts';++describe('createMergeHistoryRecorder', () => {+ let tmpDir: string;++ beforeEach(() => {+ tmpDir = realpathSync(mkdtempSync(join(tmpdir(), 'ok-merge-history-')));+ resetMetrics();+ });++ afterEach(() => {+ rmSync(tmpDir, { recursive: true, force: true });+ resetMetrics();+ });++ const recorderFor = (shadow: ShadowHandle | undefined) =>+ createMergeHistoryRecorder({+ getShadow: () => shadow,+ contentRoot: '',+ getBranch: () => 'main',+ });++ test('with no shadow repo nothing is written, keep-live resolves not saved, and both count as unavailable', async () => {+ const history = recorderFor(undefined);++ await expect(+ history.keptLive('doc', { side: 'theirs', bytes: 'outside\n' }, 'too-large'),+ ).resolves.toBe(false);+ history.bothKept('doc', { ours: 'here\n', theirs: 'outside\n' }, 1);++ expect(getMetrics().mergeHistoryUnavailable).toBe(2);+ expect(getMetrics().mergeHistoryCheckpointFailures).toBe(0);+ expect(getMetrics().mergeHistoryCheckpointCreated).toBe(0);+ });++ test('a keep-live write that fails resolves false and counts a failure, never a save', async () => {+ const history = recorderFor({+ gitDir: join(tmpDir, 'missing-git-dir'),+ workTree: tmpDir,+ });++ await expect(+ history.keptLive('doc', { side: 'theirs', bytes: 'outside\n' }, 'check-failed'),+ ).resolves.toBe(false);+ expect(getMetrics().mergeHistoryCheckpointFailures).toBe(1);+ expect(getMetrics().mergeHistoryCheckpointCreated).toBe(0);+ });++ test('a keep-live write that lands resolves true and leaves the full bytes in a checkpoint', async () => {+ const projectRoot = join(tmpDir, 'project');+ mkdirSync(projectRoot, { recursive: true });+ await simpleGit(projectRoot).init();+ configureTestGitRepository(projectRoot);+ const shadow = await initShadowRepo(projectRoot);+ const history = recorderFor(shadow);+ const outside = '# Outside\n\n indented \n';++ await expect(+ history.keptLive('doc', { side: 'theirs', bytes: outside }, 'check-failed'),+ ).resolves.toBe(true);+ history.bothKept('doc', { ours: 'here\n', theirs: outside }, 1);+ await vi.waitFor(() => expect(getMetrics().mergeHistoryCheckpointCreated).toBe(3));++ const sg = shadowGit(shadow);+ const shas = (await sg.raw('for-each-ref', '--format=%(objectname)', 'refs/checkpoints/main/'))+ .trim()+ .split('\n');+ const kinds: string[] = [];+ for (const sha of shas) {+ const parsed = parseCheckpoint(await sg.raw('log', '-1', '--format=%B', sha));+ if (+ parsed?.kind !== 'external-change-both-kept' &&+ parsed?.kind !== 'external-change-kept-live'+ ) {+ throw new Error(`unexpected checkpoint ${sha}: ${parsed?.kind}`);+ }+ kinds.push(`${parsed.kind}:${parsed.metadata.side}`);+ if (parsed.kind === 'external-change-kept-live') {+ expect(await sg.raw('show', `${sha}:doc`)).toBe(outside);+ }+ }+ expect(kinds.sort()).toEqual([+ 'external-change-both-kept:ours',+ 'external-change-both-kept:theirs',+ 'external-change-kept-live:theirs',+ ]);+ expect(getMetrics().mergeHistoryCheckpointFailures).toBe(0);+ });+});diff --git a/packages/server/src/merge-history.ts b/packages/server/src/merge-history.tsnew file mode 100644index 000000000..e874bdc47--- /dev/null+++ b/packages/server/src/merge-history.ts@@ -0,0 +1,127 @@+import type { ParsedCheckpoint } from '@inkeep/open-knowledge-core/shadow-repo-layout';+import { getLogger } from './logger.ts';+import {+ incrementMergeHistoryCheckpointCreated,+ incrementMergeHistoryCheckpointFailures,+ incrementMergeHistoryUnavailable,+} from './metrics.ts';+import {+ type InMemoryCheckpointParams,+ type ShadowHandle,+ saveInMemoryCheckpoint,+} from './shadow-repo.ts';++type KeptLiveMetadata = Extract<+ ParsedCheckpoint,+ { kind: 'external-change-kept-live' }+>['metadata'];++type MergeSide = KeptLiveMetadata['side'];++type MergeHistoryParams = Extract<+ InMemoryCheckpointParams,+ { kind: 'external-change-both-kept' | 'external-change-kept-live' }+>;++export interface MergeHistoryRecorder {+ bothKept(+ docName: string,+ versions: Readonly<Record<MergeSide, string>>,+ keptBlocks: number,+ ): void;+ keptLive(+ docName: string,+ displaced: { side: MergeSide; bytes: string },+ reason: KeptLiveMetadata['reason'],+ ): Promise<boolean>;+}++export type MergeHistoryWriter = typeof saveInMemoryCheckpoint;++export interface MergeHistoryRecorderOptions {+ getShadow: () => ShadowHandle | null | undefined;+ contentRoot: string;+ getBranch: () => string | null;+ writeCheckpoint?: MergeHistoryWriter;+}++const log = getLogger('reconcile');++const SIDE_LABEL: Record<MergeSide, string> = {+ ours: 'Version here',+ theirs: 'Outside version',+};++export function createMergeHistoryRecorder(+ options: MergeHistoryRecorderOptions,+): MergeHistoryRecorder {+ const writeCheckpoint = options.writeCheckpoint ?? saveInMemoryCheckpoint;+ const write = (shadow: ShadowHandle, params: MergeHistoryParams): Promise<boolean> =>+ writeCheckpoint(shadow, options.contentRoot, params).then(+ (sha) => {+ incrementMergeHistoryCheckpointCreated();+ log.info(+ { docName: params.docName, kind: params.kind, side: params.metadata.side, sha },+ `[reconcile] history entry saved (${params.kind}, ${params.metadata.side}): ${params.docName}`,+ );+ return true;+ },+ (err: unknown) => {+ incrementMergeHistoryCheckpointFailures();+ log.error(+ { err, docName: params.docName, kind: params.kind, side: params.metadata.side },+ `[reconcile] history entry write failed (${params.kind}, ${params.metadata.side}): ${params.docName}`,+ );+ return false;+ },+ );++ const unavailable = (docName: string, kind: MergeHistoryParams['kind']): void => {+ incrementMergeHistoryUnavailable();+ log.info(+ { docName, kind },+ `[reconcile] history entry not saved for ${docName} (${kind}): the shadow repo is unavailable`,+ );+ };++ return {+ bothKept(docName, versions, keptBlocks) {+ const shadow = options.getShadow();+ if (!shadow) {+ unavailable(docName, 'external-change-both-kept');+ return;+ }+ const branch = options.getBranch() ?? 'main';+ const at = new Date().toISOString();+ for (const side of ['ours', 'theirs'] as const) {+ const params: MergeHistoryParams = {+ kind: 'external-change-both-kept',+ docName,+ contents: versions[side],+ label: `${SIDE_LABEL[side]} before both versions were kept @ ${at}`,+ branch,+ metadata: { side, keptBlocks },+ };+ queueMicrotask(() => {+ void write(shadow, params);+ });+ }+ },++ keptLive(docName, displaced, reason) {+ const shadow = options.getShadow();+ if (!shadow) {+ unavailable(docName, 'external-change-kept-live');+ return Promise.resolve(false);+ }+ return write(shadow, {+ kind: 'external-change-kept-live',+ docName,+ contents: displaced.bytes,+ label: `${SIDE_LABEL[displaced.side]} moved to history (${reason}) @ ${new Date().toISOString()}`,+ branch: options.getBranch() ?? 'main',+ metadata: { side: displaced.side, reason },+ });+ },+ };+}diff --git a/packages/server/src/metrics.ts b/packages/server/src/metrics.tsindex 04a92aa88..ec9f8ac20 100644--- a/packages/server/src/metrics.ts+++ b/packages/server/src/metrics.ts@@ -24,6 +24,13 @@ export interface ReconciliationMetrics {conflictCount: number;reconcileInsertDedupSkipped: number;reconcileCheckFailed: number;+ reconcileConstructCheckSkipped: number;+ reconcileBothKept: number;+ reconcileBothKeptBlocks: number;+ reconcileKeptLive: number;+ mergeHistoryCheckpointCreated: number;+ mergeHistoryCheckpointFailures: number;+ mergeHistoryUnavailable: number;staleExternalWriteRefused: number;concurrentOverwriteRefused: number;batchCount: number;@@ -145,6 +152,13 @@ const counters: ReconciliationMetrics = {conflictCount: 0,reconcileInsertDedupSkipped: 0,reconcileCheckFailed: 0,+ reconcileConstructCheckSkipped: 0,+ reconcileBothKept: 0,+ reconcileBothKeptBlocks: 0,+ reconcileKeptLive: 0,+ mergeHistoryCheckpointCreated: 0,+ mergeHistoryCheckpointFailures: 0,+ mergeHistoryUnavailable: 0,staleExternalWriteRefused: 0,concurrentOverwriteRefused: 0,batchCount: 0,@@ -265,6 +279,31 @@ export function incrementReconcileCheckFailed(): void {counters.reconcileCheckFailed++;}+export function incrementReconcileConstructCheckSkipped(): void {+ counters.reconcileConstructCheckSkipped++;+}++export function incrementReconcileBothKept(keptBlocks: number): void {+ counters.reconcileBothKept++;+ counters.reconcileBothKeptBlocks += keptBlocks;+}++export function incrementReconcileKeptLive(): void {+ counters.reconcileKeptLive++;+}++export function incrementMergeHistoryCheckpointCreated(): void {+ counters.mergeHistoryCheckpointCreated++;+}++export function incrementMergeHistoryCheckpointFailures(): void {+ counters.mergeHistoryCheckpointFailures++;+}++export function incrementMergeHistoryUnavailable(): void {+ counters.mergeHistoryUnavailable++;+}+export function incrementStaleExternalWriteRefused(): void {counters.staleExternalWriteRefused++;}@@ -698,6 +737,13 @@ export function resetMetrics(): void {counters.conflictCount = 0;counters.reconcileInsertDedupSkipped = 0;counters.reconcileCheckFailed = 0;+ counters.reconcileConstructCheckSkipped = 0;+ counters.reconcileBothKept = 0;+ counters.reconcileBothKeptBlocks = 0;+ counters.reconcileKeptLive = 0;+ counters.mergeHistoryCheckpointCreated = 0;+ counters.mergeHistoryCheckpointFailures = 0;+ counters.mergeHistoryUnavailable = 0;counters.staleExternalWriteRefused = 0;counters.concurrentOverwriteRefused = 0;counters.batchCount = 0;diff --git a/packages/server/src/paired-intake-detection-wiring.test.ts b/packages/server/src/paired-intake-detection-wiring.test.tsindex c947fd2b7..90a9562bb 100644--- a/packages/server/src/paired-intake-detection-wiring.test.ts+++ b/packages/server/src/paired-intake-detection-wiring.test.ts@@ -20,6 +20,7 @@ import { RECONCILE_TEST_CONFLICTS } from './conflict-authority.test-helper.ts';import { DocumentDurabilityState } from './document-durability-state.ts';import { reconcileDiskBeforeAgentWrite } from './external-change.ts';import { mdManager } from './md-manager.ts';+import { NO_SHADOW_MERGE_HISTORY } from './merge-history.test-helper.ts';import { createWiredPreDrainRig, WIRED_PENDING_LINE } from './pre-drain-wired.test-helper.ts';const schema = getSchema(sharedExtensions);@@ -176,7 +177,7 @@ describe('paired-intake detection follows the registry at every wired site', ()try {const on = lossCollector();await seed('watcher-detect');- reconcileDiskBeforeAgentWrite(+ await reconcileDiskBeforeAgentWrite(durabilityState,hp,'watcher-detect',@@ -184,13 +185,15 @@ describe('paired-intake detection follows the registry at every wired site', ()undefined,on.reporter,RECONCILE_TEST_CONFLICTS,+ NO_SHADOW_MERGE_HISTORY,);expect(on.trips.length).toBeGreaterThan(0);const off = lossCollector();await seed('watcher-suppress');+ let suppressed: Promise<unknown> | undefined;withMode('file-watcher', 'suppress', () => {- reconcileDiskBeforeAgentWrite(+ suppressed = reconcileDiskBeforeAgentWrite(durabilityState,hp,'watcher-suppress',@@ -198,8 +201,10 @@ describe('paired-intake detection follows the registry at every wired site', ()undefined,off.reporter,RECONCILE_TEST_CONFLICTS,+ NO_SHADOW_MERGE_HISTORY,);});+ await suppressed;expect(off.trips).toEqual([]);} finally {rmSync(contentDir, { recursive: true, force: true });diff --git a/packages/server/src/reconcile-intake-loss.test.ts b/packages/server/src/reconcile-intake-loss.test.tsindex 7a2f7fb4c..42e2734c7 100644--- a/packages/server/src/reconcile-intake-loss.test.ts+++ b/packages/server/src/reconcile-intake-loss.test.ts@@ -17,6 +17,7 @@ import { RECONCILE_TEST_CONFLICTS } from './conflict-authority.test-helper.ts';import { DocumentDurabilityState } from './document-durability-state.ts';import { reconcileDiskBeforeAgentWrite } from './external-change.ts';import { mdManager } from './md-manager.ts';+import { NO_SHADOW_MERGE_HISTORY } from './merge-history.test-helper.ts';const schema = getSchema(sharedExtensions);@@ -79,7 +80,7 @@ describe('reconcileDiskBeforeAgentWrite paired-intake instrumentation', () => {trips.push({ docName: name, obs, writerId, site });};- const result = reconcileDiskBeforeAgentWrite(+ const result = await reconcileDiskBeforeAgentWrite(durabilityState,hp,docName,@@ -87,6 +88,7 @@ describe('reconcileDiskBeforeAgentWrite paired-intake instrumentation', () => {undefined,reporter,RECONCILE_TEST_CONFLICTS,+ NO_SHADOW_MERGE_HISTORY,);expect(result.reconciled).toBe(true);@@ -124,7 +126,7 @@ describe('reconcileDiskBeforeAgentWrite paired-intake instrumentation', () => {writeFileSync(join(contentDir, `${docName}.md`), DISK_EDIT);const trips: Trip[] = [];- const result = reconcileDiskBeforeAgentWrite(+ const result = await reconcileDiskBeforeAgentWrite(durabilityState,hp,docName,@@ -132,6 +134,7 @@ describe('reconcileDiskBeforeAgentWrite paired-intake instrumentation', () => {undefined,(name, obs, writerId, site) => trips.push({ docName: name, obs, writerId, site }),RECONCILE_TEST_CONFLICTS,+ NO_SHADOW_MERGE_HISTORY,);expect(result.reconciled).toBe(true);diff --git a/packages/server/src/reconcile-own-flush-window.test.ts b/packages/server/src/reconcile-own-flush-window.test.tsindex c50bebef3..02606760d 100644--- a/packages/server/src/reconcile-own-flush-window.test.ts+++ b/packages/server/src/reconcile-own-flush-window.test.ts@@ -12,6 +12,7 @@ import {type ReconcileBeforeWriteResult,reconcileDiskBeforeAgentWrite,} from './external-change.ts';+import { NO_SHADOW_MERGE_HISTORY } from './merge-history.test-helper.ts';import { createPersistenceExtension } from './persistence.ts';const BROWSER_ORIGIN = {@@ -99,6 +100,7 @@ async function drivePhantomDivergence(};let windowFired = false;+ let windowGuard: Promise<ReconcileBeforeWriteResult> | undefined;const persistence = createPersistenceExtension({contentDir: tmpDir,projectDir: tmpDir,@@ -113,7 +115,7 @@ async function drivePhantomDivergence(if (options.diskContentInWindow !== undefined) {writeFileSync(docPath, options.diskContentInWindow, 'utf-8');}- probe.windowResult = reconcileDiskBeforeAgentWrite(+ windowGuard = reconcileDiskBeforeAgentWrite(durabilityState,fakeHocuspocusWith(docName, document),docName,@@ -121,6 +123,7 @@ async function drivePhantomDivergence(undefined,undefined,RECONCILE_TEST_CONFLICTS,+ NO_SHADOW_MERGE_HISTORY,);probe.conflictAfterGuard = isDocInConflict(document as never);},@@ -129,6 +132,7 @@ async function drivePhantomDivergence(await loadDocument(persistence, document, docName);document.transact(() => replaceDocParagraphs(document, FLUSHED_PARAGRAPHS), BROWSER_ORIGIN);await storeDocument(persistence, document, docName);+ probe.windowResult = await windowGuard;expect(windowFired).toBe(true);expect(probe.baseSeenInWindow).toBe(BASE_CONTENT);@@ -180,7 +184,7 @@ describe('reconcileDiskBeforeAgentWrite — own persistence flush is not foreignexpect(durabilityState.getReconciledBase(docName)).toBe(FLUSHED_CONTENT);- const laterGuard = reconcileDiskBeforeAgentWrite(+ const laterGuard = await reconcileDiskBeforeAgentWrite(durabilityState,fakeHocuspocusWith(docName, document),docName,@@ -188,6 +192,7 @@ describe('reconcileDiskBeforeAgentWrite — own persistence flush is not foreignundefined,undefined,RECONCILE_TEST_CONFLICTS,+ NO_SHADOW_MERGE_HISTORY,);expect(laterGuard.reconciled).toBe(false);@@ -302,7 +307,7 @@ describe('reconcileDiskBeforeAgentWrite — overlapping own flushes are not forelet pendingAtWindow = 0;let headAtWindow: string | undefined;let diskMatchedPendingAtWindow = false;- let result: ReconcileBeforeWriteResult | undefined;+ let pendingResult: Promise<ReconcileBeforeWriteResult> | undefined;let conflictAfterGuard: boolean | undefined;const persistence = createPersistenceExtension({@@ -325,11 +330,15 @@ describe('reconcileDiskBeforeAgentWrite — overlapping own flushes are not foredocName,normalizeBridge(diskAtWindow),);- result = reconcileDiskBeforeAgentWrite(+ pendingResult = reconcileDiskBeforeAgentWrite(durabilityState,fakeHocuspocusWith(docName, document),docName,tmpDir,+ undefined,+ undefined,+ RECONCILE_TEST_CONFLICTS,+ NO_SHADOW_MERGE_HISTORY,);conflictAfterGuard = isDocInConflict(document as never);},@@ -339,6 +348,7 @@ describe('reconcileDiskBeforeAgentWrite — overlapping own flushes are not foredocument.transact(() => replaceDocParagraphs(document, FLUSHED_PARAGRAPHS), BROWSER_ORIGIN);await storeDocument(persistence, document, docName);await laterFlush;+ const result = await pendingResult;expect(windowFired).toBe(true);expect(pendingAtWindow).toBeGreaterThanOrEqual(2);diff --git a/packages/server/src/reconciliation.fidelity.test.ts b/packages/server/src/reconciliation.fidelity.test.tsindex 405b0bb57..5a058bf69 100644--- a/packages/server/src/reconciliation.fidelity.test.ts+++ b/packages/server/src/reconciliation.fidelity.test.ts@@ -1,6 +1,8 @@import { MarkdownManager, sharedExtensions } from '@inkeep/open-knowledge-core';import { describe, expect, test } from 'vitest';+import { getMetrics } from './metrics';import {+ blocksKeptTwice,checkMergeOutput,MAX_CONSTRUCT_CHECK_BYTES,MAX_LCS_CELLS,@@ -8,7 +10,7 @@ import {} from './reconciliation';const docName = 'fidelity-doc';-const cleanMerge = { conflicts: [], dedupSkipped: false } as const;+const cleanMerge = { dedupSkipped: false } as const;const parser = new MarkdownManager({ extensions: sharedExtensions });const rootTypes = (md: string): string[] => parser.parseToMdast(md).children.map((c) => c.type);@@ -293,7 +295,7 @@ describe('classification under the raw-byte definition of changed', () => {});});- test('a whitespace-only outside change inside a paragraph ours edited is a conflict', () => {+ test('a whitespace-only outside change inside a paragraph ours edited keeps both versions', () => {const outcome = reconcile({docName,base: 'A\n\nPara text here.\n\nC\n',@@ -301,14 +303,15 @@ describe('classification under the raw-byte definition of changed', () => {theirs: 'A\n\nPara text here.\n\nC theirs\n',});expect(outcome).toEqual({- kind: 'conflicts',- newContent: 'A\n\nPara text EDITED.\n\nC theirs\n',- conflicts: [+ kind: 'both-kept',+ newContent: 'A\n\nPara text EDITED.\n\nPara text here.\n\nC theirs\n',+ kept: [{blockIndex: 1,base: 'Para text here.',ours: 'Para text EDITED.',theirs: 'Para text here.',+ landed: 'both',},],});@@ -342,6 +345,133 @@ describe('classification under the raw-byte definition of changed', () => {});});+describe('a block both sides changed keeps both versions, ours first', () => {+ test('a mid-document paragraph lands ours then theirs, each byte-exact, with no markers', () => {+ expect(+ reconcile({+ docName,+ base: '# Title\n\nShared.\n\nTail.\n',+ ours: '# Title\n\nShared, ours.\n\nTail.\n',+ theirs: '# Title\n\nShared, theirs.\n\nTail.\n',+ }),+ ).toEqual({+ kind: 'both-kept',+ newContent: '# Title\n\nShared, ours.\n\nShared, theirs.\n\nTail.\n',+ kept: [+ {+ blockIndex: 1,+ base: 'Shared.',+ ours: 'Shared, ours.',+ theirs: 'Shared, theirs.',+ landed: 'both',+ },+ ],+ });+ });++ test("the gap between the two versions is theirs' own separator before its version", () => {+ const outcome = reconcile({+ docName,+ base: 'A\n\nB\n\nC\n',+ ours: 'A\n\nB ours\n\nC\n',+ theirs: 'A\n\n\nB theirs\n\nC\n',+ });+ expect(outcome).toMatchObject({+ kind: 'both-kept',+ newContent: 'A\n\n\nB ours\n\n\nB theirs\n\nC\n',+ });+ });++ test('a first-block collision separates the two versions with the document blank line', () => {+ expect(+ reconcile({ docName, base: 'A\n\nB\n', ours: 'A ours\n\nB\n', theirs: 'A theirs\n\nB\n' }),+ ).toMatchObject({ kind: 'both-kept', newContent: 'A ours\n\nA theirs\n\nB\n' });+ expect(+ reconcile({ docName, base: 'A\n', ours: 'A ours\n', theirs: 'A theirs\n' }),+ ).toMatchObject({ kind: 'both-kept', newContent: 'A ours\n\nA theirs\n' });+ });++ test('CRLF documents keep CRLF in both versions and between them', () => {+ expect(+ reconcile({+ docName,+ base: '# T\r\n\r\nP.\r\n\r\nQ.\r\n',+ ours: '# T\r\n\r\nP ours.\r\n\r\nQ.\r\n',+ theirs: '# T\r\n\r\nP theirs.\r\n\r\nQ.\r\n',+ }),+ ).toMatchObject({+ kind: 'both-kept',+ newContent: '# T\r\n\r\nP ours.\r\n\r\nP theirs.\r\n\r\nQ.\r\n',+ });+ });++ test('a block ours deleted and theirs changed restores the outside version in place, landed as theirs only', () => {+ expect(+ reconcile({+ docName,+ base: 'A\n\nB\n\nC\n',+ ours: 'A\n\nC\n',+ theirs: 'A\n\nB theirs\n\nC\n',+ }),+ ).toEqual({+ kind: 'both-kept',+ newContent: 'A\n\nB theirs\n\nC\n',+ kept: [{ blockIndex: 1, base: 'B', ours: '', theirs: 'B theirs', landed: 'theirs' }],+ });+ });++ test('a block ours changed and theirs deleted keeps the version here, landed as ours only', () => {+ expect(+ reconcile({+ docName,+ base: 'A\n\nB\n\nC\n',+ ours: 'A\n\nB ours\n\nC\n',+ theirs: 'A\n\nC\n',+ }),+ ).toEqual({+ kind: 'both-kept',+ newContent: 'A\n\nB ours\n\nC\n',+ kept: [{ blockIndex: 1, base: 'B', ours: 'B ours', theirs: '', landed: 'ours' }],+ });+ });++ test('only blocks that landed as both versions count as kept twice', () => {+ const outcome = reconcile({+ docName,+ base: 'A\n\nB\n\nC\n\nD\n',+ ours: 'A ours\n\nC\n\nD ours\n',+ theirs: 'A theirs\n\nB theirs\n\nC\n\nD\n',+ });+ expect(outcome).toMatchObject({+ kind: 'both-kept',+ newContent: 'A ours\n\nA theirs\n\nB theirs\n\nC\n\nD ours\n',+ });+ if (outcome.kind !== 'both-kept') return;+ expect(outcome.kept.map((pair) => pair.landed)).toEqual(['both', 'theirs']);+ expect(blocksKeptTwice(outcome.kept)).toBe(1);+ });++ test("the check accepts both versions and refuses an output that drops or alters theirs' version", () => {+ const sides = {+ ...cleanMerge,+ base: 'A\n\nB\n\nC\n',+ ours: 'A\n\nB ours\n\nC\n',+ theirs: 'A\n\nB theirs\n\nC\n',+ };+ expect(checkMergeOutput({ ...sides, newContent: 'A\n\nB ours\n\nB theirs\n\nC\n' })).toBeNull();+ expect(checkMergeOutput({ ...sides, newContent: 'A\n\nB ours\n\nC\n' })).toEqual({+ check: 'side-bytes',+ side: 'theirs',+ sideBlockIndex: 1,+ });+ expect(checkMergeOutput({ ...sides, newContent: 'A\n\nB ours\n\nB theirs\n\nC\n' })).toEqual({+ check: 'side-bytes',+ side: 'theirs',+ sideBlockIndex: 1,+ });+ });+});+describe('checkMergeOutput', () => {const base = 'Intro.\n\n code\n\nAfter.\n';const ours = 'Intro.\n\n code\n\nAfter.\n\n- new item\n';@@ -465,6 +595,34 @@ describe('documents the strict parse rejects', () => {});});+describe('a skipped construct check is counted once per merge', () => {+ test('a source the strict parse rejects counts one skip; an ordinary merge counts none', () => {+ const before = getMetrics().reconcileConstructCheckSkipped;+ reconcile({ docName, base: 'A\n\nB\n', ours: 'A ours\n\nB\n', theirs: 'A\n\nB theirs\n' });+ expect(getMetrics().reconcileConstructCheckSkipped).toBe(before);+ reconcile({+ docName,+ base: '# T\n\nA.\n\n<Foo>broken</Bar>\n\nB.\n',+ ours: '# T\n\nA ours.\n\n<Foo>broken</Bar>\n\nB.\n',+ theirs: '# T\n\nA.\n\n<Foo>broken</Bar>\n\nB theirs.\n',+ });+ expect(getMetrics().reconcileConstructCheckSkipped).toBe(before + 1);+ });++ test('an oversized merge counts one skip', () => {+ const padding = 'Filler words. '.repeat(Math.ceil((MAX_CONSTRUCT_CHECK_BYTES + 1024) / 14));+ const before = getMetrics().reconcileConstructCheckSkipped;+ const outcome = reconcile({+ docName,+ base: `${padding}\n\nA\n\nB\n`,+ ours: `${padding}\n\nA ours\n\nB\n`,+ theirs: `${padding}\n\nA\n\nB theirs\n`,+ });+ expect(outcome.kind).toBe('merged');+ expect(getMetrics().reconcileConstructCheckSkipped).toBe(before + 1);+ });+});+describe('the construct check size bound', () => {const filler = (bytes: number): string => 'Filler words. '.repeat(Math.ceil(bytes / 14));const shape = (padding: string) => ({diff --git a/packages/server/src/reconciliation.test.ts b/packages/server/src/reconciliation.test.tsindex 49a4d667e..9fefb236b 100644--- a/packages/server/src/reconciliation.test.ts+++ b/packages/server/src/reconciliation.test.ts@@ -175,37 +175,44 @@ describe('reconcile', () => {}});- test('conflicts: both sides change the same block', () => {+ test('both-kept: both sides change the same block, ours first then theirs', () => {const base = '# Title\n\nShared paragraph.\n\nAnother paragraph.\n';const ours = '# Title\n\nOur version of shared.\n\nAnother paragraph.\n';const theirs = '# Title\n\nTheir version of shared.\n\nAnother paragraph.\n';const result = reconcile({ docName, base, ours, theirs });- expect(result.kind).toBe('conflicts');- if (result.kind === 'conflicts') {- expect(result.conflicts).toHaveLength(1);- expect(result.conflicts[0].base).toBe('Shared paragraph.');- expect(result.conflicts[0].ours).toBe('Our version of shared.');- expect(result.conflicts[0].theirs).toBe('Their version of shared.');- const blocks = splitMarkdownBlocks(result.newContent);- expect(blocks).toContain('Our version of shared.');+ expect(result.kind).toBe('both-kept');+ if (result.kind === 'both-kept') {+ expect(result.kept).toHaveLength(1);+ expect(result.kept[0].base).toBe('Shared paragraph.');+ expect(result.kept[0].ours).toBe('Our version of shared.');+ expect(result.kept[0].theirs).toBe('Their version of shared.');+ expect(splitMarkdownBlocks(result.newContent)).toEqual([+ '# Title',+ 'Our version of shared.',+ 'Their version of shared.',+ 'Another paragraph.',+ ]);}});- test('conflicts: mixed — some blocks conflict, others merge cleanly', () => {+ test('both-kept: mixed — one block kept twice, the others merge cleanly', () => {const base = '# Title\n\nBlock A.\n\nBlock B.\n\nBlock C.\n';const ours = '# Title\n\nBlock A edited by us.\n\nBlock B.\n\nBlock C edited by us.\n';const theirs = '# Title\n\nBlock A.\n\nBlock B edited by them.\n\nBlock C edited by them.\n';const result = reconcile({ docName, base, ours, theirs });- expect(result.kind).toBe('conflicts');- if (result.kind === 'conflicts') {- expect(result.conflicts).toHaveLength(1);- expect(result.conflicts[0].base).toBe('Block C.');-- const blocks = splitMarkdownBlocks(result.newContent);- expect(blocks).toContain('Block A edited by us.');- expect(blocks).toContain('Block B edited by them.');+ expect(result.kind).toBe('both-kept');+ if (result.kind === 'both-kept') {+ expect(result.kept).toHaveLength(1);+ expect(result.kept[0].base).toBe('Block C.');+ expect(splitMarkdownBlocks(result.newContent)).toEqual([+ '# Title',+ 'Block A edited by us.',+ 'Block B edited by them.',+ 'Block C edited by us.',+ 'Block C edited by them.',+ ]);}});@@ -417,7 +424,7 @@ describe('reconcile', () => {}});- test('conflicts: an outcome that also skipped dedup past the LCS cap reports dedupSkipped', () => {+ test('both-kept: an outcome that also skipped dedup past the LCS cap reports dedupSkipped', () => {const perSide = Math.ceil(Math.sqrt(MAX_LCS_CELLS)) + 10;const shared = Array.from({ length: 10 }, (_, i) => `shared ${i}.`);const base = 'Anchor.\n';@@ -435,11 +442,15 @@ describe('reconcile', () => {const theirs = `${theirBlocks.join('\n\n')}\n`;const result = reconcile({ docName, base, ours, theirs });- expect(result.kind).toBe('conflicts');- if (result.kind === 'conflicts') {- expect(result.conflicts).toHaveLength(1);- expect(result.conflicts[0].ours).toBe('Anchor edited by us.');- expect(result.conflicts[0].theirs).toBe('Anchor edited by them.');+ expect(result.kind).toBe('both-kept');+ if (result.kind === 'both-kept') {+ expect(result.kept).toHaveLength(1);+ expect(result.kept[0].ours).toBe('Anchor edited by us.');+ expect(result.kept[0].theirs).toBe('Anchor edited by them.');+ expect(splitMarkdownBlocks(result.newContent).slice(0, 2)).toEqual([+ 'Anchor edited by us.',+ 'Anchor edited by them.',+ ]);expect(result.dedupSkipped).toBe(true);}});diff --git a/packages/server/src/reconciliation.ts b/packages/server/src/reconciliation.tsindex 201d1f846..1cdba876b 100644--- a/packages/server/src/reconciliation.ts+++ b/packages/server/src/reconciliation.ts@@ -8,6 +8,7 @@ import {} from './cc1-broadcast.ts';import { getLogger } from './logger.ts';import { mdManager } from './md-manager.ts';+import { incrementReconcileConstructCheckSkipped } from './metrics.ts';export interface ReconcileInput {docName: string;@@ -16,11 +17,34 @@ export interface ReconcileInput {theirs: string;}-export interface BlockConflict {+export interface KeptBlockPair {blockIndex: number;base: string;ours: string;theirs: string;+ landed: 'both' | 'ours' | 'theirs';+}++export interface KeptBlockCounts {+ readonly duplicated: number;+ readonly restoredDeletedHere: number;+ readonly restoredDeletedOutside: number;+}++export function countKeptBlocks(kept: readonly KeptBlockPair[]): KeptBlockCounts {+ let duplicated = 0;+ let restoredDeletedHere = 0;+ let restoredDeletedOutside = 0;+ for (const pair of kept) {+ if (pair.landed === 'both') duplicated++;+ else if (pair.landed === 'theirs') restoredDeletedHere++;+ else restoredDeletedOutside++;+ }+ return { duplicated, restoredDeletedHere, restoredDeletedOutside };+}++export function blocksKeptTwice(kept: readonly KeptBlockPair[]): number {+ return countKeptBlocks(kept).duplicated;}export type ReconcileRefusalReason = 'conflict-markers' | 'no-base' | 'too-large' | 'check-failed';@@ -37,7 +61,7 @@ type MergeCheckFailure =export type ReconcileOutcome =| { kind: 'clean'; newContent: string }| { kind: 'merged'; newContent: string; mergedBlocks: number; dedupSkipped?: true }- | { kind: 'conflicts'; newContent: string; conflicts: BlockConflict[]; dedupSkipped?: true }+ | { kind: 'both-kept'; newContent: string; kept: KeptBlockPair[]; dedupSkipped?: true }| { kind: 'refused'; reason: Exclude<ReconcileRefusalReason, 'check-failed'> }| { kind: 'refused'; reason: 'check-failed'; failure: MergeCheckFailure }| { kind: 'noop' };@@ -214,25 +238,22 @@ export function reconcile(input: ReconcileInput): ReconcileOutcome {const merge = mergeBlocks(baseDoc, ourDoc, theirDoc, mergeOutputLineEnding(base, ours, theirs));- const failure = checkMergeOutput({+ const verdict = verifyMergeOutput({base,ours,theirs,newContent: merge.newContent,- conflicts: merge.conflicts,dedupSkipped: merge.dedupSkipped,});- if (failure) return { kind: 'refused', reason: 'check-failed', failure };+ if (verdict.failure) return { kind: 'refused', reason: 'check-failed', failure: verdict.failure };+ if (verdict.constructCheck !== 'ran') {+ reportConstructCheckSkipped(input.docName, verdict.constructCheck);+ }- if (merge.conflicts.length > 0) {+ if (merge.kept.length > 0) {return merge.dedupSkipped- ? {- kind: 'conflicts',- newContent: merge.newContent,- conflicts: merge.conflicts,- dedupSkipped: true,- }- : { kind: 'conflicts', newContent: merge.newContent, conflicts: merge.conflicts };+ ? { kind: 'both-kept', newContent: merge.newContent, kept: merge.kept, dedupSkipped: true }+ : { kind: 'both-kept', newContent: merge.newContent, kept: merge.kept };}return merge.dedupSkipped@@ -253,7 +274,7 @@ interface EmittedBlock {interface BlockMerge {readonly newContent: string;readonly mergedBlocks: number;- readonly conflicts: BlockConflict[];+ readonly kept: KeptBlockPair[];readonly dedupSkipped: boolean;}@@ -297,7 +318,7 @@ function mergeBlocks(const theirEdits = computeEditOps(base.keys, theirs.keys);const emitted: EmittedBlock[] = [];- const conflicts: BlockConflict[] = [];+ const kept: KeptBlockPair[] = [];let dedupSkipped = false;for (let i = 0; i < base.blocks.length; i++) {@@ -329,15 +350,26 @@ function mergeBlocks(} else {const ourKey = ourText === null ? null : blockIdentity(ourText);const theirKey = theirText === null ? null : blockIdentity(theirText);+ if (ourText !== null) anchored(ourText);if (ourKey !== theirKey) {- conflicts.push({+ let landed: KeptBlockPair['landed'] = 'ours';+ if (theirText !== null) {+ if (ourText === null) {+ landed = 'theirs';+ anchored(theirText);+ } else {+ landed = 'both';+ emitted.push({ text: theirText, separator: sideSeparator(theirs, theirOp) ?? null });+ }+ }+ kept.push({blockIndex: i,base: base.blocks[i].text,ours: ourText ?? '',theirs: theirText ?? '',+ landed,});}- if (ourText !== null) anchored(ourText);}}@@ -375,7 +407,7 @@ function mergeBlocks(return {newContent: outputLineEnding ? spellLineEndings(assembled, outputLineEnding) : assembled,mergedBlocks: emitted.length,- conflicts,+ kept,dedupSkipped,};}@@ -518,10 +550,16 @@ interface MergeCheckInput {readonly ours: string;readonly theirs: string;readonly newContent: string;- readonly conflicts: readonly BlockConflict[];readonly dedupSkipped: boolean;}+type ConstructCheckRun = 'ran' | 'skipped-oversized' | 'skipped-unparsed-source';++interface MergeVerdict {+ readonly failure: MergeCheckFailure | null;+ readonly constructCheck: ConstructCheckRun;+}+type CheckedDocument = 'base' | MergeSide | 'output';interface BlockSource {@@ -542,7 +580,6 @@ interface Occurrence {readonly sources: readonly BlockSource[];readonly side: MergeSide;readonly sideBlockIndex: number;- required: boolean;consumed: boolean;}@@ -622,7 +659,6 @@ function collectOccurrences(],side: 'ours',sideBlockIndex: oi,- required: true,consumed: false,});}@@ -634,27 +670,21 @@ function collectOccurrences(sources: [{ document: 'theirs', index: ti }],side: 'theirs',sideBlockIndex: ti,- required: true,consumed: false,});}return occurrences;}-function releaseConflictingTheirs(- occurrences: ReadonlyMap<string, readonly Occurrence[]>,- conflicts: readonly BlockConflict[],-): void {- for (const conflict of conflicts) {- if (conflict.theirs === '') continue;- const occurrence = occurrences- .get(blockIdentity(conflict.theirs))- ?.find((candidate) => candidate.required && candidate.side === 'theirs');- if (occurrence) occurrence.required = false;- }+export function checkMergeOutput(input: MergeCheckInput): MergeCheckFailure | null {+ return verifyMergeOutput(input).failure;}-export function checkMergeOutput(input: MergeCheckInput): MergeCheckFailure | null {+function failed(failure: MergeCheckFailure): MergeVerdict {+ return { failure, constructCheck: 'ran' };+}++function verifyMergeOutput(input: MergeCheckInput): MergeVerdict {const outputLineEnding = mergeOutputLineEnding(input.base, input.ours, input.theirs);const expectBytes = (raw: string): string =>outputLineEnding ? spellLineEndings(raw, outputLineEnding) : raw;@@ -681,7 +711,7 @@ export function checkMergeOutput(input: MergeCheckInput): MergeCheckFailure | nucontinue;}if (output.blocks[k].text !== expectBytes(base.blocks[bi].text)) {- return { check: 'untouched-verbatim', baseBlockIndex: bi };+ return failed({ check: 'untouched-verbatim', baseBlockIndex: bi });}sources[k] = [{ document: 'base', index: bi },@@ -691,7 +721,7 @@ export function checkMergeOutput(input: MergeCheckInput): MergeCheckFailure | nunextUntouched++;}if (nextUntouched < untouched.length) {- return { check: 'untouched-verbatim', baseBlockIndex: untouched[nextUntouched] };+ return failed({ check: 'untouched-verbatim', baseBlockIndex: untouched[nextUntouched] });}const occurrences = collectOccurrences(@@ -702,7 +732,6 @@ export function checkMergeOutput(input: MergeCheckInput): MergeCheckFailure | nu!input.dedupSkipped,expectBytes,);- releaseConflictingTheirs(occurrences, input.conflicts);const unclaimed: number[] = [];for (const k of rest) {@@ -714,28 +743,30 @@ export function checkMergeOutput(input: MergeCheckInput): MergeCheckFailure | nucontinue;}const bytes = output.blocks[k].text;- const exact =- candidates.find((c) => c.required && c.expected.includes(bytes)) ??- candidates.find((c) => c.expected.includes(bytes));+ const exact = candidates.find((c) => c.expected.includes(bytes));if (!exact) {- return {+ return failed({check: 'side-bytes',side: candidates[0].side,sideBlockIndex: candidates[0].sideBlockIndex,- };+ });}exact.consumed = true;sources[k] = exact.sources;}for (const bucket of occurrences.values()) {- const missing = bucket.find((occurrence) => occurrence.required && !occurrence.consumed);+ const missing = bucket.find((occurrence) => !occurrence.consumed);if (missing) {- return { check: 'side-bytes', side: missing.side, sideBlockIndex: missing.sideBlockIndex };+ return failed({+ check: 'side-bytes',+ side: missing.side,+ sideBlockIndex: missing.sideBlockIndex,+ });}}if (unclaimed.length > 0) {- return { check: 'single-occurrence', outputBlockIndex: unclaimed[0] };+ return failed({ check: 'single-occurrence', outputBlockIndex: unclaimed[0] });}return checkConstructs(@@ -816,24 +847,32 @@ function coveringTypes(return types.join(' ');}+function reportConstructCheckSkipped(+ docName: string,+ reason: Exclude<ConstructCheckRun, 'ran'>,+): void {+ incrementReconcileConstructCheckSkipped();+ getLogger('reconcile').info(+ { docName, reason, limitBytes: MAX_CONSTRUCT_CHECK_BYTES },+ `[reconcile] ${docName} construct check skipped (${reason}); the byte checks still ran`,+ );+}+function checkConstructs(texts: Readonly<Record<CheckedDocument, string>>,sources: readonly (readonly BlockSource[])[],-): MergeCheckFailure | null {+): MergeVerdict {const largestBytes = Math.max(...Object.values(texts).map((text) => Buffer.byteLength(text)));if (largestBytes > MAX_CONSTRUCT_CHECK_BYTES) {- getLogger('reconcile').debug(- { largestBytes, limitBytes: MAX_CONSTRUCT_CHECK_BYTES },- '[reconcile] construct check skipped: a document is over the size bound',- );- return null;+ return { failure: null, constructCheck: 'skipped-oversized' };}const base = parseDocument(texts.base);const ours = parseDocument(texts.ours);const theirs = parseDocument(texts.theirs);- if (!base || !ours || !theirs) return null;+ if (!base || !ours || !theirs)+ return { failure: null, constructCheck: 'skipped-unparsed-source' };const output = parseDocument(texts.output);- if (!output) return { check: 'parse' };+ if (!output) return failed({ check: 'parse' });const parsed: Readonly<Record<CheckedDocument, ParsedDocument>> = { base, ours, theirs, output };const constructAt = (source: BlockSource): string | null => {@@ -847,7 +886,7 @@ function checkConstructs(const stable =merged !== null &&sources[k].some((source) => source.index !== -1 && constructAt(source) === merged);- if (!stable) return { check: 'construct', outputBlockIndex: k };+ if (!stable) return failed({ check: 'construct', outputBlockIndex: k });}- return null;+ return { failure: null, constructCheck: 'ran' };}diff --git a/packages/server/src/server-factory.test.ts b/packages/server/src/server-factory.test.tsindex 844bc7f3f..dd03d3e42 100644--- a/packages/server/src/server-factory.test.ts+++ b/packages/server/src/server-factory.test.ts@@ -7126,7 +7126,7 @@ describe('createServer() — disk-event reconcile insert-group dedup reporting',rig?.cleanup();});- test('a conflicting reconcile that also skipped insert dedup past the LCS cap still counts the skip', async () => {+ test('a both-kept reconcile that also skipped insert dedup past the LCS cap still counts the skip', async () => {rig = await setupReconcileRig('ok-reconcile-conflict-dedup-');const docName = 'conflict-dedup-target';const perSide = Math.ceil(Math.sqrt(MAX_LCS_CELLS)) + 10;@@ -7161,18 +7161,22 @@ describe('createServer() — disk-event reconcile insert-group dedup reporting',server.durabilityState.setReconciledBase(docName, base);const conflictsBefore = getMetrics().conflictCount;+ const bothKeptBefore = getMetrics().reconcileBothKept;const dedupSkippedBefore = getMetrics().reconcileInsertDedupSkipped;writeFileSync(docPath, theirs, 'utf-8');- await vi.waitFor(() => expect(getMetrics().conflictCount).toBe(conflictsBefore + 1), {+ await vi.waitFor(() => expect(getMetrics().reconcileBothKept).toBe(bothKeptBefore + 1), {timeout: 20_000,interval: 50,});const merged = serverDoc.getText('source').toString();+ expect(merged.startsWith('Anchor edited by us.\n\nAnchor edited by them.\n\n')).toBe(true);expect(merged).toContain('Ours 0.');expect(merged).toContain(`Theirs ${perSide - 1}.`);expect(getMetrics().reconcileInsertDedupSkipped).toBe(dedupSkippedBefore + 1);+ expect(getMetrics().conflictCount).toBe(conflictsBefore);+ expect(server.conflicts.findByDocName(docName)).toBeUndefined();conn.disconnect();} finally {diff --git a/packages/server/src/server-factory.ts b/packages/server/src/server-factory.tsindex 854420cc3..b2c9fa06a 100644--- a/packages/server/src/server-factory.ts+++ b/packages/server/src/server-factory.ts@@ -152,7 +152,9 @@ import {type FrozenResidentRevival,redactedErrorSummary,refuseStaleExternalWrite,+ reportReconcileBothKept,reportReconcileCheckFailed,+ reportReconcileKeptLive,reviveFrozenResidentDoc,serializeYDocSource,wireReasonForRefusal,@@ -231,6 +233,7 @@ import { startManagedArtifactWatcher } from './managed-artifact-watcher.ts';import { recoverPendingManagedRename } from './managed-rename-journal.ts';import type { NativeTomlMcpEditor } from './mcp-config-reconciler.ts';import { mdManager, schema } from './md-manager.ts';+import { createMergeHistoryRecorder, type MergeHistoryWriter } from './merge-history.ts';import {incrementBatch,incrementBranchSwitch,@@ -258,7 +261,7 @@ import {} from './persistence-staleness-watchdog.ts';import { loadPrincipal } from './principal.ts';import { RecentlyRemovedDocs } from './recently-removed-docs.ts';-import { type ReconcileOutcome, reconcile } from './reconciliation.ts';+import { blocksKeptTwice, type ReconcileOutcome, reconcile } from './reconciliation.ts';import { reconcileRecoveredFileTarget } from './recovered-file-target.ts';import { runRemovalRedirectGuard } from './removal-redirect-guard.ts';import { loadRemovedDocsJournal, saveRemovedDocsJournal } from './removed-docs-journal.ts';@@ -380,6 +383,7 @@ export interface ServerOptions {ephemeral?: boolean;watcherBackend?: NativeWatcherBackend;watcherClock?: WatcherClock;+ mergeHistoryWriter?: MergeHistoryWriter;generatedIndexTestHooks?: {beforePlan?: (context: { fullSweep: boolean; signal: AbortSignal }) => Promise<void> | void;beforeDecision?: (context: {@@ -2529,6 +2533,7 @@ export function createServer(options: ServerOptions): ServerInstance {hocuspocus.configuration.extensions.push(systemDocBroadcastGuard);const apiExtension = createApiExtension({+ mergeHistoryWriter: options.mergeHistoryWriter,declaredGitHubHosts,resolveSyncCredentialConfig,hocuspocus,@@ -2872,6 +2877,13 @@ export function createServer(options: ServerOptions): ServerInstance {}}+ const mergeHistory = createMergeHistoryRecorder({+ getShadow: () => shadowRef.current,+ contentRoot: contentRoot ?? '',+ getBranch: getStorageBranch,+ writeCheckpoint: options.mergeHistoryWriter,+ });+const CHECKPOINT_KIND_BY_MINT_SITE = {delete: 'external-change-rescue',rename: 'external-change-rescue',@@ -2961,7 +2973,7 @@ export function createServer(options: ServerOptions): ServerInstance {ours: string,theirs: string,): void => {- if (outcome.kind !== 'merged' && outcome.kind !== 'conflicts') return;+ if (outcome.kind !== 'merged' && outcome.kind !== 'both-kept') return;if (!outcome.dedupSkipped) return;incrementReconcileInsertDedupSkipped();log.warn(@@ -3128,7 +3140,7 @@ export function createServer(options: ServerOptions): ServerInstance {const oursLen = ours.length;const theirsLen = theirs.length;const newLen =- result.kind === 'clean' || result.kind === 'merged' || result.kind === 'conflicts'+ result.kind === 'clean' || result.kind === 'merged' || result.kind === 'both-kept'? result.newContent.length: undefined;const delta = newLen === undefined ? undefined : newLen - Math.max(oursLen, theirsLen);@@ -3201,30 +3213,30 @@ export function createServer(options: ServerOptions): ServerInstance {}break;- case 'conflicts': {- let applied = false;+ case 'both-kept': {try {applyToDoc(docName, result.newContent);- setReconciledBase(docName, result.newContent);- incrementReconcile();- incrementConflict();- applied = true;} catch (e) {log.error({ err: e, docName },- `[reconcile] failed to apply conflict content to Y.Doc for ${docName}`,+ `[reconcile] failed to apply both-kept content to Y.Doc for ${docName}; raising the conflict`,);setReconciledBase(docName, theirs);+ incrementConflict();+ conflicts.raise({+ kind: 'reconcile',+ file: conflicts.fileOf(docName),+ reason: 'merged-with-markers',+ stages: { base, ours, theirs },+ });+ break;}- conflicts.raise({- kind: 'reconcile',- file: conflicts.fileOf(docName),- reason: 'merged-with-markers',- stages: { base, ours, theirs },- });- if (applied) {- await derivedDocumentIndex.recordDiskUpsert(docName, theirs);- }+ mergeHistory.bothKept(docName, { ours, theirs }, blocksKeptTwice(result.kept));+ setReconciledBase(docName, theirs);+ incrementReconcile();+ reportReconcileBothKept(docName, result.kept, 'watcher');+ conflicts.dissolveReconcile(docName);+ await derivedDocumentIndex.recordDiskUpsert(docName, theirs);break;}@@ -3234,6 +3246,21 @@ export function createServer(options: ServerOptions): ServerInstance {break;}reportReconcileCheckFailed(docName, result, 'watcher');+ if (result.reason === 'check-failed' || result.reason === 'too-large') {+ if (+ await mergeHistory.keptLive(+ docName,+ { side: 'theirs', bytes: theirs },+ result.reason,+ )+ ) {+ setReconciledBase(docName, theirs);+ reportReconcileKeptLive(docName, result.reason, 'watcher');+ conflicts.dissolveReconcile(docName);+ await derivedDocumentIndex.recordDiskUpsert(docName, theirs);+ break;+ }+ }incrementConflict();conflicts.raise({kind: 'reconcile',@@ -4984,19 +5011,21 @@ export function createServer(options: ServerOptions): ServerInstance {docName,);if (!parked) continue;- if (parked.markdown === parked.diskSnapshot) continue;+ const parkedMarkdown = parked.markdown.trim();+ const parkedBase = parked.diskSnapshot.trim();+ if (parkedMarkdown === parkedBase) continue;const currentDisk = getReconciledBase(docName);if (!currentDisk) continue;const outcome = reconcile({docName,- base: parked.diskSnapshot,- ours: parked.markdown,+ base: parkedBase,+ ours: parkedMarkdown,theirs: currentDisk,});- reportInsertDedupSkipped(docName, outcome, parked.markdown, currentDisk);+ reportInsertDedupSkipped(docName, outcome, parkedMarkdown, currentDisk);reportReconcileCheckFailed(docName, outcome, 'branch-switch');switch (outcome.kind) {@@ -5006,21 +5035,16 @@ export function createServer(options: ServerOptions): ServerInstance {setReconciledBase(docName, outcome.newContent);restoredCount++;break;- case 'conflicts': {+ case 'both-kept': {applyToDoc(docName, outcome.newContent);setReconciledBase(docName, outcome.newContent);- incrementConflict();+ mergeHistory.bothKept(+ docName,+ { ours: parked.markdown, theirs: currentDisk },+ blocksKeptTwice(outcome.kept),+ );+ reportReconcileBothKept(docName, outcome.kept, 'branch-switch');restoredCount++;- conflicts.raise({- kind: 'reconcile',- file: conflicts.fileOf(docName),- reason: 'merged-with-markers',- stages: {- base: parked.diskSnapshot,- ours: parked.markdown,- theirs: currentDisk,- },- });break;}case 'noop':@@ -5028,10 +5052,26 @@ export function createServer(options: ServerOptions): ServerInstance {case 'refused': {const refusal = outcome.reason;switch (refusal) {+ case 'too-large':+ case 'check-failed': {+ if (+ await mergeHistory.keptLive(+ docName,+ { side: 'ours', bytes: parked.markdown },+ refusal,+ )+ ) {+ reportReconcileKeptLive(docName, refusal, 'branch-switch');+ break;+ }+ log.warn(+ { docName, reason: refusal, branch: newBranch },+ `[branch-switch] parked WIP not restored for ${docName} (reconcile refused: ${refusal}); it stays parked`,+ );+ break;+ }case 'conflict-markers':case 'no-base':- case 'too-large':- case 'check-failed':log.warn({ docName, reason: refusal, branch: newBranch },`[branch-switch] parked WIP not restored for ${docName} (reconcile refused: ${refusal})`,diff --git a/packages/server/src/shadow-repo.test.ts b/packages/server/src/shadow-repo.test.tsindex fa478d41e..66a775c97 100644--- a/packages/server/src/shadow-repo.test.ts+++ b/packages/server/src/shadow-repo.test.ts@@ -8,6 +8,7 @@ import { atomicTempPath } from '@inkeep/open-knowledge-core/server';import {CHECKPOINT_KIND_REGISTRY,CHECKPOINT_KINDS,+ CHECKPOINT_SAMPLE_BY_KIND,type CheckpointKind,formatCheckpointBodyLine,parseCheckpoint,@@ -719,8 +720,18 @@ describe('parkBranch', () => {const state = await readParkedState(shadow, 'feature', SERVICE_WRITER.id, 'guide');expect(state).not.toBeNull();- expect(state?.markdown).toBe('# Guide v2');- expect(state?.diskSnapshot).toBe('# Guide v1');+ expect(state?.markdown).toBe('# Guide v2\n');+ expect(state?.diskSnapshot).toBe('# Guide v1\n');+ });++ test('readParkedState returns the parked markdown byte for byte', async () => {+ const parked = '\n # Guide v2\r\n\nbody \n\n';+ await parkBranch(shadow, 'feature', SERVICE_WRITER.id, [+ { docName: 'guide', markdown: parked, diskSnapshot: '# Guide v1\n' },+ ]);++ const state = await readParkedState(shadow, 'feature', SERVICE_WRITER.id, 'guide');+ expect(state?.markdown).toBe(parked);});test('readParkedState returns null when no park exists', async () => {@@ -1670,6 +1681,26 @@ describe('gcCheckpointRefs (bridge-correctness SPEC §6 R7 + review iteration 5)metadata: { diskBytes: 41, discardedBytes: 66 },});return;+ case 'external-change-both-kept':+ await saveInMemoryCheckpoint(shadow, 'content/docs', {+ kind,+ docName: 'n.md',+ contents: `n${tag}\n`,+ label: 'l',+ date,+ metadata: { side: 'theirs', keptBlocks: 1 },+ });+ return;+ case 'external-change-kept-live':+ await saveInMemoryCheckpoint(shadow, 'content/docs', {+ kind,+ docName: 'o.md',+ contents: `o${tag}\n`,+ label: 'l',+ date,+ metadata: { side: 'theirs', reason: 'check-failed' },+ });+ return;case 'auto-consolidation':await writeAutoConsolidationCheckpoint(shadow, tag === '' ? 1 : Number(tag));return;@@ -1707,6 +1738,41 @@ describe('gcCheckpointRefs (bridge-correctness SPEC §6 R7 + review iteration 5)expect(result.retained).toBe(CHECKPOINT_KINDS.length);});+ test('a document-scoped history kind keeps its cap per document and never expires', async () => {+ const { gcCheckpointRefs } = await import('./shadow-repo.ts');+ const seed = (docName: string, rank: number): Promise<string> =>+ saveInMemoryCheckpoint(shadow, 'content/docs', {+ kind: 'external-change-both-kept',+ docName,+ contents: `${docName} ${rank}\n`,+ label: 'l',+ date: seedDate(rank),+ metadata: { side: 'theirs', keptBlocks: 1 },+ });+ const quiet = [await seed('quiet.md', 1), await seed('quiet.md', 2)];+ const busy: string[] = [];+ for (let rank = 3; rank <= 8; rank++) busy.push(await seed('busy.md', rank));++ const result = await gcCheckpointRefs(shadow, 'main', {+ ...DEFAULT_CHECKPOINT_RETENTION,+ maxExternalChangeBothKeptPerDocument: 3,+ ttlMs: 1,+ });++ expect(result.deletedExternalChangeBothKept).toBe(3);+ const remaining = (+ await shadowGit(shadow).raw(+ 'for-each-ref',+ '--format=%(objectname)',+ 'refs/checkpoints/main/',+ )+ )+ .trim()+ .split('\n')+ .sort();+ expect(remaining).toEqual([...quiet, ...busy.slice(3)].sort());+ });+test('every registered kind is actually REAPED at its own limit, not merely bucketed', async () => {const { gcCheckpointRefs } = await import('./shadow-repo.ts');@@ -1788,13 +1854,18 @@ describe('checkpoint chain anchoring', () => {.split(/\s+/).slice(1);- test('a kind anchors the chain exactly when its retention bucket is count-only', async () => {+ test('a kind that anchors the chain is count-only, and a single-document kind never anchors', async () => {const { GC_BUCKET_POLICY } = await import('./shadow-repo.ts');for (const kind of CHECKPOINT_KINDS) {const { chainAnchor, gcBucket } = CHECKPOINT_KIND_REGISTRY[kind];- expect(`${kind}:chainAnchor=${chainAnchor}`).toBe(- `${kind}:chainAnchor=${!GC_BUCKET_POLICY[gcBucket].applyTtl}`,- );+ if (chainAnchor) {+ expect(`${kind}:applyTtl=${GC_BUCKET_POLICY[gcBucket].applyTtl}`).toBe(+ `${kind}:applyTtl=false`,+ );+ }+ if (CHECKPOINT_SAMPLE_BY_KIND[kind].docName !== null) {+ expect(`${kind}:chainAnchor=${chainAnchor}`).toBe(`${kind}:chainAnchor=false`);+ }}});diff --git a/packages/server/src/shadow-repo.ts b/packages/server/src/shadow-repo.tsindex 3e2e4d186..2d3e0d3d5 100644--- a/packages/server/src/shadow-repo.ts+++ b/packages/server/src/shadow-repo.ts@@ -853,6 +853,22 @@ export type InMemoryCheckpointParams = (branch?: string;metadata: { diskBytes: number; discardedBytes: number };}+ | {+ kind: 'external-change-both-kept';+ docName: string;+ contents: string;+ label: string;+ branch?: string;+ metadata: Extract<ParsedCheckpoint, { kind: 'external-change-both-kept' }>['metadata'];+ }+ | {+ kind: 'external-change-kept-live';+ docName: string;+ contents: string;+ label: string;+ branch?: string;+ metadata: Extract<ParsedCheckpoint, { kind: 'external-change-kept-live' }>['metadata'];+ }) & {date?: string;};@@ -988,6 +1004,22 @@ async function saveInMemoryCheckpointInner(metadata: params.metadata,};break;+ case 'external-change-both-kept':+ parsed = {+ kind: 'external-change-both-kept',+ docName: params.docName,+ size,+ metadata: params.metadata,+ };+ break;+ case 'external-change-kept-live':+ parsed = {+ kind: 'external-change-kept-live',+ docName: params.docName,+ size,+ metadata: params.metadata,+ };+ break;}const bodyLine = formatCheckpointBodyLine(parsed);const message = `checkpoint: ${params.label}\n\n${bodyLine}`;@@ -1129,6 +1161,8 @@ export interface CheckpointRetentionPolicy {maxPersistenceDuplicationReset: number;maxPersistenceDivergenceRealign: number;maxManagedArtifactReconcile: number;+ maxExternalChangeBothKeptPerDocument: number;+ maxExternalChangeKeptLivePerDocument: number;maxAutoConsolidation: number;ttlMs: number;}@@ -1147,6 +1181,8 @@ export const DEFAULT_CHECKPOINT_RETENTION: CheckpointRetentionPolicy = {maxPersistenceDuplicationReset: 50,maxPersistenceDivergenceRealign: 50,maxManagedArtifactReconcile: 50,+ maxExternalChangeBothKeptPerDocument: 50,+ maxExternalChangeKeptLivePerDocument: 50,maxAutoConsolidation: 2,ttlMs: 30 * 24 * 60 * 60 * 1000,};@@ -1166,6 +1202,8 @@ export interface CheckpointGcResult {deletedPersistenceDuplicationReset: number;deletedPersistenceDivergenceRealign: number;deletedManagedArtifactReconcile: number;+ deletedExternalChangeBothKept: number;+ deletedExternalChangeKeptLive: number;deletedAutoConsolidation: number;retained: number;}@@ -1176,6 +1214,7 @@ interface GcBucketPolicy {limit: (policy: CheckpointRetentionPolicy) => number;counter: GcDeletionCounter;applyTtl: boolean;+ scope: 'branch' | 'document';}export const GC_BUCKET_POLICY = {@@ -1183,71 +1222,97 @@ export const GC_BUCKET_POLICY = {limit: (p) => p.maxBridgeMergeLoss,counter: 'deletedBridgeMergeLoss',applyTtl: true,+ scope: 'branch',},'producer-guard-loss': {limit: (p) => p.maxProducerGuardLoss,counter: 'deletedProducerGuardLoss',applyTtl: true,+ scope: 'branch',},'observer-a-duplication': {limit: (p) => p.maxObserverADuplication,counter: 'deletedObserverADuplication',applyTtl: true,+ scope: 'branch',},'external-change-rescue': {limit: (p) => p.maxExternalChangeRescue,counter: 'deletedExternalChangeRescue',applyTtl: true,+ scope: 'branch',},'external-change-rescue-disk-update': {limit: (p) => p.maxExternalChangeRescueDiskUpdate,counter: 'deletedExternalChangeRescueDiskUpdate',applyTtl: true,+ scope: 'branch',},'defer-exhaustion-loss': {limit: (p) => p.maxDeferExhaustionLoss,counter: 'deletedDeferExhaustionLoss',applyTtl: true,+ scope: 'branch',},'observer-a-apply-loss': {limit: (p) => p.maxObserverAApplyLoss,counter: 'deletedObserverAApplyLoss',applyTtl: true,+ scope: 'branch',},'bridge-derive-loss': {limit: (p) => p.maxBridgeDeriveLoss,counter: 'deletedBridgeDeriveLoss',applyTtl: true,+ scope: 'branch',},'bridge-backstop-trip': {limit: (p) => p.maxBridgeBackstopTrip,counter: 'deletedBridgeBackstopTrip',applyTtl: true,+ scope: 'branch',},'persistence-reconcile-loss': {limit: (p) => p.maxPersistenceReconcileLoss,counter: 'deletedPersistenceReconcileLoss',applyTtl: true,+ scope: 'branch',},'persistence-duplication-reset': {limit: (p) => p.maxPersistenceDuplicationReset,counter: 'deletedPersistenceDuplicationReset',applyTtl: true,+ scope: 'branch',},'persistence-divergence-realign': {limit: (p) => p.maxPersistenceDivergenceRealign,counter: 'deletedPersistenceDivergenceRealign',applyTtl: true,+ scope: 'branch',},'managed-artifact-reconcile': {limit: (p) => p.maxManagedArtifactReconcile,counter: 'deletedManagedArtifactReconcile',applyTtl: true,+ scope: 'branch',+ },+ 'external-change-both-kept': {+ limit: (p) => p.maxExternalChangeBothKeptPerDocument,+ counter: 'deletedExternalChangeBothKept',+ applyTtl: false,+ scope: 'document',+ },+ 'external-change-kept-live': {+ limit: (p) => p.maxExternalChangeKeptLivePerDocument,+ counter: 'deletedExternalChangeKeptLive',+ applyTtl: false,+ scope: 'document',},'auto-consolidation': {limit: (p) => p.maxAutoConsolidation,counter: 'deletedAutoConsolidation',applyTtl: false,+ scope: 'branch',},} as const satisfies Record<CheckpointKind, GcBucketPolicy>;@@ -1279,6 +1344,8 @@ async function gcCheckpointRefsInner(deletedPersistenceDuplicationReset: 0,deletedPersistenceDivergenceRealign: 0,deletedManagedArtifactReconcile: 0,+ deletedExternalChangeBothKept: 0,+ deletedExternalChangeKeptLive: 0,deletedAutoConsolidation: 0,retained: 0,};@@ -1331,6 +1398,7 @@ async function gcCheckpointRefsInner(sha: string;timestamp: number;kind: CheckpointKind | null;+ docName: string | null;}const entries: Entry[] = [];for (const record of logRaw.split('\x1e')) {@@ -1341,7 +1409,12 @@ async function gcCheckpointRefsInner(const parsed = parseCheckpoint(body);const kind = parsed?.kind ?? null;const ts = Date.parse(timestamp);- entries.push({ sha, timestamp: Number.isFinite(ts) ? ts : 0, kind });+ entries.push({+ sha,+ timestamp: Number.isFinite(ts) ? ts : 0,+ kind,+ docName: parsed?.docName ?? null,+ });}const byBucket = Object.fromEntries(@@ -1387,14 +1460,22 @@ async function gcCheckpointRefsInner(}}};+ const groupsOf = (list: Entry[], scope: GcBucketPolicy['scope']): Entry[][] => {+ if (scope === 'branch') return [list];+ const byDocument = new Map<string, Entry[]>();+ for (const entry of list) {+ const key = entry.docName ?? '';+ const group = byDocument.get(key);+ if (group) group.push(entry);+ else byDocument.set(key, [entry]);+ }+ return [...byDocument.values()];+ };for (const bucket of CHECKPOINT_KINDS) {const bucketPolicy = GC_BUCKET_POLICY[bucket];- planDeletions(- byBucket[bucket],- bucketPolicy.limit(policy),- bucketPolicy.counter,- bucketPolicy.applyTtl,- );+ for (const group of groupsOf(byBucket[bucket], bucketPolicy.scope)) {+ planDeletions(group, bucketPolicy.limit(policy), bucketPolicy.counter, bucketPolicy.applyTtl);+ }}for (const ref of deleteRefs) {@@ -1568,8 +1649,8 @@ export async function readParkedState(const msg = (await sg.raw('log', '-1', '--format=%s', refSha)).trim();if (!msg.startsWith('park:')) return null;- const markdown = (await sg.raw('show', `${refSha}:${docName}`)).trim();- const diskSnapshot = (await sg.raw('show', `${refSha}:.park-base/${docName}`)).trim();+ const markdown = await sg.raw('show', `${refSha}:${docName}`);+ const diskSnapshot = await sg.raw('show', `${refSha}:.park-base/${docName}`);return { markdown, diskSnapshot };} catch (e) {log.error({ docName, ref, err: e }, `Failed to read parked state for ${docName} from ${ref}`);diff --git a/packages/server/src/stale-external-write-gate.test.ts b/packages/server/src/stale-external-write-gate.test.tsindex 383f16b6f..fd65bc111 100644--- a/packages/server/src/stale-external-write-gate.test.ts+++ b/packages/server/src/stale-external-write-gate.test.ts@@ -4,6 +4,7 @@ import { join } from 'node:path';import type { Hocuspocus } from '@hocuspocus/server';import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest';import * as Y from 'yjs';+import { RECONCILE_TEST_CONFLICTS } from './conflict-authority.test-helper.ts';import { ConflictAuthority } from './conflict-authority.ts';import { DISPLACED_VERSION_TTL_MS, DocumentDurabilityState } from './document-durability-state.ts';import {@@ -12,6 +13,7 @@ import {serializeYDocSource,} from './external-change.ts';import { getLogger } from './logger.ts';+import { NO_SHADOW_MERGE_HISTORY } from './merge-history.test-helper.ts';import { getMetrics } from './metrics.ts';import { createPersistenceExtension } from './persistence.ts';import { MAX_LCS_CELLS } from './reconciliation.ts';@@ -94,11 +96,15 @@ describe('reconcileDiskBeforeAgentWrite — stale external write gate', () => {const before = getMetrics();writeFileSync(join(tmpDir, `${docName}.md`), STALE_CONTENT, 'utf-8');- const result = reconcileDiskBeforeAgentWrite(+ const result = await reconcileDiskBeforeAgentWrite(durabilityState,fakeHocuspocusWith(docName, document),docName,tmpDir,+ undefined,+ undefined,+ RECONCILE_TEST_CONFLICTS,+ NO_SHADOW_MERGE_HISTORY,);expect(result.reconciled).toBe(false);@@ -189,11 +195,15 @@ describe('reconcileDiskBeforeAgentWrite — stale external write gate', () => {await settleWrite();writeFileSync(join(tmpDir, `${docName}.md`), 'alpha\n\nbeta gamma\n\ndelta\n', 'utf-8');- const result = reconcileDiskBeforeAgentWrite(+ const result = await reconcileDiskBeforeAgentWrite(durabilityState,fakeHocuspocusWith(docName, document),docName,tmpDir,+ undefined,+ undefined,+ RECONCILE_TEST_CONFLICTS,+ NO_SHADOW_MERGE_HISTORY,);expect(result.reconciled).toBe(true);@@ -205,11 +215,15 @@ describe('reconcileDiskBeforeAgentWrite — stale external write gate', () => {await settleWrite(false);writeFileSync(join(tmpDir, `${docName}.md`), STALE_CONTENT, 'utf-8');- const result = reconcileDiskBeforeAgentWrite(+ const result = await reconcileDiskBeforeAgentWrite(durabilityState,fakeHocuspocusWith(docName, document),docName,tmpDir,+ undefined,+ undefined,+ RECONCILE_TEST_CONFLICTS,+ NO_SHADOW_MERGE_HISTORY,);expect(result.reconciled).toBe(true);@@ -252,11 +266,15 @@ describe('reconcileDiskBeforeAgentWrite — stale external write gate', () => {expect(refuseStaleExternalWrite(durabilityState, document, docName, STALE_CONTENT)).toBe(true);writeFileSync(join(tmpDir, `${docName}.md`), ACKNOWLEDGED_CONTENT, 'utf-8');- reconcileDiskBeforeAgentWrite(+ await reconcileDiskBeforeAgentWrite(durabilityState,fakeHocuspocusWith(docName, document),docName,tmpDir,+ undefined,+ undefined,+ RECONCILE_TEST_CONFLICTS,+ NO_SHADOW_MERGE_HISTORY,);expect(document.getMap('lifecycle').get('status')).toBeUndefined();@@ -380,20 +398,28 @@ describe('reconcileDiskBeforeAgentWrite — stale external write gate', () => {durabilityState.recordDisplacedVersion(docName, crlfStale);replaceDocParagraphs(document, ['alpha', 'beta gamma']);writeFileSync(join(tmpDir, `${docName}.md`), crlfStale, 'utf-8');- reconcileDiskBeforeAgentWrite(+ await reconcileDiskBeforeAgentWrite(durabilityState,fakeHocuspocusWith(docName, document),docName,tmpDir,+ undefined,+ undefined,+ RECONCILE_TEST_CONFLICTS,+ NO_SHADOW_MERGE_HISTORY,);expect(durabilityState.getStaleExternalWrite(docName)).toBeDefined();expect(document.getText('source').toString()).toBe(ACKNOWLEDGED_CONTENT);writeFileSync(join(tmpDir, `${docName}.md`), ACKNOWLEDGED_CONTENT, 'utf-8');- reconcileDiskBeforeAgentWrite(+ await reconcileDiskBeforeAgentWrite(durabilityState,fakeHocuspocusWith(docName, document),docName,tmpDir,+ undefined,+ undefined,+ RECONCILE_TEST_CONFLICTS,+ NO_SHADOW_MERGE_HISTORY,);expect(durabilityState.getStaleExternalWrite(docName)).toBeUndefined();});@@ -410,11 +436,15 @@ describe('reconcileDiskBeforeAgentWrite — stale external write gate', () => {const beforeReplacement = getMetrics();replaceDocParagraphs(document, ['alpha', 'beta gamma', 'next retained edit']);writeFileSync(join(tmpDir, `${docName}.md`), diskContent, 'utf-8');- reconcileDiskBeforeAgentWrite(+ await reconcileDiskBeforeAgentWrite(durabilityState,fakeHocuspocusWith(docName, document),docName,tmpDir,+ undefined,+ undefined,+ RECONCILE_TEST_CONFLICTS,+ NO_SHADOW_MERGE_HISTORY,);expect(durabilityState.getStaleExternalWrite(docName)).toBeDefined();expect(refuseStaleExternalWrite(durabilityState, undefined, docName, diskContent)).toBe(true);@@ -498,7 +528,7 @@ describe('reconcileDiskBeforeAgentWrite — stale external write gate', () => {},});- const result = reconcileDiskBeforeAgentWrite(+ const result = await reconcileDiskBeforeAgentWrite(durabilityState,fakeHocuspocusWith(docName, document),docName,@@ -506,6 +536,7 @@ describe('reconcileDiskBeforeAgentWrite — stale external write gate', () => {undefined,undefined,authority,+ NO_SHADOW_MERGE_HISTORY,);expect(result.reconciled).toBe(false);@@ -531,11 +562,15 @@ describe('reconcileDiskBeforeAgentWrite — stale external write gate', () => {const warnSpy = vi.spyOn(getLogger('reconcile'), 'warn');try {- const result = reconcileDiskBeforeAgentWrite(+ const result = await reconcileDiskBeforeAgentWrite(durabilityState,fakeHocuspocusWith(docName, document),docName,tmpDir,+ undefined,+ undefined,+ RECONCILE_TEST_CONFLICTS,+ NO_SHADOW_MERGE_HISTORY,);expect(result.reconciled).toBe(true);